Category: Network Technologies

Cisco: 2-Bytes / 4-Bytes ASN BGP scenarios

In my last post I wrote some basics about the new 4-bytes AS number which is supported on new Cisco IOS images. Now I would like to give you a brief overview about different BGP peering scenarios that you can meet in the real world. Even if the configuration examples in this article are on a Cisco hardware with the latest IOS, the BGP scenarios can be applied to any other platform.

OK, let take the following topology (the high quality PDF file can be downloaded here):

2bytesAS-4bytesAS-BGP

You can see that I think of 3 main scenarios when it comes to have in the same topology 2-Bytes only and 4-Bytes ASN BGP routers:
– 2-Bytes support ASN peering with a 4-Bytes ASN, which has a number greater that 65535 (if you remember from the previous article, this can be wrote as ASPLAIN or ASDOT)
– 4-Bytes ASN peering with a router that support 4-Bytes ASN, but in fact has a number smaller than 65535 (e.g old telecom provider which has has the ASN already asigned as 3456)
– 2-Bytes ASN peering with the same 4-Bytes supported ASN, but with a number smaller than 65535, as in the above scenario

I chose this particular scenarios because it will be interesting to see how we configure a 2-Bytes ASN BGP router which has a 4-Bytes ASN peer or how the same 2-Bytes ASN BGP router will have the AS-Path containing the 4-Bytes ASN router.

For the R1 and R3 routers I will use a C3640 with an IOS image that support only 2-Bytes ASN, and for R2 and R3 a C7200 with the newest IOS image that can support 4-Bytes ASN. All routers have a Loopback interface that will be the BGP router-id an also announced into BGP (e.g. R2 will have interface Loopback100 with IP address 2.2.2.2/32). The AS number that I will use (ignore the fact that I use public ones, it’s just to be more easy to follow):

R1 – ASN 1111
R2 – ASN 2.2 (ASPLAIN 131074)
R3 – ASN 3333
R4 – ASN 4444

I will skip all the boring point-2-point configuration and present only the BGP config.

#### R1-C3600 ####

router bgp 1111
bgp router-id 1.1.1.1
neighbor 10.10.12.2 remote-as 23456
network 1.1.1.1 mask 255.255.255.255

As you can see in BGP peering between R1 and R2 we have to use the remote-as number 23456, due to the fact that R1 is not supporting 4-Bytes ASN. If you read my previous post, or from other sources, you know that any BGP router that does not support 4-Bytes ASN, will use a transitory number for peering with 4-Bytes ASN BGP routers and this number is 23456.

#### R2-C7200 ####

First let’s be sure that the IOS is supporting 4-Bytes ASN. Simplest way:

router bgp ?
<1-65535>    Autonomous system number
<1.0-XX.YY>  4 Octets Autonomous system number

I think it’s obvious that it supports. A small note. On all Cisco platforms (usually C3600, C7200) with IOS that I tried testing 4-Bytes ASN BGP, I saw that only ASDOT format is supported. I don’t know about other software platforms (like NX-OS) or something high-end hardware if they support ASPLAIN. Now for the configuration, R2 is peering with R1 and R3:

router bgp 2.2
bgp router-id 2.2.2.2
neighbor 10.10.12.1 remote-as 1111
neighbor 10.10.23.3 remote-as 3333
network 2.2.2.2 mask 255.255.255.255

#### R3-C7200 ####

R3 supports 4-Bytes ASN, but due to the fact that it’s and “old telecom provider” it had been assigned a 2-Bytes ASN, so I will preserve this form:

router bgp 3333
bgp router-id 3.3.3.3
neighbor 10.10.23.2 remote-as 2.2
neighbor 10.10.34.4 remote-as 4444
network 3.3.3.3 mask 255.255.255.255

The difference between R3 and R1 is that R3 support for “remote-as” a 4-bytes ASN number (again only ASDOT representation). So, in it’s peering with R2, the remote ASN is 2.2

#### R4-C3600 ####

R4 only support 2-Bytes ASN, but the difference from R1 is that it has to peer with a 4-Bytes supported ASN router that in fact has a 2-Bytes AS number.

router bgp 4444
bgp router-id 4.4.4.4
neighbor 10.10.34.3 remote-as 3333
network 4.4.4.4 mask 255.255.255.255

This is the basic configuration in this topology. Let’s take now the IP address 1.1.1.1 which is advertised into BGP from R1. If you will check the AS-Path on R4 and R3, you will see a big difference. Have a look into the screen capture below to see the real environment for the above topology:

2b4b-bgp

Posted on by Calin 10

4-bytes Autonomous System Number

Last week I received a form from APNIC with a new AS numbers. When I had a look through papers I saw there something strange: AS 123456 (I replaced the original with this number). 6 digits. First I thought that there is a mistake or something, then I recall the new 4-bytes ASN. If for IPv6 the things seems to be moving slower, than for the new format of AS numbers, it seems that the things are going faster. So faster that by January 1, 2010 all BGP speaker must support this feature, according to Cisco. I didn’t understood if they refer in the document for their products or it is something that is mandatory globally. No matter how, the things are moving quite fast in this direction.

Since I have to implement a BGP configuration with this 4-Bytes ASN, I started to search with Google friend about the standards and I was surprised that there is not to much to search after. Of course there is the official RFC, some other documentation, but not real examples how to configure, troubleshoot and so on. That’s why I said it’s nice to put something together for a general understanding of what is and how does it work this 4-Bytes ASN. I assume here that reader has a basic understanding of what ASN and BGP is.

RFC 4893 is the reference for “BGP Support for Four-octet AS Number Space”. Currently the Autonomous System number is encoded as a two-octet (2-bytes) entity in BGP, meaning 16bits and this was defined in RFC 4271. The new system is using a four-octet (4-bytes) , meaning 32bits. Currently the ASN 2-bytes include a range from 1 – 65535, used in decimal plain text when configuring the BGP. The expansion from 2-bytes to 4-bytes give us 4,294,967,295 AS number which can be written in ASPLAIN or ASDOT format.

Why two formats? Mainly due to different opinions about how the 4-bytes number should be represented:

ASPLAIN representation

The RIPE NCC assigns and registers 4-byte AS Numbers in ASPLAIN format.
ASPLAIN defines the 4-byte AS Number as a basic 32-bit integer.
“2-byte only AS Numbers” refers to AS Numbers in the range 0 – 65535
“4-byte only AS Numbers” refers to AS Numbers in the range 65536 – 4294967295”
“4-byte AS Numbers” refers to AS Numbers in the range 0 – 4294967295
Advantages:
– IETF preferred notation
– continuation on how a 2-Byte AS number has been represented historically
– does not break AS-PATH REGEX
– APNIC reached consensus to adopt ASPLAIN for assignment and representation of 4-byte AS Numbers
– routers vendors appear to be supporting ASPLAIN, which will require no conversion from allocation to configuration
Disadvantages:
– long number to remember
– All existing 4-byte only assignments have been made in ASDOT

ASDOT representation

The full binary 4-byte AS number is split two words of 16 bits each. It is proposed to identify 4-byte AS Numbers using a syntax of <high
order 16 bit value in decimal>.<low order 16 bit value in decimal>:
“2-byte only AS Numbers” refers to AS Numbers in the range 0 – 65535
”4-byte only AS Numbers” refers to AS Numbers in the range 1.0 – 65535.65535
“4-byte AS Numbers” refers to AS Numbers in the range 0.0 – 65535.65535
Advantages:
– easy to read and remember
Disadvantages
-require conversion from ASPLAIN to ASDOT
-hard for regular expressions

What’s happening if in a BGP peering one router supports the new format and the other one only the old one.  The new reserved ASN 23456 is used for backward compatibility between 4-bytes and 2-bytes BGP speakers. So, if your router advertise BGP with a 4-bytes as number (doesn’t matter in which representation ASDOT or ASPLAN), the peer which does not support the new format, will translate the 4-bytes ASN into 2-bytes ASN 23456. A graphical representation of the AS path from 4-bytes to 2-bytes in BGP would be:
4bytes-2bytes-as

OK, I hope you understand the basics of 4-bytes ASN. For me, some challenge was to understand to transform the 4-bytes ASN from ASPLAIN to ASDOT. In every document that I saw on the Internet there was the same example: AS 65546 in ASPLAIN is 1.10 in ASDOT, but without no explanation. See below how I understood that the conversion takes place. If I understood it wrong, please let me know, to correct it here. After all I’m not an 4-byte ASN expert, I just try to help as much as I can.

So let’s take the number 65546.

1. 65546 / 65535 = 1 (integer) which will be the parte in front of the . (dot) in ASDOT representation.

2. 65546 – ( 65535 * 1) = 11 (see how much rest remains after 65353 going once in 65546)

3. 11 – 1 = 10 which will be the part after . (dot)

4. You obtain 1.10

As a general rule, you have an ASPLAIN number. You take 65535 and see how many times it goes, with integer in the ASPLAIN number (1 time, 2 times, 3 times…depending). This will be your decimal number before dot in ASDOT format . Then you multiple the 65535 with the integer obtained in the first step and you deduct from the ASPLAIN number. From the rest after the second operation, you deduct the the decimal you have in front of the dot. The rest in decimal that comes after the dot .

What confused me is that the number in the example was so close to the last 2-bytes ASN which is 65535.

Now for the last example, let me take a random higher number 194534 (the example from the first line) and to obtain the ASDOT format.

1. 194534 / 65535 = 2 (integer)

2 194534 – ( 65535 * 2) = 194534 – 131070 = 63464

3 63464 – 2 = 63462

4 ASDOT = 2.63462

An online converter from ASPLAIN do ASDOT you can find here: http://as4.nullroute.se/index.php

For some more detailed explanation I would like to ask you to download the Cisco and Juniper documents regarding 4-bytes ASN implementation in BGP.

If you have any useful information about this topic or if something is wrong in my post, please comment and share your knowledge.

Posted on by Calin 5

Cisco: The basics about VRF implementation

VRF, meaning Virtual Routing and Forwarding, is a technology implemented in the IP network routers that allows multiple instances of a routing table to exist on the same router in the same time. Since each VRF is independent, the same IP subnet can exist in 2 different VRFs. Basically you can overlap one IP address in 2 VRFs but without conflicting with each other. Even this is possible, I would not suggest doing so, unless you have a very good reason to do it.

Another meaning of VRF is VPN Routing and Forwarding which is a key element in Cisco’s MPLS (Multiprotocol Label Switching) VPN technology. Internet service providers often take advantage of VRF to create separate virtual private networks (VPNs) for customers. Some advantages of  using this technology is than an ISP can provision scalable IP  MPLS VPN services, generate reports (e.g. audit for services), Service Level Agreements (SLA) contracts and more…

To summarize, virtual networks enable administrators to split a physical link into multiple virtual links completely isolated one from the others. Typically, a virtual network will be dedicated to traffic from a specific application or from a specific users / customers.


Now that we clarify the basic of what is and how it works, let’s see where is VRF used the most. As you maybe guess already, this is in the MPLS VPN environment, due to the fact that in today’s business granularity is very important and VRF help network engineers to isolate and provide security for its customers in an ISP environment or to separate services in an Intranet environment. As you probably already know, MPLS functionality is based on P (Provider) routers, PE (Provider edge) routers and CE (Customer edge) routers. Each of these routers must be configured in order for MPLS to work within an enterprise’s architecture. I describe a little bit the MPLS technology, so you can understand better the topology presented below and then following configuration example:

VRF Implementation As you can see from the topology, one PE router can hold and manage multiple virtual routing table, one for each customer that an ISP have. If you are running in a private environment (e.g. Intranet), you can use MPLS VPN to separate services (e.g. office, development…) The  basic functionality is the same and I’ll show you below how to implement VRFs.

The actual configuration of VRFs  is not a complicate task to achieve. There are two main components to a VRF: The route distinguisher(RD) and the route target(RT).

The route distinguisher (RD) is a number which help identify a VPN in a provider’s network and allow for overlapping IP space.

The route target (RT) indicates the VPN membership of a route and allows VPN routes to be imported or exported into or out of your VRFs. The RT functions a little like a routing policy — determining how routes are distributed throughout the particular VPN.

The RD / RT is a 8-byte (64-bits) number which can be written down as follow:

– 16-bit AS number: your 32-bit number
(e.g.) 65000:100

or

–32-bit IP address: your 16-bit number
(e.g) 192.168.0.1:10

Usually the first method is used more often.

For some very basic VRF configuration follow the steps:
1. Enters VRF configuration mode and assigns a VRF name.

Router(config)#ip vrf vrf-name

2. Creates a VPN route distinguisher (RD) following one of the 16bit-ASN:32bit-number or 32bit-IP:16bit-number explained above

Router(config-vrf)#rd route-distinguisher

3. Creates a list of import and/or export route target communities for the specified VRF.

Router(config-vrf)# route-target {import | export | both} route-distinguisher

4. (Optional step) Associates the specified route map with the VRF.

Router(config-vrf)# import map route-map

5. Specifies an interface and enters interface configuration mode.

Router(config)# interface type number

6. Associates a VRF with an interface or subinterface.

Router(config-vrf)# ip vrf forwarding vrf-name

To check your configuration, you can use ping or traceroute tools under Cisco CLI, but remember that you have to use “vrf vrf-name” parameter:

Router# ping vrf vrf-name IP-address

Also you can check the virtual routing table:

Router# show ip route vrf vrf-name

In some of the following posts, I will present a VRF implementation following a real environment topology, but until then I hope you understood the basics of VRF functionality. It’s not hard to implement (from case to case it might be due to local topologies and technology) but it can help you to have a more granular connections and makes troubleshooting more easy, especially in the environments which have a lot of IP addresses under management.

Posted on by Calin 29

Cisco: How to use CEF for load-balancing

According to Cisco’s website, Cisco Express Forwarding (CEF) is advanced, Layer 3 IP switching technology. CEF optimizes network performance and scalability for networks with large and dynamic traffic patterns, such as the Internet, on networks characterized by intensive Web-based applications, or interactive sessions.

The term Load balancing describes a functionality in a router that distributes packets across multiple links based on layer 3 routing information.

Now, putting this two terms together we obtain Load balancing with CEF. Cisco IOS software basically supports two modes of load balancing: On per-destination or per-packet basis.
In per-destination mode all packets for a given destination are forwarded along the same path. Usage of only one path will lead to a unequal usage of the lines, as the packets to the same destination will use only one line, leaving the other unused. This can be a problem in a small environment where let’s say that a location is having 2 x E1 connection. In per-destination mode, all the packets to one destination will use one E1 connection, and the other line will remain unused. In addition to this, if this one destination is a server on which most on the site’s users will connect to, the per-destination mode will lead to an exhaust of the bandwidth available on one E1 line leaving the other one empty. On the other hand, on more developed location, where there are more destination available, the per-destination method will not have too much impact about the usage of the lines, as in this case the traffic will be split for the multiple destinations over multiple paths.

The per-packet mode guarantees equal load across all links because the forwarding process determines the outgoing interface for each packet by looking up the route table and picking the least used interface. If you read until now, than most probably you will say that this method is the best and then why the per-destination mode is the default one. Some issues with the per-packet mode would be that this method will almost always result in out-of-order packets, as advised by Ivan Pepelnjak, on his blog. What I can tell you is that this out-of-order packets has not a big impact on low-speed environment where TCP stack can deal with this problem. On the other hand on high-speed environment where video or voice traffic is expected, you can have big problems and it’s better to avoid the usage of per-packet load-balancing. Also, Cisco is advising that this ensures equal utilization of the links, but is a processor intensive task and impacts the overall forwarding performance. How much will impact on the CPU / resources usage? I cannot tell exactly because this depend on the task and traffic that the device has to handle.

OK, so if this per-packet mode is not so great, than why we should use it? The answer is that in some particular topologies or environments you cannot use other method of load-balancing and you are in desperate need of such mechanism. Below you will see an example of what kind of topology can force us to use per-packet load-balancing.

Before you configure load-balancing, you have to be sure that IP CEF is enabled on your router. In case that it is disabled, please enable it:

configure terminal
ip cef

If you want to fine tune the IP CEF load-balancing algorithm you can do this with the command:

configure terminal
ip cef load-balancing algorithm “name parameter”

where for “name” you have 3 choices:
original – Sets the load sharing algorithm to the original based on a source and destination hash.
tunnel – Sets the load sharing algorithm for use in tunnel environments or in environments where there are only a few IP source and destination address pairs.
universal – Sets the load sharing algorithm to the universal algorithm that uses a source and destination, and ID hash.
Skipping the tunnel option which you should use only if you are sure that you need it, the other choices would be universal and original. Original algorithm use IP addresses to generate the 4-bit hash. On the other hand universal algorithm add a router specific information to the hash leading to a more complex development of the hash value. Since universal option is the default one, exception the case where you know what you are doing, you should not change this value.

To enable IP CEF load-balancing on per-destination base, you don’t have to modify anything, as it’s enabled by default. For per-packet mode, you have to use the following commands:

configure terminal
interface FastEthernet x/y
ip load-balancing per-packet

ip-cef-load-balancingOne scenario where I had to use per-packet load-balancing is the one below.
Let’s assume that we have client (c-ubuntu-1, 10.10.20.100) which is sending traffic to one server (s-ubuntu-1,10.10.10.100). The routing protocol is already configured in such way that CE1 and CE2 routers are announcing, through OSPF, 2 default routes (equal cost) to SC device. The rest of the routing part is assure by BGP towards PE routers and another IGP protocols, but this has no importance for the topic discussed here.

SC1#sh ip route | i 110/1
O*E2 0.0.0.0/0 [110/1] via 172.29.190.237, 02:35:46, Vlan23
[110/1] via 172.29.190.229, 02:35:46, Vlan13

On the SC devices I used L3 interface vlan 23 to connect to CE2 and vlan 13 to connect to CE1. With the CEF enabled and per-destination load-balancing mechanism, only one  path (either vlan 23 or vlan 13) was used, leading to only one WAN serial connections to be used. Since this was client to server traffic, and quite a lot, from time to time one WAN connection was exhausted while the other one  remains unused. You can see this, in the excerpt below:

SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100    -> 10.10.10.100   : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100    -> 10.10.10.100   : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100    -> 10.10.10.100   : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100    -> 10.10.10.100   : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100    -> 10.10.10.100   : Vlan23 (next hop 172.29.190.237)

After I enabled the per-packet load-balancing, the situation has changed since both lines to CE routers where used, leading to a equal utilization of the WAN lines:

SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan13 (next hop 172.29.190.229)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan13 (next hop 172.29.190.229)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan13 (next hop 172.29.190.229)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan23 (next hop 172.29.190.237)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan13 (next hop 172.29.190.229)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan13 (next hop 172.29.190.229)
SC#sh ip cef exact-route 10.10.20.100 10.10.10.100
10.10.20.100 -> 10.10.10.100 : Vlan23 (next hop 172.29.190.237)

I had to choose this method to force the equal usage of the WAN links and I rely on TCP stack to solve the out-of-order packets as there is not so high traffic over this interfaces.

Posted on by Calin 5

Cisco: OSPF conditional inject of a Default Route

I believe most of you are familiar how OSPF is injecting a default route in a normal area. If not, you can find here all the documentation that you need. Please be familiar with this concept before reading this article.

Now, let’s assume that we have the following topology (click on image to have a more detailed view) :cisco-ospf-conditional-default-route

As you can see we have a BGP peering between PE and CE router, with CE router having and OSPF connection with the Core. CE router is injecting a default route to Core:

router ospf 1
default-information originate always

This configuration is OK, but we can run into the following issue. Imagine that for some reason the BGP peering between PE and CE is broken (e.g. line being down), the CE router will have no clue about this and will still propagate the default route to the Core. In this situation, the Core will still forward all the packets without specific route to CE where it will have no further route to reach the destination, as the CE does not receive any route from the BGP peer. As you can imagine is better to avoid this situation, especially if for some reasons you are not monitoring the connection between PE and CE and you cannot react to change manually the route in case of a failure. We are lucky because some smart engineers have developed a solution to avoid this problem, called Conditional inject of a default route in OSPF.

With this solution, OSPF is monitoring the reachability of the point-to-point IP connection between PE and CE. When OSPF process on CE router notice that the IP connection is not available anymore, it automatically retract the propagation of the default route to the Core. The solution is simple an assume use of  an ACL or prefix-list then match this on a route-map and finally use this route-map under “router ospf” process. For step-by-step configuration check below.

First we will create and ACL matching the IP subnet between PE and CE. In this example I’m using a p2p subnet 10.10.10.0 /30:

access-list 1 permit 10.10.10.0 0.0.0.3

Then I will match this into a route-map as follow:

route-map WAN-LINK permit 10
match ip address 1

Finall, we will use this route-map to implement the OSPF conditional injection of default route to Core router (192.168.0.0 /30 is the p2p IP subnet between CE and Core):

router ospf 1
log-adjacency-changes
network 192.168.10.2 0.0.0.0 area 0
default-information originate always route-map WAN-LINK

Now, the OSPF process on CE will inject a default route to the Core as long as the IP subnet between CE and PE is reachable.
IMPORTANT NOTE: This solution might not work if your connection from CE to PE is Ethernet and not Serial like in the example. I will explain why on the next post, when I’ll achieve the same behavior but using EEM together with an Ethernet based connection instead of a Serial one.

Please check below to see a small presentation how this is working on a test environment:

cisco-ospf-conditional-default-route-post

Posted on by Calin 7