Cisco STP forward-time command trick

This is more a topic for exams, like Cisco CCIE, but also it can appear in real-world environments.

Consider the following simple topology:
Simple L2 topology

SW1 is the spanning-tree root bridge for all VLANs. Imagine that you have a request which ask you that when a port becomes active, no matter of VLAN, it should wait 10 seconds until it transition to forwarding state.

You look into configuration mode and spot the “spanning-tree .. forward-time..” command. OK, simple enough, you go there and type:

spanning-tree vlan 1-4094 forward-time 10

Task completed. This is what I also thought. Looks pretty straight forward, but it’s a mistake which I learned from and I hope you will read this before doing it.
Lucky for me it was just a test lab not the real CCIE lab, but if I don’t teach this kind of things or spend the entire day fine-tuning STP, I tend to forget.

Back to this example, let’s see on the SW2 what’s happening after I did configure the above command on SW1. By the way, you need to configure this command only on the STP root bridge because the downstream devices will inherit the values.

On the SW2 I shutdown the root interface to force the alternate interface (which is in blocking state) to transition to forward state and check the output of “debug spanning-tree events” :

00:55:15: STP: VLAN0001 new root port Fa0/2, cost 19
00:55:15: STP: VLAN0001 Fa0/2 -> listening
00:55:25: STP: VLAN0001 Fa0/2 -> learning
00:55:35: STP: VLAN0001 sent Topology Change Notice on Fa0/2
00:55:35: STP: VLAN0001 Fa0/2 -> forwarding

From :15 seconds when the port entered Listening state until :35 seconds when it went into Forwarding state there are 20 seconds. We were required to have 10 seconds not 20. That’s not good.

Going back and checking the parameters of a STP for a particular VLAN did not give me too much information to clarify the issue:

show spanning-tree vlan 1 | i Forward
Hello Time   3 sec  Max Age 10 sec  Forward Delay 10 sec

OK, after reading again carefully the documentation I came to the understanding that value added to this command is applied to each state towards the forwarding state.
The STP Port States are: Blocking, Listening, Learning, Forwarding and Disabled.

When I did shutdown the root port, the alternate port when immediately to Listening state then waited for 10 seconds until it transition to Learning state and again 10 Seconds to achieve Forwarding state.

The correct solution was obvious now to decrease the forward-time to 5 seconds.

spanning-tree vlan 1-4094 forward-time 5

Let’s check again:

01:11:55: STP: VLAN0001 new root port Fa0/2, cost 19
01:11:55: STP: VLAN0001 Fa0/2 -> listening
01:12:00: STP: VLAN0001 Fa0/2 -> learning
01:12:05: STP: VLAN0001 sent Topology Change Notice on Fa0/2
01:12:05: STP: VLAN0001 Fa0/2 -> forwarding

Now the total time is 10 second.

I hope this will help others to avoid my mistakes.

EtherChannel over 802.1q Tunneling

Consider the following topology:
EtherChannel over 802.1q Tunneling Topology

We have one Customer with two distributed locations (SW1, R1 and SW2, R2) connected over Provider backbone. What we want to create is something like this:

EtherChannel over Provider L2 cloud

If Provider support 802.1q and L2 tunneling we can achieve a nice Etherchannel between our 2 remote locations with direct CDP visibility. Also STP and VTP is supported, just like when these SW1 and SW2 switches are directly connected.

First, lets configure SW1 and SW2 Customer devices.

On the three interfaces connected to provider devices we want to configure LACP Etherchannel:

SW1 / SW2 Customer

interface FastEthernet0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode active
interface FastEthernet0/2
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode active
interface FastEthernet0/3
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode active

Next we will configure the SW1 and SW2 ports connected to R1 and R2 devices:

SW1 / SW2 Customer

vtp mode transparent
vtp domain Customer
vlan 100
 name End2End
interface Fa0/10
switchport mode access
switchport access vlan 100

Of course another approach can be taken in terms of VTP, like having Server / Client configuration, but this was the simplest one to illustrate here.

Let’s add some IP addresses on the two routers R1 and R2:

R1 Customer

interface fa0/0
ip address

R2 Customer

interface fa0/0
ip address

Our job, as Customer, is done. What about the Provider configuration? Here is where “the magic” happens.

To provide our Customer with three end to end 802.1q tunnels, we need to create three VLANs, assign them to the interfaces pointing to Customer SW1 and SW2 and enable the 802.1q tunnels.

SW1 / SW2 Provider

vlan 10
vlan 20
vlan 30
interface FastEthernet0/1
 switchport access vlan 10
 switchport mode dot1q-tunnel
interface FastEthernet0/2
 switchport access vlan 20
 switchport mode dot1q-tunnel
interface FastEthernet0/3
 switchport access vlan 30
 switchport mode dot1q-tunnel

Of course SW1 and SW2 from Provider should have 802.1q trunk enable and allow the tranport of VLANs 10, 20 and 30:

SW1 / SW2 Provider

int fa0/4
switchport trunk mode dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20,30

OK, we have the dot1q tunneling enabled now:

SW1 / SW2 Provider

show dot1q-tunnel

dot1q-tunnel mode LAN Port(s)

Still, the Customer wants Etherchannel functionality, CDP visibility and the ability to transport own VLAN information (remember we did configure Vlan 100 on the interface of SW1 / SW2 Customer pointing to R1 / R2). Let’s enable also these ones:

SW1 / SW2 Provider

interface FastEthernet0/1
l2protocol-tunnel point-to-point lacp
l2protocol-tunnel cdp
l2protocol-tunnel stp
no cdp enable
interface FastEthernet0/2
l2protocol-tunnel point-to-point lacp
l2protocol-tunnel cdp
l2protocol-tunnel stp
no cdp enable
interface FastEthernet0/3
l2protocol-tunnel point-to-point lacp
l2protocol-tunnel cdp
l2protocol-tunnel stp
no cdp enable

Perfect, now let’s do some “show” commands to see that everything is working.

SW1 / SW2 Customer

show etherchannel 1 summary | b Group
Group  Port-channel  Protocol    Ports
1      Po1(SU)         LACP      Fa0/1(P)   Fa0/2(P)   Fa0/3(P)
show spanning-tree vlan 100

  Spanning tree enabled protocol ieee
  Root ID    Priority    32868
             Address     0011.20ab.6180
             Cost        9
             Port        56 (Port-channel1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32868  (priority 32768 sys-id-ext 100)
             Address     0014.a86b.f600
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Fa0/10              Desg FWD 19        128.3    P2p
Po1                 Root FWD 9         128.56   P2p

OK, the Etherchannel is UP and the STP is showing correct values. Let’s see if we can do a simple “ping” from R1 to R2


Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/20/24 ms

The Customer is happy, but what about the Provider, what does it see on the L2 infrastructure?

SW1 / SW2 Provider

show spanning-tree vlan 100
Spanning tree instance(s) for vlan 100 does not exist.

So, the Provider has no idea about Vlan 100 used by the Customer. This is because STP BPDUs from SW1 / SW2 Customer are tunneled inside dot1q-tunnel and hidden by the metro tags 10, 20 and 30.

One note for real life example, the Provider needs to support at least MTU 1504 so that Customer does not deal with packet fragmentation.

MAC address tracking with NMS

Let’s assume that I have a port open in my Cisco switch for guest users and I would like to keep track of the new mac-addresses that are added or existing mac-addresses that are removed from that port.

I have a NMS (Network Management System), capable of receiving SNMP, traps that will send me an e-mail alert whenever one of the above mentioned situations happens.

For this test I did use the following IP addresses:
– Cisco switch L3 interface from where the SNMP traps will be send: Lo10, /32
– Cisco switch L2 port where I want to track the mac-address change: Fa0/10
– NMS IP address:
– SNMP community string: Cisco

OK, this is a simple example, but it can be useful in some situation or for Cisco exam preparation.

First let’s bring the Lo10 interface UP and configure the IP address. Please keep in mind that without a L3 interface the SNMP configuration will raise an alarm and the system will not work.

interface Loopback10
 ip address

Next, I’ll configure the SNMP part:

SW(config)#snmp-server host traps Cisco
SW(config)#snmp-server enable traps mac-notification

Following is the actual configuration for mac-address change notification:

SW(config-if)#snmp trap mac-notification change ?
  added    Enable Mac Address added notification for this port
  removed  Enable Mac Address removed notification for this port

The above lines show the possible options. You can use them both if you want to receive notification for adding and removing of the mac-address or any of these options separately. I did use both for this test.

Let’s verify if the configuration is applied correctly:

SW#show mac address-table notification change interface fa0/10
MAC Notification Feature is Disabled on the switch
Interface            MAC Added Trap MAC Removed Trap
---------            -------------- ----------------
FastEthernet0/10     Enabled        Enabled

I find this helpful if you have open ports which require special attention and for some reason you cannot use some technique like dot1x or lock them on static mac-address.

CCIE Home Lab, what to do when your router has too little Flash space for IOS

I already explained in an older post my home lab for CCIE preparation. My BB1-BB3 routers are Cisco 2600 series and the rest of R1-R6 are emulated with Dynamips. The only problem is that one of the C2600 has too little Flash space to hold the required IOS. Memory is sufficient, but Flash not.

The only workaround I have is to load the IOS image from a TFTP server. I will explain here my procedure, maybe it’s useful for somebody else out there.

First, I recently upgrade to Ubuntu 12.04 and for some reason the TFTPD (default tftp server) was not working properly. I found TFTPD-HPA to be a good alernative, so I did install it:

sudo apt-get install tftpd-hpa

After installation you may want to check /etc/default/tftpd-hpa. On my system it looks like this:

# /etc/default/tftpd-hpa


Next step is to get a IOS image and copy it under /var/lib/tftpboot. If you need a hint, I’m using c2600-adventerprisek9-mz.124-25d.bin which needs only a small amount of memory to be installed on the Cisco 2600 platform and it’s enough for testing.

Now we need to get the IOS image on the C2600 using TFTP. Depending on your topology used for CCIE exam practice, this can be done in different ways.

Currently I’m using the workbooks from Micronicstraining (Narbik’s workbooks, if this sounds more familiar). Previously I used the ones from Internetwork Expert. The idea is that topology is pretty similar and it looks something like this:

Narbik’s workbook topology

My problematic router is the BB3. Somehow I need that BB3 is communicating with my Ubuntu server, as simple as possible, without changing ethernet cables all the time.
You noticed int the above diagram that BB3 has an interface on the SW1 (Fa0/13) and SW1 has F0/1 connected to my Ubuntu server as explained in this post. What I need is to have Fa0/13 and Fa0/1 on the SW1 on the same VLAN for proper communication. Usually I just default the interface and then everything is fine.

On the Ubuntu box, I can have an IP address on the physical card (this will not influence in any way the Dynamips emulated router attached to this interface):

eth1      Link encap:Ethernet  HWaddr 00:e0:b6:06:a6:3b
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::2e0:b6ff:fe06:a63b/64 Scope:Link
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:14100 (14.1 KB)

By default on the BB3 router I have an IOS image that can fit the Flash size (c2600-i-mz.123-26.bin) and in case that for some reasons the TFTP Server is not reachable, it will boot that image.

The BB3 startup-configuration looks like this:

interface FastEthernet0/0
 ip address
boot system tftp c2600-adventerprisek9-mz.124-25d.bin

What you have to remember:
1. Start your TFTP server and assure that it has the correct IP address on the interface where you will server IOS files
2. Start your switch (if you have one in between Cisco router and the TFTP server) and be sure that the interfaces are able to exchange packet (same VLAN, etc)
3. Start your Router

In case you did miss any of the above steps, you’ll see something like this:

%SYS-4-CONFIG_NEWER: Configuration from version 12.4 may not be correctly understood
 Slot is empty or does not support clock participate
 WIC slot is empty or does not support clock participate
%SYS-6-READ_BOOTFILE_FAIL: tftp:// File read failed -- Timed out.

 Hello from IFS_TYPE_ROM successful type-check

%SYS-6-BOOT_MESSAGES: Messages above this line are from the boot loader.
boot of "tftp:c2600-adventerprisek9-mz.124-25d.bin" using boot helper "flash:c2600-i-mz.123-26.bin" failed
error returned: File read failed -- Timed out
loadprog: error - on file open
boot: cannot load "tftp:c2600-adventerprisek9-mz.124-25d.bin"

and the router will boot your image stored locally on Flash.

Instead of doing all this work, which may generate some headache, I could just buy another router with enough Flash (and Memory). Currently I don’t want to make this investment, so I’ll stick with the above scenario.

The experience with my first Juniper certification

Last week I got my first Juniper certification, JNCIA-Junos, and I’m pretty excited about it. I had to start with this exam, which is somehow equivalent to CCNA,  for two reasons:

– it’s mandatory for all other Juniper exams
– as I start with a new vendor technology I need to get a good understanding of how Junos is working and how this CLI is different from Cisco one

Maybe you are curious why I wanted to go with another certification vendor than Cisco. Well, I felt the need to do something different, but still related to networking technology. I’m not thinking to move away from Cisco, as this is still my strong point, but I don’t want my mind to get used to think only “Cisco style”. If I may say like this, I want to “think  out of the box”. I went with Juniper as I was always curious about their products and the way they handle routing protocols, configuration, network interconnections and so on…

I must say it, the preparation and exam experience, was a very pleasant one. First of all, after registration, on the Juniper website, I got access to their Learning Portal from where I could download two books in PDF format, free of charge.  I already have the necessary networking fundamentals knowledge, so these two books were enough to prepare for this exam. If you are a beginner, there is another eLearning course available on the same Juniper Learning Portal called “Networking Fundamentals” which is also free of charge. I checked some chapters from this course and I was surprised how nice and clear the explanation was. Beginners can easy understand and learn the basic concepts.

The theory was fine, but I wanted to have some hands on experience, so I start looking on Internet about some cheap solutions to prepare. I was lucky enough to get two boxes with Junos on them. After connecting them, I was able to start with the basic configuration and ended configuring IGP, BGP, filtering, route manipulation, etc. I also established some connections between Cisco and Juniper and configure inter-vendor routing protocols IGP and BGP. That was fun and I will add in future some “how-to” on my blog.

Next, on the Juniper Learning Portal, any certification candidate can take a Pre-assessment exam. If you pass, you will get a voucher which offer you 50% discount from the exam price. A nice welcome gift!

Without breaking the agreement, I think I may say that the exam was “to the point” with clear questions expecting clear answers. No room for tricky interpretations of the questions, so if you did your lessons you will pass for sure.

On and on, it was a nice experience which did offer me an appreciated break from CCIE preparation. Depending on the available time, I think I will continue with JNCIS-ENT.

I have to mention only one ugly point during my entire experience with this Juniper certification. Those of you who follow me on Twitter or LinkedIN already know what I’m talking about. I had to attend this exam three times and not because I failed, but because the first two times, the testing center was thinking that my time worth nothing. Long story short, 1st time they said that there is an update ongoing and they cannot held exams. After rescheduling with help of Prometric, at the time and date suggested by the testing center, I went for the 2nd attempt just to find out that the testing center was closed and they did cancel (?!) all exams without any information. Finally, 3rd time I did book with another testing center. In the past I used this testing center for Cisco certifications. As expected the things went smooth this time. I’m not blaming Juniper or Prometric for the first two experiences, but nevertheless I’m expecting at least that they will notify the testing center about this unacceptable behavior.

Finally, my blog will remain mainly focused on Cisco, but I will also add some articles about Juniper / Junos in form of beginners how-to. I hope my articles will become more complex as I will gain more Juniper knowledge and my idea is to develop some scenarios which involve Cisco and Juniper mix environment.

If you have questions please use the comment form, but please don’t ask questions that I cannot respond to (ex. questions from exam).