The purpose of these exercises is to build Layer 2 (switched) networks utilizing the concepts explained in today's design presentations. Students will see how star topology, aggregation, virtual LANs, Spanning Tree Protocol, etc. are put to work.
There will be 6 groups of students, with 6 switches per group. The distribution of IP address space for the building (Layer 2) networks will be as follows:
|Building 1 Management||172.2X.0.0/28||2001:db8:X:3::/64|
|Building 2 Management||172.2X.0.16/28||2001:db8:X:4::/64|
You will need to replace 'X' with the number of your campus group!
The overall architecture and the full address plan can be found in the IP Address Plan
Cisco 3745 with 16 Port 10BaseT/100BaseTX EtherSwitch (NM-16ESW) module
Note: This Cisco model is actually a router, but the 16-port module provides basic Layer-2 capabilities, and we will use these as switches. Dynamips does not support the emulation of the Cisco Catalyst class of switches, unfortunately.
The following diagram shows the layout of the devices on each campus:
The following table shows the connections between each device in the campus:
|Device||Interface||Remote Device||Remote Interface|
Replace N with your building number and X with your campus number.
The instructors will assign routers and switches to each class group, and will indicate the method of access to the Dynamips server. This will usually be by wireless – if this is the case, make a note of the SSID and any password required. Also make a note of the IP address (IPv4, as Dynamips only supports IPv4 access) of the Dynamips server.
Access to Dynamips will be by telnet, to a high port, which the instructor will specify. Each participant should ensure that their device has a suitable telnet client. Linux and MacOS system have access to a shell command prompt (or Terminal) programme, which allows telnet at the command line. Windows users can use the Windows “Command Prompt” with the telnet client there, but it’s notoriously unreliable. Better to install software such as Putty, TeraTerm, HyperTerm or similar third party telnet client.
|sd1-b1-campus1||telnet s1.ws.nsrc.org 2103|
|se1-b1-campus1||telnet s1.ws.nsrc.org 2104|
|se2-b1-campus1||telnet s1.ws.nsrc.org 2105|
|sd1-b2-campus1||telnet s1.ws.nsrc.org 2106|
|se1-b2-campus1||telnet s1.ws.nsrc.org 2107|
|se2-b2-campus1||telnet s1.ws.nsrc.org 2108|
|sd1-b1-campus2||telnet s1.ws.nsrc.org 2203|
|se1-b1-campus2||telnet s1.ws.nsrc.org 2204|
|se2-b1-campus2||telnet s1.ws.nsrc.org 2205|
|sd1-b2-campus2||telnet s1.ws.nsrc.org 2206|
|se1-b2-campus2||telnet s1.ws.nsrc.org 2207|
|se2-b2-campus2||telnet s1.ws.nsrc.org 2208|
|sd1-b1-campus3||telnet s1.ws.nsrc.org 2303|
|se1-b1-campus3||telnet s1.ws.nsrc.org 2304|
|se2-b1-campus3||telnet s1.ws.nsrc.org 2305|
|sd1-b2-campus3||telnet s1.ws.nsrc.org 2306|
|se1-b2-campus3||telnet s1.ws.nsrc.org 2307|
|se2-b2-campus3||telnet s1.ws.nsrc.org 2308|
|sd1-b1-campus4||telnet s1.ws.nsrc.org 2403|
|se1-b1-campus4||telnet s1.ws.nsrc.org 2404|
|se2-b1-campus4||telnet s1.ws.nsrc.org 2405|
|sd1-b2-campus4||telnet s1.ws.nsrc.org 2406|
|se1-b2-campus4||telnet s1.ws.nsrc.org 2407|
|se2-b2-campus4||telnet s1.ws.nsrc.org 2408|
|sd1-b1-campus5||telnet s1.ws.nsrc.org 2503|
|se1-b1-campus5||telnet s1.ws.nsrc.org 2504|
|se2-b1-campus5||telnet s1.ws.nsrc.org 2505|
|sd1-b2-campus5||telnet s1.ws.nsrc.org 2506|
|se1-b2-campus5||telnet s1.ws.nsrc.org 2507|
|se2-b2-campus5||telnet s1.ws.nsrc.org 2508|
|sd1-b1-campus6||telnet s1.ws.nsrc.org 2603|
|se1-b1-campus6||telnet s1.ws.nsrc.org 2604|
|se2-b1-campus6||telnet s1.ws.nsrc.org 2605|
|sd1-b2-campus6||telnet s1.ws.nsrc.org 2606|
|se1-b2-campus6||telnet s1.ws.nsrc.org 2607|
|se2-b2-campus6||telnet s1.ws.nsrc.org 2608|
Using the client, connect to the switches you have been assigned; for example, to connect to the console port of sd1-b1-campus1:
telnet s1.ws.nsrc.org 2103
or to se2-b1-campus6:
telnet s1.ws.nsrc.org 2605
Once connected, you will see the Dynamips response, followed by the login or command prompt of the router:
bash-3.2$ telnet s1.ws.nsrc.org 2103 Trying 10.10.0.241... Connected to s1.ws.nsrc.org. Escape character is '^]'. Connected to Dynamips VM "sd1-b1-campus1" (ID 4, type c3745) - Console port Press ENTER to get the prompt. .... User Access Verification Username:
If the “Connected to Dynamips VM” won’t appear, even after hitting the Return key several times, please request help from the workshop instructors.
Our building network consists of a aggregation switch and two edge switches in each building. The backbone switches connect to the core of our campus network and serve as aggregation points for all the edge switches. Edge switches serve the end users.
Each switch will be named according to the table above: sd1-b1-campus1, se2-b1-campus5, etc
Your switches should be given a basic configuration as follows:
Router> enable Router# config terminal Enter configuration commands, one per line. End with CNTL/Z. Router(config)# hostname sd1-b1-campusX sd1-b1-campusX(config)#
Cisco devices will always try to look up the DNS for any name or address specified in the command line. You can see this when doing a trace on a router with no DNS server or a DNS server with no in-addr.arpa entries for the IP addresses. We will turn this lookup off for the labs for the time being to speed up traceroutes.
sd1-b1-campusX (config)# no ip domain-lookup
sd1-b1-campusX (config)# line con 0 sd1-b1-campusX (config-line)# transport preferred none sd1-b1-campusX (config-line)# line vty 0 4 sd1-b1-campusX (config-line)# transport preferred none
All router usernames should be cndlab with password being lab-PW. The enable password (which takes the operator into configuration mode) needs to be lab-EN.
Please do not change the username or password to anything else, or leave the password unconfigured (access to vty ports is not possible if no password is set). It is essential for a smooth operating lab that all participants have access to all routers.
sd1-b1-campusX (config)# username cndlab secret lab-PW sd1-b1-campusX (config)# enable secret lab-EN sd1-b1-campusX (config)# service password-encryption
The service password-encryption directive tells the router to encrypt all passwords stored in the router’s configuration (apart from enable secret which is already encrypted).
Note A: There is the temptation to simply have a username of cisco and password of cisco as a lazy solution to the username/password problem. Under no circumstances must any service provider operator ever use easily guessable passwords as these on their live operational network.
IMPORTANT: This sentence cannot be emphasized enough. It is quite common for attackers to gain access to networks simply because operators have used familiar or easily guessed passwords.
Note B: for IOS releases prior to 12.3, the username/secret pair is not available, and operators will have to configure username/password instead. The latter format uses type-7 encryption, whereas the former is the more secure md5 based encryption.
In order to let you telnet into your router in future modules of this workshop, you need to configure a password for all virtual terminal lines.
sd1-b1-campusX (config)# aaa new-model sd1-b1-campusX (config)# aaa authentication login default local sd1-b1-campusX (config)# aaa authentication enable default enable
This series of commands tells the router to look locally for standard user login (the username password pair set earlier), and to the locally configured enable secret for the enable login. By default, login will be enabled on all vtys for other teams to gain access.
A vital part of any Internet operational system is to record logs. The router by default will display system logs on the router console. However, this is undesirable for Internet operational routers, as the console is a 9600 baud connection, and can place a high processor interrupt load at the time of busy traffic on the network. However, the router logs can also be recorded into a buffer on the router – this takes no interrupt load and it also enables to operator to check the history of what events happened on the router. In a future module, the lab will configuration the router to send the log messages to a SYSLOG server.
sd1-b1-campusX (config)# no logging console sd1-b1-campusX (config)# logging buffer 8192 debug
which disables console logs and instead records all logs in a 8192 byte buffer set aside on the router. To see the contents of this internal logging buffer at any time, the command “sh log” should be used at the command prompt.
With the basic configuration in place, save the configuration. To do this, exit from enable mode by typing “end” or “<ctrl> Z”, and at the command prompt enter “write memory”.
sd1-b1-campusX(config)#^Z sd1-b1-campusX# write memory Building configuration... [OK] sd1-b1-campusX#
It is highly recommended that the configuration is saved quite frequently to NVRAM. If the configuration is not saved to NVRAM, any changes made to the running configuration will be lost after a power cycle or virtual machine failure
Log off the router by typing “exit”, and then log back in again. Notice how the login sequence has changed, prompting for a “username” and “password” from the user. Note that at each checkpoint in the workshop, you should save the configuration to memory – remember that powering the router off will result in it reverting to the last saved configuration in NVRAM.
Assign each switch a different IP address as follows, for example for the distribution switch in CampusX:
int vlan 1 ip address 172.2X.0.2 255.255.255.240 ipv6 address 2001:db8:X:3::2/64 no shut end
Replace the “X” with your group number:
Verify connectivity by pinging each switch within the building. Do not continue until you can ping each switch from every other switch in the building.
HINT: If ping fails, but the configuration seems OK, try doing the following:
int vlan 1 shutdown no shutdown end
(this is not normal, but most likely a bug in the IOS code somewhere)
Question: Why can't you ping a switch in Building 2 from a switch in Building 1?
Answer: They're on different subnets and we haven't configured a router to connect the buildings at this stage.
Cisco IOS has a command which let's you find out about other Cisco devices connected to the device you are on. Cisco has a proprietary protocol called Cisco Discovery Protocol (CDP). To find out about neighbouring devices connected to your switch, do:
show cdp neighbor
which will list everything connected to your switch. Note that the output of the command will list which interfaces are used to connect to which neighbouring devices.
Later in the week we're going to start using SNMP to manage the routers and switches. We'll add the necessary commands at this stage:
access-list 99 permit 100.68.X.130 ! snmp-server community NetManage RO 99 snmp ifmib ifindex persist
The access-list only allows SNMP queries from the NMM server.
If your switch doesn't take the above snmp commands, try the following instead. Even though Cisco IOS is one operating system, the implementation details on different platforms can well be different:
access-list 99 permit 100.68.X.130 ! snmp-server community NetManage RO 99 snmp-server ifindex persist
Run the following commands and pay close attention to the output:
show spanning-tree brief show spanning-tree blockedports show spanning-tree
Make a note of the answers to the above questions, as we will compare those with the answers once we do the next step. If the instructors ask you, please write them up on the classroom whiteboard too.
Configure the STP priorities explicitly for each switch, according to the plan in Appendix A.
For example, on sd1-b1-campus1:
sd1-b1-campus1(config)#spanning-tree vlan 1 priority 12288
show spanning-tree brief
Question: Why is it so important to set the priorities explicitly?
Check the output of the spanning-tree status command. What is happening on se1-bY-campusX? For example:
se1-b1-campus6#sh spanning-tree brief VLAN1 Spanning tree enabled protocol ieee Root ID Priority 12288 Address c42c.3f2c.0000 Cost 19 Port 55 (FastEthernet1/14) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 24576 Address c42d.3f2c.0000 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 300 Interface Designated Name Port ID Prio Cost Sts Cost Bridge ID Port ID -------------------- ------- ---- ----- --- ----- -------------------- ------- FastEthernet1/14 128.55 128 19 FWD 0 12288 c42c.3f2c.0000 128.53 FastEthernet1/15 128.56 128 19 BLK 0 12288 c42c.3f2c.0000 128.54
There are actually two links from se1 to sd1 in each Building. We will use these two links later on in this lab exercise.
Notice how one link is in Forwarding Mode, and the other link is in Blocking Mode. This is so we do not have a loop between the se1 and sd1 switches. If spanning tree is turned off between these two switches, we end up with a loop, traffic would not be forwarded, and the CPU load on the switches would go to 100%.
We could disable spanning tree to see what effect it has.
WARNING: Disabling spanning tree has a significant effect on the Dynamips server's CPU load. For this reason, we cannot safely demonstrate this in our virtual environment.
START OF DEMO ONLY SECTION
We'll try to set up a demo with real hardware in class - here's what we'll try on the test setup.
On a network of real switches we could type:
no spanning-tree vlan 1
Can the switches ping each other reliably now? Why?
Watch the port counters on the inter-switch links.
show interfaces stats
What happens with the counters of the connected interfaces? What is going on?
Very quickly enable STP again on all switches:
spanning-tree vlan 1
This is known as a Broadcast Storm
WARNING: Don't try this on a production network!
END OF DEMO ONLY SECTION
Disconnect sd1-b1-campusX from the rest of the network:
interface range fastEthernet 1/13 - 15 shutdown
While it is cut off from the rest, verify spanning tree status on the other switches.
a. Which switch is the root now?
b. Verify port roles and status. Verify connectivity with ping.
interface range fastEthernet 1/13 - 15 no shutdown
What happens to the spanning tree when the switch comes back online?
We now want to segment the network to separate STAFF traffic from STUDENT and network management traffic. Each of these segments will be a separate subnet.
VTP (VLAN Trunking Protocol) is a proprietary Cisco technology that allows for dynamic VLAN provisioning. We will not use it here.
Disable VTP by setting it to 'transparent mode':
vtp mode transparent
Add the vlan to the VLAN database on each switch and give them names to better identify them:
In Building 1:
vlan 41 name MGMT1
In Building 2:
vlan 42 name MGMT2
Move the IP address to the MGMT vlan
We originally set the switches so that we were using vlan 1 to manage them. It's better to manage them using a separate vlan.
In Building 1:
interface vlan 1 no ip address no ipv6 address 2001:db8:X:3::Y/64 shutdown ! interface vlan 41 ip address 172.2X.0.Y 255.255.255.240 ipv6 address 2001:db8:X:3::Y/64 no shutdown
In Building 2:
interface vlan 1 no ip address no ipv6 address 2001:db8:X:4::Y/64 shutdown ! interface vlan 42 ip address 172.2X.0.Y 255.255.255.240 ipv6 address 2001:db8:X:4::Y/64 no shutdown
Verify connectivity between switches. Can you ping? What's missing?
Configure trunk ports. Do the following for each port that needs to tag VLAN frames:
interface FastEthernet1/14 switchport mode trunk switchport trunk encapsulation dot1q
Note 1: The Cisco default is to use dot1q encapsulation (rather than the Cisco proprietary ISL). But we include the dot1q command in the configuration in any case.
Note 2: Check the table at the start of this lab to see which ports you need to modify. sd1-b1-campusX and sd1-b2-campusX are each connected to the core router, r1-core-campusX. These ports will also need to be configured as trunks.
Try pinging between switches again. It should work now.
The switches need a default route added to them so that they can forward traffic to Network Monitoring and Management server we will configure later. On each switch we add this route to forward traffic to the Core router:
In Building 1:
ip route 0.0.0.0 0.0.0.0 172.2X.0.1 ipv6 route ::/0 2001:db8:X:3::1
In Building 2:
ip route 0.0.0.0 0.0.0.0 172.2X.0.17 ipv6 route ::/0 2001:db8:X:4::1
Add the VLANs to the VLAN database on each switch and give them names to better identify them:
In Building 1:
vlan 51 name STAFF1 vlan 61 name STUDENT1
In Building 2:
vlan 52 name STAFF2 vlan 62 name STUDENT2
Designate 5 edge ports each for STAFF and STUDENT VLAN access:
On the edge (se) switches only (example is for Building 1):
interface range Fast1/1 - 5 description Access port 51 STAFF switchport mode access switchport access vlan 51 ! interface range Fast1/6 - 10 description Access port 61 STUDENT switchport mode access switchport access vlan 61
Verify which ports are members or trunks of each vlan:
show vlan-switch id <VLAN ID>
Imagine that there are computers connected to the STAFF VLAN. Would they be able to ping the switch? Explain your response.
Verify the Spanning Tree status:
show spanning-tree brief
Notice the root and bridge priorities on each VLAN (1,41,51,61) and (1,42,52,62). Are they the same?
Use the table in Appendix A to set the correct priority for each VLAN.
Note: This is called “Per-VLAN spanning tree”, or PVST. This means that the switches are creating 4 separate trees, each with its own parameters, status, calculations, etc. Imagine if you had several hundred VLANs! This is certainly not ideal. There are better standards, like “Multiple Spanning Tree” (MST), that allow the administrator to create only the desired number of trees, and map groups of VLANs to each tree. Unfortunately, this Cisco device does not support MST.
PortFast is a feature that allows end-user stations to be granted instant access to the L2 network. Instead of starting at the bottom of the Blocking-Listening- Learning-Forwarding hierarchy of states (30 seconds!), Portfast starts at the top. The port starts in Forwarding state, and if a loop is detected, STP does all its calculations and blocks the necessary ports. This feature should only be applied to ports that connect end-user stations.
Configure end-user ports on the edge (se) switches to be in PortFast mode:
interface range fast1/1 - 10 spanning-tree portfast
With PortFast, end-user ports still participate in STP. That means that anything connected to those ports can send BPDUs and participate in (and affect the status of) the spanning tree calculations. For example, if the device connected to the edge port is configured with a lower bridge priority, it becomes the root switch and the tree topology becomes suboptimal.
Another useful Cisco feature that avoids this situation is BPDUGuard. At the reception of BPDUs, the BPDU guard operation disables the port that has PortFast configured.
BPDUGuard is enabled on all ports with PortFast enabled using the following command:
spanning-tree portfast bpduguard
We now want more capacity and link redundancy between the aggregation switches. The network diagram has been updated below to include a second link between the distribution switch and the first edge switch in each building.
Configure a Port Channel between sd1-bN-campusX and se1-bN-campusX (so, for example, between sd1-b1 and se1-b1, and between sd1-b2 and se1-b2, etc). Don't forget that we need to make the Port Channel interface a trunk port too - the Aggregated Link interface (known as a LAG - Link Aggregation Group) has to be of the same type as the original underlying interfaces. Replace the N with your building number and X with your campus number.
interface fast 1/12 description First Link to se1-bN-campusX switchport mode trunk ! interface fast 1/13 description Second Link to se1-bN-campusX switchport mode trunk ! interface port-channel 1 description sd1-bN-campusX to se1-bN-campusX aggregate link switchport mode trunk ! interface range fast 1/12 - 13 channel-group 1 mode on
interface fast 1/14 description First Link to sd1-bN-campusX switchport mode trunk ! interface fast 1/15 description Second Link to sd1-bN-campusX switchport mode trunk ! interface port-channel 1 description sd1-bN-campusX to se1-bN-campusX aggregate link switchport mode trunk ! interface range fast1/14 - 15 channel-group 1 mode on
Verify the status:
show interface port-channel 1
What capacity do you have now on the new trunk? Hint: Look for the line that says BW … Kbit/sec
Disable one of the ports in the bundle on sd1-bN-campusX:
interface fast 1/12 shutdown
Is the channel still up?
Enable it again:
interface fast 1/12 no shutdown
Note: There is a standard protocol for port bundling. It's called “LACP” (Link Aggregation Control Protocol). The Cisco ESW16 network module does not support LACP, so these port channels are actually using a proprietary Cisco protocol called “EtherChannel”. All modern switches support LACP, so we strongly recommend using that, instead of any proprietary versions.
Refer to this priority table below for the appropriate priorities on each switch.
|0||Core Node||The core switches/routers will not be participating in STP... reserved in case they ever are|
|4096||Redundant Core Node||Ditto|
|12288||Building Backbone||(sd1-b1-campusX; sd1-b2-campusX)|
|20480||Secondary Backbone||This is for building complexes|
|24576||Access Switches||This is the normal edge-device priority (se1-b1-campusX; se2-b1-campusX; se1-b2-campusX; se2-b2-campusX)|
|28672||Access Switches||Used for access switches that are daisy-chained from another access switch. We're using this terminology instead of "aggregation switch" because it's hard to define when a switch stops being an access switch and becomes an aggregation switch.|
|32768||Default||No managed network devices should have this priority.|