Ccie DC Full Scale Labs PDF

CCIE Data Center Full-Scale Labs - Bootcamp
Members - CCIE Data Center Full-Scale Lab 1

CCIE DC Full-Scale Lab 1 Tasks
This workbook is not yet compatible with current DC racks for
self-paced study.
Introduction
1. Data Center Infrastructure
2. Data Center Storage Networking
3. Unified Computing
4. Data Center Virtualization
Introduction
All devices used in this scenario, with the exception of the UCS and Nexus 7K, will
be pre-configured for you with a basic initial configuration before starting. Do not
modify or remove this initial configuration, such as pre-configured MGMT0 IP
addresses, pre-configured VRFs, pre-configured routing, etc. These initial configs are
required to successfully complete this scenario.
NX-OS device logins are admin with the password Cciedc01. The UCS
Management VM's login is Administrator with the password cisco. Do not modify
the admin role on any platform, change the console speed, configure AAA, or make
any other configuration changes that would potentially lock you out of the CLI
interface. Rack rental tokens will not be refunded in cases where configuration errors
on your part cause you or the automation system to be locked out of the devices.
Pre-configured MGMT0 addresses for this scenario are as follows:
N5K1 192.168.101.51/24
N5K2 192.168.101.52/24
MDS1 192.168.101.61/24
MDS2 192.168.101.62/24
N7K1 192.168.101.71/24
Any references to "Y" in this scenario refer to the last octet of the MGMT0 interface.

1.1 UCS Initialization
Connect to UCS Fabric Interconnect A's CLI and use the following options for the
initial configuration dialog:
Enforce strong passwords: yes
Admin password: Cciedc01
Cluster: yes
Switch fabric: A
System name: UCS-FI
MGMT0 IP address: 192.168.101.201
Netmask: 255.255.255.0
Default gateway: 192.168.101.1
Cluster IP address: 192.168.101.200
Configure UCS FI B to join the cluster and use the IP address 192.168.101.202/24.
Enable both Telnet and SSH access to the Fabric Interconnects.
Score: 3 Points
1.2 Nexus 7K VDC Initialization

Create three VDCs on N7K1 as follows:
VDC 2 named N7K2
VDC 3 named N7K3
VDC 4 named N7K4
Do not inherit the VDC hostname from the default VDC's hostname.
Allocate the interfaces to these VDCs according to the diagram. Any unneeded
interfaces should be assigned to VDC 0.
Connect to these VDCs from the console and configure the admin user with the
password Cciedc01.
Configure the MGMT0 IP addresses of the VDCs as follows:
VDC 2: 192.168.101.72/24
VDC 3: 192.168.101.73/24
VDC 4: 192.168.101.74/24
Enable both telnet and SSH access to all VDCs.
Score: 5 Points
1.3 Initial IP Addressing

Configure the higher-numbered M1 port in the diagram between N7K1 and N7K3 as
a native layer 3 routed interface using the addresses 10.71.73.Y/24.
Configure the M1 ports between N7K2 and N7K4 as layer 3 Port-Channel10. Use
LACP for the Port-Channel, and the addresses 10.72.74.Y/24.
Configure N7K3 and N7K4's links to the Data Center Interconnect as layer 2 access
edge ports in VLANs 1050 and 1051, respectively. Configure interfaces VLAN 1050
and 1051 on N7K3 and N7K4, respectively, with addresses 10.50.73.0/31 and
10.51.74.0/31.
Score: 3 Points
1.4 Layer 3 Routing

Configure N7K1 and N7K2 to default to N7K3 and N7K4, respectively.
Configure N7K3 and N7K4 to peer BGP with the DCI provider. The provider uses
BGP AS 100, whereas N7K3 and N7K4 have been allocated BGP ASes 65001 and
65002, respectively. The DCI provider also requires MD5 authentication using the
password DCIPROVIDER.
Do not modify any DCI-related configuration on N5K1 or 3750G.
When complete, N7K1 and N7K2 should have IP reachability to each other over the
DCI.
Score: 5 Points
1.5 FabricPath
N5K1 and N7K4 should form Port-Channel20 using LACP on the links connecting
them according to the diagram.
Configure FabricPath on the port channel as well as the link connecting N7K4 and
N5K2 according to the diagram.
Create VLANs 200299 as FabricPath VLANs on these switches.
Authenticate all FabricPath IS-IS adjacencies using an MD5 hash of the password
FPAUTH.
Score: 6 Points
1.6 vPC+
Configure UCS-FI-A to form Port-Channel201 up to N5K1 and N5K2 using the links
in the diagram.
Configure UCS-FI-B to form Port-Channel202 up to N5K1 and N5K2 using the links
in the diagram.
From N5K1 and N5K2's perspective, these links should be vPC 201 and 202.
vPC 201 and 202 should be 802.1Q trunk links, STP edge ports, and only allow
VLANs 200299.
Use the vPC Domain ID 500 and the FabricPath Switch-ID 501.
Score: 6 Points
1.7 FabricPath Traffic Engineering

Ensure that N7K4 can use both N5K1 and N5K2 to reach their southbound Classical
Ethernet peers in VLANs 200299.

Score: 5 Points
1.8 Spanning-Tree Protocol Optimization

Modify N5K1 and N5K2's Classical Ethernet configuration so that they run the
minimum number of spanning-tree instances necessary to deliver traffic from the
northbound FabricPath domain into the southbound UCS domain.
Any new switches that are attached to the Classical Ethernet domain of N5K1 or
N5K2 that have a non-zero STP priority should not be able to be elected the STP
root bridge.
Score: 6 Points
1.9 Fabric Extenders

N7K3 has two links to each N2K1 and N2K2, which are then used to dual-home to
the UCS C200 server. Configure N7K3 to pair with N2K1 and N2K2 as FEX 131 and
132, respectively. Use Port-Channel 131 and 132, respectively.
Score: 5 Points
1.10 OTV
Configure OTV on N7K1 and N7K2 to bridge VLANs 200299 over the Data Center
Interconnect.
N7K1 should use the Site VLAN and Identifier 3001, and N7K2 should use the Site
VLAN and Identifier 3002.
Trunk the minimum number of necessary VLANs between N7K1 and N7K3, and
N7K2 and N7K4.
N7K3 and N7K4 should use PIM Sparse Mode for multicast routing with the DCI, and
use the RP address 10.0.0.51, which is hosted by the provider.
Multicast Control Plane traffic for the OTV should be tunneled over the DCI using the
group 224.71.72.0.
Multicast Data Plane traffic originating from N7K1 should use the group range
232.71.71.0/24.
Multicast Data Plane traffic originating from N7K2 should use the group range
232.72.72.0/24.
Authenticate the IS-IS adjacency between N7K1 and N7K2 using an MD5 hash of
the password OTVAUTH.
Create Interface VLAN 200 on N7K3 and N7K4 with the IP addresses
192.168.200.Y/24.
When complete, N7K3 and N7K4 should be able to ping each other over the DCI
through the OTV tunnel, as well as the VMKernel interfaces of the ESXi instances on
UCS Blades 1 and 2, and the C200 server. The ESXi addresses are
192.168.200.101, 192.168.200.102, and 192.168.200.104, respectively.
Score: 7 Points

2.1 Fibre Channel Initialization
Configure N5K1, N5K2, UCS-FI-A, and UCS-FI-B's Unified Ports in Fibre Channel
mode as shown in the diagram.
N5K1's links to MDS1 and N5K2's links to MDS2 should be configured as PortChannel101 and 102, respectively. The port channels should use dynamic
negotiation and be configured as Trunking Expansion ports.
N5K1's links to UCS-FI-A and N5K2's links to UCS-FI-B should be configured as PortChannel 103 and 104, respectively. The port channels should use dynamic
negotiation and be configured as non-trunking Fabric ports on the N5K1 and N5K2
sides.
Score: 5 Points
2.2 VSANs and Trunking

The SAN A side of the UCS blade servers will use VSAN 103, and the SAN B side
will use VSAN 104. Internal to UCS, these should map to VLANs 1103 and 1104,
respectively.
UCS-FI-A's Port-Channel103 to N5K1 and UCS-FI-B's Port-Channe104 to N5K2
should be non-trunking NP ports in VSANs 103 and 104, respectively.
N5K1's Port-Channel101 to MDS1 and N5K2's Port-Channel102 to MDS2 should be
TE ports that only forward VSANs 103 and 104, respectively.
MDS1 and MDS2's link to the SAN should be F ports in VSANs 103 and 104,
respectively.
Score: 6 Points
2.3 Fibre Channel Zoning

Configure Enhanced Zoning and Enhanced Device Aliases on both the SAN A and
SAN B sides of the UCS blade server.
Device Aliases in SAN A should be configured as follows:
Alias "FC-SAN-A" pWWN 21:00:00:1b:32:04:5e:dc
Alias "BLADE1-SAN-A" pWWN 20:00:00:cc:1e:dc:01:0a
Alias "BLADE2-SAN-A" pWWN 20:00:00:cc:1e:dc:02:0a
Device Aliases in SAN B should be configured as follows:
Alias "FC-SAN-B" pWWN 21:01:00:1b:32:24:5e:dc
Alias "BLADE1-SAN-B" pWWN 20:00:00:cc:1e:dc:01:0b
Alias "BLADE2-SAN-B" pWWN 20:00:00:cc:1e:dc:02:0b
Configure Zoning for SAN A so that both blades can reach "FC-SAN-A" on the A side.
Configure Zoning for SAN B so that both blades can reach "FC-SAN-B" on the B side.
Use the minimum amount of zones necessary to accomplish this.
Score: 5 Points
2.4 iSCSI Virtual Target

The UCS C200 is preconfigured to mount its VMware ESXi Datastores via iSCSI.
Configure the network as follows to allow for this.
The C200 uses VLAN 202 and the initiator IP address 192.168.202.104/24 for iSCSI,
and has the target address configured as 192.168.202.61.
The 3750G is preconfigured with VLAN 202 trunking toward N7K3, and an access
VLAN 202 assignment toward MDS1.
Configure N7K3 so that it trunks only VLAN 202 traffic received from the C200 server
toward MDS1.
Configure MDS1 so that the C200 server is assigned the pWWN
20:00:00:cc:1e:dc:03:0a.
Target LUNs reachable via MDS1's link in VSAN 103 to the FC SAN should be
represented with the IQN "iqn.1987-05.com.cisco:05.mds1.0101.01234567890abcde".
Ensure that the C200 is the only initiator that can use this target.
Do not add any additional zones to accomplish this.
Score: 6 Points
3.1 Address Pools
Configure default pools in the Root ORG on UCS as follows:
UUIDs 0000-000000000001 - 0000-000000000080
MAC Addresses 00:CC:1E:DC:00:01 00:CC:1E:DC:00:FF
nWWNs 20:01:00:CC:1E:DC:01:01 - 20:01:00:CC:1E:DC:01:FF
Management IPs 192.168.101.210 - 192.168.101.219 (GW 192.168.101.1)
Score: 5 Points
3.2 UCS Service Profile Templates

Create a Service Profile Initial Template that will be used for Blades 1 and 2 called
PROFILE.
UUIDs, MAC Addresses, nWWNs, and Management IPs should be pulled from the
previously created default pools.
For SAN connectivity, there should be two vHBAs, fc0 on SAN A using VSAN 103,
and fc1 on SAN B using VSAN 104.
For LAN connectivity, create five vNICs as follows:
vNIC0 named VMKernelA on Fabric A in VLAN 200
vNIC1 named VMKernelB on Fabric B in VLAN 200
vNIC2 named vMotion on Fabric B in VLAN 201
vNIC3 named VMGuestsA on Fabric A with VLANs 202 - 210
vNIC4 named VMGuestsB on Fabric B with VLANs 202 - 210
Ensure that if FI-B loses upstream connectivity that the vMotion NIC does not lose
reachability to the rest of the network.
If a change in this service profile in the future requires re-association to apply the
change, ensure that the administrator is notified before the blade is automatically
rebooted.
Score: 6 Points
3.3 Service Profiles

Create two Service Profiles from the previously created template called PROFILE1
and PROFILE2 for Blade 1 and Blade 2, respectively.
PROFILE1 should be customized as follows:
Assign vHBA FC0 the pWNN 20:00:00:cc:1e:dc:01:0a.
Assign vHBA FC1 the pWNN 20:00:00:cc:1e:dc:01:0b.
Boot to LUN 0 on the SAN target 21:00:00:1b:32:24:5e:dc via FC0 as the
primary, and then to LUN 0 on the SAN target 21:01:00:1b:32:24:5e:dc via
FC1 if booting via FC0 fails.
PROFILE2 should be customized as follows:
Assign vHBA FC0 the pWNN 20:00:00:cc:1e:dc:02:0a.
Assign vHBA FC1 the pWNN 20:00:00:cc:1e:dc:02:0b.
Boot to LUN 0 on the SAN target 21:01:00:1b:32:24:5e:dc via FC1 as the
primary, and then to LUN 0 on the SAN target 21:00:00:1b:32:24:5e:dc via
FC0 if booting via FC1 fails.
Associate PROFILE1 to Blade 1 and PROFILE2 to Blade 2. If successful, the blades
should boot their ESXi instances from the SAN.
Score: 6 Points

4.1 Nexus 1000v
Nexus 1000v VSMs are pre-installed on the ESXi instances for Blade 1 and Blade 2.
The VSM's MGMT0 IP address is 192.168.200.200, and it has a login of admin with
the password Cciedc01.
Modify the existing N1Kv configuration so that the VEM on Blade 1's ESXi host
(192.168.200.101) appears as module 10.
The VEM on Blade 2's ESXi host (192.168.200.102) should appear as module 20.
The C200's ESXi host (192.168.200.104) should dynamically choose any available
VEM slot.
Score: 5 Points
4.2 Private VLANs

Virtual Machines (VMs) Win2k8-www-1 through 6 are preconfigured with IP
addresses 192.168.255.1 through 6, and they have a pre-defined port-group on the
Nexus 1000v. These VMs can be reached through the VMware Console of the
vSphere Client and have the username/password combination Administrator/
Cciedc01.
Create Interface VLAN 204 on N7K3 with the IP address 192.168.255.73/24.
Configure Private-VLANs in such a way that all VMs can ping N7K3's VLAN 204
interface, but cannot ping each other.
Do not make changes to any other devices besides the Nexus 1000v and N7K3 to
accomplish this, including the vCenter server.
Score: 5 Points

CCIE DC Full-Scale Lab 1 Solutions

Configuration
UCS-FI-A:
Enter the configuration method. (console/gui) ?console
Enter the setup mode; setup newly or restore from backup. (setup/restore) ?setup
You have chosen to setup a new Fabric interconnect. Continue? (y/n):y
Enforce strong password? (y/n) [y]:y
Enter the password for "admin":Cciedc01
Confirm the password for "admin":Cciedc01
Is this Fabric interconnect part of a cluster(select 'no' for standalone)? (yes/no) [n]:yes
Enter the switch fabric (A/B) []:A
Enter the system name:UCS-FI
Physical Switch Mgmt0 IPv4 address :192.168.101.201
Physical Switch Mgmt0 IPv4 netmask :255.255.255.0
IPv4 address of the default gateway :192.168.101.1
Cluster IPv4 address :192.168.101.200
Configure the DNS Server IPv4 address? (yes/no) [n]:
Configure the default domain name? (yes/no) [n]:
Following configurations will be applied:
Switch Fabric=A
System Name=UCS-FI
Enforced Strong Password=yes
Physical Switch Mgmt0 IP Address=192.168.101.201
Physical Switch Mgmt0 IP Netmask=255.255.255.0
Default Gateway=192.168.101.1
Cluster Enabled=yes
Cluster IP Address=192.168.101.200
NOTE: Cluster IP will be configured only after both Fabric Interconnects are initialized
Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no):yes
Applying configuration. Please wait.
Configuration file - Ok
UCS-FI-B:
Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the c
y
Enter the admin password of the peer Fabric interconnect:Cciedc01
Connecting to peer Fabric interconnect... done
Retrieving config from peer Fabric interconnect... done
Peer Fabric interconnect Mgmt0 IP Address: 192.168.101.201
Peer Fabric interconnect Mgmt0 IP Netmask: 255.255.255.0
Cluster IP address
: 192.168.101.200
Like Nexus, UCS allows SSH access by default. Telnet can be enabled from the
UCSM GUI, or from the CLI as follows.
UCS-FI-A#scope system
UCS-FI-A /system #scope services
UCS-FI-A /system/services #enable telnet-server
UCS-FI-A /system/services* #commit-buffer
UCS-FI-A /system/services #end
UCS-FI-A#exit
1.2 Nexus 7K VDC Initialization

Configuration
First remove all interfaces from the default VDC by allowing only F2 ports. This will
force all M1 and F1 ports to be allocated to VDC 0:
N7K1#config t
N7K1(config)#feature telnet
N7K1(config)#vdc N7K1
N7K1(config-vdc)#limit-resource module-type f2
This will cause all ports of unallowed types to be removed from this vdc. Continue (y/n)? [yes]yes
N7K1(config-vdc)#show vdc membership
vdc_id: 0 vdc_name: Unallocated interfaces:
Ethernet1/1
Ethernet1/2
Ethernet1/3
Ethernet1/4
Ethernet1/5
Ethernet1/6
Ethernet1/7
Ethernet1/8
Ethernet1/9
Ethernet1/10
Ethernet1/11
Ethernet1/12
Ethernet1/13
Ethernet1/14
Ethernet1/15
Ethernet1/16
Ethernet1/17
Ethernet1/18
Ethernet1/19
Ethernet1/20
Ethernet1/21
Ethernet1/22
Ethernet1/23
Ethernet1/24
Ethernet1/25
Ethernet1/26
Ethernet1/27
Ethernet1/28
Ethernet1/29
Ethernet1/30
Ethernet1/31
Ethernet1/32
Ethernet2/1
Ethernet2/2
Ethernet2/3
Ethernet2/4
Ethernet2/5
Ethernet2/6
Ethernet2/7
Ethernet2/8
Ethernet2/9
Ethernet2/10
Ethernet2/11
Ethernet2/12
Ethernet2/13
Ethernet2/14
Ethernet2/15
Ethernet2/16
Ethernet2/17
Ethernet2/18
Ethernet2/19
Ethernet2/20
Ethernet2/21
Ethernet2/22
Ethernet2/23
Ethernet2/24
Ethernet2/25
Ethernet2/26
Ethernet2/27
Ethernet2/28
Ethernet2/29
Ethernet2/30
Ethernet2/31
Ethernet2/32
vdc_id: 1 vdc_name: N7K1 interfaces:
Now change the default VDC back to allow both M1 and F1 ports, create the other
VDCs, and allocate the needed ports.

N7K1(config)#no vdc combined-hostname
N7K1(config)#vdc N7K1
N7K1(config-vdc)#limit-resource module-type m1 f1 m1xl
This will cause all ports of unallowed types to be removed from this vdc. Continue (y/n)? [yes]yes
N7K1(config-vdc)#allocate interface Ethernet1/1-8
Moving ports will cause all config associated to them in source vdc to be removed. Are you sure you want to move the
yes
N7K1(config-vdc)#vdc N7K2 id 2
Note:
Creating VDC, one moment please ...
N7K1 %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 2 has come online N7K1(config-vdc)#

allocate interface Ethernet1/25-32
yes
yes
Note:

yes
yes
Note:

yes
N7K1(config-vdc)#allocate interface Ethernet2/5-8,Ethernet2/13-14,Ethernet2/19-20
yes
N7K1(config-vdc)#end
Now "switchto" the VDCs to configure the admin password as well as the MGMT0 IP
address.
N7K1#switchto vdc N7K2
---- System Admin Account Setup ----
Do you want to enforce secure password standard (yes/no) [y]:y

---- Basic System Configuration Dialog VDC: 2 ----
This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.
Please register Cisco Nexus7000 Family devices promptly with your

supplier. Failure to register may affect response times for initial
service calls. Nexus7000 devices must be registered to receive
entitled support services.
Press Enter at anytime to skip a dialog. Use ctrl-c at anytime

to skip the remaining dialogs.
Would you like to enter the basic configuration dialog (yes/no): n

Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2011, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php N7K2#config t
Enter configuration commands, one per line.
End with CNTL/Z. N7K2(config)#feature telnet
N7K2(config)#interface mgmt0
N7K2(config-if)#ip address 192.168.101.72/24
N7K2(config-if)#end
N7K2#switchback

of the system.



http://www.opensource.org/licenses/lgpl-2.1.php N7K3#conf t
N7K3(config)#int mgmt0
N7K3(config-if)#end
N7K3#switchback

of the system.



http://www.opensource.org/licenses/lgpl-2.1.php N7K4#config t
N7K4(config)#interface mgmt 0
N7K4(config-if)#end
N7K4#switchback
N7K1#copy running-config startup-config vdc-all
[#######
17%
[############
29%
[######################
53%
[############################
69%
[#####################################
90%
[########################################] 100%
Verification
N7K1#show vdc membership
vdc_id: 0 vdc_name: Unallocated interfaces:
Ethernet2/1
Ethernet2/2
Ethernet2/9
Ethernet2/10
Ethernet2/11
Ethernet2/12
Ethernet2/15
Ethernet2/16
Ethernet2/17
Ethernet2/18
Ethernet2/25
Ethernet2/26
Ethernet2/27
Ethernet2/28
Ethernet2/29
Ethernet2/30
Ethernet2/31
Ethernet2/32

Ethernet1/1
Ethernet1/2
Ethernet1/3
Ethernet1/4
Ethernet1/5
Ethernet1/6
Ethernet1/7
Ethernet1/8

Ethernet1/25
Ethernet1/26
Ethernet1/27
Ethernet1/28
Ethernet1/29
Ethernet1/30
Ethernet1/31
Ethernet1/32
Ethernet2/3
Ethernet2/4

Ethernet1/9
Ethernet1/10
Ethernet1/11
Ethernet1/12
Ethernet1/13
Ethernet1/14
Ethernet1/15
Ethernet1/16
Ethernet2/21
Ethernet2/22
Ethernet2/23
Ethernet2/24

Ethernet1/17
Ethernet1/18
Ethernet1/19
Ethernet1/20
Ethernet1/21
Ethernet1/22
Ethernet1/23
Ethernet1/24
Ethernet2/5
Ethernet2/6
Ethernet2/7
Ethernet2/8
Ethernet2/13
Ethernet2/14
Ethernet2/19
Ethernet2/20
Some interfaces not listed on the diagram must still be allocated to

VDCs 1 - 4 due to the port-group boundaries. Port-groupings can be
verified as shown below.
N7K1#show interface capabilities | include "Ethernet|Group"

Ethernet1/1 Port Group Members:
1,3,5,7
Ethernet1/2 Port Group Members:
2,4,6,8
Ethernet1/3
Port Group Members:
1,3,5,7
Ethernet1/4
Port Group Members:
2,4,6,8
Ethernet1/5
Port Group Members:
1,3,5,7
Ethernet1/6
Port Group Members:
2,4,6,8
Ethernet1/7
Port Group Members:
1,3,5,7
Ethernet1/8
Port Group Members:
2,4,6,8
1.3 Initial IP Addressing

Configuration
N7K1:
interface Ethernet1/2
ip address 10.71.73.71/24
no shutdown
N7K2:
feature lacp
!
channel-group 10 mode active
no shutdown
!
no shutdown
!
interface port-channel10
ip address 10.72.74.72/24
N7K3:
feature interface-vlan
!
vlan 1050
!
ip address 10.71.73.73/24
no shutdown
!
switchport access vlan 1050
spanning-tree port type edge
no shutdown
!
interface Vlan1050
no shutdown
ip address 10.50.73.0/31
N7K4:
!
feature lacp
!
vlan 1051
!
no shutdown
!
no shutdown
!
ip address 10.72.74.74/24
!
no shutdown
!
interface Vlan1051
no shutdown
ip address 10.51.74.0/31
Verification
N7K2#show port-channel summary
Flags:
D - Down
P - Up in port-channel (members)
I - Individual
H - Hot-standby (LACP only)
s - Suspended
r - Module-removed
S - Switched
R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
-------------------------------------------------------------------------------Group Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------10
Po10(RU)
Eth
LACP
Eth1/25(P)
Eth1/26(P)
N7K2#show ip route direct

IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%' in via output denotes VRF
10.72.74.0/24
, ubest/mbest: 1/0, attached *via 10.72.74.72, Po10
, [0/0], 21:47:09, direct
N7K2#ping 10.72.74.72
PING 10.72.74.72 (10.72.74.72): 56 data bytes
64 bytes from 10.72.74.72: icmp_seq=0 ttl=255 time=0.597 ms
--- 10.72.74.72 ping statistics --- 5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.295/0.422/0.597 ms
1.4 Layer 3 Routing

Configuration
N7K1:
ip route 0.0.0.0/0 10.71.73.73
N7K2:
ip route 0.0.0.0/0 10.72.74.74
N7K3:
feature bgp
!
router bgp 65001
address-family ipv4 unicast
network 10.71.73.0/24
neighbor 10.50.73.1
remote-as 100
password 0 DCIPROVIDER
N7K4:
feature bgp
!
router bgp 65002
log-neighbor-changes
network 10.72.74.0/24
neighbor 10.51.74.1
remote-as 100
password 0 DCIPROVIDER
Verification
N7K3#show ip bgp neighbors
BGP neighbor is 10.50.73.1,
remote AS 100, ebgp link,
Peer index 1
BGP version 4, remote router ID 10.0.0.50

BGP state = Established, up for 21:47:57
Peer is directly attached, interface Vlan1050 TCP MD5 authentication is enabled
N7K3#show bgp ipv4 unicast summary

BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 10.71.73.73, local AS number 65001
BGP table version is 8, IPv4 Unicast config peers 1, capable peers 1
4 network entries and 4 paths using 496 bytes of memory
BGP attribute entries [4/512], BGP AS path entries [2/16]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor
10.50.73.1
AS MsgRcvd MsgSent
100
1301
1310
TblVer
8
InQ OutQ Up/Down

0
State/PfxRcd
0 21:45:52 3
N7K3#show bgp ipv4 unicast

BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 8, local router ID is 10.71.73.73
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath
Network
Next Hop
*>e10.0.0.50/32
10.50.73.1
*>e10.0.0.51/32
10.50.73.1
*>l10.71.73.0/24
0.0.0.0
10.50.73.1
N7K3#show ip route bgp
IP Route Table for VRF "default"
Metric
LocPrf
Weight Path
0 100 i
0 100 i
100
32768 i *>e10.72.74.0/24
0 100 65002 i

'%' in via output denotes VRF
10.0.0.50/32, ubest/mbest: 1/0

*via 10.50.73.1, [20/0], 21:46:03, bgp-65001, external, tag 100
10.0.0.51/32, ubest/mbest: 1/0
*via 10.50.73.1, [20/0], 21:46:03, bgp-65001, external, tag 100 10.72.74.0/24
, ubest/mbest: 1/0 *via 10.50.73.1, [20/0], 21:45:14, bgp-65001
, external, tag 100
N7K2#ping 10.71.73.71
PING 10.71.73.71 (10.71.73.71): 56 data bytes
1.5 FabricPath
Configuration
N5K1:
install feature-set fabricpath
feature-set fabricpath
feature lacp
!
vlan 200-299
mode fabricpath
!
key chain FABRICPATH
key 1
key-string 0 FPAUTH
!
switchport
switchport mode fabricpath
fabricpath isis authentication-type md5
fabricpath isis authentication key-chain FABRICPATH
!

no shutdown
!
no shutdown
N5K2:
!
vlan 200-299
mode fabricpath
!
key 1
key-string 0 FPAUTH
!
no shutdown
N7K1:
N7K4:
!
vlan 200-299
mode fabricpath
!
key 1
key-string 0 FPAUTH
!
switchport
!

no shutdown
!
no shutdown
!
no shutdown
Verification
N7K4#show port-channel summary
Flags:
D - Down
I - Individual
s - Suspended
r - Module-removed
S - Switched
R - Routed
-------------------------------------------------------------------------------Group Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------10
Po10(RU)
Eth
LACP
Eth1/17(P)
Eth1/18(P)
20
Po20(SU)
Eth
LACP
Eth2/7(P)
Eth2/13(P)
N7K4#show fabricpath isis adjacency

Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID
SNPA
Level
State
Hold Time
Interface
N5K1
N/A
UP
00:00:24
port-channel20
N5K2
N/A
UP
00:00:29
Ethernet2/6
N7K4#show fabricpath isis interface port-channel 20

Fabricpath IS-IS domain: default
Interface: port-channel20
Status: protocol-up/link-up/admin-up
Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1 Authentication type MD5
Authentication keychain is FABRICPATH
Authentication check specified
Extended Local Circuit ID: 0x16000013, P2P Circuit ID: 0000.0000.0000.00

Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 1500
P2P Adjs: 1, AdjsUp: 1, Priority 64

Hello Interval: 10, Multi: 3, Next IIH: 00:00:04
Level
Adjs
AdjsUp
Metric
CSNP
20
60
Next CSNP
Last LSP ID
00:00:55
ffff.ffff.ffff.ff-ff
Topologies enabled:
Topology Metric
MetricConfig Forwarding
no
20
UP
1.6 vPC+
Configuration
N5K1:
feature vpc
!
vpc domain 500
peer-keepalive destination 192.168.101.52
fabricpath switch-id 501
!
no shutdown
!
no shutdown
!
switchport mode trunk
switchport trunk allowed vlan 200-299
no shutdown
!
no shutdown
!
vpc peer-link

!
spanning-tree port type edge trunk
vpc 201
!
vpc 202
N5K2:
feature vpc
feature lacp
!
vpc domain 500
peer-keepalive destination 192.168.101.51
fabricpath switch-id 501
!
no shutdown
!
no shutdown
!
no shutdown
!
no shutdown
!

vpc peer-link
!
vpc 201
!
vpc 202
Connect to the UCSM using the credentials that you previously configured. Next,
under the Fabric Interconnects on the Equipment tab, configure the Ethernet links
connecting northbound to the N5Ks in the diagram as Uplink Ports.
Now under the LAN tab, create and enable Port-Channels 201 and 202 on FI-A and
FI-B respectively.
Verification
N5K1# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id
vPC+ switch id
: 500
: 501
Peer status
: peer adjacency formed ok
vPC keep-alive status
: peer is alive
vPC fabricpath status
: peer is reachable through fabricpath
Configuration consistency status: success

Per-vlan consistency status
: success
Type-2 consistency status
: success
vPC role
: secondary
Number of vPCs configured
: 2
Peer Gateway
: Disabled
Dual-active excluded VLANs
: -
Graceful Consistency Check
: Enabled
vPC Peer-link status

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ -------------------------------------------------- 1
Po500
vPC status
--------------------------------------------------------------------------id
Port
Status Consistency Reason
Active vlans vPC+ Attrib
--
----------
------ ----------- ------
------------ -----------
201
Po201
up
200-299
202
Po202
success
up
success
success
success
200-299
DF: Partial
DF: Partial
UCS-FI-A:
UCS-FI-A#connect nxos
UCS-FI-A(nxos)#show run interface ethernet 1/4 - 5
description U: Uplink
pinning border
switchport trunk allowed vlan 1,200-299
no shutdown
description U: Uplink
pinning border
switchport trunk allowed vlan 1,200-299
no shutdown
UCS-FI-A(nxos)#show port-channel summary
Flags:
D - Down
I - Individual
s - Suspended
r - Module-removed
S - Switched
R - Routed
--------------------------------------------------------------------------------
up
200-299
Group Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------201
Po201(SU)
Eth
LACP
Eth1/4(P)
Eth1/5(P)
1.7 FabricPath Traffic Engineering

Configuration
N7K4:
fabricpath isis metric 40
Verification
N5K1 and N5K2 share the emulated FabricPath Switch-ID 501 for the vPC+, as
shown below:
N7K4#show fabricpath switch-id
FABRICPATH SWITCH-ID TABLE

Legend: '*' - this system
=========================================================================
SWITCH-ID
SYSTEM-ID
FLAGS
STATE
STATIC
EMULATED
----------+----------------+------------+-----------+-------------------501
547f.ee79.137c
501
*645
Primary
547f.ee7a.4d7c
Confirmed
Primary
No
Confirmed
Yes
No
Yes
64a0.e742.8dc4
Primary
Confirmed
No
No
1207
547f.ee79.137c
Primary
Confirmed
No
No
3550
547f.ee7a.4d7c
Primary
Confirmed
No
No
Total Switch-ids: 5
The port channel between N7K4 and N5K1 has an IS-IS metric of 20, whereas the
single 10GigE link from N7K4 to N5K2 has an IS-IS metric of 40. This means that
the shortest path from N7K4 to Switch-ID 501 (the vPC+ pair) is only via N5K1.
N7K4#show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id

'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/645/0, number of next-hops: 0

via ---- , [60/0], 0 day/s 22:19:30, local 1/501/0, number of next-hops: 1
via Po20, [115/20]
, 0 day/s 20:30:58, isis_fabricpath-default
via Po20, [115/40], 0 day/s 20:30:58, isis_fabricpath-default
via Eth2/6, [115/40], 0 day/s 22:19:16, isis_fabricpath-default
To allow for Equal Cost Multipath (ECMP), the port channel to N5K1 and the single
link to N5K2 must have equal costs. This can be configured either by raising the
cost of the port channel or by lowering the cost of the link to N5K2.
N7K4#config t
End with CNTL/Z. N7K4(config)#interface port-channel20
N7K4(config-if)#fabricpath isis metric 40
N7K4(config-if)# end
Now Switch-ID 501 is reachable via both N5K1 and N5K2 with a metric of 40.
N7K4#show fabricpath route
FabricPath Unicast Route Table

'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default

via ---- , [60/0], 0 day/s 22:19:58, local 1/501/0, number of next-hops: 2
via Po20, [115/40]

, 0 day/s 20:31:26, isis_fabricpath-default via Eth2/6, [115/40]
, 0 day/s 00:00:06, isis_fabricpath-default
via Eth2/6, [115/40], 0 day/s 22:19:44, isis_fabricpath-default
1.8 Spanning-Tree Protocol Optimization

Configuration
N5K1:
spanning-tree mode mst
spanning-tree mst 0 priority 0
spanning-tree mst configuration
name MST0
revision 1
N5K2:
spanning-tree mode mst

spanning-tree mst 0 priority 0
spanning-tree mst configuration
name MST0
revision 1
Verification
In the below output, we can see that both N5K1 and N5K2 have collapsed all of their
STP instances into the single default MST0 instance. Additionally, both switches in
the vPC+ pair should always appear as the root of the Spanning-Tree, and share
the Bridge-ID c84c.75fa.6000. Note that Spanning-Tree only forwards southbound
toward the Classical Ethernet domain, and not northbound toward the FabricPath
domain.
N5K1#show spanning-tree mst 0
##### MST0 vlans mapped:
1-4094
Bridge
address c84c.75fa.6000
Root
this switch for the CIST
priority
(0 sysid 0)
Regional Root this switch

Operational
hello time 2 , forward delay 15, max age 20, txholdcount 6
Configured
hello time 2 , forward delay 15, max age 20, max hops
20
Interface
Role Sts Cost
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------Po201
Desg FWD 200
128.4296 (vPC) Edge P2p
Po202
Desg FWD 200
128.4297 (vPC) Edge P2p
N5K2#show spanning-tree mst 0

##### MST0 vlans mapped:
1-4094
Bridge
address c84c.75fa.6000
Root
this switch for the CIST
priority
(0 sysid 0)
Regional Root this switch

Operational
hello time 2 , forward delay 15, max age 20, txholdcount 6
Configured
hello time 2 , forward delay 15, max age 20, max hops
Interface
Role Sts Cost
20
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------Po201
Desg FWD 200
128.4296 (vPC) Edge P2p
Po202
Desg FWD 200
128.4297 (vPC) Edge P2p
Eth1/11
Desg FWD 20000
128.139
1.9 Fabric Extenders

N7K1:
install feature-set fex
N7K3:
feature-set fex
!
switchport
switchport mode fex-fabric
fex associate 131
!
switchport
fex associate 132
!
switchport
fex associate 131
channel-group 131
no shutdown
P2p Bound(PVST)
!
switchport
fex associate 131
channel-group 131
no shutdown
!
switchport
fex associate 132
channel-group 132
no shutdown
!
switchport
fex associate 132
channel-group 132
no shutdown
!
interface Ethernet131/1/1
switchport
no shutdown
!
switchport
no shutdown
Verification
N7K3#show fex
FEX
Number
FEX
FEX
Description
State
FEX
Model
Serial
-----------------------------------------------------------------------131
FEX0131
Online
N2K-C2232PP-10GE
FOC17100NHX 132
N2K-C2232PP-10GE
FOC17100NHU
FEX0132
Online
N7K3#show fex detail

FEX: 131 Description: FEX0131
state: Online
FEX version: 6.0(2) [Switch version: 6.0(2)]

FEX Interim version: 6.0(2.9)
Switch Interim version: 6.0(2)
Extender Model: N2K-C2232PP-10GE,
Extender Serial: FOC17100NHX
Part No: 73-12533-05

Card Id: 82, Mac Addr: f0:29:29:ff:00:42, Num Macs: 64
Module Sw Gen: 12594
pinning-mode: static
[Switch Sw Gen: 21]

Max-links: 1
Fabric port for control traffic: Eth1/14

Fabric interface state: Po131 - Interface Up. State: Active
Eth1/13 - Interface Up. State: Active
Fex Port
State
Fabric Port Eth131/1/1
FEX: 132 Description: FEX0132
Up
Po131
state: Online
FEX version: 6.0(2) [Switch version: 6.0(2)]

FEX Interim version: 6.0(2.9)
Switch Interim version: 6.0(2)
Extender Model: N2K-C2232PP-10GE,
Extender Serial: FOC17100NHU
Part No: 73-12533-05

Card Id: 82, Mac Addr: f0:29:29:ff:02:02, Num Macs: 64
Module Sw Gen: 12594
pinning-mode: static
[Switch Sw Gen: 21]

Max-links: 1
Fabric port for control traffic: Eth1/15

Fabric interface state: Po132 - Interface Up. State: Active
Fex Port
State
Fabric Port Eth132/1/1
Up
Po132
1.10 OTV
Configuration
The OTV Site VLAN is in decimal, but the OTV Site Identifier is in hex,
which means that a decimal to hex conversion is required.
N7K1:
feature otv
!
vlan 200-299,3001
!
otv site-vlan 3001
otv site-identifier 0xbb9
!
spanning-tree vlan 3001 priority 0
!
key chain OTV
key 1
key-string 0 OTVAUTH
!
interface Overlay1
otv isis authentication-type md5
otv isis authentication key-chain OTV
otv join-interface Ethernet1/2
otv control-group 224.71.72.0
otv data-group 232.71.71.0/24
otv extend-vlan 200-299
no shutdown
!
switchport
switchport trunk allowed vlan 200-299,3001
no shutdown
!
ip igmp version 3
N7K2:
feature otv
!
vlan 200-299,3002
!
otv site-vlan 3002
otv site-identifier 0xbba
!
key chain OTV
key 1
key-string 0 OTVAUTH
!
ip igmp version 3
!
interface Overlay1
otv join-interface port-channel10

otv extend-vlan 200-299
no shutdown
!
no shutdown
N7K3:
feature pim
!
vlan 200-299,3001
!
interface Vlan200
no shutdown
ip address 192.168.200.73/24
!
interface Vlan1050
ip pim sparse-mode
!
switchport
no shutdown
!
ip pim sparse-mode
ip igmp version 3
!
ip pim rp-address 10.0.0.51 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
N7K4:
feature pim
!
vlan 3002
!
spanning-tree vlan 200-299 priority 0
!
interface Vlan200
no shutdown
ip address 192.168.200.74/24
!
interface Vlan1051
ip pim sparse-mode
!
ip pim sparse-mode
ip igmp version 3
!
no shutdown
!
Verification
To establish the OTV tunnel, the AEDs must have multicast reachability to each
other with the control group. The first step in verification, then, is to ensure that the
tree for the control multicast group is built in the DCI core. Both N7K3 and N7K4
should see the (S,G) entries for the control group 224.71.72.0.
N7K3#show ip mroute
IP Multicast Routing Table for VRF "default"
(*, 224.71.72.0/32), uptime: 00:11:06, igmp ip pim

Incoming interface: Vlan1050, RPF nbr: 10.50.73.1
Outgoing interface list: (count: 1)
Ethernet1/10, uptime: 00:11:06, igmp
(10.71.73.71/32, 224.71.72.0/32)
, uptime: 00:12:45, ip pim mrib
Incoming interface: Ethernet1/10
, RPF nbr: 10.71.73.71 Outgoing interface list

: (count: 2)
Ethernet1/10, uptime: 00:11:06, mrib, (RPF) Vlan1050
, uptime: 00:12:34, pim
(10.72.74.72/32, 224.71.72.0/32)
, uptime: 00:11:03, ip mrib pim
Incoming interface: Vlan1050

: (count: 1) Ethernet1/10
, uptime: 00:11:03, mrib
(*, 232.0.0.0/8), uptime: 00:12:54, pim ip

Incoming interface: Null, RPF nbr: 0.0.0.0
N7K4#show ip mroute
(*, 224.71.72.0/32), uptime: 00:13:47, igmp ip pim

Incoming interface: Vlan1051, RPF nbr: 10.51.74.1
port-channel10, uptime: 00:13:47, igmp
(10.71.73.71/32, 224.71.72.0/32)

: (count: 1) port-channel10
, uptime: 00:13:39, mrib
(10.72.74.72/32, 224.71.72.0/32)
Incoming interface: port-channel10

: (count: 2) Vlan1051
, uptime: 00:12:18, pim
port-channel10, uptime: 00:13:44, mrib, (RPF)
(*, 232.0.0.0/8), uptime: 00:13:53, pim ip

Incoming interface: Null, RPF nbr: 0.0.0.0
Ensure that the Site VLAN is up on both AEDs.

N7K1#show otv
OTV Overlay Information

Site Identifier 0000.0000.0bb9
Overlay interface Overlay1
VPN name
: Overlay1
VPN state
: UP
Extended vlans
: 200-299 (Total:100)
Control group
: 224.71.72.0
Data group range(s) : 232.71.71.0/24

Join interface(s)
: Eth1/2 (10.71.73.71)
AED-Capable
: Yes
Capability
: Multicast-Reachable
N7K2#show otv

Site Identifier 0000.0000.0bba
Site vlan
: 3001 (up)
VPN name
: Overlay1
VPN state
: UP
Extended vlans
: 200-299 (Total:100)
Control group
: 224.71.72.0

Join interface(s)
: Po10 (10.72.74.72)
AED-Capable
: Yes
Capability
: Multicast-Reachable
Site vlan
: 3002 (up)
Now the AEDs should be able to form an IS-IS adjacency over the OTV tunnel.
N7K1#show otv isis adjacency
OTV-IS-IS process: default VPN: Overlay1
OTV-IS-IS adjacency database:
System ID
SNPA
Level
State
Hold Time
Interface Site-ID
N7K2
64a0.e742.8dc2
UP
00:00:08
Overlay1 0000.0000.0bba
Verify that MD5 authentication for IS-IS is enabled on the Overlay1 interface.
N7K1#show otv isis interface overlay 1
OTV-IS-IS process: default VPN: Overlay1

Overlay1, Interface status: protocol-up/link-up/admin-up
IP address: none
IPv6 address: none
IPv6 link-local address: none
Index: 0x0001, Local Circuit ID: 0x01, Circuit Type: L1
Level1
Adjacency server (local/remote) : disabled / none
Adjacency server capability : multicast Authentication type is MD5
Authentication keychain is OTV
Authentication check specified
LSP interval: 33 ms, MTU: 1400

Level
Metric
CSNP
40
10
Level
1
Adjs
AdjsUp Pri
64
Next CSNP
Hello
Inactive
10
Multi
Next IIH
00:00:03
Circuit ID
Since
N7K2.01
00:15:55
N7K3 and N7K4 should now be able to reach each other's VLAN 200 interfaces,
and the OTV AEDs should learn the routes to these MAC addresses.
N7K4#show interface vlan 200 | include ddress
Hardware is EtherSVI, address is 64a0.e742.8dc4
Internet Address is 192.168.200.74/24
N7K3#ping 192.168.200.74
PING 192.168.200.74 (192.168.200.74): 56 data bytes
N7K1#show otv route
OTV Unicast MAC Routing Table For Overlay1
VLAN MAC-Address
Metric
Uptime
Owner
Next-hop(s)
---- --------------
------
--------
---------
-----------
42
00:18:25
overlay
N7K2
200 000c.29bb.9b82
200 64a0.e742.8dc3
200 64a0.e742.8dc4
200 d48c.b5bd.460c
1
42
1
00:18:15
00:18:14
00:18:23
site
Ethernet1/1
overlay
site
N7K2
Ethernet1/1
N7K2#show otv route
VLAN MAC-Address
Metric
Uptime
Owner
Next-hop(s)
---- --------------
------
--------
---------
-----------
200 000c.29bb.9b82
00:19:03
site
Ethernet2/3
200 64a0.e742.8dc3
42
00:18:24
overlay
N7K1
200 64a0.e742.8dc4
200 d48c.b5bd.460c
42
00:18:24
00:18:32
site
Ethernet2/3
overlay
N7K1
Multicast tunneling can be verified by joining a multicast group on one of the

switches and then sending ICMP pings from the remote OTV site. If successful, a
new OTV multicast tunnel should form using the OTV multicast data groups.
N7K3#config t
End with CNTL/Z. N7K3(config)#interface vlan 200
N7K3(config-if)#ip pim sparse-mode

N7K3(config-if)#ip igmp join-group 224.1.1.1
N7K4#ping multicast 224.1.1.1 interface vlan 200
PING 224.1.1.1 (224.1.1.1): 56 data bytes
--- 224.1.1.1 ping multicast statistics --5 packets transmitted, From member 192.168.200.73: 5 packets received, 0.00% packet loss
--- in total, 1 group member responded --N7K3#show ip mroute 232.72.72.0
(10.72.74.72/32, 232.72.72.0/32)
, uptime: 00:02:44, igmp ip pim

: (count: 1) Ethernet1/10
, uptime: 00:02:44, igmp

2.1 Fibre Channel Initialization
Configuration
N5K1:
feature fcoe
feature npiv
feature fport-channel-trunk
!
slot 1
port 28-32 type fc
!
interface fc1/28
channel-group 101
no shutdown
!
interface fc1/29
channel-group 101
no shutdown
!
interface fc1/30
switchport mode F
switchport trunk mode off
channel-group 103
no shutdown
!
interface fc1/31
switchport mode F
channel-group 103
no shutdown
!
interface san-port-channel 101
channel mode active
!
channel mode active
switchport mode F
N5K2:
feature fcoe
feature npiv
feature fport-channel-trunk
!
slot 1
port 28-32 type fc
!
interface fc1/28
channel-group 102
no shutdown
!
interface fc1/29
channel-group 102
no shutdown
!
interface fc1/30
switchport mode F
channel-group 104
no shutdown
!
interface fc1/31
switchport mode F
channel-group 104
no shutdown
!
channel mode active
!
channel mode active
switchport mode F
MDS1:
interface fc1/3
channel-group 101
no shutdown
!
interface fc1/4
channel-group 101
no shutdown
!
interface port-channel 101
channel mode active
MDS2:
interface fc1/3
channel-group 102
no shutdown
!
interface fc1/4
channel-group 102
no shutdown
!
channel mode active
In UCSM, go to the Equipment tab, and then, under the Fabric Interconnects, go to
Configure Unified Ports. Just like on the 5Ks, changing the port type from Ethernet
to Fibre Channel requires a reboot, so to save time, start with FI-B first, and then
configure FI-A.
When the FIs have rebooted, go to the SAN tab and configure FC uplinks on FI-A
and FI-B as SAN-Port-Channels 103 and 104, respectively. Remember to enable
the port channels when created, because like on the 5Ks, they are in the shutdown
state when created.
Verification
Changing Unified Port types between Ethernet and Fibre Channel requires a reload
of the Nexus 5000 or the UCS Fabric Interconnect on which the change was made.
N5K2#config t
End with CNTL/Z. N5K2(config)#feature fcoe
FC license checked out successfully

fc_plugin extracted successfully
FC plugin loaded successfully
FCoE manager enabled successfully
FC enabled on all modules successfully
Enabled FCoE QoS policies successfully N5K2(config)#feature npiv
N5K2(config)# ! N5K2(config)#slot 1
N5K2(config-slot)# port 28-32 type fc
N5K2(config-slot)#end
N5K2#copy running-config startup-config
[########################################] 100%
Copy complete, now saving to disk (please wait)...
N5K2# reload
WARNING: This command will reboot the system Do you want to continue? (y/n) [n]y
Shutdown Ports..
writing reset reason 9,
When the SAN port channels are configured, you may need to flap the links for the
port channels to come up, as shown below.
N5K2#show san-port-channel database
san-port-channel 102
Last membership update is successful
2 ports in total, 2 ports up
First operational port is fc1/28

Age of the port-channel is 0d:00h:10m:14s Ports:
fc1/29
fc1/28
[up] *
fc1/30
[down]
[up]
fc1/31
[down]
N5K2#conf t
End with CNTL/Z. N5K2(config)#int san-port-channel 104
N5K2(config-if)#shut
N5K2 %$ VDC-1 %$ %PORT-5-IF_DOWN_ADMIN_DOWN: %$VSAN 1%$ Interface san-port-channel 104 is down
(Administratively down)
N5K2(config-if)#no shut
N5K2(config-if)#end
N5K2 %$ VDC-1 %$ %PORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: %$VSAN 1%$ Interface san-port-channel 104 is down (No op
N5K2 %$ VDC-1 %$ Apr
6 20:48:00 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x
N5K2 %$ VDC-1 %$ %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on console0

N5K2 %$ VDC-1 %$ %PORT-5-IF_UP: %$VSAN 1%$ Interface san-port-channel 104 is up in mode F
N5K2#show san-port-channel database

fc1/29
fc1/28
[up] *
fc1/30
[up]
[up]
fc1/31
[up] *
On the UCS side, the SAN port channels are configured in Proxy Node Port (NP)
mode, for Node Port Virtualizer (NPV), or in other words, Fibre Channel End Host
Mode.
UCS-FI-A(nxos)#show run interface fc1/31 - 32
interface fc1/31 switchport mode NP

channel-group 103 force
no shutdown
interface fc1/32 switchport mode NP

no shutdown
UCS-FI-A(nxos)#show run interface san-port-channel 103

channel mode active switchport mode NP
UCS-FI-A(nxos)#show san-port-channel database
fc1/32
[up]
2.2 VSANs & Trunking

Configuration
N5K1:
vsan database
vsan 103
vsan 103 interface san-port-channel 103
!
switchport trunk allowed vsan 103
N5K2:
vsan database
vsan 104
vsan 104 interface san-port-channel 104
!
MDS1:
vsan database
vsan 103
fc1/31
[up] *
vsan 103 interface fc1/7

!
interface fc1/7
no shutdown
!
MDS2:
vsan database
vsan 104
!
interface fc1/7
no shutdown
!
UCS-FI-A:
UCS-FI-A(nxos)#show run | section "vsan database"
vsan database
vsan 103
UCS-FI-A(nxos)#show run | section "vlan 1104"

vlan 1103
fcoe vsan 103
name fcoe-vsan-1103
UCS-FI-B:
UCS-FI-B#connect nxos
UCS-FI-B(nxos)#show run | section "vsan database"
vsan database
vsan 104
UCS-FI-B(nxos)#show run | section "vlan 1104"
vlan 1104
fcoe vsan 104
name fcoe-vsan-1104
In UCSM, browse to the SAN tab, and then, under SAN Cloud, right-click VSANs to
create new VSANs. Ensure that VSAN 103 is on the Fabric A side and VSAN 104 is
on the Fabric B side.
To assign the VSANs to the SAN-Port-Channels, go back to the SAN tab, and under
SAN Cloud, right-click the appropriate FC interface and click Show Navigator. SANPort-Channel 103 should be in VSAN 103, and Port-Channel 104 should be in
VSAN 104.
Verification
When all the VSANs are created and assigned, check N5K1 and N5K2 to ensure
that the UCS FIs have performed a Fabric Login (FLOGI) on the SAN port channel
interfaces.
N5K1#show flogi database vsan 103
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------San-po103
103
0xbc0000
24:67:00:2a:6a:15:66:80 20:67:00:2a:6a:15:66:81
Total number of flogi = 1.

N5K2#show flogi database vsan 104
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------San-po104
104
0x6e0000
24:68:00:2a:6a:15:05:00 20:68:00:2a:6a:15:05:01
On MDS1 and MDS2, ensure that the Fibre Channel SAN has performed FLOGI.
MDS1#show flogi database vsan 103
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/7
103
0x0d0000
21:00:00:1b:32:04:5e:dc 20:00:00:1b:32:04:5e:dc

MDS2#show flogi database vsan 104
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/7
104
0xaa0000
21:01:00:1b:32:24:5e:dc 20:01:00:1b:32:24:5e:dc
Both N5K1 and MDS1 on the SAN A side and N5K2 and MDS2 on the SAN B side
should agree on the Fibre Channel Name Service (FCNS) database. This verifies
that both the initiators and targets are logged in and have been assigned Fibre
Channel Identifiers (FCIDs) and that VSAN trunking in the fabric is end to end.
N5K1#show fcns database vsan 103
VSAN 103:
-------------------------------------------------------------------------FCID
TYPE
PWWN
(VENDOR)
FC4-TYPE:FEATURE
-------------------------------------------------------------------------0x0d0000
21:00:00:1b:32:04:5e:dc (Qlogic)
0xbc0000
24:67:00:2a:6a:15:66:80 (Cisco)
npv
Total number of entries = 2

N5K2#show fcns database vsan 104
VSAN 104:
-------------------------------------------------------------------------FCID
TYPE
PWWN
(VENDOR)
FC4-TYPE:FEATURE
-------------------------------------------------------------------------0x6e0000
24:68:00:2a:6a:15:05:00 (Cisco)
0xaa0000
21:01:00:1b:32:24:5e:dc (Qlogic)
npv
2.3 Fibre Channel Zoning

Configuration
N5K1:
device-alias mode enhanced
device-alias database
device-alias name FC-SAN-A pwwn 21:00:00:1b:32:04:5e:dc
device-alias name BLADE1-SAN-A pwwn 20:00:00:cc:1e:dc:01:0a
!
device-alias commit
!
zone mode enhanced vsan 103
!
zone name VSAN_103_ZONE vsan 103
member device-alias FC-SAN-A
member device-alias BLADE1-SAN-A
member device-alias BLADE2-SAN-A
!
zoneset name VSAN_103_ZONESET vsan 103

member VSAN_103_ZONE
!
zoneset activate name VSAN_103_ZONESET vsan 103
zone commit vsan 103
N5K2:
device-alias mode enhanced

device-alias name FC-SAN-B pwwn 21:01:00:1b:32:24:5e:dc
device-alias name BLADE1-SAN-B pwwn 20:00:00:cc:1e:dc:01:0b
device-alias name BLADE2-SAN-B pwwn 20:00:00:cc:1e:dc:02:0b
!
device-alias commit
!
!
member device-alias FC-SAN-B
member device-alias BLADE1-SAN-B
member device-alias BLADE2-SAN-B
!
member VSAN_104_ZONE
!
Verification
Devices on the SAN A side should agree on the Device Alias database and zoneset
for VSAN 103.
MDS1#show device-alias status
Fabric Distribution: Enabled
Database:- Device Aliases 3 Mode: Enhanced
Checksum: 0x252e3d5059933b2826cabfe0ee148
MDS1#show device-alias database
device-alias name FC-SAN-A pwwn 21:00:00:1b:32:04:5e:dc

MDS1#show zone status vsan 103
VSAN: 103 default-zone: deny distribute: active only Interop: default mode: enhanced
merge-control: allow
session: none
hard-zoning: enabled broadcast: enabled
Default zone:
qos: none broadcast: disabled ronly: disabled
Full Zoning Database :
DB size: 224 bytes
Zonesets:1
Zones:1 Aliases: 0 Attribute-groups: 1 Active Zoning Database
:
DB size: 148 bytes Name: VSAN_103_ZONESET
Zonesets:1
Zones:1
Status: Activation completed at 20:55:21 UTC May 26 2013
MDS1 learned the zoning configuration applied on N5K1, but it does not yet see an
FCID for the UCS blades. This is because we haven't configured the service profiles
for the blades, which means they're not yet logged in to the fabric. When the SP
association is complete, we should see the FCIDs of the blades get dynamically
assigned, as well as the pWWNs we manually configure on them logged in to the
fabric.
MDS1#show zoneset active vsan 103
* fcid 0x0d0000 [device-alias FC-SAN-A]
device-alias BLADE1-SAN-A
2.4 iSCSI Virtual Target

Configuration
N7K3:
switchport trunk allowed vlan 202
no shutdown
MDS1:
device-alias name UCS-C200-SAN-A pwwn 20:00:00:cc:1e:dc:03:0a
!
device-alias commit
!
feature iscsi
iscsi enable module 1
!
vsan database
vsan 103 interface iscsi1/1
!
iscsi virtual-target name iqn.1987-05.com.cisco:05.mds1.01-01.01234567890abcde
pWWN 21:00:00:1b:32:04:5e:dc
initiator ip address 192.168.202.104 permit
!
iscsi initiator ip-address 192.168.202.104
static pWWN 20:00:00:cc:1e:dc:03:0a
!
member device-alias UCS-C200-SAN-A
!
!
interface GigabitEthernet1/1
ip address 192.168.202.61 255.255.255.0
no shutdown
!
interface iscsi1/1
no shutdown
Verification
When the iSCSI configuration is complete, MDS1 should see the UCS C200 server
log in as an iSCSI Initiator. The nWWN can be dynamic, but because zoning and
LUN Masking on the SAN is done based on the pWWN, this needs to be manually
assigned to the iSCSI Initiator.
MDS1#show iscsi initiator
iSCSI Node name is 192.168.202.104
iSCSI Initiator name: iqn.1998-01.com.vmware:localhost-7463f71b
iSCSI alias name:
Configured node (iSCSI)
Node WWN is 21:01:00:0d:ec:4a:21:02 (dynamic)
Member of vsans: 103
Number of Virtual n_ports: 1 Virtual Port WWN is 20:00:00:cc:1e:dc:03:0a (configured)
Interface iSCSI 1/1, Portal group tag: 0x3000
VSAN ID 103, FCID 0x0d0100
From the iSCSI Initiator's point of view, the MDS is an iSCSI Target. Note that only
the C200's IP address is allowed to use this target.
MDS1#show iscsi virtual-target
target: iqn.1987-05.com.cisco:05.mds1.01-01.01234567890abcde
* Port WWN 21:00:00:1b:32:04:5e:dc

Configured node (iSCSI)
No. of initiators permitted: 1
initiator 192.168.202.104/32 is permitted
All initiator permit is disabled
Trespass support is
disabled
Revert to primary support is
disabled
MDS1 should see the C200 server registered to the fabric in the FLOGI database.
MDS1#show flogi database
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/7
103
0x0d0000
21:00:00:1b:32:04:5e:dc 20:00:00:1b:32:04:5e:dc
[FC-SAN-A] iscsi1/1
103
0x0d0100
20:00:00:cc:1e:dc:03:0a
21:01:00:0d:ec:4a:21:02 [UCS-C200-SAN-A]

Adding the C200's pWWN to the already defined zone for VSAN 103 will allow it
access to the LUNs that the SAN is presenting for this initiator.
MDS1#show zoneset active
* fcid 0x0d0000 [device-alias FC-SAN-A]
device-alias BLADE2-SAN-A * fcid 0x0d0100 [device-alias UCS-C200-SAN-A]
The final verification for this task is to ensure that the ESXi instance has actually
mounted the iSCSI LUNs. To check this, go to the vSphere client, select the C200
host on the left, click the Configuration tab, and then click Storage Adapters.
Under the iSCSI Software Adapter, you should see the LUNs appear as shown
below.
3.1 Address Pools
UUID Pools in UCSM are configured under the Servers tab, Pools, then UUID
Suffix Pools, as shown below.
MAC Address Pools are under the LAN tab, Pools, then MAC Pools.
Node World Wide Name Pools are under the SAN tab, Pools, then WWNN Pools.
Management IP Address Pools are under the Admin tab, Communication

Management, then Management IP Pool. Note that the default gateway here is
arbitrary, because the task did not ask for a specific value, but it is still a required
field.
3.2 UCS Service Profile Templates
Create a new Service Profile Template under the Servers tab, then Service Profile
Templates. The task requires that this be an Initial Template and get its addresses
from the default pools that were created in the previous task.
Under Storage, ensure that the vHBAs are assigned to VSANs 103 and 104 on
Fabric A and Fabric B, respectively.
For vNICs, use the Expert option, and add the five new vNICs according to the task
requirements. The VLANs needed are created in this step to save time, but could
also be configured as a separate step under the LAN Cloud.
Ensure that the vMotion vNIC has Fabric Failover enabled according to the task
requirements.
The vNICs for the VMGuests are trunks that carry the rest of the VLANs.
The Maintenance Policy is where we define that the administrator must

acknowledge a change that would cause the blade to reboot.
The Operational Policies define where the Management IP addresses of the Service
Profiles come from.

To assign the service profiles, we must first enable the southbound links from the
FIs to the Blade Chassis. To do so, configure them as Server ports under the
Fabric Interconnects on the Equipment tab.
Create two copies of the Service Profile Template previously created.
Before we customize the boot options for the individual service profiles, a QoS
policy is created that will apply to the vHBAs. Note that this is just for clarity of the
configuration, so that we know for certain that the vHBAs are being assigned to a nodrop QoS policy.
Modify the vHBAs to have the appropriate pWWNs according to the task. Note that
if these values are incorrect, the blades will fail to boot from the SAN, because the
LUN masking on the SAN only allows specific initiating pWWNs to access their
LUNs.
We need to create a Boot Policy that tells the blade which SAN target it needs to
boot to.
Again, ensure 100% accuracy, because an incorrect pWWN value will cause the
blade to be unable to boot.
Repeat the above steps, but now for the backup boot target.
Don't forget to actually assign the Boot Policy to the service profile after it is
successfully created.
Repeat the above steps for the second service profile that will be assigned to blade
2.
Finally, associate the service profiles to the blades.
When the blades begin to boot, you can track their progress by connecting to their
KVMs. When the blades are fully booted, you should see the console screen for the
ESXi instances, as shown below.

4.1 Nexus 1000v
Configuration
First we need to determine which UUIDs were dynamically assigned to the blades,
and which VEMs they are currently inserted as. The below output shows us the
module number (VEM number), the UUID, and the IP address.
N1Kv#show module
Mod
Ports
Module-Type
Model
Status
---
-----
--------------------------------
------------------
------------
Virtual Supervisor Module
Nexus1000V
active *
Nexus1000V
ha-standby
248
Virtual Ethernet Module
NA
ok
248
NA
ok
248
NA
ok
Mod
Sw
Hw
---
------------------
------------------------------------------------
4.2(1)SV2(1.1)
0.0
4.2(1)SV2(1.1)
0.0
4.2(1)SV2(1.1)
VMware ESXi 5.1.0 Releasebuild-799733 (3.1)
4.2(1)SV2(1.1)
4.2(1)SV2(1.1)
Mod
MAC-Address(es)
Serial-Num
---
--------------------------------------
----------
00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8
NA
00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8
NA
02-00-0c-00-04-00 to 02-00-0c-00-04-80
NA
02-00-0c-00-05-00 to 02-00-0c-00-05-80
NA
02-00-0c-00-06-00 to 02-00-0c-00-06-80
NA
Mod
Server-IP
Server-UUID
Server-Name
---
---------------
------------------------------------
--------------------
192.168.200.200
NA
NA
192.168.200.200
NA
NA
192.168.200.102
6220349f-9cc4-e211-0000-00000000007f
NA 5
192.168.200.104
d57068d1-dfa4-11e1-a49d-30f70d088146
NA 6
192.168.200.101
6220349f-9cc4-e211-0000-00000000006f
NA
* this terminal session
The VEM number is part of the running config, as shown below.

N1Kv#sh run | b vem
vem 3
host vmware id 2e2baff2-03bd-e211-0000-00000000007f
vem 4
host vmware id 6220349f-9cc4-e211-0000-00000000007f
vem 5
host vmware id d57068d1-dfa4-11e1-a49d-30f70d088146
vem 6
host vmware id 6220349f-9cc4-e211-0000-00000000006f
vem 7
host vmware id 2e2baff2-03bd-e211-0000-00000000006f
Power cycle blade 2, which will cause its VEM to be removed. Then we can delete
its current VEM number and move it to module 20.
N1Kv#
N1Kv %PLATFORM-2-MOD_REMOVE: Module 2 removed (Serial number T5056BB0E4A)
N1Kv %VEM_MGR-2-VEM_MGR_REMOVE_NO_HB: Removing VEM 4 (heartbeats lost) N1Kv %VEM_MGR-2-MOD_OFFLINE:
Module 4 is offline
N1Kv#config t
End with CNTL/Z. N1Kv(config)#no vem 4
N1Kv(config)#vem 20
N1Kv(config-vem-slot)#host vmware id 6220349f-9cc4-e211-0000-00000000007f
N1Kv(config-vem-slot)#end
N1Kv#copy run start
[########################################] 100%
N1Kv#
N1Kv %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.200.102 detected as module 20
N1Kv %VEM_MGR-2-MOD_ONLINE: Module 20 is online
N1Kv %PLATFORM-2-MOD_DETECT: Module 2 detected (Serial number :unavailable) Module-Type Virtual Supervisor Module Mo
Finally, power cycle blade 1, which will cause its VEM to be removed. Because the
primary VSM also runs on this blade, you will have to disconnect and reconnect
your terminal session. Before blade 1 fully reboots, delete its current VEM number
and move it to module 10.
N1Kv#conf t
End with CNTL/Z. N1Kv(config)#no vem 6
N1Kv(config)#vem 10
N1Kv(config-vem-slot)#host vmware id 6220349f-9cc4-e211-0000-00000000006f
N1Kv(config-vem-slot)#end
N1Kv %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.200.101 detected as module 10
N1Kv %VIM-5-IF_ATTACHED: Interface Vethernet1 is attached to vmk0 on port 1 of module 10 with dvport id 1216
N1Kv %VEM_MGR-2-MOD_ONLINE: Module 10 is online
N1Kv#show module
Mod
Ports
Module-Type
Model
Status
---
-----
--------------------------------
------------------
------------
Nexus1000V
ha-standby
Nexus1000V
active *
248
NA
ok
10
248
NA
ok
20
248
NA
ok
Mod
Sw
Hw
---
------------------
------------------------------------------------
4.2(1)SV2(1.1)
0.0
4.2(1)SV2(1.1)
0.0
4.2(1)SV2(1.1)
10
4.2(1)SV2(1.1)
20
4.2(1)SV2(1.1)
Mod
MAC-Address(es)
Serial-Num
---
--------------------------------------
----------
00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8
NA
00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8
NA
02-00-0c-00-05-00 to 02-00-0c-00-05-80
NA
10
02-00-0c-00-0a-00 to 02-00-0c-00-0a-80
NA
20
02-00-0c-00-14-00 to 02-00-0c-00-14-80
NA
Mod
Server-IP
Server-UUID
Server-Name
---
---------------
------------------------------------
--------------------
192.168.200.200
NA
NA
192.168.200.200
NA
NA
192.168.200.104
d57068d1-dfa4-11e1-a49d-30f70d088146
192.168.200.104
10
192.168.200.101
6220349f-9cc4-e211-0000-00000000006f
192.168.200.101
20
192.168.200.102
6220349f-9cc4-e211-0000-00000000007f
192.168.200.102
4.2 Private VLANs

N7K3:
feature private-vlan
!
vlan 203
private-vlan isolated
vlan 204
private-vlan primary
private-vlan association 203
!
interface Vlan204
no shutdown
private-vlan mapping 203
ip address 192.168.255.73/24
N1Kv:
vlan 203
private-vlan isolated
vlan 204
private-vlan primary
private-vlan association 203
!
port-profile type ethernet VMGuests_UPLINK
switchport mode private-vlan trunk promiscuous
switchport private-vlan mapping trunk 204 203
!
port-profile type vethernet VLAN203
switchport mode private-vlan host
switchport private-vlan host-association 204 203

CCIE DC Full-Scale Lab 2 Tasks
Introduction
Introduction
General Lab Guidelines
You may not use any links that may physically be present but not specifically pictured
and labeled in this topology.
Name and number all VLANs, port channels, SAN port channels, service profiles,
templates, and so on exactly as described in this lab. Failure to do so will result in
missed points for that task.
You may not change any passwords on any devices unless explicitly directed to do
so.
You may not change any management IP addresses or default routes on any
devices or VDCs unless explicitly directed to do so (you may add them if they do not
exist, but you may not change existing).
You may not disable telnet on any device. Telnet must work properly on all devices
and VDCs.
You may not log on to the 3750G switch for this particular lab. It is fully functional and
pre-configured for you.
1.1 VLANs
Do not create any unnecessary VLANs on any switch.
Create VLANs 120, 125, 130, 135, 140, 200, 201, 710, and 711 on N7K1.
Create VLANs 120, 125, 130, 135, 140, 200, 201, 720, and 721 on N7K2.
Create VLANs 120, 125, 130, 135, 200, 201, and 140 on N7K3.
Create VLANs 120, 125, 130, 135, 200, 201 and 140 on N7K4
Create VLANs 120, 125, 130, 135, 200, and 201 on N5K1 and N5K2.
Name VLANs on every device they appear on according to Table 1.
Table 1
VLAN
Name
120
VM-DATA1
125
VM-DATA2
130
VM-DATA3
135
VM-DATA4
140
OTV-SITE
150
BACKUP
200
DCI-ESXI
201
DCI-VMOTION
710
DC1-ISP-1
VLAN
Name
711
DC1-ISP-2
720
DC2-ISP-1
721
DC2-ISP-2
1.2 DCI L3 Routing

Configure an L3 link over N7K1 e2/29 with the IP address and subnet mask of
10.71.71.0 255.255.255.254.
Use VLAN 710 to accomplish this.
This L3 link must belong to VRF "DC1".
Configure the L3 link to form an OSPF adjacency in area 0.0.0.5.
Use a router id of 10.71.71.71 for the OSPF process.
The OSPF process should be named "DC1".
Ensure that e2/29 will only ever run at a rate of 1Gbps.
10.71.71.2 255.255.255.254.
10.72.72.0 255.255.255.254.
10.72.72.2 255.255.255.254.

These four ports should all immediately go into a forwarding state when brought up
and should go into an errDisabled state if they receive any STP BPDUs.
Do not modify any configuration on the 3750G switch for this or any other task in this
lab.
Ensure OSPF converges with whatever means necessary
1.3 L2 Trunking and L3 Routed Interfaces

Configure trunking between N7K1 e2/1 and N7K3 e2/9.
Allow only previously created VLANs 120-140 and 200-201 over this link.
Ensure that N7K1 is the root for all STP instances.
Configure an L3 routed interface between N7K1 e1/1 using the IP address
10.13.13.0/31 and N7K3 e1/9 using the IP address 10.13.13.1/31.
Ensure that this L3 link can participate in the OSPF process and route over
the DCI.
Configure trunking between N7K2 e2/11 and N7K4 e2/20.
Only allow only previously created VLANs 120-140 and 200-201 over this
link.
Configure an L3 routed interface between N7K2 e1/17 using the IP address
10.24.24.0/31 and N7K4 e1/25 using the IP address 10.24.24.1/31.
Ensure that this L3 link can participate in the OSPF process and route over
the DCI.
1.4 Port Channels

Assuming that more links will be added later, with the desire for minimal traffic
disruption, configure the following:
Configure trunking on port channel 215 from N7K1 to UCS FI-A, and ensure
that the same port channel number is used later from the UCS side.
Configure trunking on port channel 218 from N7K1 to UCS FI-B, and ensure
that the same port channel number is used later from the UCS side.
Ensure that both of these port channels transition immediately to a state of
forwarding traffic.
Ensure that the N7K1 is the primary device in LACP negotiation.
Ensure that the hashing algorithm takes L3 and L4 for both source and
destination into account.
Trunk only previously created VLANs 120-135 and 200-201 southbound
from N7K1 to both FIs.
1.5 HSRP
Using information from Table 2, configure SVIs on N7K1 and N7K2 for all VLANs that
are present on that switch.
Assume that a second Nexus 7000 will be added to each Data Center, and with that
in mind, go ahead and provision HSRP for all SVIs at both sites, as follows:
Use the newest version of HSRP supported.
Make HSRP group numbers correspond with their respective VLAN/SVI
numbers.
Use the virtual IP address of .254 for SVIs on both switches.
Use the host IP address of .251 for each current SVIs on N7K1.
(.250 will be used in the future for the other HSRP member at DC1).
Use the host IP address of .252 for each current SVIs on N7K2.
(.253 will be used in the future for the other HSRP member at DC1).
These current SVIs will be the primary HSRP group member even after the
other N7K is put into service at each DC; ensure that these SVIs have a
higher preference for being the Active forwarder assuming the others come
online with defaults.
Have the SVIs for VLAN 200 use the fastest possible hello and hold timers.
Table 2
VLAN
IP Subnet / Mask
VRF
120
192.168.120.0 255.255.255.0
default
125
192.168.125.0 255.255.255.0
default
VLAN
IP Subnet / Mask
VRF
130
192.168.130.0 255.255.255.0
default
135
192.168.135.0 255.255.255.0
default
200
192.168.200.0 255.255.255.0
default
201
192.168.201.0 255.255.255.0
default
1.6 vPC
Configure vPC between N5K1 and N5K2 with the Domain ID 12.
Configure the peer-link with an LACP trunking over ports e1/1-2 on Port Channel 512
between N5K1 and N5K2 according to the diagram.
Ensure that any vPC numbers correspond with their designated port channel
numbers, as listed in the tasks that follow.
You are not permitted to create any additional links that are not explicitly pictured in
the diagram.
Ensure that N5K1 is the root for all STP instances; however, you may not configure
any spanning tree priority or root commands globally or at the interface level on
N5K1.
Ensure that N5K1 holds the primary role for the vPC domain.
Ensure that N5K1 always decides which links are active in any port channel.
Synchronize all ARP tables.
Ensure that if our SAN was an EMC VPLEX or VMAX using IP technologies, vPC
would not cause any problems with forwarding frames.
1.7 Port Channels, FEX, and vPC

Configure trunking on Port Channel 100 from N7K2 to N5K1 and N5K2 according to
the diagram, and ensure that the pair of N5Ks are the only ones initiating any port
channel protocol negotiation.
Configure FEX 113 using trunking on Port Channel 113 from N5K1 and N5K2
according to the diagram.
Configure FEX 123 using trunking on Port Channel 123 from N5K1 and N5K2
according to the diagram.
1.8 Mgmt VM Access

Configure a 1Gbps access link in VLAN 200 to the Management VM on N5K1 e1/11.
Ensure that traffic forwards immediately and goes into an errDisable state if it
receives any STP packets.
1.9 Access Trunking

Configure trunking on both ports individually coming from SVR1 up to N5K1 e113/1/1
and N5K2 e123/1/1 according to the diagram.
For now, trunk only previously created VLANs 120-135 and 200-201 (there may be
additional VLANs needed later).
1.10 OTV
Extend only previously created VLANs 120-135 and 200-201 between Data Centers
using OTV.
Use the OTV site VLAN of 140 on both sides of the DCI.
You may use whatever site identifiers you prefer.
The ISP supports SSM and ASM, and for ASM it provides a PIM RP of 10.10.10.25;
use this as your only RP.
OTV should be authenticated using a hashed value from the word "DCIOTV".
Any of the SVIs on N7K1 or N7K2 for the VLANs that are extended across the DCI
should be able to ping each other.
Prevent HSRP groups at DC1 from becoming active/standby members of the same
HSRP group numbers at DC2, and vice-versa.
Prevent any device ARPing at either DC from getting the virtual MAC address of the
HSRP group from the 7K at the opposite side of the DCI.
When finished, both N7K1 and N7K2 should be able to ping the actual host IP
address of the SVI at the opposite data center traversing the overlay.
Each N7K1 and N7K2 should also be able to ping the virtual IP address of .254,
which should keep traffic local to the site from which the ping originates.
2.1 VSANs and FCoE VLANs

Create VSAN 10 on MDS1, MDS2, N5K1, and N5K2.
Create VSAN 20 only on MDS1, MDS2, and N5K2.
Create VLAN 10 to carry FCoE traffic for VSAN 10 on N5K1 and N5K2.
Create VLAN 20 to carry FCoE traffic for 20 respectively only on N5K2.
2.2 UCS SAN Connectivity

Configure FC links on MDS1 as pictured in the diagram ready for both UCS FIs.
Do not use any port channeling or trunking.
Configure links coming from FI-A to MDS1 to use VSAN 10.
Configure links coming from FI-B to MDS1 to use VSAN 20.
2.3 E Port Trunking

N5K1 should be configured as an E trunk to N5K2 and should trunk only VSAN 10
over SAN Port Channel 256 using interfaces fc1/26 and fc1/27.
Configure a trunk between N5K2 fc1/28 and MDS2 fc1/3 that trunks only VSANs 10
and 20.
N5K2 fc1/32 should provide connectivity to the SAN array for VSAN 10.
MDS2 fc1/7 should provide connectivity to the SAN array for VSAN 20.
2.4 Cisco C200 P81E (VIC) CNA FLOGIs

Configure FCoE for Svr1 so that it logs in to VSAN 10 over FEX 113.
Configure FCoE for Svr1 so that it logs in to VSAN 20 over FEX 123.
Svr1 is set up to FLOGI to both fabrics.
2.5 FCIP
Configure FCIP between MDS1 and MDS2 on interfaces G1/1 and G1/2 on each
switch.
Use the IP address of 12.12.12.1/30 on MDS1 G1/1 and 12.12.12.2/30 on MDS2
G1/1 over FCIP Profile 10 and interface FCIP 10 on both sides.
Use the IP address of 12.12.12.5/30 on MDS1 G1/2 and 12.12.12.6/30 on MDS2
G1/2 over FCIP Profile 20 and interface FCIP 20 on both sides.

The 3750G switch is already configured properly; do not connect to it at all.
Configure SAN Port Channel 50 over both of these links and trunk only VSAN 10 and
VSAN 20 over it.
Optimize FCIP on MDS1 and MDS2 to account for optimum TCP window scaling
based on the approximate actual RTT (within 20% variance is allowed).
Allow FCIP to monitor the congestion window and increase the burst size to the
maximum allowed.
Ensure that there is no fragmentation of FCIP packets over the link.
2.6 Zoning
Ensure that MDS1 appears to the fabric as domain 0x61 for VSAN 10 and 20.
Ensure that MDS2 appears to the fabric as domain 0x62 for VSAN 10 and 20.
Ensure that N5K2 appears to the fabric as domain 0x52 for VSAN 10 and 20.
Ensure that N5K1 appears to the fabric as domain 0x51 for VSAN 10 and 20.
Zone according to the following information.
You may only make zoning changes for both Fabric A and Fabric B from MDS1.
According to information given in Table 3:
Zone so that "ESXi1", "ESXi2", and "ESXi3" all have access to their FCTARGET-SAN-x for the appropriate Fabrics (fc0's to Fabric A; fc1's to Fabric
B).
Fabric A uses VSAN 10.
Fabric B uses VSAN 20.
Zoning for Fabric A should use the zone name "ZONE-A".
Zoning for Fabric B should use the zone name "ZONE-B".
The zoneset for Fabric A should be named "ZoneSet_VSAN10".
The zoneset for Fabric B should be named "ZoneSet_VSAN20".
Aliases must be created according to Table 3 and must be used in the zoning
configuration.
Many pWWN's are the same below. They are sorted first by FC-4
Type and then by Fabric.
Table 3
Fabric
pWWN
LUN
20:aa:00:25:b5:01:01:01
FC-4
Type
Description
Alias
N/A
ESXi1 vHBA
"fc0"
ESXi1A-fc0
Init
20:aa:00:25:b5:01:01:02
N/A
ESXi2 vHBA
"fc0"
ESXi2A-fc0
Init
20:00:d4:8c:b5:bd:46:0e
N/A
ESXi3 vHBA
"fc0"
ESXi3A-fc0
Init
20:bb:00:25:b5:01:01:01
N/A
ESXi1 vHBA
"fc1"
ESXi1B-fc1
Init
20:bb:00:25:b5:01:01:02
N/A
ESXi2 vHBA
"fc1"
ESXi2B-fc1
Init
20:00:d4:8c:b5:bd:46:0f
N/A
ESXi3 vHBA
"fc1"
ESXi3B-fc1
Init
ESXi1 Boot
Volume
FCTARGETSAN-A
Target
ESXi2 Boot
Volume
FCTARGETSAN-A
Target
FC_Datastore
1
FCTARGETSAN-A
Target
FC_Datastore
2
FCTARGETSAN-A
Target
21:03:00:1b:32:64:5e:dc
21:03:00:1b:32:64:5e:dc
21:03:00:1b:32:64:5e:dc
21:03:00:1b:32:64:5e:dc
Fabric
pWWN
21:01:00:1b:32:24:5e:dc
21:01:00:1b:32:24:5e:dc
21:01:00:1b:32:24:5e:dc
21:01:00:1b:32:24:5e:dc
Alias
FC-4
Type
ESXi1 Boot
Volume
FCTARGETSAN-B
Target
ESXi2 Boot
Volume
FCTARGETSAN-B
Target
FC_Datastore
1
FCTARGETSAN-B
Target
FC_Datastore
2
FCTARGETSAN-B
Target
LUN
Description
Initialize both UCS Fabric Interconnects (FIs).
Fabric Interconnect A should use the IP address of 192.168.101.201/24.
Fabric Interconnect B should use the IP address of 192.168.101.202/24.
Both Fabric Interconnects should use a VIP of 192.168.101.200.
3.2 SAN Uplinks and VSANs

Disable all confirmation messages for creation and deletion of objects.
Configure individual FC uplinks as instructed earlier in the Storage Networking
section and according to the diagram.
Do not use any port channeling or trunking.
Create VSAN 10 and name it "VSAN10" and ensure that it uses VLAN 10 for FCoE.
Create VSAN 20 and name it "VSAN20" and ensure that it uses VLAN 20 for FCoE.
Configure links coming from FI-A to MDS1 to use VSAN 10.
Configure links coming from FI-B to MDS1 to use VSAN 20.

Disable any unused FC ports according to diagram.
3.3 LAN Uplinks and VLANs

Configure port channels for all links from FIs to IOM/FEXs in UCS chassis according
to the diagram.
Configure a port channel from each FI to N7K1 according to the diagram, and use
the same port channel number as previously instructed from the N7K side.
Create VLANs 120-135 and 200-201 and VLAN 150 from Table 1, with correct
names on both UCS FIs (only the ones in the table).
Only allow the BACKUP VLAN to traverse the 1Gbps ports designated in the
diagram toward the 3750G switch, and ensure that it is in an UP state.
3.4 Disk Policies

Create a hard disk policy named "MAXRAID" that specifies a method that both
mirrors and then stripes local disks.
Ensure that if any service policy ever uses this policy and tries to associate with a
blade whose hard drives are not already provisioned with this RAID method, the
association will fail.
Do not associate this policy with any service profiles.
3.5 Pools
Create a UUID pool called "Global-UUIDs" and allocate suffixes from the range of
0001-000000000101 to 0001-00000000010f.
Create a MAC address pool called "Global-MACs" ranging from 00:25:b5:0a:0a:01 to
00:25:b5:0a:0a:11.
Create an nWWN pool called "Global-nWWNs" ranging from 20:ff:00:25:b5:01:01:01
to 20:ff:00:25:b5:01:01:11.
Create a Management IP address pool ranging from 192.168.101.210 to
192.168.101.219 with the default gateway of 192.168.101.1.

Configure a service profile named "ESXi1" with the following values.
Anything changed in this service profile template should never affect any service
profiles instantiated from it.
UUIDs should be dynamically allocated from the Global-UUIDs pool.
2 vHBAs should be created with the following information:
Name them "fc0" and "fc1".
"fc0" must be assigned the initiator pWWN of 20:aa:00:25:b5:01:01:01.
"fc1" must be assigned the initiator pWWN of 20:bb:00:25:b5:01:01:01.
Both vHBAs must be able to dynamically obtain nWWNs from the GlobalnWWNs pool.
Neither of these vHBAs should be allowed to re-attempt FLOGIs more than
3 times.
Configure a specific boot policy to boot from SAN with the following information:
"fc0" should attempt first to boot from Fabric A using the pWWN for "ESXi1
Boot Volume" in Table 3.
"fc1" should attempt first to boot from Fabric B using the pWWN for "ESXi1
Boot Volume" in Table 3.
5 vNICs should be created with the following information:
Name them "eth0", "eth1", "eth2", "eth3", and "eth4".
"eth0" and "eth3" should only be allowed to ever use Fabric A.
"eth1" and "eth4" should only be allowed to ever use Fabric B.
"eth2" primarily uses Fabric A, but should automatically use Fabric B if all
uplinks on FI-A are down.
MAC addresses should must be allocated dynamically from the GlobalMACs pool.
All VLANs should be allowed on all vNICs except for VLAN 1 and VLAN
150; these should not be allowed on any vNICs.
All hosts will explicitly tag their VLAN IDs.
Any changes to the service profile requiring a reboot should force the administrator to
manually allow it.
Any service profile created from this template should not automatically associate with
any blades in the chassis.
Only allow this service profile to ever associate with blades that have a Palo
mezzanine adapter.
Do not allow blade to automatically boot after this service profile is associated.
Ensure that when booting, the KVM console viewer can see the FC disk that
attaches directly after the FC drivers load.
Configure the management IP addresses to be dynamically assigned from the global
pool.
Manually associate this profile with blade 1 and boot the blade.
3.7 Cloning Service Profiles

Create a clone of the previous service profile and call it "ESXi2".
Change what is necessary for the vHBAs to be set up as follows:
"fc0" must be assigned the initiator pWWN of 20:aa:00:25:b5:01:01:02.
"fc1" must be assigned the initiator pWWN of 20:bb:00:25:b5:01:01:02.
Ensure that this service profile always uses links fc1/30 on Fabric A and
fc1/28 Fabric B for its SAN traffic.
Manually associate this profile with blade 2 and boot the blade.
3.8 Traffic Monitoring

Measure traffic in a policy called "Over_3Gbps" on vNIC "eth2" in Service Profile
"ESXi1", and raise an informational alert if the traffic received by the vNIC rises
above 3Gbps.
Do not change the collection interval for any device in the system.

4.1 VSM and VEM Connectivity
Ensure reachability to both VSMs that are running on both UCS blades.
Ensure that the VEMs running on both UCS blades insert into the Nexus1000v
chassis properly.
Ensure that service profile ESX1 shows up as VEM 4 and service profile ESX2
shows up as VEM 5.
Do not worry about the UCS C200 VEM for this lab.
4.2 N1Kv QoS

Ensure that all traffic coming from vNIC "eth2" on both blades is marked with CoS 4
only by the use of Nexus1000v, and that the UCS trusts that marking.
You are not permitted to attach any policy directly to that interface.

CCIE DC Full-Scale Lab 2 Solutions
Introduction
Introduction
General Lab Guidelines
You may not use any links that may physically be present but are not specifically
pictured and labeled in this topology.
Name and number all VLANs, port channels, SAN port channels, service profiles,
templates, and so on exactly as described in this lab. Failure to do so will result in
missed points for that task.
You may not change any passwords on any devices unless explicity directed to do
so.
You may not change any management IP addresses or default routes on any
devices or VDCs unless explicitly directed to do so (you may add them if they do not
exist, but you may not change existing).
You may not disable telnet on any device. Telnet must work properly on all devices
and VDCs.
You may not log on to the 3750G switch for this particular lab. It is fully functional and
pre-configured for you.
1.1 VLANs
Configuration
This task doesn't seem like it would be a difficult one, and it obviously isn't hard to
create VLANs. What can be difficult is getting them all in exactly as instructed. Other
than VLAN 1, if you entered additional VLANs beyond what was asked of you, or if
you simply mis-typed a name, you would have missed the points for this task.
N7K1:
vlan 120
name VM-DATA1
vlan 125
name VM-DATA2
vlan 130
name VM-DATA3
vlan 135
name VM-DATA4
vlan 140
name OTV-SITE
vlan 200
name DCI-ESXI
vlan 201
name DCI-VMOTION
vlan 710
name DC1-ISP-1
vlan 711
name DC1-ISP-2
N7K2:
vlan 120
name VM-DATA1
vlan 125
name VM-DATA2
vlan 130
name VM-DATA3
vlan 135
name VM-DATA4
vlan 140
name OTV-SITE
vlan 200
name DCI-ESXI
vlan 201
name DCI-VMOTION
vlan 710
name DC1-ISP-1
vlan 711
name DC1-ISP-2
N7K3:
vlan 120
name VM-DATA1
vlan 125
name VM-DATA2
vlan 130
name VM-DATA3
vlan 135
name VM-DATA4
vlan 140
name OTV-SITE
vlan 200
name DCI-ESXI
vlan 201
name DCI-VMOTION
N7K4:
vlan 120
name VM-DATA1
vlan 125
name VM-DATA2
vlan 130
name VM-DATA3
vlan 135
name VM-DATA4
vlan 140
name OTV-SITE
vlan 200
name DCI-ESXI
vlan 201
name DCI-VMOTION
N5K1:
vlan 120
name VM-DATA1
vlan 125
name VM-DATA2
vlan 130
name VM-DATA3
vlan 135
name VM-DATA4
vlan 200
name DCI-ESXI
vlan 201
name DCI-VMOTION
N5K2:
vlan 120
name VM-DATA1
vlan 125
name VM-DATA2
vlan 130
name VM-DATA3
vlan 135
name VM-DATA4
vlan 200
name DCI-ESXI
vlan 201
name DCI-VMOTION
1.2 DCI L3 Routing

Configuration
Before we even start with the configuration instructed, you may notice that not all
ports are allocated to their proper VDC, which was a part of our inherent
troubleshooting. When you are moving ports, it is recommended to do so one at a
time, because if you move a sequential range of ports, it may move more than you
intended and upset other port groups. Remember that on M1 modules, port groups
are groups of 4 odd and groups of 4 even ports, so moving a sequential range can
certainly have adverse effects.
N7K1:
feature ospf
vdc N7K1 id 1
allocate interface Ethernet1/1-8,Ethernet1/18,Ethernet1/20,Ethernet1/22,Ethernet1/24
allocate interface Ethernet2/1-2,Ethernet2/7-8,Ethernet2/17-18,Ethernet2/27-32

vdc N7K2 id 2
allocate interface Ethernet1/17,Ethernet1/19,Ethernet1/21,Ethernet1/23
allocate interface Ethernet2/3-6,Ethernet2/11-12,Ethernet2/21-24
vdc N7K3 id 3
allocate interface Ethernet2/9-10,Ethernet2/25-26
vdc N7K4 id 4
allocate interface Ethernet2/13-16,Ethernet2/19-20
system jumbomtu 9000
vrf context DC1
interface Vlan710
no shutdown
vrf member DC1
ip address 10.71.71.0/31
ip ospf mtu-ignore
ip router ospf DC1 area 0.0.0.5
ip pim sparse-mode
interface Vlan711
no shutdown
vrf member DC1
ip address 10.71.71.2/31
ip ospf mtu-ignore
ip pim sparse-mode
spanning-tree bpduguard enable
speed 1000
mtu 9000
no shutdown
speed 1000
mtu 9000
no shutdown
router ospf DC1

vrf DC1
router-id 10.71.71.71
N7K2:
feature ospf
system jumbomtu 9000
vrf context DC2
interface Vlan720
no shutdown
vrf member DC2
ip address 10.72.72.0/31
ip ospf mtu-ignore
ip pim sparse-mode
interface Vlan721
no shutdown
vrf member DC2
ip address 10.72.72.2/31
ip ospf mtu-ignore
ip pim sparse-mode
spanning-tree bpdufilter enable
speed 1000
mtu 9000
no shutdown
spanning-tree bpdufilter enable
speed 1000
mtu 9000
no shutdown
router ospf DC2

vrf DC2
router-id 10.72.72.72
Verification
N7K1(config)#sh ip ospf neighbors vrf DC1
OSPF Process ID DC1 VRF DC1
Total number of neighbors: 2
Neighbor ID
10.10.10.25
10.10.10.25
Pri State
1 FULL/DR
1 FULL/DR
Up Time
Address
01:27:26 10.71.71.3
01:27:20 10.71.71.1
Interface
Vlan711
Vlan710
N7K1(config)#sh ip route vrf DC1

IP Route Table for VRF "DC1"
'%<string>' in via output denotes VRF <string>
10.0.0.25/32, ubest/mbest: 2/0

*via 10.71.71.1, Vlan710, [110/41], 01:28:28, ospf-DC1, inter
10.10.10.25/32, ubest/mbest: 2/0
*via 10.71.71.1, [1/0], 01:28:34, static
*via 10.71.71.3, [1/0], 01:28:34, static
10.71.71.0/31, ubest/mbest: 1/0, attached
*via 10.71.71.0, Vlan710, [0/0], 01:28:35, direct
*via 10.71.71.0, Vlan710, [0/0], 01:28:35, local
*via 10.71.71.2, Vlan711, [0/0], 01:28:35, direct
*via 10.71.71.2, Vlan711, [0/0], 01:28:35, local
10.72.72.0/31, ubest/mbest: 2/0
10.72.72.2/31, ubest/mbest: 2/0
N7K1(config)#
N7K2(config)#sh ip ospf neighbors vrf DC2

OSPF Process ID DC2 VRF DC2
Total number of neighbors: 2
Neighbor ID
10.10.10.25
10.10.10.25
Pri State
1 FULL/BDR
1 FULL/BDR
Up Time
Address
01:28:17 10.72.72.3
01:28:18 10.72.72.1
Interface
Vlan721
Vlan720

10.0.0.25/32, ubest/mbest: 2/0

10.10.10.25/32, ubest/mbest: 2/0
*via 10.72.72.1, [1/0], 01:28:32, static
*via 10.72.72.3, [1/0], 01:28:32, static
10.71.71.0/31, ubest/mbest: 2/0
10.71.71.2/31, ubest/mbest: 2/0
*via 10.72.72.0, Vlan720, [0/0], 01:28:32, direct
*via 10.72.72.0, Vlan720, [0/0], 01:28:32, local
*via 10.72.72.2, Vlan721, [0/0], 01:28:32, direct
*via 10.72.72.2, Vlan721, [0/0], 01:28:32, local
N7K2(config)#
1.3 L2 Trunking and L3 Routed Interfaces

Configuration
To ensure that the L3 interfaces in our aggregation layer VDCs have the ability to
route, we must include them in their site's respective VRFs. We don't neccessarily
need to set these specific VRFs up in the OTV layer if we don't want to (we weren't
instructed to), so we will leave them in the default VRF.
N7K1:
vrf member DC1
ip address 10.13.13.0/31
no shutdown
switchport trunk allowed vlan 120,125,130,135,140,200-201
no shutdown
N7K3:
ip address 10.13.13.1/31
no shutdown
no shutdown
N7K2:
vrf member DC2
ip address 10.24.24.0/31
no shutdown
no shutdown
N7K4:
ip address 10.24.24.1/31
no shutdown
no shutdown
Verification
10.0.0.25/32, ubest/mbest: 2/0

10.10.10.25/32, ubest/mbest: 2/0
*via 10.71.71.1, [1/0], 01:28:34, static
*via 10.71.71.3, [1/0], 01:28:34, static
*via 10.13.13.0, Eth1/1, [0/0], 01:27:19, direct
*via 10.13.13.0, Eth1/1, [0/0], 01:27:19, local
10.24.24.0/31, ubest/mbest: 2/0
*via 10.71.71.0, Vlan710, [0/0], 01:28:35, direct
*via 10.71.71.0, Vlan710, [0/0], 01:28:35, local
*via 10.71.71.2, Vlan711, [0/0], 01:28:35, direct
*via 10.71.71.2, Vlan711, [0/0], 01:28:35, local
10.72.72.0/31, ubest/mbest: 2/0
10.72.72.2/31, ubest/mbest: 2/0
N7K1(config)#


10.0.0.25/32, ubest/mbest: 2/0

10.10.10.25/32, ubest/mbest: 2/0
*via 10.72.72.1, [1/0], 01:28:32, static
*via 10.72.72.3, [1/0], 01:28:32, static
10.13.13.0/31, ubest/mbest: 2/0

*via 10.24.24.0, Eth1/17, [0/0], 01:27:23, direct
*via 10.24.24.0, Eth1/17, [0/0], 01:27:23, local
10.71.71.0/31, ubest/mbest: 2/0
10.71.71.2/31, ubest/mbest: 2/0
*via 10.72.72.0, Vlan720, [0/0], 01:28:32, direct
*via 10.72.72.0, Vlan720, [0/0], 01:28:32, local
*via 10.72.72.2, Vlan721, [0/0], 01:28:32, direct
*via 10.72.72.2, Vlan721, [0/0], 01:28:32, local
N7K2(config)#
1.4 Port Channels

Configuration
N7K1:
lacp system-priority 1
port-channel load-balance src-dst ip-l4port
switchport
switchport trunk allowed vlan 120,125,130,135,200-201
switchport
switchport
no shutdown
switchport
no shutdown
Verification
These won't show up until you complete the UCS side, but once you have, they should show:
N7K1(config)# sh port-channel summary

Flags:
D - Down
I - Individual
s - Suspended
r - Module-removed
S - Switched
R - Routed
-------------------------------------------------------------------------------Group Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------215
Po215(SU)
Eth
LACP
N7K1(config)#
1.5 HSRP
Configuration
N7K1:
interface Vlan120
no shutdown
ip address 192.168.120.251/24
hsrp version 2
hsrp 120
priority 110
preempt
ip 192.168.120.254
interface Vlan125
no shutdown
ip address 192.168.125.251/24
hsrp version 2
hsrp 125
priority 110
preempt
ip 192.168.125.254
interface Vlan130
no shutdown
Eth1/22(P)
218
Po218(SU)
Eth
LACP
Eth1/24(P)
ip address 192.168.130.251/24
hsrp version 2
hsrp 130
priority 110
preempt
ip 192.168.130.254
interface Vlan135
no shutdown
ip address 192.168.135.251/24
hsrp version 2
hsrp 135
priority 110
preempt
ip 192.168.135.254
interface Vlan200
no shutdown
ip address 192.168.200.251/24
hsrp version 2
hsrp 200
priority 110
preempt
timers msec 250 msec 750
ip 192.168.200.254
interface Vlan201
no shutdown
ip address 192.168.201.251/24
hsrp version 2
hsrp 201
priority 110
preempt
ip 192.168.201.254
N7K2:
interface Vlan120
no shutdown
ip address 192.168.120.252/24
hsrp version 2
hsrp 120
priority 110
preempt
ip 192.168.120.254
interface Vlan125
no shutdown
ip address 192.168.125.252/24
hsrp version 2
hsrp 125
priority 110
preempt
ip 192.168.125.254
interface Vlan130
no shutdown
ip address 192.168.130.252/24
hsrp version 2
hsrp 130
priority 110
preempt
ip 192.168.130.254
interface Vlan135
no shutdown
ip address 192.168.135.252/24
hsrp version 2
hsrp 135
priority 110
preempt
ip 192.168.135.254
interface Vlan200
no shutdown
ip address 192.168.200.252/24
hsrp version 2
hsrp 200
priority 110
preempt
timers msec 250 msec 750
ip 192.168.200.254
interface Vlan201
no shutdown
ip address 192.168.201.252/24
hsrp version 2
hsrp 201
priority 110
preempt
ip 192.168.201.254
Verification
These HSRP group are shown after task 1.10 has been completed. At
that time, even though the vlans are extended, because both the
HSRP Hellos and the Virtual MAC addresses are both blocked by
ACLs, each side responds to ARP requests locally and also considers
itself the Active router.
N7K1(config)# sh hsrp
Vlan120 - Group 120 (HSRP-V2) (IPv4)
Local state is Active, priority 100 (Cfged 100), may preempt
Forwarding threshold(for vPC), lower: 1 upper: 100
Hellotime 3 sec, holdtime 10 sec
Next hello sent in 0.634000 sec(s)
Virtual IP address is 192.168.120.254 (Cfged) Active router is local
Standby router is unknown
Authentication text "cisco"
Virtual mac address is 0000.0c9f.f078 (Default MAC)
2 state changes, last state change 01:34:10
IP redundancy name is hsrp-Vlan120-120 (default)

Virtual mac address is 0000.0c9f.f07d (Default MAC)




Hellotime 250 msec, holdtime 750 msec
Virtual mac address is 0000.0c9f.f0c8 (Default MAC)

N7K1(config)#
N7K2(config)# sh hsrp


Virtual mac address is 0000.0c9f.f07d (Default MAC)





1.6 vPC
Configuration
N7K2:
N5K1:
feature lacp
feature vpc
vpc domain 12
role priority 1
system-priority 1
peer-keepalive destination 192.168.101.52 source 192.168.101.51
peer-gateway
ip arp synchronize
spanning-tree port type network
speed 10000
vpc peer-link
N5K2:
feature lacp
feature vpc
vpc domain 12
system-priority 1
peer-keepalive destination 192.168.101.51 source 192.168.101.52
peer-gateway
ip arp synchronize
spanning-tree port type network
speed 10000
vpc peer-link
Verification
N5K1(config)# sh vpc
Legend:
vPC domain id
: 12
Peer status
: peer is alive Configuration consistency status: success
: success
: success vPC role
: 67 Peer Gateway
Peer gateway excluded VLANs
: -
: -
: Enabled
: primary
: Enabled

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po512
up
1,120,125,130,135,200-201
Legend:
vPC domain id
: 12
Peer status
: peer is alive

: success
: success vPC role
: 67
Peer Gateway
: Enabled
: -
: -
: Enabled

---------------------------------------------------------------------
: secondary
id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po512
up
1,120,125,130,135,200-201
This spanning-tree show command is after the next task has been performed.
Remember the whole topology when thinking about where the STP domain will
reach. Only one VLAN is shown, although all have same result.
N5K1(config)# sh spanning-tree vlan 120
VLAN0120
Spanning tree enabled protocol rstp
Root ID
32888
Address
547f.ee7a.4d7c This bridge is the root
Hello Time
Priority
32888
Address
547f.ee7a.4d7c
Hello Time
Bridge ID
Priority
Interface
sec
sec
Role Sts Cost
Max Age 20 sec
Forward Delay 15 sec
(priority 32768 sys-id-ext 120)
Max Age 20 sec
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------Po100
Desg FWD 1
128.4195 (vPC) P2p
Po512
Desg FWD 1
128.4607 (vPC peer-link) Network P2p
Eth113/1/1
Desg FWD 1
128.1
(vPC) Edge P2p
Eth123/1/1
Desg FWD 1
128.1
(vPC) Edge P2p
N5K1(config)#
VLAN0120
Root ID
Bridge ID
Interface
Priority
32888
Address
547f.ee7a.4d7c
Cost
Port
4607 (port-channel512)
Hello Time
Priority
61560
Address
547f.ee79.137c
Hello Time
sec
sec
Role Sts Cost
Max Age 20 sec
Max Age 20 sec
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------Po100

1
128.4195 (vPC) P2p Po512
Desg FWD 1
Root FWD
128.4607 (vPC peer-link) Network P2p
Eth113/1/1
Desg FWD 1
128.2689 (vPC) Edge P2p
Eth123/1/1
Desg FWD 1
128.3969 (vPC) Edge P2p
N5K2(config)#
VLAN0120
Root ID
32888
Address
547f.ee7a.4d7c
Cost
Port
4195 (port-channel100)
Hello Time
Priority
57464
Address
64a0.e742.8dc2
Hello Time
Bridge ID
Priority
Interface
sec
sec
Role Sts Cost
Max Age 20 sec
Max Age 20 sec
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------- Po100

1
128.4195 P2p
Eth2/11
Desg FWD 2
128.267
P2p
N7K2(config)#
1.7 Port Channels, FEX, and vPC

Configuration
N7K2:
switchport
Root FWD
channel-group 100 mode passive

no shutdown
channel-group 100 mode passive
no shutdown
N5K1:
fex 113
pinning max-links 1
description "FEX0113"
fex 123
pinning max-links 1
slot 113
provision model N2K-C2232P
slot 123
speed 10000
vpc 100
fex associate 113
vpc 113
fex associate 123
vpc 123
speed 10000

fex associate 113
channel-group 113
shutdown
fex associate 123
channel-group 123
shutdown
N5K2:
fex 113
pinning max-links 1
fex 123
pinning max-links 1
slot 113
slot 123
speed 10000
vpc 100
fex associate 113
vpc 113
fex associate 123

vpc 123
speed 10000
fex associate 113
channel-group 113
shutdown
fex associate 123
channel-group 123
shutdown
Verification
N5K1(config)# sh fex
FEX
Number
FEX
FEX
Description
State
FEX
Model
Serial
-----------------------------------------------------------------------113
FEX0113
Online
N2K-C2232PP-10GE
SSI165204YC
123
FEX0123
Online
N2K-C2232PP-10GE
SSI16510AWF
Legend:
vPC domain id
: 12
Peer status
: peer is alive

: success
: success
vPC role
: primary
: 67
Peer Gateway
: Enabled
: -
: -
: Enabled

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po512
up
1,120,125,130,135,200-201
vPC status
---------------------------------------------------------------------------id
Port
Active vlans
------ ----------- ------ ----------- -------------------------- ----------100
Po100
up
success
success
1,10,120,12
5,130,135,2
00-201
113
Po113
up
success
success
123
Po123
up
success
success
N5K2(config)# sh fex
FEX
Number
FEX
FEX
Description
State
FEX
Model
Serial
-----------------------------------------------------------------------113
FEX0113
Online
N2K-C2232PP-10GE
SSI165204YC
123
FEX0123
Online
N2K-C2232PP-10GE
SSI16510AWF
Legend:
vPC domain id
: 12
Peer status
: peer is alive

: success
: success
vPC role
: secondary
: 67
Peer Gateway
: Enabled
: -
: -
: Enabled

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po512
up
1,120,125,130,135,200-201
vPC status
---------------------------------------------------------------------------id
Port
Active vlans
------ ----------- ------ ----------- -------------------------- ----------100
Po100
up
success
success
1,10,20,120
,125,130,13
5,200-201
113
Po113
up
success
success
123
Po123
up
success
success

Flags:
D - Down
I - Individual
s - Suspended
r - Module-removed
S - Switched
R - Routed
-------------------------------------------------------------------------------Group Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------100
Po100(SU)
Eth
LACP
Eth1/3(P)
113
Po113(SU)
Eth
NONE
Eth1/12(P) 123
512
Po512(SU)
Eth
LACP
Eth1/1(P)
Po123(SU)
Eth
NONE
Eth1/2(P)
N5K1(config)#

Flags:
D - Down
I - Individual
s - Suspended
r - Module-removed
S - Switched
R - Routed
-------------------------------------------------------------------------------Group Port-
Type
Protocol
Member Ports
Eth1/14(P)
Channel
-------------------------------------------------------------------------------100
Po100(SU)
Eth
LACP
Eth1/3(P)
113
Po113(SU)
Eth
NONE
Eth1/12(P) 123
512
Po512(SU)
Eth
LACP
Eth1/1(P)
Po123(SU)
Eth
NONE
Eth1/2(P)
N5K2(config)#
1.8 Mgmt VM Access

Configuration
N5K1:
speed 1000
Verification
N5K1(config)# sh int e1/11 | in up|1000 Ethernet1/11 is up
Hardware: 1000/10000 Ethernet, address: 547f.ee7a.4d32 (bia 547f.ee7a.4d32)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
full-duplex, 1000 Mb/s
, media type is 10G

0 jumbo packets
0 storm suppression bytes
N5K1(config)#
N5K1(config)# sh spanning-tree interface e1/11
Vlan
Role Sts Cost
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------VLAN0200
Desg FWD 4
128.139
1.9 Access Trunking
Edge P2p
Eth1/14(P)
Configuration
N5K1:
N5K2:
1.10 OTV
Configuration:
The L2 inside and L3 join interfaces have already been configured from both the
Aggregation and OTV layers, but we are displaying them here again just to bring
everything necessary into contextual view, and adding the necessary PIM and IGMP
configuration where necessary. You may notice that the ISP RP address of
10.10.10.25 doesn't appear in our RIB; this is because OSPF didn't advertise it to
us. We weren't instructed not to use static routes, so those will certainly suffice here
to point us toward it. We can only hope it has a route back, and when we confiure
redundant static routes out each of our links, we ping it and see that it, in fact, does.
Note that we must do this under our VRF, as well as set up PIM there.
N7K1:
vrf context DC1

ip route 10.10.10.25/32 10.71.71.1
ip route 10.10.10.25/32 10.71.71.3
vrf member DC1
ip address 10.13.13.0/31
ip pim sparse-mode
ip igmp version 3
no shutdown
no shutdown
N7K3:
feature otv
ip access-list ALL_IPs
10 permit ip any any
ip access-list HSRP_IP
10 permit udp any 224.0.0.2/32 eq 1985
20 permit udp any 224.0.0.102/32 eq 1985
vlan access-map HSRP_Localization 10
match ip address HSRP_IP
action drop
match ip address ALL_IPs
action forward
vlan filter HSRP_Localization vlan-list 120,125,130,135,200-201
mac-list OTV_HSRP_VMAC_deny seq 10 deny 0000.0c07.ac00 ffff.ffff.ff00

mac-list OTV_HSRP_VMAC_deny seq 20 deny 0000.0c9f.f000 ffff.ffff.f000
mac-list OTV_HSRP_VMAC_deny seq 30 permit 0000.0000.0000 0000.0000.0000
route-map OTV_HSRP_filter permit 10

match mac-list OTV_HSRP_VMAC_deny
key chain OTV

key 0
key-string DCIOTV
otv site-vlan 140

otv site-identifier 0x1
interface Overlay1
otv extend-vlan 120, 125, 130, 135, 200-201
no otv suppress-arp-nd
no shutdown
ip address 10.13.13.1/31
ip igmp version 3
no shutdown
no shutdown
otv-isis default
vpn Overlay1
redistribute filter route-map OTV_HSRP_filter
ip route 0.0.0.0/0 10.13.13.0
N7K2:
vrf context DC2

ip route 10.10.10.25/32 10.72.72.1
ip route 10.10.10.25/32 10.72.72.3
vrf member DC2
ip address 10.24.24.0/31
ip pim sparse-mode
ip igmp version 3
no shutdown
no shutdown
N7K4:
feature otv
ip access-list ALL_IPs
10 permit ip any any
ip access-list HSRP_IP
10 permit udp any 224.0.0.2/32 eq 1985
20 permit udp any 224.0.0.102/32 eq 1985
match ip address HSRP_IP
action drop
match ip address ALL_IPs
action forward
vlan filter HSRP_Localization vlan-list 120,125,130,135,200-201
mac-list OTV_HSRP_VMAC_deny seq 10 deny 0000.0c07.ac00 ffff.ffff.ff00

mac-list OTV_HSRP_VMAC_deny seq 20 deny 0000.0c9f.f000 ffff.ffff.f000
mac-list OTV_HSRP_VMAC_deny seq 30 permit 0000.0000.0000 0000.0000.0000
route-map OTV_HSRP_filter permit 10

match mac-list OTV_HSRP_VMAC_deny
key chain OTV

key 0
key-string DCIOTV
otv site-vlan 140

otv site-identifier 0x2
interface Overlay1
otv extend-vlan 120, 125, 130, 135, 200-201
no otv suppress-arp-nd
no shutdown
ip address 10.24.24.1/31
ip igmp version 3
no shutdown
no shutdown
otv-isis default
vpn Overlay1
redistribute filter route-map OTV_HSRP_filter
ip route 0.0.0.0/0 10.24.24.0
Verification
First, let's look at general OTV information from both sides of the tunnel.
N7K3(config)#sh otv

Site Identifier 0000.0000.0001
VPN name
: Overlay1
VPN state
: UP
Extended vlans
: 120 125 130 135 200-201 (Total:6)
Control group
: 224.1.1.1

Join interface(s)
: Eth1/9 (10.13.13.1)
Site vlan
: 140 (up)
AED-Capable
: Yes
Capability
: Multicast-Reachable N7K3(config)#sh otv adjacency
Overlay Adjacency database
Overlay-Interface Overlay1
Hostname
System-ID
N7K4
64a0.e742.8dc4 10.24.24.1
Dest Addr
OTV Extended VLANs and Edge Device State Information (* - AED)
Up Time
State
13:18:15
UP N7K3(config)#sh otv vlan
VLAN
Auth. Edge Device
Vlan State
Overlay
----
-----------------------------------
----------
-------
120*
N7K3
active
Overlay1
125*
N7K3
active
Overlay1
130*
N7K3
active
Overlay1
135*
N7K3
active
Overlay1
200*
N7K3
active
Overlay1
201*
N7K3
active
Overlay1
N7K3(config)#
N7K4(config)#sh otv

Site Identifier 0000.0000.0002
VPN name
: Overlay1
VPN state
: UP
Extended vlans
: 120 125 130 135 200-201 (Total:6)
Control group
: 224.1.1.1

Join interface(s)
: Eth1/25 (10.24.24.1)
Site vlan
: 140 (up)
AED-Capable
: Yes
Capability
: Multicast-Reachable N7K4(config)#sh otv adjacency
Overlay Adjacency database
Overlay-Interface Overlay1
Hostname
System-ID
Dest Addr
N7K3
64a0.e742.8dc3 10.13.13.1
Up Time
State
13:35:44
UP N7K4(config)#sh otv vlan
OTV Extended VLANs and Edge Device State Information (* - AED)
VLAN
Auth. Edge Device
Vlan State
Overlay
----
-----------------------------------
----------
-------
120*
N7K4
active
Overlay1
125*
N7K4
active
Overlay1
130*
N7K4
active
Overlay1
135*
N7K4
active
Overlay1
200*
N7K4
active
Overlay1
201*
N7K4
active
Overlay1
N7K4(config)#
We will ping the SVI for VLAN 200 on N7K2 in DC2, but first let's see what its MAC
and IP are.
N7K2(config)# sh int vlan200 | in ddress
Hardware is EtherSVI, address is
64a0.e742.8dc2
Internet Address is 192.168.200.252/24

N7K2(config)#
Let's look at the OTV routing table before we ping this particular IP; notice that the
MAC is not in the table.
N7K3(config)# sh otv route
VLAN MAC-Address
Metric
Uptime
Owner
Next-hop(s)
---- --------------
------
--------
---------
-----------
200 000c.29bb.9b82
42
13:18:17
overlay
N7K4
200 0025.b50a.0a06
13:18:16
site
Ethernet2/9
200 0025.b50a.0a0b
13:18:12
site
Ethernet2/9
200 d48c.b5bd.460c
42
13:18:17
overlay
N7K4
N7K3(config)#
Now we try to ping 192.168.200.252 and see that after a while, the OTV tunnel
routes traffic for the newly learned MAC address of 64a0.e742.8dc2.
N7K1(config)# ping 192.168.200.252

PING 192.168.200.252 (192.168.200.252): 56 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
--- 192.168.200.252 ping statistics --5 packets transmitted, 2 packets received, 60.00% packet loss
N7K1(config)#
Verify that route was added. We also see our local address (64a0.e742.8dc1) as
now known by our internal interface.
N7K3(config)# sh otv route
VLAN MAC-Address
Metric
Uptime
Owner
Next-hop(s)
---- --------------
------
--------
---------
-----------
200 000c.29bb.9b82
42
13:19:00
overlay
N7K4
200 0025.b50a.0a06
13:18:59
site
Ethernet2/9
200 0025.b50a.0a0b
13:18:55
site
Ethernet2/9
200 64a0.e742.8dc1
00:00:26
site
Ethernet2/9
200 64a0.e742.8dc2
42
00:00:26
overlay
N7K4
200 d48c.b5bd.460c
42
13:19:00
overlay
N7K4
N7K3(config)#

2.1 VSANs and FCoE VLANs
Configuration
N5K1:
feature fcoe
vlan 10
fcoe vsan 10
vsan database
vsan 10
N5K2:
feature fcoe
vlan 10
fcoe vsan 10
vlan 20
fcoe vsan 20
vsan database
vsan 10
vsan 20
MDS1:
vsan database
vsan 10
vsan 20
MDS2:
vsan database
vsan 10
vsan 20
2.2 UCS SAN Connectivity

Configuration
MDS1:
feature npiv
vsan database

interface fc1/1
switchport mode F
no shutdown
interface fc1/2
switchport mode F
no shutdown
interface fc1/9
switchport mode F
no shutdown
interface fc1/10
switchport mode F
no shutdown
Verification
This verification is pulled after the UCS side has been configured.
MDS1(config)# sh flogi d
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/1
10
0x610000
20:1d:00:2a:6a:15:66:80 20:0a:00:2a:6a:15:66:81
fc1/1
10
0x610002
20:aa:00:25:b5:01:01:01 20:ff:00:25:b5:01:01:0f
[ESXi1-A-fc0]
fc1/2
10
0x610001
20:1e:00:2a:6a:15:66:80 20:0a:00:2a:6a:15:66:81
fc1/2
10
0x610003
20:aa:00:25:b5:01:01:02 20:ff:00:25:b5:01:01:0e
[ESXi1-B-fc1]
fc1/9
20
0x610000
20:1b:00:2a:6a:15:05:00 20:14:00:2a:6a:15:05:01
fc1/9
20
0x610002
20:bb:00:25:b5:01:01:01 20:ff:00:25:b5:01:01:0f
[ESXi2-A-fc0]
fc1/10
20
0x610001
20:1c:00:2a:6a:15:05:00 20:14:00:2a:6a:15:05:01
fc1/10
20
0x610003
20:bb:00:25:b5:01:01:02 20:ff:00:25:b5:01:01:0e
[ESXi2-B-fc1]
MDS1(config)# sh int fc1/1-2

fc1/1 is up
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Port WWN is 20:01:00:0d:ec:4a:21:00
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is F, FCID is 0x610000

Port vsan is 10
Speed is 2 Gbps
Transmit B2B Credit is 16
Receive B2B Credit is 16
fc1/2 is up
Port WWN is 20:02:00:0d:ec:4a:21:00
Port vsan is 10
Speed is 2 Gbps
MDS1(config)# sh int fc1/9-10

fc1/9 is up
Port WWN is 20:09:00:0d:ec:4a:21:00
Port vsan is 20
Speed is 2 Gbps
fc1/10 is up
Port WWN is 20:0a:00:0d:ec:4a:21:00
Port vsan is 20
Speed is 2 Gbps
MDS1(config)#
2.3 E Port Trunking

Configuration
N5K1:
slot 1
port 26-32 type fc (reboot)
vsan database

channel mode active
switchport mode E
switchport trunk mode on
interface fc1/26
switchport mode E
no shutdown
interface fc1/27
switchport mode E
no shutdown
interface fc1/32
switchport mode F
no shutdown
N5K2:
slot 1
port 26-32 type fc (reboot)

channel mode active
switchport mode E
interface fc1/26
switchport mode E
no shutdown
interface fc1/27
switchport mode E
no shutdown
interface fc1/28
switchport trunk allowed vsan add 20
switchport mode E
no shutdown
MDS2:
vsan database
interface fc1/3
switchport mode E
no shutdown
interface fc1/7
switchport mode F
no shutdown
Verification
N5K1(config)# sh int fc1/26-27
fc1/26 is trunking
Port WWN is 20:1a:54:7f:ee:7a:4d:40
Peer port WWN is 20:1a:54:7f:ee:79:13:40
Admin port mode is E, trunk mode is on
snmp link state traps are enabled Port mode is TE

Port vsan is 1
Speed is 2 Gbps
Receive data field Size is 2112
Beacon is turned off
Trunk vsans (admin allowed and active) (10) Trunk vsans (up)
Trunk vsans (isolated)
()
Trunk vsans (initializing)
()
(10)
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
250 frames input, 24460 bytes
13 discards, 0 errors
0 CRC,
0 unknown class
0 too long, 0 too short

197 frames output, 18776 bytes
2 input OLS, 2 LRR, 2 NOS, 0 loop inits
8 output OLS, 3 LRR, 1 NOS, 0 loop inits
last clearing of "show interface" counters never
16 receive B2B credit remaining
16 transmit B2B credit remaining
0 low priority transmit B2B credit remaining
Interface last changed at Wed Mar 24 15:25:18 2010
fc1/27 is trunking
Port WWN is 20:1b:54:7f:ee:7a:4d:40
Peer port WWN is 20:1b:54:7f:ee:79:13:40 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 2 Gbps
()
()

(10)
0 CRC,
0 unknown class

N5K1(config)#
N5K1(config-if)# sh san-port-channel summary
U-Up D-Down B-Hot-standby S-Suspended I-Individual link
summary header
-------------------------------------------------------------------------------Group
Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------_256
San-po256
FC
PCP
(U)
FC
fc1/26(P)
fc1/27(P)
_
N5K1(config-if)#
N5K1(config-if)# sh int san-port-channel 256

san-port-channel 256 is trunking
Hardware is Fibre Channel
Port WWN is 25:00:54:7f:ee:7a:4d:40 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 8 Gbps
()
()

0 CRC,
0 unknown class

(10)

Member[1] : fc1/26
Member[2] : fc1/27
N5K2(config)# sh int fc1/26-27

fc1/26 is trunking
Port WWN is 20:1a:54:7f:ee:79:13:40
Peer port WWN is 20:1a:54:7f:ee:7a:4d:40 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 2 Gbps
()
()

0 CRC,
0 unknown class

Interface last changed at Mon Apr 20 15:27:31 2009
fc1/27 is trunking
Port WWN is 20:1b:54:7f:ee:79:13:40
Peer port WWN is 20:1b:54:7f:ee:7a:4d:40 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 2 Gbps
(10)

()
()
(10)

0 CRC,
0 unknown class

N5K2(config)#
N5K2(config-if)# sh san-port-channel summary
U-Up D-Down B-Hot-standby S-Suspended I-Individual link
summary header
-------------------------------------------------------------------------------Group
Port-
Type
Protocol
Member Ports
Channel
-------------------------------------------------------------------------------256
San-po256
FC
PCP
(U)
FC
fc1/26(P)
fc1/27(P)
N5K2(config-if)#
N5K2(config-if)# sh int san-port-channel 256

san-port-channel 256 is trunking
Port WWN is 25:00:54:7f:ee:79:13:40 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 8 Gbps
(10)
()
()

0 CRC,
0 unknown class

Member[1] : fc1/26
Member[2] : fc1/27
N5K2(config-if)# sh int fc1/28

fc1/28 is trunking
Port WWN is 20:1c:54:7f:ee:79:13:40
Peer port WWN is 20:03:00:0d:ec:28:cf:00 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 2 Gbps
Trunk vsans (admin allowed and active) (10,20) Trunk vsans (up)
()
()

0 CRC,
0 unknown class

(10,20)

Interface last changed at Sun Apr 19 21:30:28 2009
N5K2(config-if)# sh int fc1/32

fc1/32 is up
Port WWN is 20:20:54:7f:ee:79:13:40 Admin port mode is F, trunk mode is on
Port mode is F, FCID is 0x520100 Port vsan is 10
Speed is 4 Gbps
0 CRC,
0 unknown class

Interface last changed at Sun Apr 19 21:30:27 2009
N5K2(config-if)#
N5K2(config-if)#
N5K2(config-if)# sh flogi d
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/32
10
0x520100
21:03:00:1b:32:64:5e:dc 20:03:00:1b:32:64:5e:dc
[FC-TARGET-SAN-A]
N5K2(config-if)#
MDS2(config)# sh int fc1/3
fc1/3 is trunking
Port WWN is 20:03:00:0d:ec:28:cf:00
Peer port WWN is 20:1c:54:7f:ee:79:13:40 Admin port mode is E, trunk mode is on
Port vsan is 1
Speed is 2 Gbps
()
()
5 minutes input rate 3360 bits/sec, 420 bytes/sec, 2 frames/sec

5 minutes output rate 3192 bits/sec, 399 bytes/sec, 2 frames/sec
0 CRC,
0 unknown class

Interface last changed at Sat Jun
8 21:20:15 2013
MDS2(config)#
MDS2(config)# sh flog d
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/7
20
0x620000
21:01:00:1b:32:24:5e:dc 20:01:00:1b:32:24:5e:dc
[FC-TARGET-SAN-B]
MDS2(config)#
(10,20)
2.4 Cisco C200 P81E (VIC) CNA FLOGIs

Configuration
N5K1:
fex 113
fcoe
vsan database
vsan 10 interface vfc113
interface vfc113
bind interface e113/1/1
switchport mode F
N5K2:
fex 123
fcoe
vsan database
vsan 20 interface vfc123
interface vfc123
bind interface e123/1/1
switchport mode F
Verification
--------------------------------------------------------------------------------
INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------vfc113
10
0x510001
20:00:d4:8c:b5:bd:46:0e 10:00:d4:8c:b5:bd:46:0e
[ESXi3-A-fc0]
N5K1(config-if)#
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/32
10
0x520100
21:03:00:1b:32:64:5e:dc 20:03:00:1b:32:64:5e:dc
[FC-TARGET-SAN-A]
vfc123
20
0x520004
20:00:d4:8c:b5:bd:46:0f 10:00:d4:8c:b5:bd:46:0f
[ESXi3-B-fc1]
N5K2(config-if)#
2.5 FCIP
Configuration
MDS1:
feature fcip
fcip profile 10
ip address 12.12.12.1
tcp max-bandwidth-mbps 1000 min-available-bandwidth-mbps 300
round-trip-time-us 300
tcp cwm burstsize 100
fcip profile 20
channel mode active
switchport mtu 3000

switchport mode E
switchport rate-mode dedicated
no shutdown
interface fcip10
use-profile 10
peer-info ipaddr 12.12.12.2
switchport mode E
no shutdown
interface fcip20
use-profile 20
switchport mode E
no shutdown
ip address 12.12.12.1 255.255.255.252
switchport mtu 3000
no shutdown
ip address 12.12.12.5 255.255.255.252
switchport mtu 3000
no shutdown
MDS2:
feature fcip
fcip profile 10
fcip profile 20
channel mode active

switchport mode E
switchport rate-mode dedicated
no shutdown
interface fcip10
use-profile 10
switchport mode E
no shutdown
interface fcip20
use-profile 20
switchport mode E
no shutdown
ip address 12.12.12.2 255.255.255.252
switchport mtu 3000
no shutdown
ip address 12.12.12.6 255.255.255.252
switchport mtu 3000
no shutdown
Verification
MDS1(config)# sh int fcip10 - 20
fcip10 is trunking
Hardware is GigabitEthernet
Port WWN is 20:10:00:0d:ec:4a:21:00
Peer port WWN is 20:10:00:0d:ec:28:cf:00
Port vsan is 1
Speed is 1 Gbps Belongs to port-channel 50
(10,20)
()
()
Using Profile id 10
(interface GigabitEthernet1/1)
Peer Information
Peer Internet address is 12.12.12.2 and port is 3225
Write acceleration mode is configured off

Tape acceleration mode is configured off
Tape Accelerator flow control buffer size is automatic
FICON XRC EMulator mode is configured off
Ficon Tape acceleration configured off for all vsans
IP Compression is disabled
Maximum number of TCP connections is 2
QOS control code point is 0
QOS data code point is 0
TCP Connection Information
2 Active TCP connections
Control connection: Local 12.12.12.1:3225, Remote 12.12.12.2:65532
Data connection: Local 12.12.12.1:3225, Remote 12.12.12.2:65534
2 Attempts for active connections, 2 close of connections
TCP Parameters
Path MTU 3000 bytes
Current retransmission timeout is 200 ms

Round trip time: Smoothed 2 ms, Variance: 1 Jitter: 160 us
Advertized window: Current: 35 KB, Maximum: 27 KB, Scale: 4
Peer receive window: Current: 25 KB, Maximum: 25 KB, Scale: 4
Congestion window: Current: 100 KB, Slow start threshold: 742 KB
Current Send Buffer Size: 27 KB, Requested Send Buffer Size: 0 KB
CWM Burst Size: 100 KB
Measured RTT : 0 us Min RTT: 0 us Max RTT: 0 us
32676 Class F frames input, 3690456 bytes
524408 Class 2/3 frames input, 738852052 bytes
0 Reass frames
0 Error frames timestamp error 0
32952 Class F frames output, 3017512 bytes
205230 Class 2/3 frames output, 135347900 bytes
0 Error frames
fcip20 is trunking
Port WWN is 20:14:00:0d:ec:4a:21:00
Peer port WWN is 20:14:00:0d:ec:28:cf:00
Port vsan is 1
Speed is 1 Gbps Belongs to port-channel 50
()
()
Using Profile id 20
Peer Information

TCP Parameters
Path MTU 3000 bytes

0 Reass frames
0 Error frames
MDS1(config)#
MDS1(config)# sh port-channel summary

------------------------------------------------------------------------------
(10,20)
Interface
Total Ports
Oper Ports
First Oper Port
-----------------------------------------------------------------------------port-channel 50
MDS1(config)#
MDS1(config)# sh int port-channel 50
port-channel 50 is trunking
Port WWN is 24:32:00:0d:ec:4a:21:00
Port mode is TE
Port vsan is 1
Speed is 2 Gbps
Trunk vsans (admin allowed and active) (10,20)
Trunk vsans (up)
(10,20)
()
()

0 Reass frames
0 Error frames
Member[1] : fcip10
Member[2] : fcip20
MDS1(config)#
MDS2(config)# sh int fcip10-20

fcip10 is trunking
Peer port WWN is 20:10:00:0d:ec:4a:21:00
Port mode is TE
Port vsan is 1
Speed is 1 Gbps
Belongs to port-channel 50
Trunk vsans (up)
(10,20)
fcip10
()
()
Using Profile id 10
Peer Information
TCP Parameters
Path MTU 3000 bytes
0 Reass frames
0 Error frames
fcip20 is trunking
Peer port WWN is 20:14:00:0d:ec:4a:21:00

Port mode is TE
Port vsan is 1
Speed is 1 Gbps
Belongs to port-channel 50
Trunk vsans (up)
(10,20)
()
()
Using Profile id 20
Peer Information
TCP Parameters
Path MTU 3000 bytes
0 Reass frames
0 Error frames
MDS2(config)# sh port-channel sum

-----------------------------------------------------------------------------Interface
Total Ports
Oper Ports
First Oper Port
-----------------------------------------------------------------------------port-channel 50
fcip10
MDS2(config)# sh int po50

port-channel 50 is trunking
Port vsan is 1
Speed is 2 Gbps
()
()

0 Reass frames
0 Error frames
Member[1] : fcip10
Member[2] : fcip20
MDS2(config)#
2.6 Zoning
Configuration
MDS1:
fcdomain domain 97 preferred vsan 10

(10,20)
device-alias name ESXi1-A-fc0 pwwn 20:aa:00:25:b5:01:01:01
device-alias name ESXi1-B-fc1 pwwn 20:aa:00:25:b5:01:01:02
device-alias name ESXi2-A-fc0 pwwn 20:bb:00:25:b5:01:01:01
device-alias name ESXi2-B-fc1 pwwn 20:bb:00:25:b5:01:01:02
device-alias name ESXi3-A-fc0 pwwn 20:00:d4:8c:b5:bd:46:0e
device-alias name ESXi3-B-fc1 pwwn 20:00:d4:8c:b5:bd:46:0f
device-alias name FC-TARGET-SAN-A pwwn 21:03:00:1b:32:64:5e:dc
device-alias name FC-TARGET-SAN-B pwwn 21:01:00:1b:32:24:5e:dc
device-alias commit

zone name ZONE-A vsan 10

member pwwn 20:aa:00:25:b5:01:01:01
!
[ESXi1-A-fc0]
member pwwn 20:bb:00:25:b5:01:01:01
[ESXi2-A-fc0]
member pwwn 20:00:d4:8c:b5:bd:46:0e
[ESXi3-A-fc0]
member pwwn 21:03:00:1b:32:64:5e:dc
[FC-TARGET-SAN-A]
zoneset name ZoneSet_VSAN10 vsan 10

member ZONE-A
zoneset activate name ZoneSet_VSAN10 vsan 10
zone commit vsan 10
zone name ZONE-B vsan 20

member pwwn 20:aa:00:25:b5:01:01:02
!
[ESXi1-B-fc1]
member pwwn 20:bb:00:25:b5:01:01:02
[ESXi2-B-fc1]
member pwwn 20:00:d4:8c:b5:bd:46:0f
[ESXi3-B-fc1]
member pwwn 21:01:00:1b:32:24:5e:dc
[FC-TARGET-SAN-B]
member ZONE-B
zoneset activate name ZoneSet_VSAN20 vsan 20
zone commit vsan 20

MDS2:


N5K2:


N5K1:
Verification
MDS1(config)# sh zoneset active
* fcid 0x610002 [pwwn 20:aa:00:25:b5:01:01:01] [ESXi1-A-fc0]
* fcid 0x610003 [pwwn 20:bb:00:25:b5:01:01:01] [ESXi2-A-fc0]
* fcid 0x510001 [pwwn 20:00:d4:8c:b5:bd:46:0e] [ESXi3-A-fc0]
* fcid 0x520100 [pwwn 21:03:00:1b:32:64:5e:dc] [FC-TARGET-SAN-A]

* fcid 0x610002 [pwwn 20:aa:00:25:b5:01:01:02] [ESXi1-B-fc1]
* fcid 0x610003 [pwwn 20:bb:00:25:b5:01:01:02] [ESXi2-B-fc1]
* fcid 0x520004 [pwwn 20:00:d4:8c:b5:bd:46:0f] [ESXi3-B-fc1]
* fcid 0x620000 [pwwn 21:01:00:1b:32:24:5e:dc] [FC-TARGET-SAN-B]
MDS1(config)#
MDS2(config)# sh zoneset active


MDS2(config)#
N5K1(config)# sh zoneset active


N5K1(config)#
N5K2(config)# sh zoneset active



N5K2(config)#
Configuration
UCS-FI-A:
Enter the setup mode; setup newly or restore from backup. (setup/restore) ?setup
You have chosen to setup a new Fabric interconnect. Continue? (y/n):y
Enforce strong password? (y/n) [y]:y
Is this Fabric interconnect part of a cluster(select 'no' for standalone)? (yes/no) [n]:yes
Enter the switch fabric (A/B) []:A
Enter the system name:UCS-FI
Physical Switch Mgmt0 IPv4 netmask :255.255.255.0
IPv4 address of the default gateway :192.168.101.1
Cluster IPv4 address :192.168.101.200
Configure the DNS Server IPv4 address? (yes/no) [n]:
Configure the default domain name? (yes/no) [n]:
Following configurations will be applied:
Switch Fabric=A
System Name=UCS-FI
Enforced Strong Password=yes
Physical Switch Mgmt0 IP Address=192.168.101.201
Physical Switch Mgmt0 IP Netmask=255.255.255.0
Default Gateway=192.168.101.1
Cluster Enabled=yes
Cluster IP Address=192.168.101.200
NOTE: Cluster IP will be configured only after both Fabric Interconnects are initialized
UCS-FI-B:
Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the c
y
Enter the admin password of the peer Fabric interconnect:Cciedc01
Connecting to peer Fabric interconnect... done
Retrieving config from peer Fabric interconnect... done
Peer Fabric interconnect Mgmt0 IP Address: 192.168.101.201
Peer Fabric interconnect Mgmt0 IP Netmask: 255.255.255.0
Cluster IP address
: 192.168.101.200
3.2 SAN Uplinks and VSANs

Configuration
From any screen, click Options, clear all Confirm options on this page, and click
OK.
Choose Fabric Interconnect B and click Configure Unified Ports.
Click Yes.
Slide the slider to the right to just under port 27/28, as shown in the diagram. When
you click Finish, the FI will reboot.
Choose Fabric Interconnect A and click Configure Unified Ports.
Click Yes.
When you click Finish, the FI will reboot.
After the FIs both return to an UP state, on FI-A click SAN Uplinks Manager.
Click the VSANs tab and then the Fabric A tab.
Fill in the information as shown below.
Click the VSANs tab and then the Fabric B tab.
Fill in the inforomation as shown below.
On FI-A, disable port 31, and do the same for port 32.
On FI-B, disable port 29, and do the same for ports 30-32
On FI-A, select port 29 and click Show Navigator.
Change the VSAN to VSAN10. Do the same for port 30 on FI-A.
Note that both ports now show up, assuming that the MDS1 was configured properly
before in F mode.
On FI-B, select port 27 and click Show Navigator.
Change the VSAN to VSAN20. Do the same for port 28 on FI-B.
Note that both ports now show up, assuming that the MDS1 was configured properly
before in F mode.
3.3 LAN Uplinks and VLANs

Configuration
On FI-A or FI-B, click LAN Uplinks Manager.
On the VLANs tab, click the All tab.
Fill in the information as shown below for VLAN 120, and repeat for the rest of the
VLANs.
This should be the result.
To configure the links from the FIs to the FEX/IOMs, click the Equipment tab, click
Equipment category on left, click the Policies tab, click the Global Policies tab,
and choose Port Channel.
Back on FI-A, Unconfigured Ethernet Ports, choose ports 1 and 2, right-click, and
change them to Server Ports.
Click Yes.
Choose port 8 and Configure as Uplink for your primary port.
Choose port 11 and Configure as Uplink for your BACKUP VLAN port. Do the
same for ports 8 and 11 on FI-B.
It sShould look like this when finished.
vem 4 hosttype
N1Kv(config)#
policy-map
vmware
shqos
modMod
id
SetCoS4
625366c3-3bc9-e211-0001-00000000010evem
Ports
class
Module-Type
class-default
set cos 4port-profile
5 host
Model
vmware
type
idethernet
625366c3-3bc9-e211-0001-00000000010f
Status--vMotion_UPLINK
----- policy-map
-----------------------SetCoS4
Back
215
It
218
M
R
VLANs
VLAN
Port
BACKUP
Interface
Do
Fabric
Everything
Enable
Best
Navigator
Show
1Gbps
UP
Local
The
Admin
U
Create
Change
FLOGI
Note
IExpert
eth0
eth1
eth3
D
eth4
.Repeat
Things
Next
Down
Assign
Choose
Management
Pooled
S
Yes
Note
Before
Fill
E
From
fc0
fc1
SAN
Save
Repeat
vHBA
,Change
OK
VMWare
We
Servers
Threshold
F
C
vNIC
O
Add
So
T
Finish
Click
Assign
ESXi1
ESXi2
If
On
LAN
Create
N
Host
.A
Full
eth2
ESX1
ESX2
class
UI
MPORTANT:
(obscured
tab.
at
class.
tab,
to
on
and
service
but
If
Click
and
change
boot
ill
hings
elect
nter
ssign
ight-click
o
se
ouble-click
ote
lick
hoose
ame
ur
ove
should
you
configure
3.
you
add
be
the
((3,000,000,000
in
for
the
left
the
both
won't
calculation
Backup
results
the
SAN
to
then
Select
this
to
right-click
click
Channel
Boot
Pin
itto
on
that
Effort
that
Control
in
click
Stats
the
as
Changes
port
change
pulled
the
to
Disk
itInterface
Manager
150
fc0
fc1
that
Blade
one
bottom
going
same
we
A
B
should
this
the
your
to
do
Retries
right,
itleft,
Port
UUID
MAC
WWNN
Block
Service
Maintenance
BIOS
SAN
a
Threshold
QoS
the
VLAN
Primary
Secondary
for
the
Later
and
profiles.
UUIDs
to
blade
the
this
the
service
World
Service
shown
use
appear
time
as
the
ESXi
should
Group
Local
Clone
newly
itchange
name.
add
11
8
server.
the
VLAN
Boot
this
not
8do
configure
the
port.
new
should
in
you
Policies
newly
should
should
LAN
all
Config
proper
Equipment
a
Fibre
over
for
VLAN
Channel
change
Policy
from
Pin
storage,
jumbo
and
1.
through
Pool
thing
for
this
Suffix
Policy
anything
from
the
device
itfill
IP
of
right
If
2.
215
look
other
port
step
5.1
the
just
pool.
is:
Pool
Wide
QoS
to
below
below.
cloned
Disk
for
Target
Profile
you
profiles
boot.
the
port
Uplinks
up
150
this
After
IP
Address
Group
and
Profile
click
picture),
this,
created
VLANs
Channel
create
Delta
look
should
find
service
for
look
Policy
the
uses
channel
port
like
when
created.
Addresses
frames
ifis
/8)
VLANs
Pool
accidentally
LAN
policy
11
in,
from
Config
association.
upstream
and
Port
Policy
click
the
you
to
but
completely
the
with
*service
like
this
tab,
like
channel
(expert)
the
on
=
a60)
Manager...
create
Association
finished.
allowed
connectivity
rest
threshold
boot.
choose
we
((Gbps/8)*SamplingInterval)
1Gbps
which
and
profile
have
proper
Name
Adapter
this
with
to
FI-B.
when
service
this.
and
system
on
Click
for
Policy
need
of
choose
profile,
the
Server
when
Nexus
the
an
enable
clicked
ID
to
the
Then
select
completed.
link,
on
boot
the
right,
optional
MTU
from
boot.
profiles,
to
rising
Policy
will
policy
VLANs
each
finished.
WWNN
know
as
so
expand
1,
is
they
to
LUN
still
the
it.before:
and
of
configured
that
we
value
from
to
of
9000.
assign
BACKUP
should
that
let's
ifand
vNIC
need
click
the
Click
the
your
pool
the
this
of
itclick
port
service
blade
is
22,500,000,000
vHBAs,
CoS
to
previously
upstream
both
window.
prperly
60
go
channel
VLAN
has
seconds
3appear
into
profiles
and
right-click
a
from
(150),
(You
the
zoning
service
created.
but
be associated
to
before.Again,
put
again
may
because
GB/Hr.
calculate
was
in
profile
have
the
from
completed
proper
of
to
association,
opted
our
the
the
the
click
value
disjointed
802.1Qbb
blades
toproperly.
dointhis
and
our
one
L2
before
PFC
vNIC
that
atuplinks,
aitlane.
time,
Thresho
is
beginni
curre
you
an
Configuration
Verification
3.4
3.5
3.6
3.7
3.8
4.1
4.2
Disk Policies
Pools
Service
Cloning
Traffic
VSM
N1Kv
and
QoS
Monitoring
Profiles
Service
VEM
Connectivity
Profiles
4. Data
Center
Virtualization

Ccie DC Full Scale Labs PDF

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Ccie DC Full Scale Labs PDF

Încărcat de

Drepturi de autor:

Formate disponibile

CCIE Data Center Full-Scale Labs - Bootcamp

Members - CCIE Data Center Full-Scale Lab 1

1. Data Center Infrastructure

1.2 Nexus 7K VDC Initialization

1.3 Initial IP Addressing

1.4 Layer 3 Routing

1.7 FabricPath Traffic Engineering

Ethernet peers in VLANs 200299.

1.8 Spanning-Tree Protocol Optimization

1.9 Fabric Extenders

2. Data Center Storage Networking

2.2 VSANs and Trunking

2.3 Fibre Channel Zoning

2.4 iSCSI Virtual Target

3.2 UCS Service Profile Templates

3.3 Service Profiles

4. Data Center Virtualization

4.2 Private VLANs

CCIE Data Center Full-Scale Labs - Bootcamp

1. Data Center Infrastructure

Configure the DNS Server IPv4 address? (yes/no) [n]:

Configure the default domain name? (yes/no) [n]:

Following configurations will be applied:

Physical Switch Mgmt0 IPv4 address :192.168.101.202

Applying configuration. Please wait.

1.2 Nexus 7K VDC Initialization

vdc_id: 0 vdc_name: Unallocated interfaces:

vdc_id: 1 vdc_name: N7K1 interfaces:

VDCs, and allocate the needed ports.

Creating VDC, one moment please ...

N7K1 %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 2 has come online N7K1(config-vdc)#

Creating VDC, one moment please ...

N7K1 %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 3 has come online N7K1(config-vdc)#

Creating VDC, one moment please ...

N7K1 %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 4 has come online N7K1(config-vdc)#

---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no) [y]:y

---- Basic System Configuration Dialog VDC: 2 ----

Please register Cisco Nexus7000 Family devices promptly with your

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime

Would you like to enter the basic configuration dialog (yes/no): n

End with CNTL/Z. N7K2(config)#feature telnet

---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no) [y]:y

---- Basic System Configuration Dialog VDC: 3 ----

Please register Cisco Nexus7000 Family devices promptly with your

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime

Would you like to enter the basic configuration dialog (yes/no): n

End with CNTL/Z. N7K3(config)#feature telnet

N7K1#switchto vdc N7K4

---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no) [y]:y

---- Basic System Configuration Dialog VDC: 4 ----

Please register Cisco Nexus7000 Family devices promptly with your

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime

Would you like to enter the basic configuration dialog (yes/no): n

End with CNTL/Z. N7K4(config)#feature telnet

vdc_id: 1 vdc_name: N7K1 interfaces:

vdc_id: 2 vdc_name: N7K2 interfaces:

vdc_id: 3 vdc_name: N7K3 interfaces:

vdc_id: 4 vdc_name: N7K4 interfaces:

Some interfaces not listed on the diagram must still be allocated to