FabricPath RFULLER PDF

Flexible Data Centre Fabric -
FabricPath/TRILL, OTV, LISP

and VXLAN
Ron Fuller– CCIE #5851 (R&S/Storage)

Technical Marketing Engineer, Nexus 7000
rfuller@cisco.com
Agenda
 The Evolving Data Centre Fabric
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 LISP Host Mobility
 OTV LAN Extension
 Mobility with Extended Subnets
 Nexus Fabric
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
Goals of the Fabric
Addressing Concurrent Workloads, Mobility and Latency
Port Density Priority Flow Control
Adequate Buffer Capacity Early Congestion Notification
Adequate Table Sizes FabricPath Multiple Trees
Low Latency Switching ECMP L2 & L3
Cut-through Switching Multi-tenancy
: :
: :
Architecture is evolving Rapidly – in the next 24 months

L2/L3 Boundary becomes less relevant
Clos Topologies dominate new implementations
HA models shift
Server Edge becomes more intelligent
DC Fabric becomes more scalable
Goals of the Fabric
Addressing High Availability and Fate Sharing
L3
L3/L2 L3/L2
L2 L2
East-West traffic – Fate Sharing Domain

Larger POD East-West Traffic – Fate Sharing Domain
STP is the protocol of choice
N+1 redundancy
1+1 redundancy – limited forwarding paths
IS-IS is the protocol of choice
Broad forwarding paths
East-West across L3 boundaries Broader Adjacency Support
OSPF/EIGRP are protocols of choice
N+1 redundancy – Broad forwarding Paths Same number of physical boxes and links
Protocol behavior is L3-like
North-South traffic Multi-pathing over L2 and L3
OSPF/EIGRP are protocols of choice More flexible L2 adjacency, better scale capacity
N+1 redundancy – Broad forwarding paths Better latency consistency within POD
Goals of the Fabric
Not a L2 vs. L3 debate
L2/L3
 The traditional L2 vs. L3 debate has been based on a number of issues

 Scalability
 Availability
 Requirements for the scalable design moving forward is a scalable,

highly available switching fabric with the advantages of both L2 and L3
“Plug-and-Play” and Mobility vs. Availability and Scaling
Advantages of Layer 2 Disadvantages of Layer 2
 Practically “plug-n-play” – No user  MAC address consumption

configuration is required to build forwarding
database  BPDU generation is CPU intensive with increasing
number of VLANs
 It makes it simple to support teaming or L2
 VLAN sprawl causes flooding and broadcasts to
multicast for clusters
propagate even where they are not needed
 Easy to segment traffic with VLANs  Half of the links in the topology are blocking
 Very fast movement of end station addresses  Misconfigurations can cause Layer 2 loops which
(ability to update MAC address tables after a may make switches unmanageable
vMotion-type event)
MAC Table MAC Table
A A
Layer 2
Domain
MAC Table MAC Table MAC Table
A MAC Table A A
A
Availability and Scaling vs. Restricted Workload Flexibility
 Layer 3 Routed Topologies alleviate the

consumption of L2 tables via route
summarization
 Layer 3 Routed topologies provide for a
degree of fault isolation and
 “Routed Access” provides the logical L3
extension of the design philosophy L2
 “Scaling Up” of the Access Switch via
such mechanism as the FEX provide a
degree of workload mobility
 “L2” domain extension of some form is
required for most workload mobility
requirements Workload Domain for most Hypervisor and Clustering
based solutions is restricted by the Traditional
Layer 2/3 boundary
Segment-ID: Scaling Logical Groupings of Connectivity
S1 Web S2 App S3 Database

Server Server Server
S4
802.1Q
VLAN ID 802.1Q
VLAN ID 802.1ad
12-bits 12-bits standardized
frame format
SegmentId
VLAN ID VLAN ID
12-bits 24-bits 12-bits
Location Identity Separation
• Location reachability determined by
traditional routing mechanisms in the
Fabric
L2/L3 Fabric • Identity is mapped to location
Location addresses
Identity • All these technologies leverage

Location/Identity Mapping
FabricPath / VXLAN OTV LISP

TRILL
Location Switch-ID IP address IP address IP address
(IS-IS) (IP protocols) (IP protocols) (IP protocols)
Identity Client MAC Client MAC Client MAC Client IP/MAC
(Flooding) (Flooding) (IS-IS) (Mapping DB)
Multi-tenancy
© 2010 Cisco and/or its affiliates. All rights reserved. 24-bit Segment Identifier Cisco Confidential 10
FabricPath, LISP, VXLAN & OTV
Requirement Intra-DC Inter-DC

Scale
Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS
IP Mobility LISP LISP
Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS)
LISP
IP mobility
IP Network
DC-west DC-east
POD POD POD POD
App App App App App App

OTV/VPLS
OS OS OS (Inter-DC x-L3) OS OS OS
Fabric Path VXLAN/OTV Fabric Path VXLAN/OTV

(Intra-DC L2) (Intra-DC x-L3)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential 11
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
Cisco FabricPath
NX-OS Innovation Enhancing L2 with L3
Switching Routing
 Easy Configuration  Multi-pathing (ECMP)
 Plug & Play  Fast Convergence
 Provisioning Flexibility  Highly Scalable
FabricPath
“FabricPath brings Layer 3 routing benefits to

flexible Layer 2 bridged Ethernet networks”
MAC-in-MAC Optimal MAC Learning IS-IS
• Creates hierarchical layer 2 • Prevent potential MAC table • Scalable routing protocol with
address scheme with additional MAC overflow in large scale L2 domain proven implementation for fast
header • Traditional source-learning only on convergence upon network changes
• Source and destination Switch_ID Edge port for locally connected MAC • Link-state protocol ensures optimal
written into outer MAC header at addresses path between any 2 nodes
L2MP edge • Learning is disabled on Core port to • Built-in authentication mechanism
• Forwarding inside L2MP core reduce MAC table utilization enhances network security and
network is based on destination • Non-local source-MAC only learned stability
Switch_ID if destination-MAC is already learned • Inherent support for ECMP and
• Embedded path selector (FTAG) as local entry multi-topology maximize link
provides multi-pathing for even utilization
broadcast and multicast
• Built-in protections (TTL and
multicast RPF) minimize impact of
transient network issues
New Control Plane
Plug-n-Play L2 IS-IS manages forwarding topology
• IS-IS assigns addresses to all FabricPath switches automatically
• Compute shortest, pair-wise paths
• Support equal-cost paths between any FabricPath switch pairs
S10 S20 S30 S40

FabricPath
Routing Table
Switch IF
S10 L1
S20
S30
L2
L3
FabricPath
S40 L4 L1 L2 L3
S200 L1, L2, L3, L4 L4
… …
S400 L1, L2, L3, L4
S100 S200 S300 S400
New Data Plane
• The association MAC address/Switch ID is maintained at the edge
S10 S20 S30 S40
Switch ID space: S300: FabricPath

Routing decisions A  B S100  S300 Routing Table
are made based on Switch IF
the FabricPath … …
routing table S100 FabricPath S200
(FP) S300 S100 L1, L2, L3, L4
MAC adress space: 1/1 1/2 S300: CE MAC

Switching based on Classical Ethernet (CE) Address Table
MAC IF
MAC address tables A B B 1/2
…
A …
S100
• Core fabric leverages an independent routing topology from the edge

• Scales MAC learning
• Scales Core topology state
New Control and Data Plane
• Edge switches maintain both MAC address table and Switch ID table
• Ingress switch uses MAC table to determine destination Switch ID
• Egress switch uses MAC table (optionally) to determine output switchport
S10 S20 S30 S40
FabricPath
MAC Table on S100
MAC IF/SID
Local MACs point
to switchports
A
B
e1/1
e1/2
S100 S101 FabricPath S200
Remote MACs point C S101
to Switch IDs D S200

MAC A MAC B MAC C MAC D
New Control and Data Plane
• FabricPath IS-IS manages Switch ID (routing) table
• All FabricPath-enabled switches automatically assigned Switch ID (no user configuration

required)
• Algorithm computes shortest (best) paths to each Switch ID based on link metrics
• Equal-cost paths supported between FabricPath switches

S10 S20 S30 S40
FabricPath
Routing Table on S100
Switch IF
One „best‟ path S10 L1
to S10 (via L1) S20 L2
S30 L3 L1 L2 L3 L4
S40 L4
Four equal-cost S101 L1, L2, L3, L4
paths to S101 … … FabricPath
S200 L1, L2, L3, L4
S100 S101 S200
Scaling – Conversational Learning
MAC IF MAC IF
A e1/1 A s1,e1/1
… … … …
B s8, e1/2
FabricPath B e1/2
s3 s5 s8
e1/1 e1/2
A B
MAC IF
… …
• Edge switch only learn the MAC of remote hosts when there are two way communications
between remote hosts and local hosts
• Unknown unicast flooding alone won‟t have all switches within VLAN learn the source MAC
• Intermediate switches don‟t learn the MAC
• Hardware based MAC learning
Cisco FabricPath
Terminology
 Interface connected to another FabricPath device
 Sends/receives traffic with FabricPath header
 Does not run spanning tree
 Does not perform MAC learning!
 Exchanges topology info through L2 ISIS adjacency
FP Core Ports  Forwarding based on „Switch ID Table‟
S10 S20 S30 S40
Spine Switch
FabricPath (FP)
S100 S200 S300
Leaf Switch
1/1 1/2
Classical Ethernet (CE)
A B
CE Edge Ports  Interface connected to traditional network device

 Sends/receives traffic in standard 802.3 Ethernet frame format
 Participates in STP domain
 Forwarding based on MAC table
Configuration Simplicity
Automatically handled
by IS-IS
FabricPath
V10 V20 V30 V30 V10 V20 V10 V30
V10 V20 V30

• Multidestination traffic constrained to
Root for Root for loop-free trees touching all FabricPath
Tree 1 Tree 2 switches
S10 S20 S30 S40
• Root switch assigned for each
multidestination tree in FabricPath
domain
• Loop-free tree built from each Root
and assigned a network-wide
identifier (Ftag)
FabricPath • Support for multiple multidestination

S100 S101 S200 trees provides multipathing for multi-
destination traffic
Two trees supported in NX-OS release
5.1
S100 S20 S100 S10
S10 S101 S30 S40 S101 S20
Root S200 S40 Root S200 S30

Logical Logical
Tree 1
Tree 2 Cisco Confidential 22
Multi-Topology Support
FabricPath
Topology „0‟ VLAN 20 (DC Wide)
Common across entire
Data Center
FabricPath
Topologies FabricPath FabricPath
Topology Topology
„1‟ „2‟
VLAN 20 – DC Wide VLAN 20 – DC Wide

VLAN 30 – POD Local (and non-unique) VLAN 30 – POD Local (and non-unique)
VLAN 10 – POD Local (and unique) VLAN 40 – POD Local (and unique)
• Extending FabricPath to the edge switches without requiring a redesign of the VLAN topology
• Each FP switch can have up to 2 Topology ID‟s defined (Topology ID‟s does not have to be unique).
• Each Topology will have 2 Multi-Destination Trees defined
Mac-in-Mac Header
Classical Ethernet Frame DMAC SMAC 802.1Q Etype Payload CRC
16 bytes Original CE Frame
Outer Outer FP
Cisco FabricPath DA SA Tag DMAC SMAC 802.1Q Etype Payload
CRC
(new)
Frame (48) (48) (32)
6 bits 1 1 2 bits 1 1 12 bits 8 bits 16 bits 16 bits 10 bits 6 bits

OOO/DL
RSVD
Endnode ID Endnode ID Sub Etype

U/L
I/G
Switch ID LID Ftag TTL

(5:0) (7:6) Switch ID 0x8903
• Switch ID – Unique number identifying each FabricPath switch

• Sub-Switch ID – Identifies devices/hosts connected via VPC+
• LID – Local ID, identifies the destination or source interface
• Ftag (Forwarding tag) – Unique number identifying topology and/or distribution tree
• TTL – Decremented at each switch hop to prevent frames looping infinitely
Putting it all together – Host A to Host B
(1) Broadcast ARP Request
Root for Root for
Multidestination Tree 1 Tree 2
Trees on Switch 10 S10 S20 S30 S40
4 Tree IF
DA→FF
Ftag → 1 po100,po200,po300
Ftag→1
po300
2 po100
SA→100.0.12
DA→FF
DMAC→FF
po100 po200
Ftag→1
SMAC→A SA→100.0.12
Multidestination Payload DMAC→FF
Trees on Switch 100 po20 po30 po40

SMAC→A
po10 po20 po30
3 Tree IF po40 po10 Payload
Broadcast → 1 po10 S100 S200 Multidestination S300

2 po10,po20,po30,po40 Trees on Switch 300
5 Tree IF 6
FabricPath
MAC Table on S100 DMAC→FF
e1/13
Ftag → 1 po10,po20,po30,po40 e2/29 Payload
SMAC→A
2 po40
MAC IF/SID SMAC→A
DMAC→FF
A e1/13 (local) 2 Payload
FabricPath
MAC A MAC B
1 MAC Table on S200
© 2010 Cisco and/or its affiliates. All rights reserved. MAC IF/SID Cisco Confidential 25
(1) Broadcast ARP Request
• S100:
S100# sh mac address-table dynamic
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID MAC A learned as
---------+-----------------+--------+---------+------+----+------------------ local entry on e1/13
* 10 0000.0000.000a dynamic 0 F F Eth1/13
S100#
• S10 (and S20, S30, S40, S200, S300):

MAC A not learned
Legend:
on other switches
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
(2) Broadcast ARP Reply
Root for Root for
Multidestination Tree 1 Tree 2
S10 S20 S30 S40
Trees on Switch 10
10 Tree IF
Ftag → 1 po100,po200,po300
po300
2 po100 DA→MC1
DA→MC1 Ftag→1
Ftag→1 po100 po200
SA→300.0.64
SA→300.0.64
DMAC→A
DMAC→A
Multidestination
SMAC→B
Trees on Switch 100 SMAC→B
po20 po30 po40
po10 po20 po30
Payload
Payload
11 Tree IF po40 po10
Ftag → 1 po10 S200 Multidestination S300

2 po10,po20,po30,po40 Trees on Switch 300
9 Tree IF 7
FabricPath
MAC Table on S100 Payload
e1/13
Unknown → 1 po10,po20,po30,po40 e2/29
DMAC→A
2 po40 SMAC→B
MAC IF/SID SMAC→B
Payload
A e1/13 (local) 12 DMAC→A
MAC A
FabricPath MAC Table on S300 MAC B
B 300.0.64 (remote)
MAC IF/SID
8
MISS
© 2010 Cisco and/or its affiliates. All rights reserved. B e2/29 (local) Cisco Confidential 27
MAC Address Table after the first ARP frame
• S100:
Legend:
---------+-----------------+--------+---------+------+----+------------------ S100 learns MAC B as
* 10 0000.0000.000a dynamic 90 F F Eth1/13 remote entry reached
10 0000.0000.000b dynamic 60 F F 300.0.64 through S300
S100#
• S300:
Legend:
age - seconds since last seen,+ - primary entry using vPC Peer-Link MAC B learned as
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID local entry on e2/29
---------+-----------------+--------+---------+------+----+------------------
• 10 0000.0000.000b dynamic 0 F F Eth2/29
FabricPath Routing
Table on S30
S10 S20 S30 S40
Switch IF
… …
S300 → S300 po300 16

po300
DA→300.0.64 DA→300.0.64
FabricPath Routing Ftag→1 Ftag→1
Table on S100 SA→100.0.12
SA→100.0.12
Switch IF
DMAC→B DMAC→B
S10 po10
SMAC→A SMAC→A
po10 po20 po30 po20 po30 po40
S20 po20
Payload Payload
Hash po40 po10
S30 po30
S40 po40 S200 FabricPath Routing S300

S100
po10, po20, Table on S300
S200
po30, po40 17
15 Switch IF
po10, po20, … …
S300 →
e1/13 e2/29
S300 Payload
po30, po40
S300 → S300 Use LID (64)
SMAC→A
DMAC→B
FabricPath
DMAC→B
MAC Table on S100 SMAC→A FabricPath
MAC A MAC B
MAC IF/SID Payload MAC Table on S300
A e1/13 (local) 14 13 MAC IF/SID
18
A S100.0.12 (remote)
B→ B 300.0.64 (remote) If DMAC is known, then
B e2/29 (local)
© 2010 Cisco and/or its affiliates. All rights reserved. learn remote MAC Cisco Confidential 29
Unicast forwarding
Legend:
---------+-----------------+--------+---------+------+----+------------------
* 10 0000.0000.000a dynamic 90 F F Eth1/13
10 0000.0000.000b dynamic 60 F F 300.0.64

Legend:
---------+-----------------+--------+---------+------+----+------------------ S100 learns MAC A as
remote entry reached
10 0000.0000.000a dynamic 30 F F 100.0.12
through S100
• 10 0000.0000.000b dynamic 90 F F Eth2/29
Unicast Forwarding
S100# sh fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
Topology (ftag), Switch
ID, Sub-Switch ID
FabricPath Unicast Route Table for Topology-Default Administrative distance,
routing metric
0/100/0, number of next-hops: 0
via ---- , [60/0], 0 day/s 04:43:51, local
1/10/0, number of next-hops: 1 Route age
via Po10, [115/20], 0 day/s 02:24:02, isis_fabricpath-default
via Po20, [115/20], 0 day/s 04:43:25, isis_fabricpath-default Client protocol
via Po30, [115/20], 0 day/s 04:43:25, isis_fabricpath-default Next-hop interface(s)
via Po40, [115/20], 0 day/s 04:43:25, isis_fabricpath-default FabricPath
via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-default S10 S20 S30 S40

po10
1/300/0, number of next-hops: 4 po20
via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-default po30
po40
S100 S200 S300
A B C
FabricPath Design
STP Interaction
FabricPath
(no STP)
FabricPath
Classical
Ethernet STP
(STP) Domain
STP
Domain 1 BPDU ✖ STP
BPDU Domain 2
CE Edge Ports
 FabricPath domain appears as single Spanning-Tree bridge
 All FabricPath bridges share a common (static) bridge ID
Cisco reserved MAC c84c.75fa.6000
 STP BPDUs are not carried through the FabricPath network
 Configure all FabricPath edge switches using “spanning-tree vlan <x> root primary” (or manually configure bridge
priority lower than any STP bridge)
Each FabricPath edge switch must be the root for all connected STP domains
Strongly recommended to use the same bridge priority on all FabricPath edge switches 32
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
FabricPath
L2/L3 Boundary Location
Layer 3 Boundary at the Spine Layer 3 Integration at the Leaf/Edge
 Straightforward with two spine switches  Provides a “cleaner” spine design
 Considerations with more than two spines:  Traffic distributed equally across spines (no hot
 HSRP: Traffic polarized to spines on a per VLAN basis spot)
(South-North)
 GLBP to distribute servers to different default gateways  Increased number of hops to reach gateway
(latency)
 Anycast FHRP future solution
L3
FabricPath
FabricPath
L3 L3
FabricPath L2/L3 Boundary Location
Classic Two Switch Spine
• Simplest migration from most existing
designs
L3 Domain
• The spine is also used for routing with
+ MAC learning for

M1/F1 in the same VDC L3
Switch-id based
• Consideration – MAC Learning and Scaling edge/spine
routed traffic
forwarding
s M1+F1 M1+F1
• Compared to classic ethernet designs you
gain:
Ease of configuration
MAC address table increased scalability and more
efficient learning
Traffic distribution on all uplinks edge
Possibility to offload the spine by providing direct
communication paths between the edge layer devices
[…]
Conversational Learning Conversational Learning
FabricPath L2/L3 Boundary Location
Leaf/Spine/Boundary Architecture
L3 Domain
• By separating the L3 function
from the spine, the F1 card in L3 edge
the spine performs pure switch-
id forwarding M1/F1 M1/F1
FP port FP port
• The L3 edge will need both
M1/F1 in order to connect with
Switch-id based
Fabricpath ports to the spine
spine spine
forwarding
• The M1/F1 L3 edge will need to
perform learning for the remote
mac addresses
• L3 edge and spine can be

combined in the same chassis
by means of VDCs edge
Conversational Conversational Learning
© 2010 Cisco and/or its affiliates. All rights reserved. Learning Cisco Confidential 35
Nexus Edge, Core & Boundary Nodes
Large Scale Fabric 4K VLAN’s, 128K MAC Address,

512K Routes
blade1 blade1 blade1 blade1 blade1 blade1

slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 blade1 blade1 blade1 blade1
slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 blade1 blade1 slot 1 slot 1 slot 1
blade2 slot 1
blade2
slot 3 slot 3 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 1
blade2 slot 1
blade2 blade2
slot 2 blade2
slot 2 slot 2
blade3 slot 2
blade3
blade4 blade4 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 2
blade3 slot 2
blade3 blade3 blade3 slot 3
blade4 slot 3
blade4
slot 4
blade5 slot 4
blade5 slot 5 slot 5 slot 5
blade6 slot 5
blade6 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 4
blade5 slot 4
blade5
slot 5
blade6 slot 5
blade6 blade6
slot 6 blade6
slot 6 slot 6
blade7 slot 6
blade7 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 5
blade6 slot 5
blade6
slot 6
blade7 slot 6
blade8 slot 7
blade8 slot 5
blade6 slot 5
blade6 slot 5
blade6 slot 5
blade6 slot 6
blade7 slot 6
blade7
slot 7
blade8 slot 7
blade8 slot 7
blade8 slot 7
blade7 slot 6
blade7 slot 6
blade7 slot 6
blade7 slot 7 slot 7
slot 8 slot 8 slot 8 slot 8 slot 7
blade8 slot 7
blade8 slot 7
blade8 slot 7
blade8 blade8
slot 8 blade8
slot 8
slot 8 slot 8 slot 8 slot 8
slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade2 slot 1
blade2
blade4 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 1
blade2 slot 1
blade2 blade2
slot 2 blade2
slot 2 slot 2
blade3 slot 2
blade3
blade5 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 2
blade3 slot 2
blade4 slot 3
blade4
slot 4
blade5 slot 4
blade6 slot 5
blade6 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 4
blade5 slot 4
blade5
slot 5
blade6 slot 5
blade6 blade6
slot 6 blade6
slot 6 slot 6
blade7 slot 6
blade7 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 5
blade6 slot 5
blade6
slot 6
blade7 slot 6
blade8 slot 7
blade8 slot 5
blade6 slot 5
blade6 slot 5
blade6 slot 5
blade6 slot 6
blade7 slot 6
blade7
slot 7
blade8 slot 7
blade8 slot 7
blade8 slot 7
blade7 slot 6
blade7 slot 6
blade7 slot 6
blade8 slot 7
blade8 slot 7
blade8 slot 7
blade8 blade8
slot 8 blade8
slot 8
slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
blade2 slot 1
slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade3 slot 2
blade2 slot 1
blade2
blade4 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 1
blade2 slot 1
blade2 blade2
slot 2 blade2
slot 2 slot 2
blade3 slot 2
blade3
blade5 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 2
blade3 slot 2
blade4 slot 3
blade4
slot 4
blade5 slot 4
blade6 slot 5
blade6 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 3
blade4 slot 4
blade5 slot 4
blade5
slot 5
blade6 slot 5
blade6 blade6
slot 6 blade6
slot 6 slot 6
blade7 slot 6
blade7 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 4
blade5 slot 5
blade6 slot 5
blade6
slot 6
blade7 slot 6
blade8 slot 7
blade8 slot 5
blade6 slot 5
blade6 slot 5
blade6 slot 5
blade6 slot 6
blade7 slot 6
blade7
slot 7
blade8 slot 7
blade8 slot 7
blade8 slot 7
blade7 slot 6
blade7 slot 6
blade7 slot 6
blade8 slot 7
blade8 slot 7
blade8 slot 7
blade8 blade8
slot 8 blade8
slot 8
Standards Based + Cisco Extensions
• Nexus 5500, F1, F2
and all future HW are
3 Cisco Forwarding 0
3 TRILL Forwarding 0
1 1
capable of IETF Outer CDCE DA Outer MAC DA
standards TRILL Outer CDCE DA Outer CDCE SA Outer MAC DA Outer MAC SA NextHop
Outer CDCE SA Outer MAC SA
Header
• Support for TRILL in
ET = DTAG FTAG TTL ET = 802.1Q Outer VLAN
NX-OS is pending
completion of Inner MAC DA ET = TRILL V/R/M, HopCnt TRILL
extensions to the Inner MAC DA Inner MAC SA Egress RB Ingress RB Header
baseline protocol Inner MAC SA Inner MAC DA
ET = 802.1Q Inner VLAN Inner MAC DA Inner MAC SA Ethernet

• Multi-topology, VRRP
Header
interaction, … Inner MAC SA
Payload… ET = 802.1Q Inner VLAN
Payload...
Flexibility in the Fabric - Layer 2 Routing
L3 Core L2+L3
FabricPath
Core
FabricPath
POD vPC POD
vPC+ POD vPC+ POD
Path
Fabric
Site 1
FabricPath FabricPath FabricPath FabricPath

Site 4 Site 2
Path
Fabric
Site 3
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
• Customer Requirement
Tenant Network
Secure movement of vApps across cloud infrastructure (VLAN)
• Solution: VXLAN vApp1

Web vApp2 Web
VM VM
Millions of dedicated LAN segments
Security at Scale App App
VM VM
vApp mobility across data centers & clouds
DB DB
VM VM
• VXLAN is network friendly
Efficient load sharing of links (port channel)
Supports NAT; better security controls
VXLAN IETF Draft: http://datatracker.ietf.org/doc/draft-mahalingam-dutt-dcops-vxlan/

 Ethernet in IP overlay network  Tunnel between VEMs
 Entire L2 frame encapsulated in UDP  VMs do NOT see VXLAN ID
 50 bytes of overhead
 IP multicast used for L2 broadcast/multicast,
 Include 24 bit VXLAN Identifier unknown unicast
 16 Million logical networks  Technology submitted to IETF for
standardization (Cisco, VMware, Citrix, Red
 VXLAN can cross Layer 3 (IPv4 currently) Hat, Broadcom, Arista, and Others)
VXLAN Encapsulation Original Ethernet Frame

Outer Outer VXLAN Inner InnerM Optional Original
Outer Outer Outer Outer
MAC MAC Header (8 MAC AC Inner Ethernet CRC
802.1Q IP DA IP SA UDP
DA SA bytes) DA SA 802.1Q Payload
VXLAN
Flags8 Networker Reserved Res.
bits Identifier (VIN) 24 bits 8 bits
24 bits
• The Nexus 1000V VEMs act as the VXLAN Tunnel Endpoints (VTEP)
• Nexus 1000V uses a VMKNIC to terminate VTEP traffic
• VM to VM traffic on different access switches is encapsulated in a VXLAN header + UDP

+ IP
• VTEPs use multicast to deliver unknown destination VM MAC addresses to all VTEPs
participating in a given VXLANs
• VM MAC to VTEP IP address mappings are gleaned from encapsulated packets

Similar to Ethernet bridge flood and learn behavior
• Known destination VM MAC addresses are carried over point to point tunnels between
VTEPs
Access Access
Switch Switch
End End
Bridge Bridge System
System
Domain Domain
Switch VTEP IP Multicast VTEP Switch
Enabled Underlying
Network
End End
System System
Direct Unicast tunnels between VTEPs VTEP = VXLAN Tunnel End Point
(Carries known unicast frames) VNI = VXLAN Network Identifier
VTEP VTEP
VXLAN‟s IP Any Source Multicast Group (*,G)
acts as a bus for delivery to all relevant VTEPs
for a given VNI
(Carries unknown/broadcast/multicast frames)
VTEP VTEP
43
VTEP Use Of IGMP
IGMP Used to Join Each VXLANs Assigned Multicast Group on Demand
Web DB DB Web
VM VM VM VM
Join Multicast Join Multicast

Group 239.1.1.1 Group 239.2.2.2
Join Multicast
Join Multicast
Group 239.2.2.2
Group 239.1.1.1
VXLAN Example Data Flow
VM1 Communicating with VM2 in a VXLAN
MAC: VM 3
VM 1 abc
VM 2 MAC:
xyz
VXLAN VXLAN VXLAN

VMKNIC VMKNIC VMKNIC
1.1.1.1 2.2.2.2 3.3.3.3
VEM 1 VEM 2 VEM 3
Multicast Multicast
Multicast
MAC: VM 3
VM 1 abc
VM 2 MAC:
xyz
VXLAN VXLAN VXLAN

1.1.1.1 2.2.2.2 3.3.3.3
Unicast
MAC Table: VEM 2

VM Source MAC Remote Host
VXLAN IP
Layer 3 VM1:abc 1.1.1.1
MAC: VM 3
VM 1 abc
VM 2 MAC:
xyz
VXLAN VXLAN VXLAN

1.1.1.1 2.2.2.2 3.3.3.3
VEM 1 VEM 2 VEM 3
MAC Table: VEM 1 MAC Table: VEM 2

VM Source MAC Remote Host VM Source MAC Remote Host
VXLAN IP VXLAN IP
VM2:xyz 2.2.2.2 VM1:abc 1.1.1.1
MAC: VM 3
VM 1 abc
VM 2 MAC:
xyz
VXLAN VXLAN VXLAN

1.1.1.1 2.2.2.2 3.3.3.3
Unicast
MAC Table: VEM 1 MAC Table: VEM 2

VM Source MAC Remote Host VM Source MAC Remote Host
VXLAN IP VXLAN IP
VM2:xyz 2.2.2.2 VM1:abc 1.1.1.1
Multiple VXLANs Can Share One Multicast Group
Blue & Red VXLANs Share The 239.1.1.1 Multicast Group
Web App DB App

VM VM VM VM
• Encapsulate with Blue VXLAN ID VEM Discards Since No VM with

• Multicast to Servers Registered for Blue VXLAN ID
239.1.1.1 Multicast Group
VM Broadcast Frames Sent to More Servers

But Broadcast Domain Respected Within VXLAN Segment
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
Single Network Architecture Delivers:
 VM Mobility (topology independent addressing)
 Security: VPNs/Multi-tenancy
 Route Scalability (on demand routing)
 IPv6 enablement,
 Routing Policy simplification
Benefits Use-Cases
 Services integrated in a single architecture  DCI route optimization/mobility
 Services can be offered across organizational  Workload Portability to Cloud

boundaries (multiple providers)  Secure Multi-tenancy across organizations
 Very large scale  Rapid IPv6 Deployment
 Open model to integrate with cloud orchestrators  Route scaling
LISP Use Cases
Consolidated Architecture with Multiple Applications
Efficient Multi-Homing IPv6 Transition Support
v6
LISP
v6 Services LISP Router
Internet Router
IPv4 IPv6
Internet Internet
LISP LISP
Site v6 v4 v6
Routers
 IP Portability  v6-over-v4, v6-over-v6

 Ingress Traffic Engineering without BGP  v4-over-v6, v4-over-v4
Multi-Tenancy and VPNs Host-Mobility

LISP Site LISP Site
IP Network IP Network
West-DC East-DC West-DC East-DC
 Reduced CapEx/OpEx  Cloud / Layer 3 VM moves

 Large scale Segmentation  Segmentation
Location Identity Separation Protocol
What Do We Mean by “Location” and “Identity”?
Today‟s IP Behavior
Loc/ID “Overloaded” Semantic
IP core
10.1.0.1 When the Device Moves, It Gets a
New IPv4 or IPv6 Address for Its
Device IPv4 or IPv6
New Identity and Location
Address Represents 20.2.0.9
Identity and Location
LISP Behavior
Loc/ID “Split”
IP core
10.1.0.1 When the Device Moves, Keeps
Device IPv4 or IPv6 1.1.1.1 Its IPv4 or IPv6 Address.
Address Represents 2.2.2.2 It Has the Same Identity
10.1.0.1
Identity Only.
Its Location Is Here!
Only the Location Changes
A LISP Packet Walk
How Does LISP Operate?
3
EID-prefix: 10.2.0.0/24
Mapping Locator-set:
Entry Non-LISP site
1 2.1.1.1, site
Non-LISP priority: 1, weight: 50 (D1) This Policy Controlled
DNS Entry: by Destination Site
2.1.2.1, priority: 1, weight: 50 (D2)
D.abc.com A 10.2.0.1
10.1.0.0/24
LISP Site
S ITR PITR
2 1.1.1.1 5.4.4.4
10.1.0.1 -> 10.2.0.1 IP Network 5.3.3.3
EID-to-RLOC
4 mapping
5.1.1.1 5.2.2.2
1.1.1.1 -> 2.1.1.1
10.1.0.1 -> 10.2.0.1 2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1
ETR
5
10.1.0.1 -> 10.2.0.1
West-DC East-DC
D 10.2.0.0/24 10.3.0.0/24
A LISP Packet Walk
How About Non-LISP Sites? 3
EID-Prefix: 10.2.0.0/24
Mapping Locator-Set:
1 Entry 2.1.1.1, priority: 1, weight: 50 (D1)
DNS Entry:
D.abc.com A 10.2.0.1 2.1.2.1, priority: 1, weight: 50 (D2)
Non-LISP Site
Non-LISP Site
S
2
192.3.0.1 -> 10.2.0.1 PITR
4.4.4.4
4 5.3.3.3
4.4.4.4- > 2.1.2.1 EID-to-RLOC
192.3.0.1 -> 10.2.0.1 mapping
5.1.1.1 5.2.2.2
IP Network
2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1
ETR
5
192.3.0.1 -> 10.2.0.1
West-DC East-DC
D 10.2.0.0/24 10.3.0.0/24
LISP Roles and Address Spaces
Mapping EID
What Are the Different Components Involved? DB a.a.a.0/24
b.b.b.0/24
RLOC
w.x.y.1
x.y.w.2
c.c.c.0/24 z.q.r.5
d.d.0.0/16 z.q.r.5
EID RLOC
a.a.a.0/24 w.x.y.1
b.b.b.0/24 x.y.w.2
LISP Roles EID Space EID

c.c.c.0/24
d.d.0.0/16
RLOC
z.q.r.5
z.q.r.5
a.a.a.0/24 w.x.y.1
• Tunnel Routers - xTRs b.b.b.0/24

c.c.c.0/24
d.d.0.0/16
x.y.w.2
z.q.r.5
z.q.r.5
• Edge devices in charge of

ITR
encap/decap Non-LISP ALT
Prefix Next-hop
•
w.x.y.1 e.f.g.h
Ingress/Egress Tunnel Routers x.y.w.2
z.q.r.5
e.f.g.h
e.f.g.h
z.q.r.5 e.f.g.h
(ITR/ETR)
• EID to RLOC Mapping DB PxTR RLOC Space
• Contains RLOC to EID ETR
mappings
• Distributed across multiple Map EID Space
Servers (MS)
• MS may connect over an ALT
network Address Spaces
• Proxy Tunnel Routers - PxTR • EID = End-point Identifier
• Coexistence between LISP and • Host IP or prefix
non-LISP sites
• RLOC = Routing Locator
• Ingress/Egress: PITR, PETR
• IP address of routers in the backbone
LISP Mapping Database
The Basics – Registration and Resolution
LISP Site
Mapping Cache Entry (on ITR):
10.2.0.0/16-> (2.1.1.1, 2.1.2.1)
ITR
Map Server / Resolver: 5.1.1.1
Map-Reply
10.2.0.0/16 -> (2.1.1.1, 2.1.2.1)
2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1

ETR ETR ETR ETR
Database Mapping Entry (on ETR): Database Mapping Entry (on ETR):
10.2.0.0/16 -> (2.1.1.1, 2.1.2.1) 10.3.0.0/16 -> (3.1.1.1, 3.1.2.1)
West-DC East-DC
10.2.0.0 /16 10.3.0.0/16
Y
X Y Z
© 2010 Cisco and/or its affiliates. All rights reserved. 10.2.0.2 Cisco Confidential 58
Basic LISP Configuration Servers
ip lisp map-resolver
ip lisp map-server
lisp site west-DC
authentication-key 0 s3cr3t
eid-prefix 10.2.0.0/24
Border Routers Between Backbones

ip lisp proxy-itr
ip lisp ITR map-resolver 5.3.3.3
Non-LISP Sites
LISP Site PITR

ITR
Branch Routers 5.3.3.3
ip lisp itr-etr 1.1.1.1
Mapping DB
ip lisp ITR map-resolver 5.3.3.3
5.1.1.1
DC Aggregation Routers 5.2.2.2
IP Network
ip lisp itr-etr
ip lisp database-mapping 10.2.0.0/24 2.1.1.1 p1 w50
ip lisp database-mapping 10.2.0.0/24 2.1.2.1 p1 w50 2.1.1.1 2.1.2.1
ip lisp ETR map-server 5.1.1.1 key s3cr3t
ip lisp ETR map-server 5.2.2.2 key s3cr3t ETR
West-DC East-DC
10.2.0.0/24
Usually Devices Will Be Configured as ITRs and ETRs
to Handle Traffic in Both Directions;
We Illustrate Only One Direction for Simplicity
© 2010 Cisco and/or its affiliates. All rights reserved. RLOC EID LISP Encap/Decap
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
LISP Host-Mobility
Needs:
• Global IP-Mobility across subnets
Non-LISP Sites
• Optimized routing across extended subnet sites
LISP Site PxTR
LISP Solution:
xTR
• Automated move detection on xTRs Mapping DB
• Dynamically update EID-to-RLOC mappings IP Network
• Traffic Redirection on ITRs or PITRs
LAN Extensions
Benefits:
• Direct Path (no triangulation) LISP-VM (xTR)
• Connections maintained across move West-DC East-DC
• No routing re-convergence
• No DNS updates required
• Transparent to the hosts RLOC EID LISP Encap/Decap
• Global Scalability (cloud bursting)
• IPv4/IPv6 Support
Host-Mobility Scenarios
Moves Without LAN Extension Moves With LAN Extension
LISP Site Non-LISP LISP Site
xTR Site
xTR
DR Location or
Mapping DB Mapping DB
Cloud Provider IP Network
Internet or DC
Shared WAN LAN Extension
LISP-VM (xTR) LISP-VM (xTR)

West-DC East-DC West-DC East-DC
IP Mobility Across Subnets Routing for Extended Subnets
Disaster Recovery Active-Active Data Centers
Cloud Bursting Distributed Clusters

Application Members in One Location Application Members Distributed
(Broadcasts across sites)
LISP Host-Mobility - Move Detection
Monitor the Source of Received Traffic
• The new xTR checks the source of received traffic
• Configured dynamic-EIDs define which prefixes may roam

lisp dynamic-eid roamer Received a Packet …
database-mapping 10.2.0.0/24 <RLOC-C> p1 w50
database-mapping 10.2.0.0/24 <RLOC-D> p1 w50 … It’s from a “New” Host
map-server 5.1.1.1 key abcd … It’s in the Dynamic-EID Allowed
interface vlan 100
Range
lisp mobility roamer
Mapping DB
5.1.1.1 5.2.2.2 …It’s a Move!
A B C D Register the /32 with LISP
LISP-VM (xTR)
West-DC East-DC
10.2.0.0 /16 10.3.0.0/16
Y
X Y Z
LISP Host-Mobility - Traffic Redirection
Update Location Mappings for the Host System Wide
• When a host move is detected, updates are triggered:
The host-to-location mapping in the Database is updated to reflect the new location
The old ETR is notified of the move
ITRs are notified to update their Map-caches
• Ingress routers (ITRs or PITRs) now send traffic to the new location
10.2.0.0/16 – RLOC A, B
LISP Site
xTR
Mapping DB
10.2.0.2/32 – RLOC C, D
A B C D
LISP-VM (xTR)
West-DC East-DC
10.2.0.0 /16 10.3.0.0 /16
Y
X Y Z
LISP Host-Mobility - First Hop Routing
Across Different Subnets
• SVI (Interface VLAN x) and HSRP configured as usual (Consistent GWY-MAC configured across all dynamic subnets)
• The lisp mobility <dyn-eid-map> command enables proxy-arp functionality on the SVI
The LISP-VM router services first hop routing requests for both local and roaming subnets
• Hosts can move anywhere and always talk to a local gateway with the same MAC
interface vlan 100
interface vlan 100 ip address 10.3.0.7/24
interface vlan 200
ip address 10.2.0.5/24 lisp mobility roamer
ip address 10.2.0.8/24
interface Ethernet2/4 ip proxy-arp
ip proxy-arp
ip address 10.1.0.6/24 ip proxy-arp hsrp 201
hsrp 101
lisp mobility roamer mac-address 0000.0e1d.010c
hsrp 201
ip proxy-arp mac-address 0000.0e1d.010c ip 10.3.0.1
mac-address 0000.0e1d.010c
hsrp 101 ip 10.2.0.1
ip 10.3..0.1
ip 10.2.0.1
A B C D
LISP-VM (xTR)
HSRP Active HSRP Active
West-DC East-DC
10.2.0.0 /24 10.3.0.0 /24
HSRP HSRP
ARP ARP
GWY-MAC GWY-MAC
Null0 host routes indicate the host is “away”
10.2.0.0/16 – RLOC A, B
6 10.2.0.2/32 – RLOC C, D
Map-Register
10.2.0.2/32 <C,D>
Map-Notify Mapping DB
10.2.0.2/32 <C,D> 5.1.1.1 5.2.2.2
Routing Table: Routing Table:

7 5
10.2.0.0/16 – Local 10.3.0.0/16 – Local
10.2.0.2/32 – Null0 4 10.2.0.2/32 – Local
10 A B Routing Table: C D
10.3.0.0/16 – Local
2 10.2.0.2/32 – Local
Routing Table: 3
9 10.2.0.0/16 – Local
10.2.0.0 /16 10.3.0.0 /16
8 10.2.0.2/32 – Null0 1 East-DC
West-DC
Y
Map-Notify X Map-Notify
Y
10.2.0.2/32 <C,D> 10.2.0.2 10.2.0.2/32 <C,D>
Map Cache @ ITR
10.2.0.0/16 – RLOC A,B

1. ITRs and PITRs with cached mappings
continue to send traffic to the old locators LISP site
The old xTR knows the host has moved (Null0 route). ITR
10.2.0.2/32 – RLOC C,D
2. Old xTR sends Solicit Map Request (SMR)
messages to any encapsulators sending
Mapping DB
traffic to the moved host
3. The ITR then initiates a new map request

process
4. An updated map-reply is issued from the A B C D

new location LISP-VM (xTR)
5. The ITR Map Cache is updated West-DC East-DC

10.2.0.0 /16 10.3.0.0 /16
• Traffic is now re-directed Y

X Y Z
• SMRs are an important integrity measure to 10.2.0.2
avoid unsolicited map responses and spoofing
LISP Host-Mobility Configuration
Across Subnets (No LAN Extensions)
ip lisp ITR-ETR
ip lisp ITR-ETR ip lisp database-mapping 10.3.0.0/16 <RLOC-C>
ip lisp database-mapping 10.2.0.0/16 <RLOC-A> ip lisp database-mapping 10.3.0.0/16 <RLOC-D>
ip lisp database-mapping 10.2.0.0/16 <RLOC-B>
lisp dynamic-eid roamer
lisp dynamic-eid roamer database-mapping 10.2.0.0/24 <RLOC-C>
database-mapping 10.2.0.0/24 <RLOC-A> database-mapping 10.2.0.0/24 <RLOC-D>
database-mapping 10.2.0.0/24 <RLOC-B> map-server 1.1.1.1 key abcd
map-notify-group 239.2.2.2
map-server 1.1.1.1 key abcd
interface vlan 100
map-notify-group 239.1.1.1 ip address 10.3.0.11 /16
interface vlan 100 lisp mobility roamer
ip address 10.2.0.10 /16 ip proxy-arp
lisp mobility roamer hsrp 201
ip proxy-arp mac-address 0000.0e1d.010c
hsrp 101 ip 10.3.0.1
ip 10.2.0.1 Mapping DB
A B C D
LISP-VM (xTR)
West-DC East-DC
10.2.0.0 /16 10.3.0.0 /16
X
Y Z
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
Simplifying LAN Extensions
• Ethernet LAN Extension over any Network
Works over dark fiber, MPLS, or IP Many Physical Sites –
Multi-data center scalability
One Logical Data Center
• Simplified Configuration & Operation

Seamless overlay - No network re-design
Single touch site configuration
• High Resiliency
Failure domain isolation
Seamless Multi-homing
Any Workload, Anytime, Anywhere
• Maximizes available bandwidth Unleashing the Full Potential of Compute Virtualization
Automated multi-pathing
Optimal multicast replication
OTV Data Plane
Inter-Site Packet Flow
1. Layer 2 lookup on the destination MAC. 4. The Edge Device on site East receives
MAC 3 is reachable through IP B and decapsulates the packet
2. The Edge Device encapsulates the frame 5. Layer 2 lookup on the original frame.
3. The transport delivers the packet to the MAC 3 is a local MAC
Edge Device on site East 6. The frame is delivered to the destination
3
MAC TABLE MAC TABLE
Transport
VLAN MAC IF VLAN MAC IF
Infrastructure Decap
100 MAC 1 Eth 2 IP A 2 4 IP B 100 MAC 1 IP A
1 100
OTV
MAC 2 Eth 1
OTV OTV
100 MAC 2
OTV
IP A 5
Encap
MAC 1  MAC 3 IP A  IP B
Layer 2 100 MAC 3 IP B MAC 1  MAC 3 IP A  IP B 100 MAC 3 Eth 3 Layer 2
Lookup 100 MAC 4 IP B 100 MAC 4 Eth 4 Lookup
West East MAC 1  MAC 3 6

MAC 1  MAC 3 MAC 1
© 2010 Cisco and/or its affiliates. All rights reserved. Site Site MAC 3 Cisco Confidential 71
The OTV Control Plane
• OTV proactively advertises MAC reachability (control-plane learning)
• MAC addresses advertised in the background once OTV has been configured
• IS-IS is the OTV Control Protocol running between the Edge Devices
• No specific configuration is required
OTV
MAC Addresses OTV
Advertisements
IP A IP B
West East
IP C OTV

South Cisco Confidential 72
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
Ingress Routing Challenge in DCI
Extending Subnets Creates a Routing Challenge
• A subnet usually implies location
• Yet we use LAN extensions to stretch

subnets across locations LISP site
Location semantics of subnets are lost xTR
• Traditional routing relies on the location

semantics of the subnet
IP Network
Can‟t tell if a server is at the East or West
location of the subnet
LAN Extension
• More granular (host level) information is
required
LISP provides host level location semantics West-DC East-DC
Host-Mobility and Multi-homing
ETR updates – Extended Subnets
Null0 host routes indicate the host is “away”
10.2.0.0 /24 is the dyn-EID
10.2.0.0/16 – RLOC A, B
6 10.2.0.2/32 – RLOC C, D
Map-Register
10.2.0.2/32 <C,D>
Mapping DB
5.1.1.1 5.2.2.2
Routing Table: Routing Table:
10.2.0.0/16 – Local Routing Table: 10.2.0.0/16 – Local
10.2.0.0/24 – Null0 5 10.2.0.0/24 – Null0
10.2.0.0/16 – Local
4 10.2.0.2/32 – Null0 10.2.0.0/24 – Null0 4 10.2.0.2/32 – Local
A B 2 10.2.0.2/32 – Local C D
Routing Table:
10.2.0.0/16 – Local
10.2.0.0/24 – Null0
4 10.2.0.2/32 – Null0 3
10.2.0.0 /16 3 10.2.0.0 /16
1
OTV East-DC
West-DC Y
X Map-Notify
Y
Map-Notify 10.2.0.2/32 <C,D>
10.2.0.2/32 <C,D> 10.2.0.2
Refreshing the map caches Map Cache @ ITR
10.2.0.0/16 – RLOC A,B
1. ITRs and PITRs with cached mappings continue to LISP site

send traffic to the old locators ITR
1. The old xTR knows the host has moved (Null0 route). 10.2.0.2/32 – RLOC C,D
2. Old xTR sends Solicit Map Request (SMR)

messages to any encapsulators sending traffic to Mapping DB
the moved host
3. The ITR then initiates a new map request process
4. An updated map-reply is issued from the new
location
A B C D
5. The ITR Map Cache is updated LISP-VM (xTR)
• Traffic is now re-directed West-DC OTV East-DC
10.2.0.0 /16 10.2.0.0 /16
• SMRs are an important integrity measure to avoid
unsolicited map responses and spoofing Y
X Y Z
10.2.0.2
LISP Host-Mobility - First Hop Routing
With Extended Subnets
• Consistent GWY-IP and GWY-MAC configured across all sites
Consistent HSRP group number across sites  consistent GWY-MAC
• Servers can move anywhere and always talk to a local gateway with the same
IP/MAC
interface vlan 100 interface vlan 100

ip address 10.2.0.5/24 interface vlan ip 200
address 10.2.0.7/24
lisp mobility roamer ip address lisp
10.2.0.8/24
mobility roamer
interface Ethernet2/4
lisp extended-subnet-mode lisp mobilitylisproamer
extended-subnet-mode
ip address 10.2.0.6/24
lisp extended-subnet-mode
hsrp 101
lisp mobilityhsrp 101
roamer LAN Ext.
ip 10.2.0.1 hsrp 101 ip 10.2.0.1
lisp extended-subnet-mode
ip 10.2.0.1
hsrp 101
A B C D
ip 10.2.0.1
LISP-VM (xTR)
HSRP Active HSRP Active
West-DC East-DC
10.2.0.0 /24 10.2.0.0 /24
HSRP HSRP
ARP ARP
GWY-MAC GWY-MAC
LISP VM-Mobility Configuration
With Extended Subnets  Use “Extended-Subnet-Mode”
ip lisp ITR-ETR ip lisp ITR-ETR
ip lisp database-mapping 10.2.0.0/16 <RLOC-A> ip lisp database-mapping 10.2.0.0/16 <RLOC-A>
ip lisp database-mapping 10.2.0.0/16 <RLOC-B> ip lisp database-mapping 10.2.0.0/16 <RLOC-B>
ip lisp database-mapping 10.2.0.0/16 <RLOC-C> ip lisp database-mapping 10.2.0.0/16 <RLOC-C>
ip lisp database-mapping 10.2.0.0/16 <RLOC-D> ip lisp database-mapping 10.2.0.0/16 <RLOC-D>
lisp dynamic-eid roamer lisp dynamic-eid roamer

database-mapping 10.2.0.0/24 <RLOC-A> … database-mapping 10.2.0.0/24 <RLOC-C>
database-mapping 10.2.0.0/24 <RLOC-B> database-mapping 10.2.0.0/24 <RLOC-D>
map-server 1.1.1.1 key abcd map-server 1.1.1.1 key abcd
map-notify-group 239.10.10.10 map-notify-group 239.10.10.10
interface vlan 100 interface vlan 100
ip address 10.2.0.10 /16 ip address 10.2.0.11 /16
lisp mobility roamer lisp mobility roamer
lisp extended-subnet-mode lisp extended-subnet-mode
hsrp 101 hsrp 101
ip 10.2.0.1 ip 10.2.0.1
Mapping DB
LAN Ext. 1.1.1.1 2.2.2.2
A B C D
LISP-VM (xTR)
West-DC East-DC
10.2.0.0/16
X
Y Z
Off-Subnet Client-Server Traffic
All Off-Subnet/Off-Site Traffic Is LISP Encapsulated
• Clients (192.168.0.1 & 192.168.2.1 CLIENT

192.168.2.1
communicate with Server 10.2.0.2 Non-LISP Sites
CLIENT 192.168.2.1  10.2.0.2
• Client-server traffic is LISP 10.1.0.1 LISP Site PxTR
xTR G
encapsulated at the ITRs or PITRs 10.1.0.1  10.2.0.2
F Mapping DB
Client-to-server:
to ETRs C or D GD 192.168.2.1  10.2.0.2
Server-to-client:
FC 10.1.0.1  10.2.0.2
to ETR (F) for LISP sites
to PETR (G) for non-LISP sites
A B C D
• Server-Server off-subnet traffic across LISP-VM (xTR)
sites is also LISP encapsulated
West-DC East-DC
10.2.0.0 /16 10.3.0.0 /16
10.1.0.1  10.2.0.2
192.168.2.1  10.2.0.2
Y
X Y
On-Subnet Server-Server Traffic
On Subnet Traffic Across L3 boundaries
With LAN Extension Without LAN Extensions
• Live moves and cluster member • Cold moves, no application dispersion
dispersion • X- Y traffic is sent to the LISP-VM
router & LISP encapsulated
• Traffic between X & Y uses the
LAN Extension • Need LAN extensions for link-local
multicast traffic
• Link-local-multicast handled by the
LAN Extension
BC 10.2.0.3  10.2.0.2
Mapping DB
LAN Ext. 10.2.0.3  10.2.0.2
A B C D A B C D
LISP-VM (xTR) LISP-VM (xTR)

West-DC West-DC
10.2.0.0/16 East-DC East-DC
10.2.0.0/16
10.3.0.0/16
10.2.0.3 Y 10.2.0.3 Y
X Y Z X Y Z
10.2.0.2
10.2.0.2 Cisco Confidential 80
Agenda
 FabricPath
 VXLAN 1K
Cisco
Nexus
x8
 LISP 6
 Nexus Fabric
• Enhance application availability by distributing Cluster members across PODs and across locations
• Distance limited by application latency budget and storage replication
• Two types of traffic specific to the cluster:

Non-routable heartbeats: FabricPath (Intra-DC) & OTV (Inter-DC) provide LAN connectivity
Front-end IP connectivity: LISP provides mobility for cluster virtual-IP failover
LISP
IP mobility
IP Network
DC-west DC-east
OTV
POD POD (Inter-DC) POD POD
App Cluster Distributed App (GeoCluster)

OS OS OS
Fabric Path
(Intra-DC)
• VXLAN & FP provide elasticity within the DC within a L2 POD and across PODs
• OTV extends the LAN across DC sites without compromising network stability
• LISP integrates with SLBs and balances traffic across the SLBs (Future)
Intra-DC Inter-DC
Virtual Machines VXLAN (x-L3), FabricPath (L2) OTV (x-L3)
Physical Machines FabricPath (L2), VXLAN GWY (future) OTV (x-L3)
LISP
IP mobility
IP Network
DC-west DC-east
POD POD POD POD
App App App App

OTV
OS OS OS OS
(Inter-DC x-L3)
Fabric Path VXLAN SLB

• Reduce Disaster Recovery Bring-up times - Less Network Changes/Operations = Faster recovery times
• Preserve IP addressing with LISP host mobility

No reconfiguration of applications or network service policies
No routing re-convergence
Automatic routing re-localization (upon application bring-up at DR)
• VXLAN segments move along with the applications (vApps)
LISP
IP mobility
IP Network
DC-west DC-east
POD POD POD POD

OS OS OS OS OS OS
VXLAN VXLAN
(Intra-DC x-L3)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
(Intra-DC x-L3) 84
IP1
GWY
vxlan 1
• Move virtual Applications (vApps) to private cloud PODs V

web M
Move VMs and virtual Segments (VXLANs) vxlan 2
V VSG
• LISP host mobility allows the vApp GWY to roam app M
Maintain GWY IP address and optimal reachability vxlan 3

V
db M
• VXLAN segments move along with the applications (vApps)
Very large scale of virtual segments can move and extend across L3 boundaries vApp = Collection of
VMs and segments
LISP
IP mobility
with a GWY
IP Network
DC-west DC-east
POD POD POD POD

GWY
GWY
vxlan
GWY 1 GWY GWY
vxlan 1
VM vxlan 1 vxlan 1 vxlan 1
VM
web vxlan 2
web VM
vxlan 2
VM VM
web VM vxlan 2 web vxlan 2 web vxlan 2
VM
app vxlan 3
app VM
vxlan 3
VM VM
appVM vxlan 3 app vxlan 3 app vxlan 3
db VM
© 2010 Ciscodb
and/or itsVMaffiliates. All rights reserved. VM VM Cisco Confidential 85
db db db
Complimentary Capabilities
FabricPath, VXLAN, LISP
Requirement Intra-DC Inter-DC

Scale
Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS
IP Mobility LISP LISP
Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS)
LISP
IP mobility
IP Network
DC-west DC-east
POD POD POD POD
OS OS OS OTV/VPLS OS OS OS
(Inter-DC x-L3)
Fabric Path VXLAN/OTV Fabric Path VXLAN/OTV
Q&A
#CiscoPlusCA
We value your feedback.
Please be sure to complete the Evaluation Form for this session.
Access today‟s presentations at cisco.com/ca/plus
Follow @CiscoCanada and join the #CiscoPlusCA conversation

FabricPath RFULLER PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

FabricPath RFULLER PDF

Încărcat de

Drepturi de autor:

Formate disponibile

Flexible Data Centre Fabric -

FabricPath/TRILL, OTV, LISP

Ron Fuller– CCIE #5851 (R&S/Storage)

 The Evolving Data Centre Fabric

 LISP Host Mobility

 OTV LAN Extension

 Mobility with Extended Subnets

Architecture is evolving Rapidly – in the next 24 months

East-West traffic – Fate Sharing Domain

 The traditional L2 vs. L3 debate has been based on a number of issues

 Requirements for the scalable design moving forward is a scalable,

 Practically “plug-n-play” – No user  MAC address consumption

 Layer 3 Routed Topologies alleviate the

S1 Web S2 App S3 Database

Identity • All these technologies leverage

FabricPath / VXLAN OTV LISP

Requirement Intra-DC Inter-DC

Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS

IP Mobility LISP LISP

Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS)

POD POD POD POD

App App App App App App

Fabric Path VXLAN/OTV Fabric Path VXLAN/OTV

 The Evolving Data Centre Fabric

 LISP Host Mobility

 OTV LAN Extension

 Mobility with Extended Subnets

“FabricPath brings Layer 3 routing benefits to

S10 S20 S30 S40

S100 S200 S300 S400

Switch ID space: S300: FabricPath

MAC adress space: 1/1 1/2 S300: CE MAC

• Core fabric leverages an independent routing topology from the edge

• Ingress switch uses MAC table to determine destination Switch ID

• Egress switch uses MAC table (optionally) to determine output switchport

S10 S20 S30 S40

© 2010 Cisco and/or its affiliates. All rights reserved.

• All FabricPath-enabled switches automatically assigned Switch ID (no user configuration

• Equal-cost paths supported between FabricPath switches

• Intermediate switches don‟t learn the MAC

• Hardware based MAC learning

CE Edge Ports  Interface connected to traditional network device

V10 V20 V30 V30 V10 V20 V10 V30

V10 V20 V30

FabricPath • Support for multiple multidestination

S10 S101 S30 S40 S101 S20

Root S200 S40 Root S200 S30

VLAN 20 – DC Wide VLAN 20 – DC Wide

16 bytes Original CE Frame

6 bits 1 1 2 bits 1 1 12 bits 8 bits 16 bits 16 bits 10 bits 6 bits

Endnode ID Endnode ID Sub Etype

Switch ID LID Ftag TTL

• Switch ID – Unique number identifying each FabricPath switch

Multidestination Payload DMAC→FF

Trees on Switch 100 po20 po30 po40

3 Tree IF po40 po10 Payload

Broadcast → 1 po10 S100 S200 Multidestination S300

• S10 (and S20, S30, S40, S200, S300):

Ftag → 1 po10 S200 Multidestination S300

S300 → S300 po300 16

S40 po40 S200 FabricPath Routing S300

S300# sh mac address-table dynamic

via Po20, [115/40], 0 day/s 04:43:06, isis_fabricpath-default