VXLAN

VXLAN Encapsulation VXLAN Encapsulation(Detail)

VXLAN 0 bit 16 bit 31 bit
RSVD VNI RSVD
Flags Destination MAC Address
8 bit 24 bit 24 bit 8 bit

Outer Outer Outer VXLAN Original Layer 2 Frame FCS

MAC IP UDP Header (new) Source MAC Address

14 bytes 20 bytes 8 bytes 8 bytes 4 bytes Optional: 802.1Q VLAN Header

+
(4 bytes optional)

VXLAN Encapsulation Ethertype = 0x0800 (ipv4)

Underlay
DMAC SMAC 802.1Q Etype Payload FCS Version IHL TOS Total Length

6 bytes 6 bytes 4 bytes 2 bytes 46~9000 bytes 4 bytes Identification Flags Fragment Offset
* VXLAN adds 50 bytes of overhead Classical Ethernet Frame (Original L2 Frame)
Time to Live Protocol = 17 Header Checksum
Terminology IPv4 Source Address
VXLAN Overlay layer 2 overlay on top of Layer 3 underlay, identified by
Ipv4 Destination Address
VNID & extends/tunnels traffic from one VTEP to another
Source Port = (random) Dest Port = 4789
VXLAN Underlay services such as OSPF, IS-IS, EIGRP, Multicast & BGP
that provides the transport for VXLAN UDP Length UDP Checksum

VXLAN Tunnel End Point (VTEP) a device that perform VXLAN R|R|R|R|I|R|R|R Reserved
encap/decapsulation, could be hardware or software
VXLAN Network Identifier (VNI) Reserved
VNI/VNID each VXLAN segment identified by 24-bit segment ID, only
Destination MAC Address
hosts on the same VNI are allowed to communicate with each other. It
overcome 4094 VLAN scale limitation. Thus, segment IDs are globally

Overlay
significant and VLAN IDs are locally significant
Source MAC Address
VXLAN Gateway VTEP that bridge layer 2/3 traffic between VXLAN segments
Optional: 802.1Q VLAN Header
Network virtualization Edge (NVE) logical representation of the VTEP,
Ethertype
I.e. NVE is the tunnel interface
Original Ethernet Payload
BUM Traffic Broadcast, Unknown Layer-2 Unicast and Multicast

Flood & Learn vs EVPN Control Plane

Peer Peer Host Route Host Route
Encapsulation
Discovery Authentication Distribution Learning
Data driven No route Local & Remote host:
Flood & Learn MAC in UDP
flood & Learn
Not available
distribution Data driven flood & Learn

EVPN Local host: Data driven

MAC in UDP MP-BGP MP-BGP MP-BGP
Control Plane Remote host: MP-BGP

VXLAN - Flood & Learn VXLAN - EVPN Control Plane

ARP IP_B MAC_A ALL VTEP 3 VTEP 1 VTEP 2
Host C MAC VXLAN ID NH MAC, IP L2VNI L3VNI NH MAC, IP L2VNI L3VNI NH
MAC_A 10 VTEP1 MAC_A, IP_A 3001 5001 Local MAC_A, IP_A 3001 5001 VTEP1
3
MAC_B, IP_B 3001 5001 VTEP2 MAC_B, IP_B 3001 5001 Local
MAC_C, IP_C 3002 5001 VTEP2 Spine MAC_C, IP_C 3002 5001 Local
VXLAN Original Frame
BGP Route Reflector
2
Multicast/Unicast Host B
Host A Host B Host A MAC_B
Replication
1 3 MAC_A / IP_A IP_B
2
ARP IP_B MAC_A ALL VXLAN Original Frame ARP IP_B MAC_A ALL
Host C
VTEP 1 VTEP 2
Leaf Leaf MAC_C
MAC VXLAN ID NH MAC VXLAN ID NH
MAC_B 10 VTEP2 4 MAC_A 10 VTEP1 * VTEPs advertise End-Host reachability information (Mac, IP) within MP-BGP IP_C

Data Plane learning technique for VXLAN, VTEP will flood the MP-BGP EVPN introduces control-plane learning for end hosts
packet to all neighbor and will learn the remote end. behind remote VTEPs. Provides control & data plane separation
Last update Jan 5, 2017 (version 1.00)
References: https://cloudpacket.net/bookmarks/ Page 1/2 Prepared By Shakib Shaygan
 VXLAN
Gateway Types Leaf Node Configuration - L2 VNI
# 1-Feature enablement # 5-Anycast addresses
VXLAN Red VLAN 10
feature bgp/pim/interface-vlan fabric forwar any-cast-
vn-segment-vlan-based gateway 0001.0001.0001
Egress packet is
nv overlay/nxapi/lldp
802.1q tagged
VXLAN L2 fabric/fabric forwar interface vlan 100
Gateway nv overlay evpn vrf member EVPN-TENANT
ip add 100.1.1.254/24
VXLAN Red VXLAN Blue # 2-Map VLAN to VXLAN fabric forw mode anycast
vlan 100
Egress packet is vn-segment 10000 # 6-Configure BGP
another vxlan segment router bgp 100
VXLAN L3 # 3-Create L2 VNI router-id 192.168.1.1
Gateway/Router evpn address-family l2vpn evpn
vni 10000 l2 neighbor 192.168.10.10
Layer 2 Gateway is required when layer 2 traffic (802.1q) rd 10000:1 remote-as 100
comes from VLAN into VXLAN segment (encapsulation) or route-target import 10000:1 update-source lo0
the ingress VXLAN packet egresses out an 802.1q tagged route-target export 10000:1 address-fam l2vpn evpn
interface (decapsulation) where packet is bridged to a new vlan send-comm extended
# 4-Configure NVE Interface vrf EVPN-TENENT
Layer 3 Gateway is used when there is a VXLAN to VXLAN interface nve1 address-fam ipv4 unicast
routing.The ingress packet is a VXLAN packet on a routed source-interface lo 0 advertise l2vpn evpn
segment but the packet egresses out on a tagged 802.1q host-reachability prot bgp
interface and the packet is routed to a new VLAN member vni 10000
mcast-group 239.1.1.1
Asymmetric IRB supress-arp

Leaf Node Configuration - L3 VNI

Leaf Leaf
VTEP 1 VTEP 2 # 1-Feature enablement # 5-Configure interface
SVI 200

SVI 300

SVI 200

SVI 300

feature bgp/pim/interface-vlan interface vlan 200

L2VNI 3000
vn-segment-vlan-based vrf member EVPN-TENANT
nv overlay/nxapi/lldp ip forward
fabric/fabric forwar
nv overlay evpn interface loopback 200
vrf member EVPN-TENANT
# 2-Map VLAN to VXLAN ip add 200.1.1.1/32
vlan 200
Host A Host B Host C Host D vn-segment 20000 # 6-Configure BGP
VLAN 200 VLAN 300 VLAN 200 VLAN 300 router bgp 100
VXLAN 2000 VXLAN 3000 VXLAN 2000 VXLAN 3000
# 3-Create L3 VNI vrf EVPN-TENANT
Routing and Bridging on the ingress VTEP, Bridging on vrf context EVPN-TENANT address-family ipv4 un
the egress VTEP, both source and destination VNI need to vni-20000 network 200.1.1.1/32
rd 20000:1 advertise l2vpn evpn
reside on the ingress VTEP. Similar to Inter-VLAN routing
address-famil ipv4 unicast
route-target imp 20000:1
Symmetric IRB route-target imp 20000:1 evpn
route-target exp 20000:1
L3VNI 50001 route-target exp 20000:1 evpn

Leaf Leaf # 4-Configure NVE Interface

VTEP 1 VTEP 2 interface nve1
SVI 200

SVI 300

SVI 200

SVI 300

source-interface lo 0
host-reachability prot bgp
member vni 20000 associate-vrf

Troubleshooting & Debugging

show bgp l2vpn evpn
Host A Host B Host C Host D show nve [interface | vni | peers | vxlan-param]
VLAN 200 VLAN 300 VLAN 200 VLAN 300 show interface nve1
VXLAN 2000 VXLAN 3000 VXLAN 2000 VXLAN 3000 show forwarding nve l3 peers
show l2route evpn mac-ip evi 100
Routing on both ingress and egress VTEPs, Ingress VTEP show ip arp suppression-cache [local | remote]
routes packets onto the layer 3 VNI, Egress VTEP routes show ip route vrf EVPN-TENANT
packet to the destination layer 2 VNI debug nve [errors | events | pim-library | all]
Last update Jan 5, 2017 (version 1.00)
References: https://cloudpacket.net/bookmarks/ Page 2/2 Prepared By Shakib Shaygan