BRKSPG 3334

Carrier Grade NAT44 on IOS-XR
Deployment Experience
BRKSPG-3334
Nicolas Fevrier
Rajendra Chayapathi
Syed Hassan
Agenda
Introduction
NAT Principles and Mechanisms
Bulk-Port Allocation
Port limit
Static Port Forwarding
ALG
Logging
Hardware
Deployment feedback
Routing consideration and Best Practices
Redundancy
BRKSPG-3334 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
INTRODUCTION
Introduction
Do you think CGN is evil?
Yes but its a necessary one
IPv4 address exhaustion
End-to-end IPv6 traffic, are you ready?
The same cards can be used for:
NAT44
But also for smooth transition to IPv6
Lets jump directly into the deep end
Facts About IPv4 Shortage
LIR are allocating their last blocks

On 14 September 2012, the RIPE NCC
started allocating from the last /8 of
IPv4 addresses received from IANA
IPv4 grey/black market is flourishing
Ciscos Strategy: 3 Pillars
Ciscos strategy relies on three pillars
Preserve (Business Continuity)
NAT44 / CGN
Optimize the IPv4 resource and allow growth
Prepare (Encourage Adoption)
Offer IPv6 to the customers
6rd: transport IPv6 on top of a IPv4 infrastructure
Prosper (Interworking)
DS-Lite, MAP-T/E: transport of the remaining IPv4 traffic
on top of a IPv6 backbone
NAT64: translate to the IPv4 at the border
Among IOS-XR products, the ISM and VSM (ASR9000) and CGSE and CGSE+
(CRS) cards are the tools used to build these three pillars.
Vocabulary
i2o / o2i: inside to outside / outside to input
NAT/NAPT: Network Address (and Port) Translation
CGx: carrier grade (CGN: Carrier Grade NAT)
LSN: Large Scale NAT
ALG: Application Layer Gateway
GRT: Global Routing Table
SL/SF: Stateless/Stateful
Translation Protocols Illustrated
Stateful vs Stateless
Example: we have 16 public addresses
4 4 4 4
Stateless translation Stateful translation

1 external IP : 1 internal IP 1 external IP : n internal IP
No multiplexing no DB needed Multiplexing DB to maintain
Stateless vs Stateful
Stateless Stateful
1:1 translation 1:n translation (port multiplexing)
i2o or o2i initiated sessions are Needs translation DB maintenance
treated equally
Logging scalability can be an issue
Allows asymmetrical traffic
Need static port forwarding or PCP to
Better convergence time accept o2i initiated sessions
Potential inline implementation May need ALGs
No logging required In case of failover, we need to re-
establish sessions on a new device
Protocols: NAT64SL, 6rd, MAP-T Protocols: NAT44, NAT64SF, DS Lite
NAT44
Introduction
Preserve the investment: buy time to prepare migration to IPv6

Not the solution but meets a vast majority of user current needs
NAT vs NAPT
Defined since 2001 (RFC3022, RFC4787, RFC5382, RFC5508)
Unicast
TCP/UDP/ICMP
Stateful
Permit multiplexing: several internal hosts will use the same external
address, maximizing the IPv4 resource
ALGs
NAT44 Overview
IPv4 Traffic Outside Address = 170.0.0.1
Source Address = 10.1.1.10
IPv4
Internet
IPv4 CGN
Backbone
Stateful translation protocol from an IPv4 space to another IPv4 space

IPv4 space public or private
Usually, from private (RFC1918) to public but not necessary
Translation table or database (DB) maintained on the CGN card
BRKSPG-3334 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
NAT444 or Double NAT44
IPv4 Traffic Outside Address = 170.0.0.1
Source Address = 10.1.1.10
IPv4
Internet
CPE
IPv4 CGN
Backbone
Translated Address = 10.8.1.111

Double step stateful translation:
At CPE level
Between home network and ISP access network
At CGN level
Between ISP network and public address network
From CGN perspective: NAT44 = NAT444
NAT PRINCIPLES and MECHANISMS
NAT Mechanisms
Inside VRF NAT Engine Outside VRF
Source Source
10.10.10.2:2493 100.2.1.24:8442
Destination Destination
5.20.3.2:80 5.20.3.2:80
Web Client Web Server: 5.20.3.2
10.10.10.2
Collector
Translation table
10.10.10.2:2493100.2.1.24:8442 Logging
Record
Syslog
Netflow
Inside VRF NAT Engine Outside VRF

Source Source
5.20.3.2:80 5.20.3.2:80
Destination Destination
Web Client 10.10.10.2:2493 100.2.1.24:8442 Web Server: 5.20.3.2
10.10.10.2
NAT Principles
EIM/EIF vs EDM/EDF
EIM: End-point Independent Mapping

destination address and port for i2o traffic Dest
A:80
Dest
B:80
Dest
B:80
not tracked
If multiple destinations but source address
and port are the same Source
no other entry created Y:1430
Inside Outside Destination
Sometime referred as full cone NAT X:4828 Y:1430 *
EDM: End-point Dependent Mapping

Opposite of EIM
Source
Destination info is maintained in DB X:4828
NAT Principles
EIM/EIF vs EDM/EDF
EIF: End-point Independent Filtering

Once entry is present in the table Source
A:80
Source
B:4234
Source
C:80
For o2i traffic, we dont verify source
address/port
Better scalability and larger support Dest
Y:1430
EDF: End-point Dependent Filtering
Opposite of EIF Inside Outsid Destinatio
n
Check the source addresses for o2i traffic e
Required in some situation: bill shock effect Dest
X:482 Y:1430 *
X:4828
8
NAT Principles
Paired IP Address Assignment
We use the same external IP address mapping for all sessions associated
with the same internal IP address (RFC4787)
Each inside odd port is mapped to an outside odd port number
Each inside even port is mapped to an outside even port number
Inside Outsid
e
Source Source
Source Source
Outside Source A:11238 A:10985 B:1045
B:1491 Source X:2104 A:1030
A:10302 B:1228 2
X:2334 A:11238
Inside 2
X:4827 A:1098
Source Source Source Source Source Source
X:2104 X:23342 X:48271 Y:29301 Y:43017 Y:1024 1 5
Y:29301 B:1045
Y:43017 B:1491
BRKSPG-3334 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public Y:1024 B:1228
NAT Principles
Hair Pinning
Two endpoints on inside NAT can communicate to each others using external
NAT IPv4 addresses and ports.
Outside A:10302 B:11237
Inside
Source Source
X:2104 Y:11003
NAT Principles
Address Allocation
First flow per Inside source address Used

port
Free
port
CGN picks an Outside address that ? NAT ?

has at least 1/3 of its ports free
IP1 IP2 IP3 IP4 IP5 IP6 IP7 IP8
All subsequent Flows from that Inside
source will use the same Outside Ok
address. No
NAT Principles
Address Allocation
If that Outside address is completely exhausted, then a random selection is
made from the remaining addresses, repeated until an address is chosen or it is
determined that none are available (which results in an ICMP error message)
NAT Used Free NAT
port port
? ? ?
IP1 IP2 IP3 IP4 IP5 IP6 IP7 IP8 IP1 IP2 IP3 IP4 IP5 IP6 IP7 IP8
ICMP
error
No
NAT Principles
Port Allocation
Ports are randomly picked from the list of

available (unused) ports associated with Used Free
IP1
port port
the chosen Outside IP address
Each port is allocated once, regardless NAT
of which L4 protocol (UDP, TCP) is being ?
used in the Flow ? Inside Outside
CGN creates a Translation binding IPa:2104 IP1:10302
(state) between
Inside source IP address + port
and
Outside source IP address + port
NAT Principles
Port Allocation
If the randomly chosen port is already
being used, the selection increments port-limit=8
IP1 IP1
(around a ring) until an available port
is found; if none are available then an Inside Outside
IPa:Pa IP1:P1
ICMP error message is sent
IPa:Pb IP1:P2
NAT
If the Inside source already has a IPa:Pc IP1:P3
?
number of Flows equal to the IPa:Pd IP1:P4
IPa:Pe IP1:P5
configured per-user limit, then the IPa:Pf IP1:P6
allocation is rejected and an ICMP IPa:Pg IP1:P7
message is returned IPa:Ph IP1:P8
IPa:Pi No
ICMP
error
Algorithm-based / Predefined NAT
Often referred as Deterministic NAT, coming in future releases
Opposite approach than random allocation mechanisms described before
Allows predictable mapping of source addresses/ports between the inside
and outside world
Based on an algorithm, each internal address will be allocated an external
address and range
Predefined NAT is still stateful (translations are still stored in DB)
Main benefit: logging is no longer necessary (but will still be possible)
Main flaw: sub-optimal address allocation
Addresses and port ranges are allocated regardless of the presence or usage of the
internal users
To meet requirements of certain ALGs, it will be necessary to allocate contiguous ports
SDNAT (stateless) draft has been discontinued
BULK PORT ALLOCATION
Bulk Port Allocation
Aims at reduces data generated by logging Outside
Bulk port allocation behavior IP1
A subscriber creates the first connection
N contiguous ports are pre-allocated Logging
Record
(ex: 2064 to 2080 if N=16)
Bulk-allocation message (NFv9 and/or syslog) is logged NAT Collector
for the port-range

Additional connections (up to N) will use one of the Syslog
pre-allocated ports Netflow
New pool allocated if subscriber creates > N concurrent

connections
Bulk-delete message is logged when subscriber terminates all
sessions from pre-allocated pool
When bulk size is changed, all current dynamic translations will be deleted
Ports below dynamic start range (< 1024) are not allocated to bulk
It can take one of the following values:
16, 32, 64, 128, 256, 512, 1024, 2048, 4096 (8 in IOS XR 4.3.1)
port-limit / 4 bulk-port-alloc port-limit x 2
Recommendation: closest value to half the port-limit
Orthogonal with Destination Based Logging, can NOT be configured
together
Port range allocation is random, in following examples we picked 1024-1039
and 1040-1055 for the sake of simplicity only
BPA Illustrated
Example Bulk=16
IPv4 Traffic
Source Address = 10.1.1.1 Outside Address
NAT from pool = 99.0.0.1
IPv4
Internet
CGN
10.1.1.2
BPA Illustrated
Example Bulk=16
10.1.1.1 NAT IPv4
Internet
10.1.1.2
1 1 packet from 10.1.1.1 to 30.1.1.1:80
2 1 packet from 10.1.1.1 to 30.1.1.1:25
3 1 packet from 10.1.1.2 to 40.1.1.1:80
BPA Illustrated
Example Bulk=16
10.1.1.1 NAT IPv4
Internet
10.1.1.2
1 1 packet from 10.1.1.1 to 50.1.1.1:80
2 1 packet from 10.1.1.1 to 60.1.1.1:80
BPA Illustrated
Example Bulk=16
Same rules for init and active timeout apply for bulk ports
1 2 3
4 5 6
7 1 packet from 10.1.1.1 to 30.1.1.1:80
BPA=16 can reduce the logging volume MUCH more than by 16

With NAT444, its very likely that at least one device is connected behind the
CPE at any given time
Consequently, logging for the port allocation is generated once and the port
block is never de-allocated or de-allocated many weeks or months later
Its exactly what the protocol is supposed to do, but it creates some issues
Potential issue with logging collector correlator
Another issue could be the security. It makes one CPU always use the same port
range and reduces the scope for attackers
Workaround: DHCP lease time reduced to re-assign a different IP to the CPE
every couple of weeks.
Bulk Port Allocation: Configuration
Config parser will enforce the selection respecting:
8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096
port-limit / 4 bulk-port-alloc port-limit x 2
Recommendation: closest value to half the port-limit
service cgn POC-1
service-type nat44 nat44-1
inside-vrf Inside-1
bulk-port-alloc size 256
PORT LIMIT
Per-user Port Limit
For stateful translation protocols (NAT44, NAT64 SF, DS Lite), each user can
be assigned a maximum number of ports. It prevents a single user to
consume all port resources port-limit=8
IP1
Inside Outside
IPa:Pa IP1:P1
IPa:Pb IP1:P2
NAT IPa:Pc IP1:P3
IPa:Pd IP1:P4
IPa:Pe IP1:P5
? IPa:Pf IP1:P6
IPa:Pg IP1:P7
IPa:Ph IP1:P8
IPa:Pi No
Per-user Port Limit
Port-limit can be defined per protocol
But also per VRF
allows different treatment for different type of customers
Finding the proper port-limit is a very tricky exercise
No simple rule of the thumb
Different for each type of customer (ADSL, Mobile, Cable, Enterprise)
Different for each theater (Asia, Europe, Russia, Americas)
Scripts can be used to collect average and maximum port usage
Per-user Port Limit on CGSE
Exceeding the port limit will trigger a syslog message:
[Portblockrunout 17 10.1.11.202 ivrf- 2005 - - ]
Portblockrunout: event name signifying the port limit hit event
17: it was hit by a UDP packet requesting the translation
10.1.11.202: is the subscribers private IP
ivrf: name of the inside VRF
2005: private port number
These messages are throttled
For 10.1.11.202, once we report this message, we will not repeat them for the same
subscriber until it goes below 70% of max limit and then goes up again and hits the
port limit
Can be used to quickly user consuming a lot of ports
Configuring Port-Limit
Its a safety net preventing one user to use
all resources
For stateful translation protocols each user service cgn demo
can be assigned a maximum number of service-location preferred-active 0/1/CPU0
ports portlimit 512
NAT44 and NAT64SF will use keyword inside-vrf iVRF1
portlimit 256
portlimit inside-vrf iVRF2
!
We can use every value between 1 to !
65535, default is 100
Defined per protocol or globally since 4.3.1
STATIC PORT FORWARDING and PCP
Session Initiated From the Outside ?
IPv4 Traffic Map pool = 99.0.0.0/24
IPv4
10.1.1.1 Internet
CGN
30.0.0.1
Inside Outside TCP
0 state
1
2 No entry in the NAT DB, 1
o2i packets are discarded
With stateful translation mechanisms, a traffic initiated from the outside will be
discarded
Static Port Forwarding or Port Control Protocol necessary
Static Port Forwarding
IPv4
10.1.1.1 Internet
CGN
30.0.0.1
Inside Outside TCP
0 state
1 6000 entries max

service cgn demo
service-type nat44 nat1 Inside Outside TCP
inside-vrf insidevrf1 2 state
protocol tcp
static-forward inside address 10.1.1.1 port 80 11.1.1.1:80 99.0.0.1:80 static
4
3
Static-port-forwarding creates an entry in the NAT DB

Verifying Static-Port-Forwarding
External address is picked by the system, not the user (based on hashing of
inside address)
RP/0/RP0/CPU0:R#sh cgn demo inside-translation protocol tcp inside-vrf Inside inside-address

10.12.0.250 port s 10000 e 10000
Inside-translation details
---------------------------
CGN instance : demo
Inside-VRF : Inside
--------------------------------------------------------------------------------------------
Outside Protocol Inside Outside Translation Inside Outside
Address Source Source Type to to
Port Port Outside Inside
Packets Packets
--------------------------------------------------------------------------------------------
100.0.0.58 tcp 10000 15819 static 0 0
RP/0/RP0/CPU0:R#
Port Control Protocol
PCP client IPv4 Traffic Map pool = 99.0.0.0/24
30.0.0.1
on private network IPv4
10.1.1.1 Internet
CGN Host on
public
network
PCP Server
PCP allows applications to create mappings from an external IP address+proto+port to
an internal IP address+proto+port
PCP Server is a software instance via which clients request and manage explicit
mappings
PCP Client issues requests to a server
A PCP Client can issue PCP requests on behalf of a third party device
A PCP request is transported on UDP(v4/v6) packet with destination port 5351
Supported on CGSE cards for NAT44, NAT64 and DS-Lite
http://tools.ietf.org/html/draft-ietf-pcp-base-29
IPv4
10.1.1.1 Internet
CGN
Inside Outside TCP
0 state
1 MAP Request
99.0.0.1 TCP 80
MAP Response 2
0: SUCCESS
Inside Outside TCP state
3
10.1.1.1:80 99.0.0.1:80 pcp_explicit
4 FIN or RST
Inside Outside TCP state
5
10.1.1.1:80 99.0.0.1:80 pcp_explicit
PCP Req/Resp IPv4
10.1.1.1 Internet
CGN
Inside Outside TCP
0 state
10.1.1.1:80 99.0.0.1:80 dynamic
1 MAP Request
99.0.0.1 TCP 80 Other result codes could be:
1:UNSUPP_VERSION 8:NO_RESOURCES
MAP Response 2
2:NOT_AUTHORIZED 9:UNSUPP_PROTOCOL
11:CANNOT_PROVIDE_EXTERNAL 3:MALFORMED_REQUEST 10:USER_EX_QUOTA
Available external port: 84 4:UNSUPP_OPCODE 11:CANNOT_PROVIDE_EXTERNAL
5:UNSUPP_OPTION 12 ADDRESS_MISMATCH
6:MALFORMED_OPTION 13:EXCESSIVE_REMOTE_PEERS
7:NETWORK_FAILURE
IPv4
10.1.1.1 Internet
CGN
Inside Outside TCP
0 state
1 PEER Request
99.0.0.1 TCP 80
PEER Response 2
0: SUCCESS Inside Outside TCP state
3
10.1.1.1:80 99.0.0.1:80 pcp_implicit
4 FIN or RST
Inside Outside TCP
5 state
DB entry removed
APPLICATION LAYER GATEWAYS
Need for ALG
ALG are features allowing upper layer inspection to track a particular
behavior (port negotiation, ) and make sure the protocol will be unaffected by
the translation
Ciscos position is to discourage the pursue of ALGs
Applications are regularly rewritten and keeping track of each change is challenging
NAT traversal is more generally handled at the application level
Supported ALGs in CGN cards
Active FTP (passive FTP doesnt need ALG)
RTSP (used for some streaming services)
PPTP (for legacy VPN applications)
Active FTP ALG
In active mode FTP
the client connects from a random unprivileged port (N > 1023)
to the FTP server's command port 21
then, client starts listening to port N+1
and sends the FTP command PORT N+1 to the FTP server
the server will then connect back to the client's specified data port from its local data
port, which is port 20
ALG converts the network Layer address information found inside an
application payload
Note: Passive FTP Mode does NOT need any ALG
RTSP ALG
Real-Time Streaming Protocol is not a streaming protocol
Its a remote control protocol for streamers (which use RTP/RTCP or RDT)
a text-based protocol based on methods (like requests) and transported on
port554
RTSP session is not a connection per say since its not tied to a transport-
level connection, even if transported by TCP
Our implementation considers the server is located outside and clients are
inside
RTSP is used in many streamers like QuickTime or RealNetworks
(less and less used with generalization of HTML5)
PPTP ALG
Point to Point Tunneling Protocol is used by legacy VPN solutions
Encapsulate PPP packets in IP GRE
Translation of PPTP packet is challenging because we dont translate source
ports but a peer caller ID field contained in the GRE header
PAC: PPTP Access Concentrator, in the public side (Outside)
PNS: PPTP Network Server, in the private side (Inside)
PPTP
NAT
IPv4
PNS Internet PAC
Control Connection (TCP1723)
Configuring ALGs
We currently support three ALGs types for NAT44 (none for NAT64SF and
only FTP for DS Lite)
ActiveFTP (not needed for PassiveFTP)
RSTP (for Real Audio G2 and windows media player), default port is 554
PPTP (for legacy VPN systems)
service cgn demo
alg ActiveFTP
alg rtsp port 10000
alg pptpAlg
!
Verify ALG Activity
When a translation database entry will be allocated based on ALG, it will
appear like:
RP/0/RP0/CPU0:R#sh cgn demo inside-translation protocol tcp inside-vrf Inside inside-address
10.13.0.29 port s 1 e 65535
---------------------------
CGN instance : demo
Inside-VRF : Inside
--------------------------------------------------------------------------------------------
Packets Packets
--------------------------------------------------------------------------------------------
100.0.0.221 tcp 1043 41493 dynamic 51 55
100.0.0.221 tcp 55000 26236 dynamic 6 5
100.0.0.221 tcp 55001 16300 dynamic 6 5
100.0.0.221 tcp 55002 28942 alg 23 22
100.0.0.221 tcp 55003 4373 dynamic 5 5
RP/0/RP0/CPU0:R#
LOGGING
Need for Logging
Entries in NAT table are of temporary nature
Any Stateful protocol (NAT44, NAT64SF, DS-Lite) requires logging
Directive 2006/24/EC - Data Retention: EU Law
Logging preserves the mapping information between an internal and external
CGSE and ISM cards supports Netflow v9 and Syslog
NAT
IPv4
Internet
Logging
Record
Syslog
Netflow
What CGN information needs to be stored by ISPs ?
Source IP address and port translation history
to be able to reliably identify the private IP translated to public IP at one precise
moment
further inspection of RADIUS or DHCP database can be performed to provide the
identity of subscriber (e.g. MAC address of device or username)
Format of the information (as long as translation can be inverted based on the
input parameters):
ASCII format
Compressed text/binary files or relational database that contain translation history
details
Outcome of an algorithmic mapping of private IP address to public IP address/port
Dynamic or Pre-defined NAT ?
No definitive and easy answer
The logging solutions Pre-defined NAT
Dynamic NAT
Per-session logging (w/Syslog or
w/Netflow)
Bulk Port Allocation logging (w/Syslog or
w/Netflow)
Destination Based Logging w/Syslog or
w/NetFlow
Pre-defined NAT
Each choice is optimizing subset of
requirements at the expense of others
Destination-Based Logging
DBL permits to specifically log destination address and port
X1
X2
A NAT X3
Internal External
X4
Logging
Record
Syslog
Netflow
Tim Inside Outside Destination

e IP/Port IP/Port IP/Port
T1 A:Pa IP1:P1 X1:Pd1
T2 A:Pb IP1:P2 X2:Pd2
T3 A:Pc IP1:P3 X3:Pd3
T4 A:Pd IP1:P4 X4:Pd4
Why would you like to use DBL? Why should you avoid using DBL?
Legal regulations in country Privacy considerations
Many web servers are not logging port Country regulations
information for each session (not respecting
RFC6302 Logging Recommendations for Interpretation of EU directive
Internet-Facing Servers) Conflicts with Bulk Port Allocation and
Others Deterministic NAT
Need for data analytics solution e.g. Increased storage requirements
Offers very detailed info on user behavior
6 additional bytes in NFv9 to store A+P
draft-ietf-behave-lsn-requirements
REQ-12: A CGN SHOULD NOT log
destination addresses or ports unless required
to do so for administrative reasons
The CGN card will generate templates 271 for Add records and templates 272
for Delete records
service cgn POC-1

inside-vrf Inside-1
map address-pool 150.0.0.0/17
external-logging netflow version 9
server
address 172.16.255.254 port 5000
session-logging
!
Syslog or Netflow v9 ?
Two options in CGN cards today:
Netflow v9 Syslog
Syslog
Format Binary ASCII
Netflow v9 Template based format RFC52432
Netflow is preferred since lighter Transport UDP UDP
Some customers select syslog: Sequence

number
Yes in header No
existing collection infrastructure based Scalabilit High (tested) Need BPA

on syslog y
to guarantee multi-vendor interoperability
IPFIX doesnt bring anything to the CGN logging hence isnt considered
Both NFv9 and Syslog can be configured simultaneously in a CGN system
Syslog or Netflow v9 ?
Keep in mind before selecting your collector
Traditional use of NFv9 or syslog requires much lower data rates (< 50k fps)
NAT is still a relatively new application using NF hence there is no existing data
analysis tool box available
NAT requires the records to be stored in a Database
Most NF collectors store only the analysis results in a DB, but not the records
themselves and are therefore not suitable
Templates for
NAT44
NAT64SF
DS Lite
with or without
Bulk-Allocation
Destination-based-logging.
Syslog for CGN
Message needs to comply to RFC5424 format
Field are separated by space and non-applicable field are -
<Priority> <Version> <Time stamp> <host name> - - <Application name
(NAT44 or DSLITE)> - [Record 1][Record 2]
[EventName <L4> <Original Source IP> <Inside VRF Name>

<Original Source IPv6> < Translated Source IP> <Original Port>
<Translated First Source Port> <Translated Last Source Port>]
Example: NAT44 with Bulk-Port-Alloc
1 2011 May 31 10:30:45 192.168.2.3 - - NAT44 - [UserbasedA - 10.1.32.45 INVRFA - 100.1.1.28 -
12544 12671]
Netflow v9 for CGN
Netflow v9 supports flexible field definition
Light weight transport via UDP
NFv9 records are in binary
Based on templates containing IPFIX entities
(http://www.iana.org/assignments/ipfix/ipfix.xml)
Supported since the first days on CGN
Different behavior than Netflow on routers
Record creation / deletion of NAT entries
Doesnt count packets
Doesnt sample packets headers
Generated by the CGN card and not the MSC in the CRS or the LC in ASR9K
Netflow v9 templates for CGN
A few examples
Netflow Packet Generation
With default path MTU = 1500B, one netflow packet can hold around 50
creation records
Generation is handled at the CPU core level
An event (new translation or deletion of an existing one) will trigger the creation
of a NF packet but its not sent directly
If other events happen for the same core, records are added to the NFv9
packet
Packet is sent if we reach the MTU size or if we exceed one second
Configuring NFv9 Options
NFv9 is supported for all stateful translation protocol. Only a single server can be
defined for instance
Templates are regenerated and sent by default every 500 packets or 30 minutes
service cgn ISM
inside-vrf Inside-1
external-logging netflow version 9
server
address 1.2.3.4 port 123
path-mtu 2000
! can be configured from 100 to 2000
refresh-rate 100
! Regenerate NF record with template flowset every 100 logging packets
timeout 10
! Regenerate NF record with template flowset every 10 minutes
session-logging
! Session logging Enable Flag
!
HARDWARE
Service Cards on IOS XR Routers
Carrier Grade Service Engine (CGSE) for all CRS routers
CGSE-PLUS for CRS-3 and CRS-X routers
Integrated Service Module (ISM) for ASR9000 routers
Virtualized Service Module (VSM) for ASR9000 routers with RSP440
Same form-factor than any Line Card
No physical port / interfaces (except CGSE+ and VSM for future usage)
Multi-purpose cards, they can be used for different applications
Very similar to Intel server, they run a Linux distribution
Use virtual interfaces to communicate with the rest of the system
VSM introduces the Virtual Machines and the service chaining capability
Carrier Grade Service Engine (CGSE)
Supported with
CRS-1 / CRS-3 / CRS-X fabric
4-slot / 8-slot / 16-slot single/multi chassis
Up to 12 cards in the 16-slot chassis
Multi-purpose service card
CGN
Arbor TMS
Monte Vista Linux distribution but
configuration via IOS-XR
20M translations
1M sessions established per second
20Gbps
Carrier Grade Service Engine (CGSE)
GLIK M M
FPGA I
iPSE IngressQ I F
D D A
P P B
PLA L L R
A A I
N N
GLIK E EgressQ ePSE FabQs E
C
FPGA
CGSE PLIM MSC40/FP40

Paired with MSC40 or FP40.
Load-Balancing Traffic inside CGSE Cores
64 cores are available an each CGSE card (2^6) and LB

decision is performed by the egress PSE ASIC (eMetro)
For i2o traffic, the least 6 bits of the source IP address will be
used
For o2i traffic, the least 6 bits of the destination IP address will
be used.
It implies that we can not assign a map pool prefix longer
than /26 to use each core of the system:
/26: each core will handle a single IP address from the map pool
range (outside)
/24: each core will handle 4 IP addresses from the map pool range
(outside)
Carrier Grade Service Engine PLUS (CGSE+)
Supported with
CRS-3 / CRS-X fabric
8-slot / 16-slot single/multi chassis
Up to 12 cards in the 16-slot chassis
CGN
Arbor TMS (future)
DPI / Analytics (future)
Monte Vista Linux distribution but
Current supports: NAT44 / 6rd
80M translations
1M+ sessions established per second
70+ Gbps
Carrier Grade Service Engine PLUS (CGSE+)
DDR Netlogic
M M
16GB NPU I
iPSE IngressQ I F
D D A
Beluga P P B
L L R
PLA A A I
N N
DDR Netlogic EgressQ ePSE FabQs C
E E
16GB NPU
CGSE+ PLIM MSC140/FP140

Paired with MSC140 or FP140 in a CRS-3 or CRS-X chassis
Not supported in CRS-1 chassis
Integrated Service Module (ISM)
Supported with
RSP2 and RSP440
9006 and 9010 chassis (not in 9001 or 99xx)
CGN
CDS-IS/TV (discontinuated)
RedHat Linux distribution but
20M translations
1M sessions established per second
14Gbps
ISM Architecture
PPC
DRAM B
Bridge Bridge A
C
K
24GB I/O Fabric P
Hub ASIC L
A
Bridge Bridge N
E
24GB Intel CPU
Application Domain IOS-XR Domain

Virtualized Service Module (VSM)
Supported with
RSP440 (and future RSPs)
All 9x00 chassis except 9001
CGN
IPsec
Mobile GW
Service chaining
KVM virtualized environment
Current CGN Supports: NAT44
60M translations
10M+ sessions established per second
60Gbps
VSM Architecture
SFP+
SFP+ Quad Crypto/DPI XAUI
SFP+ PHY Assist
SFP+ PCIe
32GB Ivy
DDR3 Bridge Niantic
Crypto/DPI Typhoon Fabric B
Assist NPU ASIC 0 A
32GB Ivy Niantic C
DDR3 Bridge Niantic
48 K
32GB Niantic
ports P
Ivy L
DDR3
Bridge Niantic 10GE A
Crypto/DPI Typhoon Fabric N
Assist
NPU ASIC 1 E
32GB Ivy Niantic
DDR3
Bridge Niantic
Crypto/DPI
Assist
Application Processor Module (APM) Service Infra Module (SIM)

Performance / Scalability
Platform CGN Card
CGSE CGSE+ ISM VSM
CRS 4-slot 3 x CGSE or CGSE+
Sessions 20M 80M 20M 60M
Target: 80M+ CRS 8-slot 6 x CGSE or CGSE+
Establishment 1M/s 1M/s 1M/s Up to 13M/s CRS 16-slot 12 x CGSE or CGSE+
Rate
CRS Multi- Supported
Bandwidth 20Gbps 70Gbps 14Gbps 60Gbps Chassis since 4.3.1
(IMIX)
ASR9001 Not supported
Physical No 2x10G No 4x10G
Interfaces (future) (Future) ASR9006 3 x ISM or VSM
ASR9010 6 x ISM or VSM
ASR9922 VSM only
9k nV VSM is compatible
Satellite
9k nV VSM support
Cluster targeted for 5.2.0
DEPLOYMENT FEEDBACK
Deployment Tips
CGSE(+) PLIM are considered high powered PLIMs
Their power consumption is higher
But more important, they generate more heat than other PLIMs
(heat will naturally go up)
In 16-slot chassis, their position must be thought carefully
Some PLIMs are considered Thermally sensitive and upper
can not be positioned above high powered PLIMs: shelf
CRS-1 OC768 (C/L-band) DWDM PLIM
CRS-1 OC768 DPSK C/L-BAND STD CHAN PLIM
lower
So, CGSE should be positioned ideally in upper shelf shelf
If necessary, they can be positioned in lower shelf but
in that case its important to make sure another
high-powered PLIM is inserted above it in upper shelf.
Key Deployment Takeaways
Most majority of the ISM and CGSE deployments are done for
NAT44
6rd
Some new customers or customers with internal IPv4 shortage issues are now
looking at DS-Lite (and MAP)
MAP is interesting (stateless in the router / inline performance at 240G per card) but
not much CPE yet
DS-Lite is stateful (implies logging) but CPEs are very common
Many customers are testing NAT64 but some applications are not supported at
all on IPv6 (ex: Skype)
Logging
both syslog and netflow are used
Some customers using both simultaneously
Mobile are usually using far less ports (true for handheld, not for dongles)
Monitoring Options
Prime Performance Manager supports CGSE/ISM NAT44/NAT64 monitoring
Active Translation / Creating Rate
I2O and O2I Forward Rate
I2O Drop Port Limit Exceeded
I2O Drop System Limit Reached
Pool address totally free / used
Expect scripts can be used to collect counters from show commands

More scripts can be used to figure out the port user port usage
(very important to figure out the proper port-limit)
First, Get all IP outside addresses in use with a sh cgn nat44 NAT statistics
Then, for each IP address, run a sh cgn nat44 NAT outside-translation proto $Prot
outside-address $IP port start 1 end 65535 with $Prot: TCP/UDP/ICMP
Logs can be used to spot customers exceeding the limits
Scripts
Script will collect info on used ports per external address
RP/0/RSP0/CPU0:R1#sh cgn nat44 nat44 pool-utilization inside-vrf IN address-r$
Public address pool utilization details

----------------------------------------------- 1.2.3.16 65517 18
NAT44 instance : nat44 1.2.3.20 65535 0
VRF : IN 1.2.3.24 65534 1
----------------------------------------------- 1.2.3.28 65469 66
Outside Number Number 1.2.3.32 65522 13
Address of of 1.2.3.36 65530 5
Free ports Used ports 1.2.3.40 65529 6
----------------------------------------------- 1.2.3.48 65533 2
1.2.3.0 65528 7 1.2.3.52 65534 1
1.2.3.4 65525 10
1.2.3.8 65529 6
1.2.3.12 65533 2
Scripts
RP/0/RP0/CPU0:R1#sh cgn nat44 NAT-1 outside-translation protocol tcp outside-address
196.219.0.3 port start 1 end 65535
--------------------------------------------------------------------------------------------
Inside Protocol Inside Outside Translation Inside Outside
Packets Packets
--------------------------------------------------------------------------------------------
10.193.114.195 tcp 1114 46599 dynamic 110 129
10.193.114.195 tcp 1525 59248 dynamic 26 26
10.193.208.195 tcp 1691 54882 dynamic 6 4
10.193.114.195 tcp 1845 46393 dynamic 6 6
10.193.169.131 tcp 1980 63344 dynamic 12 21
10.193.248.131 tcp 2581 51821 dynamic 25 29
10.193.254.67 tcp 2873 1469 dynamic 12 15
10.193.117.67 tcp 2958 50417 dynamic 12 11
10.193.24.131 tcp 3016 50279 dynamic 8 8
10.193.247.3 tcp 3248 32869 dynamic 27 32
10.193.114.195 tcp 3479 58883 dynamic 29 28
10.193.114.195 tcp 3664 49916 dynamic 6 6
Scripts
10.193.114.195 10.193.24.131 46599 45 1 1
10.193.114.195 10.193.114.195 59248 57 32 1
10.193.208.195 10.193.114.195 54882 53 45 2
10.193.114.195 10.193.114.195 46393 45 49 2
Divide
10.193.169.131 10.193.114.195 63344 61 50 1
10.193.248.131 Sort 10.193.114.195 51821
by BPA 50
Count 53 1
10.193.117.67 10.193.117.67 1469 and 1 57 2
10.193.24.131 10.193.169.131 50417 round 49 61 1
10.193.247.3 10.193.208.195 50279 down 49
10.193.114.195 10.193.247.3 32869 32
10.193.114.195 10.193.248.131 58883 57
10.193.24.131 1
10.193.114.195 5
10.193.117.67 1
Per user port usage For a BPA=1024
10.193.169.131 1 - Top X users - Number of ports used
10.193.208.195 1 - Average per block ID BPA
10.193.247.3 1 - - Top X blocks tweaking
10.193.248.131 1 Port-limit - Average usage
tweaking -
Sizing the Port-Limit and BPA
No rule of thumb to define port-limit, BPA, timers
Example for a broadband ISP in LATAM (using a script)
18 ports average per user Can not be used to determine the best port-limit
i2o 50kpps per card
o2i 70kpps per card
Avg i2o packet size: 200B
Avg o2i packet size:1200B
percentage of users using less than X ports (starts at 99.8%)

Impact of CGN on Applications?
Several customers have been testing extensively the most popular applications
and successfully, for example:
TFTP, SSH, Telnet
IPSec VPN (Cisco Client), SSL VPN (AnyConnect Client)
HTTP/HTTPS on popular sites (CNN, Facebook, Youtube, Google services, )
WebMail (Java)
Skype, SkypeUpdate, Audio/Video/FileTransfer/Chat
MSN
Bit Torrent
Netflix
Video web sites like Crunchyroll.com, ign.com
iTunes store browsing, upgrade,
Sony Media Go Store
Steam Install and Update
StarCraft 2, World of Warcraft, MineCraft,
Application Impacted by CGN
A vast majority of users only need their internet connection for
Web surfing
Emails
Skype
Mobile Phone Apps on Wifi
Occasionally p2p download
These customers will never realize they are NATed
Per complaint behavior:
When customers are complaining about their connection (latency, applications not
working mainly for hardcore gamers who need to be a node for multiplayer games), the
ISP move them into a different VRF which is not NATed
Service Impacted by CGN
Geo-localization services
IP tracking services (advertisement system, not based on cookies)
ROUTING CONSIDERATION AND BEST PRACTICES
Types of Routing
Two types of routing should be differentiated
Intra-chassis routing
Packets candidate for translation or tunnel encapsulation/decapsulation, when
received on the router, should be forwarded to and from the CGN card
Static routes and Access-List Based Forwarding will be use
Extra-chassis routing
Packets should also be attracted by the CGN system able to handle them properly
Dynamic routing protocols (BGP or IGP) will be used to advertise the prefix
CGN Routing
ServiceApp1 ServiceApp2
CGN
Card
IPv4 Te0/1/0/0
IPv4
Te0/0/0/0
Backbone inside VRF outside VRF Internet
IGP
Static
Static
Intra-Chassis ABF IGP/BGP

Extra-Chassis
Intra-Chassis Routing
Aimed at forwarding packets candidate for translation or tunnel
encapsulation/decapsulation, to and from the CGN card
For i2o traffic, two methods available
Based on destination: static routes to the serviceApp interface
in the global table to the serviceApp
in the global table to the serviceApp in a named VRF
in a named VRF table to the serviceApp
should be advertised in IGP and/or iBGP
Based on source or destination: Access-list Based Forwarding
applied in ingress on the interface, could be VRF-aware or not
For o2i traffic
usually, we will rely on static routes to advertised a route back to the map pool
range into the outside serviceApp
should be advertised in external IGP or BGP
Extra-Chassis Routing
Its necessary to attract traffic to the CGNAT device and determine which traffic
is actually candidate to translation
Asymmetrical traffic is not possible with CGNAT routing, o2i must follow the
path of the i2o traffic
Thats why its mandatory to advertise the map pool ranges to the external
world to guarantee the symmetry
Some example:
Default Map pool
NAT Core
Access Internet
BNG CGN Public IP
Extra-Chassis Routing
A few other examples
Default Map pool Aggregate
Full Table
NAT
Core Peering
Private IP CGN Network Internet
Default Map pool

Full Table
NAT
L3VPN
Internet
VRF CGN
NAT44 Static Route Configuration
Create one static route in each VRF (inside and outside)
All packets arriving in vrf inside should be directed to the CGN card through the
serviceApp1 interface
All packets arriving in vrf outside and targeted to addresses in the map pool
range should be directed to the serviceApp2 interface
RP/0/RSP0/CPU0:router(config)# Translate to
router static 100.0.0.0/24
vrf inside ServiceApp1 ServiceApp2

address-family ipv4 unicast CGN
0.0.0.0/0 ServiceApp1 Card
! Te0/1/0/0
Te0/0/0/0
vrf outside inside VRF outside VRF
address-family ipv4 unicast
100.0.0.0/24 ServiceApp2
!
NAT44 Static Route Configuration
In many situations, physical interfaces can not be in a inside VRF but must
be in the global routing table
We could simply use a static default in the global ipv4 table pointing to
serviceApp in the inside VRF, but a global default route is not recommended:
ALL traffic with no route in the RIB will be attracted
if the router has a full BGP table, no packets will be routed to serviceApp1
Translate to
100.0.0.0/24
RP/0/RSP0/CPU0:router(config)# ServiceApp1 ServiceApp2

router static
CGN
address-family ipv4 unicast Card
0.0.0.0/0 vrf inside ServiceApp1 Te0/0/0/0 Te0/1/0/0
inside VRF outside VRF
!
NAT44 ABF Configuration
Routing based on ACL enables decision based on source addresses
Public sources can avoid NAT // Private can be sent for NAT translation
RP/0/RSP0/CPU0:router(config)#
ipv4 access-list ABF
10 permit ipv4 10.0.0.0 0.255.255.255 any nexthop1 vrf inside ipv4 1.1.1.2
20 permit ipv4 any any
interface ServiceApp1 interface ServiceApp2
vrf inside vrf outside
ipv4 address 1.1.1.1/30 ipv4 address 2.1.1.1/30
service cgn demo service-type nat44 Translate to
service cgn demo service-type nat44
! 100.0.0.0/24 !
interface TenGigE0/0/0/0 interface TenGigE0/1/0/0
ipv4 address 20.1.1.1/24 ServiceApp1 ServiceApp2 ipv4 address 30.1.1.1/24
ipv4 access-group ABF ingress CGN
Card
Te0/0/0/0 inside outside Te0/1/0/0
10.0.0.0/8
VRF VRF
30.0.0.0/8
NAT44 ABF Configuration
Return traffic
When you configure ABF for the i2o traffic, you dont need to do it for the o2i traffic
o2i traffic must be routed to the correct Inside (default) VRF when it comes out of
the Inside Service App
RP/0/RSP0/CPU0:router(config)# RP/0/RSP0/CPU0:router(config)#
router static router static
vrf inside address-family ipv4 unicast
address-family ipv4 unicast 100.0.0.0/24 vrf outside serviceApp2
10.0.0.0/8 vrf default 20.1.1.2 Translate to
100.0.0.0/24
CGN
20.1.1.2
Card
10.0.0.0/8
VRF VRF
30.0.0.0/8
NAT44 ABF Limitations
What if the next-hop address in GRT isnt reachable (interface down for example)?
100.0.0.0/24
CGN
20.1.1.2
Card
10.0.0.0/8
VRF VRF
30.0.0.0/8
Even if another path is available to reach 10.0.0.0/8 in the GRT, traffic is lost
What if the next-hop router points to the CGN router to reach 10.0.0.0/8?
100.0.0.0/24
CGN
20.1.1.2
Card
10.0.0.0/8
VRF VRF
30.0.0.0/8
In this case, the traffic will eventually find its way to 10.0.0.0/8 but via a sub-
optimal path
ABF is performed before MPLS labels are stripped from packets
Consequently, packets are not matched
Example, the CGN in PE case
Workaround: loop fiber
0/0/CPU0 Translate to
151.0.0.0/24
VRF SA1 CGN SA2

Global
Inside-1 251.5 Card 250.5
Te0/6/0/2
P
151.0.1.0/24
VRF SA3 CGN SA4

Global
2 Labels 1 Label
Transport VRF PE
VRF
Other example, the CSC case (CGN in CE)
151.0.0.0/24
VRF SA1 CGN SA2

Global
Te0/6/0/2
P PE
151.0.1.0/24
VRF SA3 CGN SA4

Global
3 Labels 2 Labels 1 Label

Transport VRF CSC CE
VRF CSC
CSC
REDUNDANCY
CGSE/ISM Redundancy
On both CRS/CGSE and ASR9000/ISM, we support 1:1 warm standby
redundancy (not supported on CGSE+ today)
Warm-standby
translation state is not synchronized between active and standby, all connections
will be re-established
Pros: simple to configure, a single map pool is used
Cons: only 1:1, one card on two will not be used 99% of the time
An alternative with ABF is available
Pros: offers more options like n:1 redundancy, converges very quickly
Cons: we can not re-use the same map pool range, so we need to configure a second
range
1:1 Warm-Standby Redundancy
Configuration
RP/0/RSP0/CPU0:CGN(config)#
service cgn demo
service-location preferred-active 0/1/CPU0 preferred-standby 0/3/CPU0
RP/0/RP0/CPU0:CGN#show services redundancy

Service type Name Pref. Active Pref. Standby
--------------------------------------------------------------------------------
ServiceInfra ServiceInfra1 0/1/CPU0 Active
ServiceInfra ServiceInfra2 0/3/CPU0 Active
ServiceCgn demo 0/3/CPU0 Standby 0/1/CPU0 Active
RP/0/RP0/CPU0:CGN#
CGSE/ISM Redundancy
service cgn mets-cgn Translate to

151.0.0.0/24
service-location preferred-active 0/1/CPU0
service-type nat44 nat44-1 VRF SA1 CGN SA2
inside-vrf Inside-1 Global
!
service cgn mets-cgn-2 Translate to
service-location preferred-active 0/3/CPU0 151.0.1.0/24
service-type nat44 nat44-2 SA3 SA4
inside-vrf Inside-2 Te0/6/0/2 VRF CGN Te0/6/0/3
Global
map address-pool 151.0.1.0/24 10.1.1.1/24 Inside-2 51.5 Card 52.5 100.1.1.1/24
!
service cgn mets-cgn-backup
service-location preferred-active 0/7/CPU0 Translate to
service-type nat44 nat44-backup 151.0.2.0/24
inside-vrf iBackUp VRF SA5 CGN SA6

map address-pool 151.0.2.0/24 Global
iBackUp 53.5 Card 54.5
CGSE/ISM n:1 Redundancy
10 permit ipv4 10.2.0.0/24 any nexthop1 vrf Inside-1 ipv4 192.168.251.6 nexthop2 vrf iBackUp ipv4 192.168.53.6
!
router static
110.1.0.0/16 100.1.1.2 description Ixia-i2o-Default
151.0.0.0/24 ServiceApp2 description Ixia-o2i-ABF
Translate to
151.0.0.0/24
SA1 CGN SA2 Packets sourced

VRF VRF
Inside-1 from 150.0.0.x
251.5 Card 250.5 Outside-1
10.2.0.0/24 Translate to
151.0.1.0/24
Te0/6/0/2 VRF SA3 CGN SA4 VRF Te0/6/0/3 110.1.0.0/16

10.1.1.1/24 Inside-2 51.5 Card 52.5 Outside-2 100.1.1.1/24
10.2.1.0/24
Translate to
151.0.2.0/24
VRF SA5 CGN SA6

Default
CGSE/ISM n:1 Redundancy
!
router static
110.1.0.0/16 100.1.1.2 description Ixia-i2o-Default
Translate to
151.0.0.0/24
VRF SA1 CGN SA2 VRF

Inside-1 251.5 Card 250.5 Outside-1
10.2.0.0/24 Translate to
151.0.1.0/24
Te0/6/0/2 VRF SA3 CGN SA4 VRF Te0/6/0/3 110.1.0.0/16

10.1.1.1/24 Inside-2 51.5 Card 52.5 Outside-2 100.1.1.1/24
10.2.1.0/24
Translate to
151.0.2.0/24
VRF SA5 CGN SA6 Packets sourced

Default
iBackUp 53.5 Card 54.5 from 150.0.2.x
CGSE/ISM n:1 Redundancy Limitations
1:1 warm standby N:1 ABF based
redundancy redundancy
Convergence time Up to 7s <1s
Needs a standby card for Needs only a single backup
CAPEX
every active one card per router
Impact on other resources No map pool necessary for No map pool necessary for
(address map pools) the backup card the backup card
Preemption when the The initial active card regains
No preemption, the new active
first card gets back card stays active
the active role and create a 2nd
online impact
No problem, the standby Since the backup card uses a
Static port forwarding re-populates the table with the different map pool, a new static
static entry entry will be created
Extra-Chassis Redundancy
ipv4 access-list ABF-1 If routers are not directly
10 permit ipv4 any any nexthop1 vrf Inside-1 ipv4 192.168.251.6
nexthop2 vrf Inside-2 ipv4 192.168.51.6 nexthop3 ipv4 10.10.1.1 connected, a GRE tunnel can be
used to avoid routing loops
151.0.0.0/24
VRF SA1 CGN SA2

Global
Te0/6/0/2 Te0/6/0/3
10.1.1.1/24 0/1/CPU0 Translate to
100.1.1.1/24
151.0.1.0/24
VRF SA3 CGN SA4

Global
151.0.2.0/24
Te0/0/0/0 SA5 SA6 Te0/0/0/1
VRF CGN
10.10.1.1/24 Global 100.1.2.1/24
Logging Redundancy
CGN cards are generating syslog and NFv9 on UDP
No mean to send backpressure if the server cant cope
One single destination per type and inside-VRF
Workarounds exist at the collector level:
Virtual IP addresses on the collector
Port SPAN on the switch were is connected the collector to replicate the logging flow
(second server needs some tweaking to accept the trafffic)
Directed-Broadcast on the last router (ex: the last interface is 10.100.1.1/30 and we will
generate the logging traffic to 10.100.1.4, the broadcast address of this network. Only
10.100.1.0/24 will be advertised in IGP)
RAID / DB redundancy is highly recommended at the server level
CONCLUSION
Conclusion
CGN offers tools to buy time for your IPv6 preparation
The same line cards can also be used for IPv6 migration (NAT64, 6rd,
DS-lite)
For the vast majority of usages: it just works
Deployment must be considered carefully for
Routing
Logging infrastructure for collection and storage
Timers, BPA, Port-Limit,
Complete Your Online Session Evaluation
Complete your online session
evaluation
Complete four session evaluations
and the overall conference evaluation
to receive your Cisco Live T-shirt
BACKUP SLIDES
UNDERSTANDING TIMERS
Stateful Protocols
Understanding the Stateful Translation
NAT44 (like NAT64SF and DS Lite) performs a stateful translation

Packet source address and port are rewritten
Details are stored in a translation database
A new packet from inside to outside will create a new entry in the table
No activity during a configurable period of time will trigger the suppression of
this entry
We use different timers for different packet types and different situations
NAT44: TCP Establishment
Source Address = 10.1.1.1 IPv4 Traffic Outside Address IPv4
NAT
from pool = 99.0.0.1 Internet
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

Dst: 30.0.0.1:80 Inside Outside TCP Dst: 30.0.0.1:80
0 state
1 SYN
Inside Outside TCP
2 state
10.1.1.1:12345 99.0.0.1:1025 Inactive 3
SYN/ACK
Inside Outside TCP
4 state
5 10.1.1.1:12345 99.0.0.1:1025 Active

ACK
Now, as long as TCP traffic is received in any direction within the active timer,
state is maintained as Active. This behavior can be changed by configuration,
considering only the i2o traffic to refresh the timers.
NAT44: End of TCP Session
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0 state
10.1.1.1:12345 99.0.0.1:1025 Active
FIN or RST 1
Inside Outside TCP
2 state
10.1.1.1:12345 99.0.0.1:1025 Inactive
3 ACK
3 Initial timer expires
DB is cleaned up
Default timers: Inside Outside TCP
4
TCP init: 120s state
10.1.1.1:12345 99.0.0.1:1025 Inactive
Note: We are not checking the sequence numbers in the NAT engine.
NAT44: TCP Initial Timeout
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0 state
1 SYN
Inside Outside TCP
2 state
10.1.1.1:12345 99.0.0.1:1025 Inactive

DB is cleaned up
Default timers: Inside Outside TCP
4
TCP init: 120s state
10.1.1.1:12345 99.0.0.1:1025 Inactive
Note: we are checking all timers every 10ms to clean up the time-outs
NAT44: TCP Active Timeout
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0 state
10.1.1.1:12345 99.0.0.1:1025 Active
No traffic matching the DB entry flows through the system

DB is cleaned up
Default timers: 2
Inside Outside TCP
state
TCP active: 1800s
10.1.1.1:12345 99.0.0.1:1025 Inactive
Note: We are not sending any FIN/RST to either side (inside nor outside), the
translation entry is simply removed from the table.
NAT44: Security Behavior
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0 state
If we send TCP data packet before a complete TCP handshake

2
1 TCP Data
this packet is considered invalid and

dropped without ICMP being generated.
NAT44: Security Behavior
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0 state
1 SYN
Inside Outside TCP
2 state
10.1.1.1:12345 99.0.0.1:1025 Inactive
If we receive a TCP data packet before a complete TCP handshake
TCP Data 3
Inside Outside TCP

4 state
10.1.1.1:12345 99.0.0.1:1025 Inactive
this packet is translated back and passed to the host, but table state isnt
changed from Inactive to Active. It stays at Inactive.
NAT44: UDP Packets
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

Dst: 30.0.0.1:80 Inside Outside UDP state Dst: 30.0.0.1:80
0
1 UDP
Inside Outside UDP state
2
10.1.1.1:12345 99.0.0.1:1025 Inactive
UDP 3
Inside Outside UDP state

4
10.1.1.1:12345 99.0.0.1:1025 Active
Now, as long as UDP traffic is received in any direction within the active timer,
state is maintained as Active.
NAT44: UDP Timeout Case 1
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0
10.1.1.1:12345 99.0.0.1:1025 Inactive
0 UDP
Only I2O traffic passes through CGN, UDP state is Inactive
1 Now, no more I2O UDP traffic is received
Default timers:
UDP init: 30s Initial timer expires
2 DB is cleaned up
Inside Outside UDP
4 state
10.1.1.1:12345 99.0.0.1:1025 Inactive
NAT44: UDP Timeout Case 2
NAT
30.0.0.1
Src: 10.1.1.1:12345 CGN Src: 99.0.0.1:1025

0
10.1.1.1:12345 99.0.0.1:1025 Active
0 UDP
UDP 0
1 Now, both I2O and O2I UDP stop flowing through the CGN
Default timers:
UDP active: 120s Initial timer expires
2 DB is cleaned up
Inside Outside UDP
4 state
10.1.1.1:12345 99.0.0.1:1025 Active
NAT44: ICMP
NAT
30.0.0.1
Src: 10.1.1.1 CGN Src: 99.0.0.1

Dst: 30.0.0.1 NAT Dst: 30.0.0.1
0 Info
1 ICMP
NAT
No state in ICMP translation 2 Info Only a DB entry.
10.1.1.1 99.0.0.1 ICMP
ICMP 3
Now, as long as ICMP traffic is received in any direction within the

timer, this entry will be maintained in the DB.
NAT44: ICMP Timeout Case
NAT
30.0.0.1
Src: 10.1.1.1 CGN Src: 99.0.0.1

Dst: 30.0.0.1 NAT Dst: 30.0.0.1
0 Info
1 ICMP
NAT
2 Info
10.1.1.1 99.0.0.1 ICMP
Now, no more I2O and O2I ICMP flow through the CGN
Default timers:
ICMP: 60s ICMP timer expires
3 DB is cleaned up
NAT
4 Info
10.1.1.1 99.0.0.1 ICMP
Fine Tuning Timers
For stateful translation protocols (NAT44, NAT64 SF, DS Lite), the NAT DB
maintains timers for each entry
service cgn demo service cgn demo service cgn demo
service-type nat44 nat44-1 service-type nat64 stateful nat64-1 service-type ds-lite ds-lite1
protocol udp protocol udp protocol udp
session initial timeout 10 timeout 30 session active timeout 30
session active timeout 30 v4-init-timeout 10 session init timeout 10
protocol tcp protocol tcp protocol tcp
session initial timeout 30 session initial timeout 30 session active timeout 120
session active timeout 120 session active timeout 120 session init timeout 30
protocol icmp protocol icmp protocol icmp
timeout 30 timeout 30 timeout 30
Default Initial Active

TCP 120s 1800s
UDP 30s 120s
ICMP 60s
Refresh Direction
Timers are refreshed when packets are translated in i2o or o2i direction.
But an external attacker could send regularly one packet for every DB entry
and eventually create a resource depletion
To change this default behavior, we can make the timer refresh to only take
into consideration Inside-to-Outside (i2o) packets
This feature is not available for DS Lite
service cgn POC-1
refresh-direction Outbound
!
service-type nat64 stateful nat64-1
refresh-direction Outbound
!
BACKUP SLIDES
LOAD BALANCING
Load-balancing Traffic Between CGSEs
BRIDGE iPSE IngressQ At egress PSE level:
PLA Hashing on source
BRIDGE Egress ePSE FabQs address to loadbalance
Q
traffic between 64 cores
M M F
I I A
BRIDGE D iPSE IngressQ D
P P B
PLA L L R At ingress PSE level:
A A
BRIDGE N Egress
Q
ePSE FabQs N I Two static routes for one
E E C NH address pointing to two
SPA serviceApps interfaces (L3
SPA PLA iPSE IngressQ or L4 LB is used depending
SPA on the configuration)
SPA
Egress ePSE FabQs ABF is possible too and is a
SPA PLA Q
SPA better option.
Note: using static routes will break the principle of same external IP address mapping for all sessions
associated with the same internal IP address (RFC4787) we recommend ACL Based Forwarding.
RP/0/RP0/CPU0:router(config)#
DDR Netlogic router static
16GB NPU iPS IngressQ vrf inside
E
PLA 0.0.0.0/0 ServiceApp11 192.168.11.2
DDR Netlogic 0.0.0.0/0 ServiceApp21 192.168.21.2
Egress ePSE FabQs 0.0.0.0/0 ServiceApp21
16GB NPU Q
192.168.21.3
192.168.21.4
M M F 0.0.0.0/0 ServiceApp21
I I A
BRIDGE iPS IngressQ 192.168.21.5
D E D
!
P P B vrf outside
PLA L L R address-family ipv4 unicast
A A
Egress ePSE FabQs 100.0.0.0/24Translate
ServiceApp12
to
BRIDGE N Q N I 100.1.0.0/16100.0.0.0/24
ServiceApp22
E E C
SPA 192.168.11.1/2
CGSE 192.168.12.1/2
SPA PLA 4 4
iPS IngressQ
E inside VRF outside VRF
SPA
SPA CGSE
Egress ePSE FabQs ServiceApp22
SPA PLA Q
ServiceApp21 PLUS
192.168.22.1/2
192.168.21.1/2
SPA 4 4
Translate to
100.1.0.0/16
DDR Netlogic + ACL definition here
16GB NPU iPS IngressQ + ABF applied on ingress interface here
E
!
PLA vrf outside
DDR Netlogic address-family ipv4 unicast
Egress ePSE FabQs 100.0.0.0/24 ServiceApp12
16GB NPU Q
M M F
I I A
BRIDGE D iPS IngressQ D
E
P P B
PLA L L R
A A
BRIDGE Egress ePSE FabQs I Translate to
N Q N 100.0.0.0/24
E E C
SPA 192.168.11.1/2
CGSE 192.168.12.1/2
SPA PLA 4 4
iPS IngressQ
E inside VRF outside VRF
SPA
SPA CGSE
Egress ePSE FabQs ServiceApp22
SPA PLA Q
ServiceApp21 PLUS
192.168.22.1/2
192.168.21.1/2
SPA 4 4
Translate to
100.1.0.0/16
Load-balancing Traffic inside ISM
Based on the number of cores, we cant allocate a range
24Gb more specific than /30 (4 public addresses)
Load-balancing is different on the ISM than CGSE:
First, its performed by the ingress NPU (Trident or Typhoon on in
the ingress card) where lookup is performed and a VQI is
assigned for the destination
24Gb Each VQI is linked to a particular Niantic port, hence to a
particular dispatcher process on a CPU.
(2 CPUs, 2 dispatchers running on 2 different ports 4 options).
Second, the dispatcher process will determine which CGv6
application process should be handle this packet:
- i2o traffic: hash is performed on the source address 32 bits
- o2i traffic: hash is performed on the destination address 32 bits
For DS-Lite, hash will be done on the B4 ipv6 address for i2o
traffic and on the destination ipv4 address for o2i traffic.
BACKUP SLIDES
NAT CONFIGURATION
Virtual Service Interfaces
Interconnecting CGSE/ISM card to the rest of the system
Configuration is only needed on the router/XR side, addresses on the
CGN/Linux side will be automatically created
To direct traffic into the CGN card, well need one or several of these options:
static routes ServiceInfra1
redistribution
ACL based forwarding rules ServiceApp1
CGN
ServiceApp2
ServiceInfra interface Physical

Card
Physical
VRF VRF
For CGN card management Interface
or or
Interface
One per card mandatory VLAN

address-family address-family
VLAN
ServiceApp interfaces
To interconnect GRT address-family
or VRF inside and outside to the CGN card
NAT44 Configuration
To avoid routing loops, VRF are mandatory with NAT44
Inside VRF must be non-default
Outside VRF is optional, we can use the Default or a named VRF
RP/0/RSP0/CPU0:Router(config)# interface ServiceApp1
vrf inside vrf inside
address-family ipv4 unicast ipv4 address 1.1.1.1 255.255.255.252
! service cgn demo service-type nat44
vrf outside !
address-family ipv4 unicast interface ServiceApp2
! vrf outside
interface te0/0/0/0 ipv4 address 2.1.1.1 255.255.255.252
vrf inside service cgn demo service-type nat44
ipv4 add 10.1.1.1/24
!
interface te0/1/0/0
vrf outside
CGN
ipv4 add 100.1.1.1/24 Card
! Te0/0/0/0 Te0/1/0/0
inside VRF outside VRF
NAT44 Configuration
Create a nat44 instance nat1 and associate an outside pool (Public IPv4
addresses) to a given inside VRF
A single nat44 instance can be created per CGN card
Several mechanisms exist to push traffic in2out into ServiceApp1
A static route with the map pool range will be necessary to send out2in traffic to
the CGN card via ServiceApp2
Translate to
service cgn demo 100.0.0.0/24
service-type nat44 nat1 ServiceApp1 ServiceApp2

inside-vrf inside CGN
map address-pool 100.0.0.0/24 Card
! Mapping to the default VRF in public side outside VRF
Inside VRF or
Default
service cgn demo
service-type nat44 nat1
inside-vrf inside
map outside-vrf outside address-pool 100.0.0.0/24
! Mapping to the VRF outside in public side
NAT44 Configuration Tips
In current XR release, we can not configure two map pools under one VRF
inside (coming in the near future)
RP/0/RP0/CPU0:Router(config-cgn-invrf)#show
Fri Jun 15 16:54:52.430 PDT
service cgn demo
inside-vrf Inside-2
!
RP/0/RP0/CPU0:Router(config-cgn-invrf)#map address-pool 151.0.1.0/24
Fri Jun 15 16:56:23.669 PDT
service cgn demo
inside-vrf Inside-2
!
RP/0/RP0/CPU0:Router(config-cgn-invrf)#
NAT44 Configuration Tips
To overcome this limit we can configure several inside VRFs:
Fri Jun 15 16:54:52.430 PDT
service cgn demo
inside-vrf Inside-1
!
inside-vrf Inside-2
!
RP/0/RP0/CPU0:Router(config-cgn-invrf)#
Challenge will now reside in directing the traffic to both inside VRF
Total of all map pools can not be larger than 65535 addresses
It doesnt need to be into a single /16 or contiguous ranges
NAT44 Show Commands
RP/0/RP0/CPU0:Router#show cgn demo stat sum
Statistics summary of NAT44 instance: demo' Translation entries allocated in DB

Number of active translations: 2250000
Number of sessions: 11500028
Translations create rate: 0 Additional flows inside
Translations delete rate: 0
Inside to outside forward rate: 12600 these translations
Outside to inside forward rate: 0
Inside to outside drops port limit exceeded: 0 Rate in sessions per second
Inside to outside drops system limit reached: 0
Inside to outside drops resorce depletion: 0
No translation entry drops: 0 Rate in packets per second
PPTP active tunnels: 0
PPTP active channels: 0
PPTP ctrl message drops: 0
Number of subscribers: 0 Packets dropped because of
Drops due to session db limit exceeded: 0 port-limit for inside user is reached
Pool address totally free: 25268
Pool address used: 7500 Packets discarded because we
-------------------------------------------------
External Address Ports Used reached the limit of 20M sessions
------------------------------------------------- or 1M internal users
160.0.0.8 300
160.0.0.36 300
160.0.0.52 300 Packets dropped because no public
L4 Port could be allocated
NAT44 Show Commands
RP/0/RP0/CPU0:Router#show cgn demo stat sum
out2in drops because of
Statistics summary of NAT44 instance: demo' no entry in the translation DB
Number of active translations: 2250000
Number of sessions: 11500028
Translations create rate: 0 PPTP/GRE sessions/tunnels info
Translations delete rate: 0
Inside to outside forward rate: 12600
Outside to inside forward rate: 0 Private addresses having at
Inside to outside drops port limit exceeded: 0
Inside to outside drops system limit reached: 0
least one active translation
Inside to outside drops resorce depletion: 0 Packets dropped after
No translation entry drops: 0
PPTP active tunnels: 0 exceeding the 20M sessions
PPTP active channels: 0
PPTP ctrl message drops: 0 Addresses available in the pool
Number of subscribers: 0
Drops due to session db limit exceeded: 0
Addresses used in the pool
Pool address totally free: 25268
Pool address used: 7500
------------------------------------------------- External addresses
External Address Ports Used
------------------------------------------------- and ports allocated
160.0.0.8 300
160.0.0.36 300
160.0.0.52 300

NAT44 Show Commands
Pool utilization statistics
RP/0/RP0/CPU0:Router#show cgn demo pool-utilization inside-vrf Inside address-range 100.0.0.90 100.0.0.95
Public address pool utilization details

-------------------------------------------------------
CGN instance : demo
VRF : Inside
-------------------------------------------------------
Outside Number Number
Address of of
Free ports Used ports
-------------------------------------------------------
100.0.0.90 64512 0
100.0.0.91 64512 0
100.0.0.92 63139 1373
100.0.0.93 63138 1374
100.0.0.94 64512 0
100.0.0.95 64512 0
...
NAT44 Show Commands
Translation statistics from an inside address perspective
RP/0/RP0/CPU0:router#sh cgn demo inside-translation protocol tcp inside-vrf Inside inside-address 10.12.0.29
port start 1 end 65535
---------------------------
CGN instance : demo
Inside-VRF : Inside
--------------------------------------------------------------------------------------------
Packets Packets
--------------------------------------------------------------------------------------------
100.0.0.93 tcp 1405 58529 dynamic 7 4
100.0.0.93 tcp 1406 34188 dynamic 7 4
100.0.0.93 tcp 1407 41851 dynamic 7 4
100.0.0.93 tcp 2156 38317 dynamic 7 4
100.0.0.93 tcp 2157 30504 dynamic 7 4
100.0.0.93 tcp 2158 40039 dynamic 7 4
100.0.0.93 tcp 2907 42745 dynamic 7 4
...
NAT44 Show Commands
Translation statistics from an outside address perspective
RP/0/RP0/CPU0:router#sh cgn demo outside-translation protocol tcp outside-vrf Outside outside-address
100.0.0.93 port start 1024 end 65535
Outside-translation details
---------------------------
CGN instance : demo
Outside-VRF : Outside
--------------------------------------------------------------------------------------------
Inside Protocol Outside Inside Translation Inside Outside
Address Destination Destination Type to to
Packets Packets
--------------------------------------------------------------------------------------------
10.12.0.221 tcp 1032 56742 dynamic 7 4
10.12.0.157 tcp 1033 43804 dynamic 7 4
10.12.0.157 tcp 1055 54299 dynamic 7 4
10.12.0.157 tcp 1206 41550 dynamic 7 4
10.12.0.157 tcp 1274 64801 dynamic 7 4
10.12.0.221 tcp 1306 10243 dynamic 7 4
10.12.0.221 tcp 1359 8738 dynamic 7 4
...
BACKUP SLIDES
CONFIGURATION AND TROUBLESHOOTING TIPS
Protecting ServiceInfra Interface w/ an ACL
ServiceInfra interfaces are virtual tunnels between the router and the CGN
card and are mandatory to boot and manage it
Even if the prefix used for this card isnt supposed to be advertised outside of
the router, its recommended to configure a filter to protect it from potential
DoS attack
ipv4 access-list ServiceInfraFilter

100 permit ipv4 host 1.1.1.1 any
101 permit ipv4 host 1.1.1.2 any
!
interface ServiceInfra1
ipv4 address 1.1.1.1 255.255.255.0 service-location 0/0/CPU0
ipv4 access-group ServiceInfraFilter egress
!
Sending Logging Reports in a VRF
ServiceInfra interfaces are part of the global routing table and they are the
source interfaces for syslog or netflow messages. If the collector is located in
the Inside VRF, its not possible to send it any reports by default
We need to use ABF to overcome this limitation
interface GigabitEthernet0/3/1/0
vrf Inside
ipv4 address 10.1.0.1 255.255.255.0
!
service cgn cgn1
service-location preferred-active 0/0/CPU0 preferred-standby 0/2/CPU0
service-type nat44 NAT44
inside-vrf Inside
external-logging syslog
server
address 10.1.0.3 port 3000
session-logging
Sending Logging Reports in a VRF
We define and apply an ABF on the serviceInfra interface
ipv4 access-list acl1
10 permit udp 101.100.11.0/24 host 10.1.0.3 nexthop1 vrf Inside
!
interface ServiceInfra2
ipv4 address 101.100.11.1 255.255.255.0
service-location 0/2/CPU0
ipv4 access-group acl1 ingress
!
!
router static
vrf Inside
10.1.0.3/32 GigabitEthernet0/3/1/0
!
!
Dynamic Port Range
For stateful translation protocols, the dynamic translations start from 1024. We
can change this starting value from 1 to 65535
service cgn POC-1
dynamic-port-range start 2000
!
ICMP Rate-Limiting
We can define an ICMP rate-limiter for CGN card (ISM, CGSE)
For CRS/CGSE: should be a multiple of 64, less than 65472
For ASR9K/ISM: should be a multiple of 8, less than 8184
It can be 0 (zero)
service cgn ISM

protocol icmp
rate-limit 8184
!
!
Using these Features Creatively
How to reduce the number of users per external address?
A customer requested to limit the number of internal users allowed to used
each external addresses of their map pool. Only for NAT44 (no dynamic-range
config in DS-Lite)
Step 1: define port-limit and bulk-port-range to the same value.
Ex: 4096 ports: rounddown[(65535-1024)/4096]=15 potential inside addresses for each
external address
Ex: 2048 ports: rounddown[(65535-1024)/2048]=31
BPA=1024 63
BPA=512 126,
Step 2: if we need to reduce the number of users to something smaller than 15,
let define the dynamic-port-range to an higher value
Ex: BPA/port-limit=4096, dynamic-range start=24575
rounddown[(65535-24575)/4096]=10
Changing Logging DSCP Marketing
Not possible to change the DSCP marking of syslog or netflow packets
generated by ISM or CGSE card. But a remarking can be done at the egress
interface level with the proper QoS policy
RP/0/RP1/CPU0:Yanks#show policy-map interface gig 0/6/3/0.2 RP/0/RP1/CPU0:Yanks#sh run policy-map
GigabitEthernet0/6/3/0.2 direction input: Service Policy not installed Wed Sep 5 03:46:20.324 PDT
GigabitEthernet0/6/3/0.2 output: NF policy-map NF
Class NF class NF
Classification statistics (packets/bytes) (rate - kbps) set dscp cs5
Matched : 37991/53199036 838 !
Transmitted : 37991/53199036 838 class class-default
Total Dropped : 0/0 0 !
Queueing statistics end-policy-map
Queue ID : 23 !
Taildropped(packets/bytes) : 0/0
Class class-default
Classification statistics (packets/bytes) (rate - kbps)
Matched : 0/0 0
Transmitted : 0/0 0
Total Dropped : 0/0 0
Queueing statistics
Queue ID : 23
High watermark (bytes)/(ms) : 0/0
Inst-queue-len (bytes)/(ms) : 0/0
Avg-queue-len (bytes)/(ms) : 0/0
Taildropped(packets/bytes) : 0/0
Changing Logging DSCP Marketing
Syslog / CS5
NetFlow v9 / CS5
Troubleshooting Tips
Makes sure the traffic is indeed pushed to and from the CGN cards
Show interface serviceApp * is always expressed from the router
perspective, so
Pkts out: going into the CGN cards
Pkts in: coming from the CGN cards into the router
RP/0/RSP0/CPU0:Nets#sh int serviceapp * accounting

ServiceApp1
Protocol Pkts In Chars In Pkts Out Chars Out
IPV4_UNICAST 2810763 348534612 37102220124 37515911766210
ServiceApp2
Protocol Pkts In Chars In Pkts Out Chars Out
IPV4_UNICAST 36742436201 37162233422198 0 0
RP/0/RSP0/CPU0:Nets#
We can use show interface serviceApp * accounting rates to get some trends
on the traffics going through the system
When using ABF: configure interface TenGigE0/0/5/0
vrf LOOPBACK
hardware count in ABF in order to ipv4 address 12.1.7.10 255.255.255.0
load-interval 30
see ABF match statistics ipv4 access-group ABF ingress hardware-count
!
You should see Hits increase as
RP/0/RP0/CPU0:router#show access-lists ABF hardware
ingress traffic is directed to ingress detail location 0/0/CPU0
ServiceApp NH ACL name: ABF
Sequence Number: 10
Grant: permit
Logging: OFF
Per ace icmp: ON
Next Hop Enable: ON
VRF Table Id: 4096
Next-hop: 1.1.1.2
Default Next Hop: OFF
Hits: 4063640803
Statistics pointer: 0x7ff5f
Number of TCAM entries: 1
Troubleshooting Tips on ISM
Be extra careful with the unix level commands, one is very useful though:
RP/0/RSP0/CPU0:BNG#run attach 0/5/cpu0
Sat Dec 22 06:33:02.403 UTC
attach: Starting session 1 to node 0/5/cpu0

#
#
# show_nat44_stats
CORE-ID #SESSIONS(%UTIL) #USERS(%UTIL)

------------------------------------------------------------------------
0 563100(19.6%) 1877(1.43%)
1 561000(19.5%) 1870(1.43%)
2 563400(19.6%) 1878(1.43%)
3 562500(19.6%) 1875(1.43%)
4 0(0.0%) 0(0.00%)
5 0(0.0%) 0(0.00%)
6 0(0.0%) 0(0.00%)
7 0(0.0%) 0(0.00%)
------------------------------------------------------------------------
Total Sessions: 2250000 Total users: 7500
Main DB size is 2875008 and User DB size is 131072
#exit
RP/0/RSP0/CPU0:BNG#
Troubleshooting Tips on CGSE
# show_nat44_stats
31 41428(11.5%) 5081(31.01%)
CORE ID #SESSIONS(UTIL) #USERS(UTIL) 32 43292(12.0%) 5028(30.69%)
----------------------------------------------------------------- 33 40294(11.2%) 5077(30.99%)
0 40194(11.2%) 5109(31.18%) 34 40734(11.3%) 5066(30.92%)
1 40541(11.3%) 5085(31.04%) 35 43167(12.0%) 5083(31.02%)
2 44626(12.4%) 5143(31.39%) 36 43519(12.1%) 5110(31.19%)
3 42984(12.0%) 5121(31.26%) 37 42372(11.8%) 5116(31.23%)
4 44286(12.3%) 5171(31.56%) 38 44425(12.4%) 5035(30.73%)
5 43361(12.1%) 5154(31.46%) 39 42546(11.8%) 5063(30.90%)
6 43394(12.1%) 5048(30.81%) 40 40284(11.2%) 5072(30.96%)
7 39203(10.9%) 5124(31.27%) 41 42166(11.7%) 5068(30.93%)
8 43285(12.0%) 5122(31.26%) 42 40136(11.2%) 5110(31.19%)
9 44728(12.4%) 5091(31.07%) 43 44040(12.3%) 5084(31.03%)
10 41258(11.5%) 5128(31.30%) 44 38744(10.8%) 5115(31.22%)
11 43362(12.1%) 5108(31.18%) 45 37815(10.5%) 5078(30.99%)
12 44791(12.5%) 5218(31.85%) 46 42205(11.7%) 5075(30.98%)
13 44026(12.2%) 5147(31.41%) 47 42783(11.9%) 5068(30.93%)
14 41399(11.5%) 5146(31.41%) 48 40146(11.2%) 5105(31.16%)
15 45238(12.6%) 5148(31.42%) 49 40471(11.3%) 5080(31.01%)
16 45989(12.8%) 5087(31.05%) 50 40798(11.4%) 5107(31.17%)
17 42037(11.7%) 5068(30.93%) 51 44311(12.3%) 5110(31.19%)
18 40363(11.2%) 5125(31.28%) 52 40794(11.3%) 5119(31.24%)
19 39819(11.1%) 5136(31.35%) 53 40354(11.2%) 5136(31.35%)
20 44321(12.3%) 5133(31.33%) 54 41776(11.6%) 5016(30.62%)
21 40380(11.2%) 5159(31.49%) 55 42932(11.9%) 5115(31.22%)
22 44183(12.3%) 5137(31.35%) 56 43001(12.0%) 5022(30.65%)
23 43153(12.0%) 5164(31.52%) 57 40488(11.3%) 5026(30.68%)
24 44762(12.5%) 5098(31.12%) 58 41422(11.5%) 5072(30.96%)
25 44317(12.3%) 5092(31.08%) 59 39293(10.9%) 5064(30.91%)
26 45482(12.7%) 5153(31.45%) 60 43408(12.1%) 5044(30.79%)
27 38451(10.7%) 5127(31.29%) 61 44388(12.3%) 5083(31.02%)
28 40848(11.4%) 5149(31.43%) 62 40447(11.3%) 5100(31.13%)
29 44388(12.3%) 5116(31.23%) 63 42022(11.7%) 5073(30.96%)
30 42729(11.9%) 5120(31.25%)
Online Diagnostics
Optionally, configure Diagnostics on the CGSE card
If we use redundant cards, active being in 0/0/CPU0
RP/0/RP0/CPU0:CRS(config)#
service-plim-ha location 0/0/CPU0 datapath-test
service-plim-ha location 0/0/CPU0 core-to-core-test
service-plim-ha location 0/0/CPU0 pci-test
service-plim-ha location 0/0/CPU0 coredump-extraction
service-plim-ha location 0/0/CPU0 linux-timeout 500
service-plim-ha location 0/0/CPU0 msc-timeout 500
!
An error detected will trigger the reload of the PLIM.
If the card is in stand-alone (no redundancy), we add the configuration:

RP/0/RP0/CPU0:CRS(admin-config)#
hw-module reset auto disable location 0/0/CPU0
!
Online Diagnostics
Optionally, configure Diagnostics on the ISM card
RP/0/RP0/CPU0:ASR9000(config)#
service-cgv6-ha location 0/2/CPU0 puntpath-test
service-cgv6-ha location 0/2/CPU0 datapath-test
!
Performance / Scalability
Per Blade Limits CGSE CGSE+ ISM VSM
NAT44 instances supported 1 per card 1 per card 1 per card 1 (at FCS)
DS Lite instances 64 per chassis N/A 64 per chassis Future
supported
6rd instances supported 64 per chassis 64 per chassis ? Future
NAT64 instances supported 64 per chassis N/A ? Future
Number of service infra 1 1 1 1
Number of service app 890 (2000 per ? 244 (per system) 4096
system)
IP pool supported /16 to /26 /16 to /26 /16 to /30 /16 to /30
(max 65535 addresses) (max 65535 (max 65535 (max 65535
addresses) addresses) addresses)
Future: longer prefix
Max Static Port forwarding 2K tested 6K 6K 6K
Max number of NAT users 1M 1M (2M) 1M 4M
Comparing the CGN Platforms
Parameter CGSE CGSE+ ISM VSM
Configuration CLIs Same Same Same Same
Uses SVI Yes Yes Yes Yes
Network Processor Yes (Metro) Yes (Pogo) No, handled by a Yes (Typhoon)
dedicated process
Packet distribution One level: One level: Two levels ?
NAT44 load-balancing NAT44 load-balancing a) by ingress LC using VQI
b) NAT44 load-balancing
on egress Metro on egress Pogo within Dispatcher process
Egress FIB Lookup On iMetro On iPogo Within CGv6 App On

ServiceApp placement Anywhere Anywhere Associated with Niantic Associated with
port/VQI NP ports / Niantic ports
# of CGv6 instances 64 (4 octeons) 8 (2 Westmeres) 48 (in 2 logical groups)
Stateless protocols (in CGN 6rd, NAT64SL 6rd, (NAT64SL future) 6rd, MAP-T/E Future: 6rd, MAP-T/E
card)
Inline support No No Yes for SL protocols Future
BACKUP SLIDES
PPTP ALG DETAILS
PPTP ALG
PPTP
NAT
IPv4
PNS Internet PAC
Inside Call-ID
Outgoing Call Request
Outbound
Outgoing Call Reply Inside Call-ID Outside Call-ID Call
Translation
DataBase
Two tuples are mapped and an entry is created in the translation DB
PPTP ALG
PPTP
NAT
IPv4
PNS Internet PAC
Incoming Call Request Outside Call-ID

Inbound
Inside Call-ID
Outside Call- Incoming Call Reply Call
ID
Translation
DataBase
Two tuples are mapped and an entry is created in the translation DB
PPTP ALG
PPTP
NAT
IPv4
PNS Internet PAC
Call Disconnect Notify Outside Call-ID

Disconnect
Inside Call-ID
Call Clear Request
Translation
DataBase
Depending on the side initiating the disconnection, the Inside-Call-ID or

Outside-Call-ID tuple will be marked for deletion from the translation DB

BRKSPG 3334

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

BRKSPG 3334

Încărcat de

Drepturi de autor:

Formate disponibile

Carrier Grade NAT44 on IOS-XR

LIR are allocating their last blocks

Example: we have 16 public addresses

Stateless translation Stateful translation

Preserve the investment: buy time to prepare migration to IPv6

Stateful translation protocol from an IPv4 space to another IPv4 space

Translated Address = 10.8.1.111

Inside VRF NAT Engine Outside VRF

EIM: End-point Independent Mapping

EDM: End-point Dependent Mapping

EIF: End-point Independent Filtering

Outside A:10302 B:11237

First flow per Inside source address Used

CGN picks an Outside address that ? NAT ?

Ports are randomly picked from the list of

CGN creates a Translation binding IPa:2104 IP1:10302

for the port-range

New pool allocated if subscriber creates > N concurrent

2 1 packet from 10.1.1.1 to 30.1.1.1:25

3 1 packet from 10.1.1.2 to 40.1.1.1:80

1 1 packet from 10.1.1.1 to 50.1.1.1:80

2 1 packet from 10.1.1.1 to 60.1.1.1:80

7 1 packet from 10.1.1.1 to 30.1.1.1:80

BPA=16 can reduce the logging volume MUCH more than by 16

NAT IPa:Pc IP1:P3

o2i packets are discarded

1 6000 entries max

Static-port-forwarding creates an entry in the NAT DB

RP/0/RP0/CPU0:R#sh cgn demo inside-translation protocol tcp inside-vrf Inside inside-address

Control Connection (TCP1723)

Tim Inside Outside Destination

T2 A:Pb IP1:P2 X2:Pd2

T3 A:Pc IP1:P3 X3:Pd3

T4 A:Pd IP1:P4 X4:Pd4

service cgn POC-1

Some customers select syslog: Sequence

existing collection infrastructure based Scalabilit High (tested) Need BPA

[EventName <L4> <Original Source IP> <Inside VRF Name>

CGSE PLIM MSC40/FP40

64 cores are available an each CGSE card (2^6) and LB

CGSE+ PLIM MSC140/FP140

Application Domain IOS-XR Domain

Application Processor Module (APM) Service Infra Module (SIM)

Expect scripts can be used to collect counters from show commands

Public address pool utilization details

percentage of users using less than X ports (starts at 99.8%)

Intra-Chassis ABF IGP/BGP

Default Map pool

vrf inside ServiceApp1 ServiceApp2

RP/0/RSP0/CPU0:router(config)# ServiceApp1 ServiceApp2

VRF SA1 CGN SA2

VRF SA3 CGN SA4

VRF SA1 CGN SA2

VRF SA3 CGN SA4

3 Labels 2 Labels 1 Label

RP/0/RP0/CPU0:CGN#show services redundancy

service cgn mets-cgn Translate to

inside-vrf iBackUp VRF SA5 CGN SA6

SA1 CGN SA2 Packets sourced

Te0/6/0/2 VRF SA3 CGN SA4 VRF Te0/6/0/3 110.1.0.0/16

VRF SA5 CGN SA6

VRF SA1 CGN SA2 VRF

Te0/6/0/2 VRF SA3 CGN SA4 VRF Te0/6/0/3 110.1.0.0/16