pseudowires

a short introduction
Yaakov (J) Stein July 2010
Chief Scientist
RAD Data Communications
 Contents

„ p
pseudowires
„ PW encapsulations
„ TDM PWs
„ Ethernet PWs
„ L2VPNs
„ OAM for PWs
„ PWE control protocol

Y(J)S PWE short Slide 2

 Pseudowires

Pseudowire (PW): A mechanism that emulates the

essential attributes of a native service while transporting
over a packet switched network ((PSN))

Y(J)S PWE short Slide 3

 Pseudowires

Packet Switched Network (PSN)

– a network that forwards packets
– IPv4, IPv6, MPLS, Ethernet

a pseudowire (PW) is a mechanism to tunnel traffic through a PSN

PWs are usually bidirectional (unlike MPLS LSPs)

PW architecture is an extension of VPN architecture

Y(J)S PWE short Slide 4

 Basic (L2
(L2,L
,L3
3)VPN model

customer physical link customer

network network

emulated link

Customer Provider Provider Customer

customer Edge Edge provider Edge Edge customer
network network network
(CE) (PE) (PE) (CE)

AC = Attachment Circuit AC = Attachment Circuit

transported traffic may be L3 (e.g. IP) or L2 (e.g. Ethernet)

Y(J)S PWE short Slide 5
 (L2
(L 2,L
,L33)VPN in more detail

C C C
C CE
CE
AC C
C AC

customer 1 network P P P customer 2 network

PE PE
P P
AC
AC
C C C
C provider network CE
CE
C C
Key
C Customer router/switch
customer 2 network CE Customer Edge router/switch customer 1 network
P Provider router/switch
PE Provider Edge router/switch Y(J)S PWE short Slide 6
 Pseudowire Emulation
(provider) Edge to Edge
PWE3
Customer
Edge
Ed
provider’s
(CE) PSN Customer
Edge
Customer
Edge Provider Provider ((CE))
Edge Edge
(CE)
(PE) (PE) Customer
Customer
C stomer Edge
Ed
native
Edge service
native Pseudowires AC (CE)
(CE) service
AC (PWs)
(PW )
Y(J)S PWE short Slide 7
Native services defined in IETF PWE3
PWE3
The PWE3 Working Group in the IETF
has defined the following native services :

„ ATM (port mode, cell mode, AAL5-specific modes) RFC 4717, 4816
„ Frame Relay RFC 4619
„ HDLC/PPP
C/ RFC
C 4618
6 8
„ TDM (E1, T1, E3, T3) RFC 4553, 5086, 5087
„ SONET/SDH (CEP) RFC 4842
„ Fiber channel
„ Multiprotocol packet service
„ Ethernet (raw, VLAN-aware) RFC 4448

Note that most are legacy services

but the most interesting service today is Ethernet

Y(J)S PWE short Slide 8

 What else ?
PWs emulate the native service –
but may not completely reproduce it (applicability statement)
PW packets
k t are nott self-describing
lf d ibi (like
(lik MPLS
MPLS, unlike
lik IP or Ethernet)
Eth t)

An demultiplexing identifier is provided to uniquely identify PWs

We may also need :
„ Native Service Processing (NSPs)
„ PW-layer OAM (at least Continuity Check)
„ PW control protocol
„ Load balancing
„ Protection (redundancy) mechanism
„ Multisegment PWs (MS
(MS-PWs)
PWs)

Y(J)S PWE short Slide 9

 Simplistic MPLS solution
CE
CE
ACs ACs
CE PE P P PE CE

CE CE

each customer network mapped to pair of (unidirectional) LSPs

supports various AC technologies
each
h native
ti packet/frame
k t/f encapsulated
l t d with
ith MPLS llabel
b l

scaling problem:
„ requires large number of LSPs
„ P-routers need to be aware of customer networks

Y(J)S PWE short Slide 10

 (Martini) Pseudowires

CE CE
ACs transport tunnel ACs
CE PE PE CE

CE CE
PWs are bidirectional

MPLS (outer) label

transport MPLS tunnel set up between PEs
PW (inner) label
p PWs may
multiple y be set up
p inside tunnel
payload
native packet/frame encapsulated with 2 labels

PEs contain the PW interworking function

P-routers are unaware of individual customer networks

Y(J)S PWE short Slide 11

 Pseudowire
encapsulations

Encapsulation: In order to enable transport over the PSN

PSN,
native service Protocol Data Units (PDUs) must be inserted
into packets of the appropriate format.
This is usually accomplished by adding headers
headers.

Y(J)S PWE short Slide 12

 Generic PWE3
PWE3 packet format

PSN / multiplexing

optional RTP header

optional control word (CW)

higher llayers

native service payload

We will ignore the RTP header in the following

Y(J)S PWE short Slide 13

 MPLS PSN
MPLS PSN

tunnel PW CW
P l d
Payload
label(s) label

MPLS over Ethernet

Ethernet MAC header ((DA SA,, …))
MPLS label stack
PW label (bottom of label stack S=1)
Control Word

native service payload

Ethernet FCS
Y(J)S PWE short Slide 14
 IP PSN using L2
L2TPv
TPv3
3
L2TPv3 – RFC 3931 ((without UDP))

IP header (5*4 B)
IP p
protocol 115

session ID (4 B)
optional
i l cookie
ki (4 or 8 B)
control word (4 B)

native service payload

Y(J)S PWE short Slide 15

 IP PSN using UDP
with
ith PW label
l b l in
i destination
d ti ti portt
UDP/IP for TDM PWs

IP header (5*44 B)
(5
P header ((8B)

return PW label (2 B)
PW label (2 B)
UDP

UDP length and checksum (4 B)

control word (4 B)

native service payload

p y

PW labels between C000 and FFFF Y(J)S PWE short Slide 16

 IP PSN using UDP
with
ith PW label
l b l in
i source portt
UDP/IP - 5087

IP header (5*44 B)
(5
P header ((8B)

PW label (2 B)
well known port (085E) (2 B)
UDP

UDP length and checksum (4 B)

control word (4 B)

native service payload

p y

PW labels between C000 and FFFF Y(J)S PWE short Slide 17

 IP PSN using RFC 4023

MPLS over IP using RFC 4023

IP hheader
d (5*4 B)
IP protocol 47(GRE) or 137(MPLS)

optional GRE header (8 B)

GRE protocol 08847(MPLS Ethertype)

PW label (4 B)
control word (4 B)

native service payload

Y(J)S PWE short Slide 18

 PWE Control Word (RFC 4385
4385))

0000 flags FRG Length Sequence Number

0000
– identifies packet as PW (not IP – which has 0100 or 0110)
– ggives clue to ECMP mechanisms
– 0001 for PWE associated channel (ACh) used for OAM
Flags (4 b)
– not all encapsulation define
– used to transport native service fault indications
FRG
– may be used to indicate payload fragmentation
z 00 = unfragmented 01 = 1st fragment
z 10 = last fragment 11 = intermediate fragment
Length
L th (6 b)
– used when packet may be padded by L2
Sequence
q Number ((16 b))
– used to detect packet loss / misordering
– processing slightly different in TDM PWs
Y(J)S PWE short Slide 19
TDM PWs

Y(J)S PWE short Slide 20

 TDM PW Protocol Processing
TDM PSN Packets PSN Packets
TDM

PSN

Steps in TDM PW processing

„ The synchronous bit stream is segmented

„ The TDM segments may be adapted

„ TDMoIP control word is prepended

„ PSN headers are prepended (encapsulation)

„ Packets are transported over PSN to destination

„ PSN headers are utilized and stripped

„ Control word is checked, utilized and stripped

pp
„ TDM stream is reconstituted (using adaptation) and played out

Y(J)S PWE short Slide 21

 Flags
0000 flags FRG Length Sequence Number

L R M(2b)
( )

The PWE control word has 2 flags: L and R

and a 2-bit field: M
Theyy are used in the following
g way y:
„ L is set to indicate a forward defect (AIS)
„ R may be set to indicate a reverse defect (RDI)
„ M can modify
dif the
th meaningi off the
th FDI

Y(J)S PWE short Slide 22

 TDM Structure
handling of TDM depends on its structure
unstructured TDM (TDM = arbitrary stream of bits)
…
structured TDM
framed (8000 frames per second)
S S S
Y Y Y
N N N
C C C
channelized
h l d (single byte timeslots)

SYNC TS1 TS2 TS3 … signaling

bits
… TSn
(1 byte)
multiframed

frame frame frame … frame

multiframe Y(J)S PWE short Slide 23

 TDM transport types
Structure-agnostic
g transport
p (SAToP – RFC4553)
• for unstructured TDM
• even if there is structure, we ignore it
• simplest
i l t way off making
ki payload
l d
• OK if network is well-engineered
Structure-aware transport (CESoPSN – RFC 5086, TDMoIP – RFC 5087)
• take TDM structure into account
• must decide which level of structure (frame,
(frame multiframe,
multiframe …))
• can overcome PSN impairments (PDV, packet loss, etc)

The Frame Alignment Signal (FAS) is maintained at PSN egress

Overhead bits may be transported

Y(J)S PWE short Slide 24

 Structure Agnostic Transport

SAToP encapsulates N bytes of TDM in each packet

There is no TDM frame alignment !
N must be constant and preconfigured
If packets are lost, the egress knows how many TDM bytes to fill in
Default values for N :
„ E1 – 256 B
„ T1 – 192 B
„ E3 and T3 – 1024 B

For T1 there is an optional special mode called octet aligned mode

that adds 7 bits of padding to every 193 consecutive bits (to make 25 B)

Y(J)S PWE short Slide 25

 Structure aware encapsulations

Structure-locked encapsulation (CESoPSN)

headers TDM structure TDM structure TDM structure TDM structure

Structure-indicated encapsulation (TDMoIP – AAL1 mode)

headers AAL1 subframe AAL1 subframe AAL1 subframe AAL1 subframe

Structure-reassembled encapsulation (TDMoIP – AAL2 mode)

headers AAL2 minicell
i i ll AAL2 minicell
i i ll AAL2 minicell
i i ll AAL2 minicell
i i ll

Y(J)S PWE short Slide 26

 Structure indication - AAL
AAL11
For robust emulation:
„ adding a packet sequence number
„ adding a pointer to the next superframe boundary
„ only sending timeslots in use
„ allowing multiple frames per packet

UDP/IP seqnum ptr T1/E1 frames (only timeslots in use)

(with CRC)
for example 7 @ TS1 TS2 TS5 TS7 TS1 TS2 TS5 TS7

Y(J)S PWE short Slide 27

 Structure reassembly - AAL
AAL22

TDM frame TDM frame TDM frame TDM frame TDM frame
1 1 1 2 2 2 3 3 3 4 4 4 5 5 5

PSN hdrs CW hdr 1 2 3 4 5 hdr 1 2 3 4 5 hdr 1 2 3 4 5

TS1 TS2 TS3

AAL1 is inefficient when timeslots are dynamically allocated

„ each minicell consists of a header and buffered data
„ minicell header contains:
– CID (Channel IDentifier)
– LI (Length Indicator) = length-1
– UUI (User
(User-User
User Indication) counter + payload type ID

Y(J)S PWE short Slide 28

 CAS and CCS signaling
Channel Associated Signaling is carried in the T1/E1
(T1 uses robbed bits , E1 uses a dedicated time slot - TS16)
Unlike VoIP, TDM PWs transparently transport CAS
and may add a separate signaling substructure (ATM
(ATM-like)
like)
that carries the CAS signaling bits
CESoPSN must respect CAS multiframe boundaries
Thus it may fragment the mutiframe (using the CW FRG bits)
and append the substructure to the last fragment

With HDLC-based trunk associated Common Chanel Signaling

(e.g., ISDN PRI signaling, SS7)
The CCS may simply be left where it is
But sometimes it is worthwhile to extract it
p it using
and transport g a separate
p HDLC PW

Y(J)S PWE short Slide 29

 PSN - Delay and PDV
„ PSNs do not carry timing
„ clock recovery required for TDMoIP
„ PSNs introduce delay and packet delay variation (PDV)
„ Delayy degrades
g p
perceived voice q
quality
y
„ PDV makes clock recovery difficult

E1/T1 VOICE E1/T1 VOICE

TDMoIP
TDMoIP PSN
GW GW DATA
DATA

The arrival
time is not
constant!!!

Y(J)S PWE short Slide 30

 Jitter Buffer
Arriving
g TDMoIP p
packets written into jjitter buffer
Once buffer filled 1/2 can start reading from buffer
Packets read from jitter buffer at constant rate
How do we know the right rate?
How do we guard against buffer overflow/underflow?

E1/T1 VOICE E1/T1 VOICE

TDMoIP PSN TDMoIP

GW GW DATA
DATA

Jitter Buffer
Y(J)S PWE short Slide 31
 Adaptive Clock Recovery
The packets are injected into network ingress at times Tn
For TDM the source packet rate R is constant
Tn = n / R
The network delay Dn can be considered to be the sum of
typical delay d and random delay variation Vn
The packets are received at network egress at times tn
tn = Tn + Dn = Tn + d + Vn
By proper averaging/filtering
<tn > = Tn + d =n/R+ d
and the packet rate R has been recovered

Y(J)S PWE short Slide 32

Differential (common clock) Clock Recovery
Sometimes we have an reference clock frequency y available
at both IWFs (PEs) (e.g., physical layer clock, GPS, PRCs_
Then at ingress we can encode the frequency difference
b t
between the
th TDM source frequency
f and
d the
th reference
f
And at egress reconstruct the TDM source frequency using the reference

TDMoIP
TDMoIP PSN
GW GW

Y(J)S PWE short Slide 33

 Handling of packet loss

IIn order
d tto maintain
i t i TDM ti
timing
i att egress
SOMETHING must be output
towards the TDM interface when a ppacket is lost

PSN

Packet Loss Concealment methods:

„ fixed
„ replay
„ interpolation

Y(J)S PWE short Slide 34

 Mis--ordering
Mis
In a perfect network all packets should arrive in proper order
In real networks, some packets are delayed (or even duplicated!)
Misordering is caused by parallel paths
– aggravated by load balancing mechanisms

1 2 4
1 2 3 4 5 1 2 4 3 5
router router
3 5

Misordering can be handled by

„ Reordering (from jitter buffer)
„ Handling as packet loss and dropping later

Y(J)S PWE short Slide 35

Ethernet PWs

Y(J)S PWE short Slide 36

 Ethernet limitations
Ethernet LAN is the most popular LAN
but Ethernet can not be made into a WAN
„ Ethernet is limited in distance between stations
„ Ethernet is limited in number of stations on segment
„ Ethernet is inefficient in finding destination address
„ Ethernet only prunes network topology
topology, does not route

so the architecture that has emerged is Ethernet private networks

connected by public networks of other types (e.g. IP)

LAN LAN

WAN

Y(J)S PWE short Slide 37

 Traditional WAN architecture
this model is sensible when traffic contains a given higher layer
Ethernet header is removed at ingress and a new header added at egress
this model is not transparent Ethernet LAN interconnect
„ Ethernet LANs with multiple higher layer packet types
(e.g. IPv4, IPv6, IPX, SNA, CLNP, etc.) can’t be interconnected
„ raw L2 Ethernet frames can not be sent

the Ethernet layer is terminated at WAN ingress

the traffic is no longer Ethernet at all

t e et
Ethernet t e et
Ethernet

WAN

not Ethernet
Y(J)S PWE short Slide 38
 Tunneling Ethernet frames

users with multiple sites want to connect their LANs

so that all locations appear to be on the same LAN
this requires
req ires tunneling
t nneling of all Ethernet L2 frames (not
( t only
l IP)
between one LAN and another
th entire
the ti Ethernet
Eth t fframe needs
d tto b
be preserved
d
(except perhaps the FCS which can be regenerated at egress)

Ethernet Ethernet
X

Ethernet inside X

Y(J)S PWE short Slide 39

 Ethernet over X

Ethernet frames can be carried over various WANs

HDLC: not standardized, Cisco-HDLC

FR: RFC2427 / STD0055 (ex 1490)

ATM: RFC2684 / (ex 1483), LANE

SONET/SDH/PDH: PoS (RFC 2615 ex RFC1619),

LAPS ((X.85/X.86),
), GFP (G.7041
( )
PSN: Ethernet PW
Y(J)S PWE short Slide 40
 Ethernet PW (RFC 4448
4448))

can transport tagged or untagged Ethernet frames

if tagged encapsulation can be “raw
raw mode”
mode or “tagged
tagged mode”
mode
tagged mode processes (swaps) SP tags

control word is optional

even if control word is used,, sequence
q number if optional
p

standard mode – FCS is stripped and regenerated

FCS retention mode (RFC 4720) allows retaining FCS

Y(J)S PWE short Slide 41

 Ethernet Pseudowire packet (MPLS)

ttunnell PW t l
control
Ethernet Frame
label label word

Ethernet Frame usually has FCS stripped, but may retain it

SP tags
S ags may
ay be modified
od ed

optional
p control word
generation and processing of sequence number is optional

0000 reserved Sequence Number (16b)

Y(J)S PWE short Slide 42

L2VPNs

Y(J)S PWE short Slide 43

 VPWS

AC PE AC
CE PE CE

provider
network

Virtual Private Wire Service is a L2 point-to-point service

pp g the Ethernet p
it emulates a wire supporting physical
y layer
y
set up MPLS tunnel between PEs
set up Ethernet PW inside tunnel
CEs appear to be connected by a single L2 circuit
(can also make VPWS for ATM, FR, etc.)

Y(J)S PWE short Slide 44

 VPLS
AC
PE CE

AC
CE PE

for clarity only one VPN is shown

PE AC CE

VPLS emulates a LAN over an MPLS network

set up MPLS tunnel between every pair of PEs (full
(f ll mesh)
h)
set up Ethernet PW inside tunnels, for each VPN instance
CEs appear to be connected by a single LAN
PE must know where to send Ethernet frames …
but this is what an Ethernet bridge does

Y(J)S PWE short Slide 45

 VPLS

V B CE

CE B V

V B CE

a VPLS-enabled PE has, in addition to its MPLS functions:

„ VPLS code module (IETF drafts)
„ Bridging module (standard IEEE 802.1D
802 1D learning bridge)

SP network (inside rectangle) looks like a single Ethernet bridge!

Note: if CE is a router, then PE only sees 1 MAC per customer location
Y(J)S PWE short Slide 46
 VPLS bridge
PE maintains a separate bridging module for each VPN (VPLS instance)

VPLS bridging
b id i module d l mustt perform:
f
„ MAC learning
„ MAC aging
„ flooding of unknown MAC frames
„ replication (for unknown/multicast/broadcast frames)

unlike true bridge, Spanning Tree Protocol is not used

„ limited traffic engineering capabilities
„ scalability limitations
„ slow convergence

forwarding loops are avoided by split horizon

„ PE never forwards packet from MPLS network to another PE
„ nott a limitation
li it ti since
i th
there iis a ffullll mesh
h off PW
PWs
so always send directly to the right PE
Y(J)S PWE short Slide 47
 Bridge - both ways
CE

CE
V B CE

CE
CE B V

CE

V B CE

CE
a packet from a CE:
may be sent back to a CE
may be sent to a PE via a PW
a packet from a PE:
is onlyy sent to a CE (split
( p horizon))
is sent to a particular CE based on 802.1D bridging
Y(J)S PWE short Slide 48
 L2VPN vs. L3
L3VPN

PE CE

CE PE
?
PE CE

in L2VPN CEs appear to be connected by single L2 network

PEs are transparent to L3 routing protocols
CEs are routing peers

in L3VPN CE routers appear to be connected by a single L3 network

CE is routing peer of PE, not remote CE
PE maintains routing table for each VPN

Y(J)S PWE short Slide 49

PW OAM

Y(J)S PWE short Slide 50

 PWE Associated Channel
PW associated channel fate-shares with user data
Inside the channel we can run different OAM mechanisms
The use of the Ach was extended to MPLS-TP as the GACh

ACh differentiated by control word format (RFC 4385)

0001 VER RES=0 Channel Type

The channel types are defined in the

Pseudowire Associated Channel Types IANA registry
1 Management Communication Channel (MCC)
2 Signaling Communication Channel (SCC)
7 BFD Control without IP/UDP Headers
021 IPv4 packet
057 IPv6 packet
Y(J)S PWE short Slide 51
 VCCV

VC (old
( ld name ffor PW) CV (incorrect
(i t name for
f CC)
VCCV is set up by PWE control protocol, if used
VCCV can run in the ACH, but there are also other methods
VCCV enables p
pings,
g ,pperiodic CC,, loopback,
p ,…
VCCV has several CV types :
– ICMP (RFC 5085)
– LSP ping (RFC 5085)
– BFD (RFC 5885)

Y(J)S PWE short Slide 52

PWE control protocol

Y(J)S PWE short Slide 53

 PWE (Martini) control protocol

„ PWE control
t l protocol
t l (RFC 4447) used
d tto sett up / configure
fi PWs
PW
„ used only by PW end-points (PEs in standard model)
intermediate nodes (e.g. P routers) don’t participate or see
„ based on LDP
– targeted LDP is used to communicate with remote end-point
– 2 new FECs for PWs
– new TLVs added for PW-specific functionality
– associates two labels with PW
P P
PE PE
P P P

Y(J)S PWE short Slide 54

 PWE control
a PW is a bidirectional entity (two LSPs in opposite directions)
a PW connects
t two
t f
forwarders
d
2 different LDP TLVs can be used
– PWid FEC (128)
– Generalized ID FEC (129)

FEC 128
– both end-points of PW must be provisioned with a unique (32b) value
– each PW end-point independently initiates LSP set up
– LSPs bound together into a single PW

FEC 129
– used when autodiscovering PW end-points
– each end-point has attachment identifier (AI) …
Y(J)S PWE short Slide 55
 Generalized ID
for each forwarder we have a PE-unique Attachment Identifier (AI)
<PE,, AI> must be globally
g y unique
q
frequently useful to group a set of forwarders into a attachment group
where PWs may only be set up among members of a group
then Attachment Identifier (AI) consists of
– Attachment Group Identifier (AGI) (which is basically a VPN-id)
– Attachment Individual Identifier (AII)
the LSPs making up the (two directions of the) PW are
< PE1, (AGI, AII1), PE2, (AGI, AII2) > and
< PE2, (AGI, AII2), PE1, (AGI, AII1) >

we also need to define

– Source Attachment Identifier (SAI = AGI+SAII)
– Target Attachment Identifier (TAI = AGI+TAII)
receiving PE can map TAI uniquely to AC

Y(J)S PWE short Slide 56