MPLS CoS

Clarence Filsfils
cfilsfil@cisco.com

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

DiffServ Architecture

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

DiffServ Architecture
RFC 2475

Thisarchitecture
architecture achieves
achieves scalability
scalabilityby
by
This
implementing complex
complexclassification
classificationand
and
implementing
conditioningfunctions
functionsonly
onlyat
atnetwork
network boundary
boundary
conditioning
nodes,and
andby
byapplying
applyingper-hop
per-hopbehaviors
behaviorsto
to
nodes,
aggregates of
of traffic
trafficwhich
whichhave
havebeen
been
aggregates
appropriatelymarked
markedusing
usingthe
the DS
DSfield
field in
inthe
theIPv4
IPv4or
or
appropriately
IPv6headers
headers [DSFIELD].
[DSFIELD]. Per-application
Per-applicationflow
flow or
orperperIPv6
customerforwarding
forwardingstate
stateneed
need not
notbe
bemaintained
maintained
customer
withinthe
thecore
coreof
ofthe
the network.
network.
within
RFC2475Architecture
Architecturefor
forDifferentiated
DifferentiatedServices
Services
RFC2475

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DiffServ Architecture
SLS/TCS

0. Negociation and agreement of an SLS/TCS

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DiffServ Architecture

1. Pre-marking in the source domain

- per-application/host basis
- per-default-gateway basis

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DiffServ Architecture
SLS/TCS

2. Egress- boundary DS node of source domain

applies traffic conditioning to ensure SLS/TCS
compliance, hence causing possible re-marking,
dropping and shaping

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DiffServ Architecture
SLS/TCS

3. Classification according to SLS

4. Conditioning according to TCS
5. Assignment to a BA (DSCP setting)

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DiffServ Architecture

6. Forwarding according to PHB mapped to set DSCP

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DiffServ Architecture

SLS/TCS

If downstream DS domain support same service

provisioning policy, same PHBs and DSCP/PHB mappings
Then

7: No-op

Else

7a: SLS/TCS negotiation

7b: Conditioning according to TCS

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

DS field
RFC 2474

DS field

DSCP

CU

DS field replaces IPv4 ToS, IPv6 Traffic

Class
DSCP = 6 bits : xxxxxx notation

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

10

EF PHB definition
RFC2598

EF PHB ensures a minimum

departure rate
DSCP: 101110
EF PHB can be used to build a low loss, low
latency, low jitter, assured bandwdith, e2e
service through DS domains

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

11

AF PHB definition
RFC2597

AF Class 1: 001dd0
AF Class 2: 010dd0
AF Class 3: 011dd0
AF Class 4: 100dd0
01: Low Drop
10: Medium Drop
11: High Drop

4 independently forwarded AF classes

Within each AF class, 3 levels of drop prec
01 < 10 < 11, with active Q mgt (RED)
4 independent capacity management plans
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

12

AF PHB definition (Cont)

Assured Forwarding (AF) PHB group is a means
for a provider DS domain to offer different levels of
forwarding assurances for IP packets received
from a customer DS domain
Olympic Service (Gold, Silver, Bronze)
gold (C1) >= silver (C2) >= bronze (C3)

No quantifiable timing requirements!

delay or delay variation

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

13

AF PHB definition (Cont)

Forwarding assurance of an IP packet:
resources allocated to the AF class
local resource allocation config (buffer and link BW)

the current load of that AF class

conditioning at the DS domain edge

the drop precedence of the packet

conditioning at the DS domain edge

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

14

DiffServ Architecture
Functional Blocks
Classifier

Conditioner

Forwarding

Metering
Dropping
Marking
Accounting

PHB
Scheduling
Dropping

Conditioner
Shaping

Cisco DiffServ Architecture

ACL
QPPB

MPLS99
Clarence Filsfils

CAR
TS
Netflow

1999, Cisco Systems, Inc.

CEF

CBWFQ
FBWFQ
WRED

TS

15

MPLS/DiffServ: the obvious fit!

Scalability!
1000s
of flows

MPLS: FEC to
Label Imposition
DS: Behavior
Aggregates
DSCP
Imposition

MPLS: Label
Switching

DS: PHB based

on DSCP

Different BAs of the same FEC follow

the same LSP
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

16

MPLS/DiffServ: the obvious fit!

Enhanced Services
1000s
of flows

MPLS/DiffServ:
per-cos per-FEC
Label Imposition
with DHCP
imposition

MPLS: Label
Switching

DS: PHB based

on DHCP

Per-DSCP FEC would allow for new

services (eg. per-CoS TE)
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

17

MPLS/DiffServ: the obvious fit!

Enhanced Services
MPLS
Stack

IP Routing

PNNI Routing

LDP Signalling

UNI/NNI Signal.

ATM Forum
Stack

VPI/VCI crosstable
IP QoS

ATMF QoS

ATM switch runs IP Routing Protocol and IP

QoS functions
Ship in the Night Model

More scalability. IP DiffServ intelligence.

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

18

MPLS DiffServ

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

19

Coloring MPLS Frames

Two methods are possible

Using the EXP bits in the MPLS header
and mapping DSCP to EXP
convenient for Frame-based Interface

Mapping a label per-CoS per-FEC

convenient for ATM-based interface
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

20

Using the EXP bits

0
1
2
3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Label
| EXP |S|
TTL
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Copy of Precedence into EXP

Mapping of DSCP into EXP
Non-MPLS
Domain
IPv4 Packet

Prec: xyz

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

MPLS Domain
MPLS

Hdr

MPLS
EXP: xyz

Prec: xyz
21

Label-inferred CoS
draft-ietf-mpls-diff-ext-01.txt

0
1
2
3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Label
| EXP |S|
TTL
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

DSCP to Label mapping

Dest-CoS
IPv4 Packet
Prec: xyz
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

Label

P/p CoS1

17

P/p CoS2

22

P/p CoS3

25

P/p CoS4

12
22

Enforcing PHBs on
non-ATM interfaces

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

23

Frame MPLS CoS

Straightforward
Same Mechanisms as IP CoS
Link Sharer: IOS CBWFQ
Active Q mgnt and differentiated drop: IOS WRED
AF and EF PHBs

Class lookup from either

MPLS CoS/EXP
MPLS Label inferred CoS

Undistinguishable from IPv4 DiffServ

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

24

Enforcing PHBs on
ATM interfaces

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

25

ATM MPLS CoS

Great Opportunity!

Peer Model!

IP Routing

PNNI Routing

LDP Signalling

UNI/NNI Signal.

VPI/VCI crosstable
IP QoS

ATMF QoS

IP intelligence at every hop

IP-friendly mech. on ATM switches!

Diffserv instead of per-VC ATM QoS

Superior Resource Utilisation
Simpler Resource Allocation
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

26

Two Modes

Multi-LSP in TBR mode

Single LSP in ABR mode
Each has advantage and drawbacks

TBR: Tag Bit Rate: ATM service

category designed for Differv/MPLS
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

27

Multi-VC TBR mode

Control Plane
ATM LSR

Parallel TBR LSPs

TDP signals up to 4 parallel LSPs for the

same prefix
CoS <--> LSPs mapping at the edge LSR
Optional setting of CLP (based on DHCP)
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

28

Multi-VC TBR mode

Data Path
ATM LSR

Parallel TBR LSPs

Edge LSR:
per CoS WFQ + per CoS WRED

ATM-LSR
per CoS WFQ + per CoS WEPD

NO per-LVC management!!!
Scalability and better muxing
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

29

Multi-VC TBR mode

Example

Per COS
WFQ

Queuing on all links is per-class WFQ (not per

LSP)
Resource allocation

MPLS99
Clarence Filsfils

Assign weight to each class on per-link basis

(e.g. Premium gets 80% of link, Standard gets 20%)
Choice of weights based on expected load & desired
performance PER CLASS
No per-router-pair configuration (config independent of
topology & geography)

1999, Cisco Systems, Inc.

30

Single `VC ABR mode

ABR LSP
ATM LSR

1 single LVC per FEC

ABR control algorithms are enabled on LSPs
Extention of IPATMCoS feature
ATM-LSRs push congestion towards edge LSRs
Edge-LSRs: WRED/WFQ per-LSP queues
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

31

Single VC ABR mode

ATM-LSR Scheduling = per-VC ABR
ABR parameters:
MCR is effectively zero (to avoid
loss/blocking)
Relative bandwidth
bandwidth parameter
carried by TDP and used by ABR
algorithm
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

32

Single VC ABR
Example
KleinStadt

B
Paris

London

A
Tarifa

Per VC
ABR

Equal sharing of link A-B is not always desirable:

Configure relative bandwidth on router-pair basis,
e.g. Tarifa-KleinStadt = 1; London-Paris = 100

Resource Allocation :
Sharing of Bandwidth across Edge Pairs via Relative BW on a
per LSP basis
MPLS99
Clarence Filsfils

Sharing of Bandwidth across COS performed

1999, Cisco Systems, Inc.

33

Single-ABR vs Multi-TBR
Multi-VC TBR Mode:
Congestion managed directly at every hop (IP and ATM hops)
Possible Discard at every hop
Resource Allocation per COS per link; does not have to concern
itself with topology and geography

Single-VC ABR:
No Loss in the ATM fabric
Discard/Scheduling possible only on the Edge performed by
Routers
Resource Allocation optionally per Pair of Edge Routers.

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

34

MPLS over ATMF PVC

A special Case
ATM Forum ATM

ATM Forum PVC

LDP

Generic Frame MPLS CoS Case!

A Frame LSR uses ATM-F PVC with chosen ATM QoS
Service Differentiation on Frame LSRs at edge of ATM

Use IPATMCoS features!

features

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

35

MPLS over IPATMCoS

per-VC IP QoS

per-VC ATM-F shaping

CBWFQ/WRED on IP
per-VC Queue

ATM-F service class

enforced

Congestion pushed back at the edge

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

36

MPLS over IPATMCoS

Bundle IP QoS

per-VC ATM-F shaping

DSCP to VC mapping
WRED on IP per-VC Q

ATM-F service class

enforced

1! IGP adajacency
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

37

MPLS VPN CoS

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

38

MPLS VPN QoS Architecture

2 very distinct point of views:
How the SP will market the service (SLA)
What are the mechanisms for SP to meet the
commitments/SLA

FR analogy:
sell 64 kb/s CIR for 99.5% of the time
reserve 64/overbooking kb/s + admission control +
selective discard +

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

39

How to market MPLS VPN CoS?

ICR concept - hose model

VPN_A
site 4

ECR
512k

VPN_A
site 2

ECR
128k

VPN
SP

ICR
512k

ECR
128k

ICR
256k
ICR
256k

VPN_A
site 3

Hose Model (point-to-multipoint commodity)

Draft-duffield-vpn-qos-framework.txt, AT&T
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

40

Proposed SLA for CoS C1

As long as for each site S of VPN X:

S sends less than ICR
S receives less than ECR
(optional: double-ended SLA)

Then:
loss property is 10^(-n1)
RTT is < m1 ms

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

41

Extensible to multiple CoS!

MPLS99
Clarence Filsfils

CoS X: [nx, mx],

price Px

Gold:

[-10, 100ms],

$$$

Silver:

[-8, 200ms],

$$

BE:

[be, be],

1999, Cisco Systems, Inc.

42

How it should not be marketed

Should not be marketed as Frame Relay QoS:
N1 kb/s guaranteed from Site 1 to Site 2
N2 kb/s guaranteed from Site 1 to Site 3
N3 kb/s guaranteed from Site 2 to Site 3

Layer 2 based VPNs (ie FR or ATM) address

that need

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

43

Advantage of this SLA model

Any to any connectivity
Without requiring the customer to have a precise
and complete knowledge of its traffic matrix
Matrix of ICR/ECR allows the provider to better
engineer his network (hence, to lower the cost of
the commodity)
Per-usage billing is still possible (ICR/ECR then
only serve as boundaries)

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

44

How to meet SLA

Enforcement of ICR:
CAR: policing in/out of profile

Enforcement of ECR
CAR/TS

MPLS CoS in the SPs backbone

single-ABR, multi-TBR mode
DiffServ engineering

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

45

DiffServ Engineering
Scalability: no per-VPN QoS in BB!!!
This is a pure diffserv design!

Per-Class Scheduling/Discarding at every hop

Resource Allocation based on ICR/ECR sold
share each trunk between different Classes
start conservative
then monitor traffic per class and fine tune
Optimise with per-class Traffic Engineering
Cisco Service Management tool for Tag VPN QoS
provisioning
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

46

DiffServ Engineering
Capacity Management

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

47

DiffServ Architecture
The Service Offer
1. The routing topology
2. The speed of the links
3. The link sharing ratios
(CBWFQ %)

THE Service Offer

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

48

DiffServ Architecture
The Service Demand
1. The matrix of ICR
2. The matrix of ECR

THE Service Demand

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

49

DiffServ Architecture
There is no magic

Service Offer (per class) >= Service

Demand (per class)
How to achieve this?
provisioned QoS

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

50

Connectionless Difficulty

Temporal gambling (of course)

In a connectionless world (eg, IP),
this provioning is more complex:
spacial gambling!

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

51

Temporal Gambling
Same as in ATM/FR
FR PVC, CIR =
48kbps

Trunk is 64kbps

FR PVC, CIR =
48kbps

The path is known (connection-oriented)

The gamble is done on the TEMPORAL traffic distribution
The red and green dlcis should not be loaded at the
same time

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

52

Spacial Gambling

The path is NOT known. The destination is not

known (anycast service)
The gamble is done on the SPACIAL (and
temporal) traffic distribution
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

53

Offer >= Demand

There are three models

1. Huge Over-provisionning
2. Constant Monitoring and
reprovisioning
3. = 2 + Per-CoS Traffic Engineering
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

54

Offer >> Demand

Backbone has lots of BW

QoS is only enforced at the edge

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

55

Offer >~ Demand

Market Research
Strategical QoS Provisioning
topological network
Size of trunks

NETSYS

CBWFQ %
Tactical QoS Provisioning
Traffic Engineering
CBWFQ %
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

SLA monitoring
RTR project

56

Per-CoS MPLS Traffic Engineering

Monitor the POP to POP matrix per CoS
eg.: for yesterday peak rates

At 00h00, provision POP to POP per-CoS

MPLS tunnels for 110% of
max(yesterdays peek, 7-days agos peek)

Optimization could be every hour, every 15-minutes

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

57

Per-CoS MPLS Traffic Engineering

MPLS per-CoS TE is an additional tactical tool

that allows you to master the space gambling
just as in connection-oriented networks
Benefit of:
MPLS for tunnelization
RRR for Constraint based routing
RSVP for resource reservation
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

58

MPLS VPN QoS

Pipe Model

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

59

MPLS VPN QoS: hose model

Remember
Relies on Diff-Serv model supported over MPLS
provides statistical guarantees to each Diff-Serv Class/Service in the
Hose Model (Provisioned QoS)
customer does not have to know/specify his traffic matrix, change
inside traffic matrix accommodated by SP without change in the
hose contract
offers Diff-Servs benefits in scalability (can be deployed on any
number of VPN Sites, does not require any increase in number of
labels, ...)
offers Diff-Servs benefits in simplicity (can be deployed immediately)

! The model for 95% of the VPNs !

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

60

PIPE model for special sites

Guaranteed Bandwidth
Bandwidth
Guarantees

5 Mb/s

1 Mb/s

VPN Green
HQ1

VPN Green
HQ back-up

MPLS VPN

ICR
k
128

ICR

k
VPN Green,
128
100s of branches,
each with n kb/s of
28k
Ingress Committed Rate ICR 1
into a Diff-Serv Class
MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

61

Guaranteed PIPE Model

The QoS benefits of Virtual Leased Line

per-PIPE guarantee

Maintain Scalability
Aggregation via DiffServ and MPLS

Scalable Admission Control

RSVP aggregation

Optimization of Resource Usage

Traffic Enginering and RRR

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

62

MPLS QoS
Conclusion

Course Number
Presentation_ID

1999, Cisco Systems, Inc.

63

MPLS QoS
Enable enhanced services
Same QoS support as IP
DiffServ, IntServ support

Plus Extra functionality

MPLS-CoS over ATM in peer model
(multi-vc TBR feature)
Traffic Engineering and RRR with per-CoS extensions
MPLS-VPN CoS (hose + pipe)

MPLS99
Clarence Filsfils

1999, Cisco Systems, Inc.

64

Presentation_ID

1999, Cisco Systems, Inc.

65