Sunteți pe pagina 1din 142

ACME Checks and Learning

24th January 2016

Table of contents

ACME Test Scenario


Network Architecture
VOLTE Call Flow
Multiband Settings and Learnings
Neighbours Deletion
eNode B Prechecks
True Call Based Analysis
CSL Based X2 Analysis
GTP Loss
X2 Learnings
S1AP Critical for Uptime
S1AP Preservation
S5 Interface Learnings
Core Network Learnings

ACME TEST Scenarios


V to V

VOLTE to VOLTE

V to C

VOLTE to CS

C to V

CS to VOLTE

V to P

VOLTE to PSTN

Around 300 to 400 Calls for each scenario


PASS CRITERIA AS MINIMUM 98% SUCCESS
Call Set Up and Call Drops added together to be less then 2%
2016 Samsung Electronics

ENB Interface and Protocol


The Protocols that run between UE and eNB
are called Access Stratum Protocols or AS
Protocols
The Protocols that run between UE and Core
Network are called as NAS Protocols or Non
Access Stratum Protocols

2016 Samsung Electronics

MME Interface and Protocol

2016 Samsung Electronics

MME Interface and Protocol

2016 Samsung Electronics

SAE GW Interface and Protocol

2016 Samsung Electronics

SAE GW Interface and Protocol

Ip Packet is tunneled between P GW and UE using GPRS Tunneling Protocol GTP


Each Bearer has associated QCI
Each QCI has GBR/ Non GBR, Priority, packet delay , Packet Error Rate defined
PCRF decides the Bearer Type
P GW performs the DHCP functionality = allocates IP addresses to each UE

2016 Samsung Electronics

VoLTE Call Flow

VoLTE Logical Architecture


ENUM Electronic Number Mapping System
To translate a telephone number into IP address

2016 Samsung Electronics

10

VoLTE Logical Architecture

2016 Samsung Electronics

11

VoLTE - UE Attachment and IMS Registration message

CCR Credit Control Request


CCA Credit Control Answer
2016 Samsung Electronics

Part 1 of 2
12

VoLTE - UE Attachment and IMS Registration message

Part 2 of 2
2016 Samsung Electronics

13

VoLTE - UE to UE Call establishment (Originating Side)

2016 Samsung Electronics

14

VoLTE - UE to UE Call establishment (Terminating Side)

2016 Samsung Electronics

15

VoLTE - UE to UE Call clearing (Initiated message)

2016 Samsung Electronics

16

VoLTE - UE to UE Call clearing (Received message )

2016 Samsung Electronics

17

VoLTE - VoLTE UE to CS call establishment (Originating Side )

2016 Samsung Electronics

18

VoLTE - VoLTE UE to CS call establishment (Originating Side )

2016 Samsung Electronics

19

VoLTE - VoLTE UE to CS call establishment (Terminating

2016 Samsung Electronics

Side )

20

VoLTE - VoLTE UE to CS call Clearing (Initiated )

2016 Samsung Electronics

21

VoLTE - VoLTE UE to CS call Clearing (Received )

2016 Samsung Electronics

22

Multiband Settings & Learnings

2016 Samsung Electronics

23

Idle Mode (2300 and 1800)


Service
Target
Idle Mode

2.3G

Parameter

2.3G
Priority
QrxLevMin
S_NonIntraSearch
Thresh_Serving Low

1.8G
1.8G

7
64
-128
12
-104 dBm
12
-104 dBm

ThrehX_Low
ThrehX_High

1.8G

2.3G

6
64
-128
12
-104 dBm
12
-104 dBm

12
-104 dBm
14
-100 dBm

(-Q_Rxlevmin + 2 * Thresh_Serv_Low) dBm


1.
2.
3.
4.
5.

Equal priority Band and higher priority Bands, are always measured
UE on 1.8 G will always measure 2.3G
The UE starts measuring other lower priority band neighbors when Serving Cell < S_nonintrasearch
UE on 2.3G measures 1.8G when 2.3G is lower than -104 dBm
UE does cell reselection from 2.3G to 1.8G when 2.3G is lower than -104 dBm and 1.8G is better
than -104 dBm
6. UE returns to 2.3G when 2.3G is better than -100 dBm
2016 Samsung Electronics

24

Idle Mode Parameters Behavior


-75dBm

-80dBm

-85dBm

-90dBm

-95dBm

-100dBm

-105dBm

-110dBm

-115dBm

-70dBm

-75dBm

-80dBm

-85dBm

-90dBm

-95dBm

-100dBm

-105dBm

Band 40
(2300MHz)
-65dBm

Band 3
(1800MHz)

The UE starts measuring other lower


priority band neighbors when Serving
Cell < S_nonintrasearch.

Equal priority band And higher priority


Bands, are always measured.

1. For Cell Reselection from 2300 to 1800 happens when


2300 is weaker than Thresh_Serving_Low and 1800 is
stronger than Thresh_Xlow (higher to lower priority)
2. For Cell Reselection from 1800 to 2300 happens when
2300 is stronger than ThreshX_High (lower to higher
priority)

Example
Idle Mode

Band 3
(1800MHz)

Band 40
(2300MHz)

2300 has higher priority than 1800


-104 dBm
2016 Samsung Electronics

25

Handover : Event in Active Mode


Event Type
Intra RAT (Intra FA / Inter FA) : Event A

Event A1 (Serving becomes better than threshold)


Event A2 (Serving becomes worse than threshold)
Event A3 (Neighbor becomes offset better than serving)
Event A4 (Neighbor becomes better than threshold)
Event A5 (Serving becomes worse than threshold1 and neighbor
becomes better than threshold2)

2016 Samsung Electronics

26

LTE Handover Events


Event A1
Serving becomes better
than threshold
RSRP
TimetoTrigger
Thresh

Hys

Event A2

Hys
RSRP
Serving Cell
Serving Cell
Leave A1
Condition
Enter A1
Condition

Time
Enter A1
Condition

Event A1

Hys
Thresh
Hys

TimetoTrigger

Serving becomes worse than threshold


Time
Enter A2
Condition

2016 Samsung Electronics

Enter A2
Condition
Leave A2
Condition

Event A2

27

LTE Handover Events


Event A3

Neighbor becomes offset better than serving

RSRP/RSRQ

Serving Cell
Neighbor Cell
TimetoTrigger

A3 Offset
Hys
Hys

Hys

Time
Enter A3 Condition
When Ofn, Ocn, Hys,
Ofs, Ocs, Off are all zero

2016 Samsung Electronics

Event A3
Leaving A3 Condition
When Ofn, Ocn, Hys,
Ofs, Ocs, Off are all zero

Enter A3 Condition
When Ofn, Ocn, Hys,
Ofs, Ocs, Off are all zero

28

LTE Handover Events


Event A5
Serving becomes worse than threshold1 and neighbor becomes better
than threshold2
RSRP/RSRQ

Neighbour Cell
Stronger than
Threshold 2

TimetoTrigger
Serving Cell

Thresh2
Thresh1

Hys
Hys
Hys
Hys

Enter Condition1

Leave Condition1
Enter Condition2
Enter Condition1

2016 Samsung Electronics

Time

Event A5

Serving Cell weaker


than Threshold 1

29

-140 dBm + A2

Active Mode 2300-1800


Service
Target
GBR Service (VoLTE)

2.3G
QCI based A2 Threshold
QCI based A1 threshold (Meas
Gap Deactivation)
QCI based A3 threshold

Non-GBR Service (FTP


DL)

2.3G

Parameter

A2 Threshold
A1 threshold(Meas Gap
Deactivation)
A3 Threshold
A5 Threshold (Th1)
A5 Threshold (Th2)

1.8G
1.8G

1.8G

35
-105 dBm

35
-105 dBm

-90 dBm

-90 dBm

6
3 dB

6
3 dB

6
3 dB

27

80

-113 dBm

-60 dBm

-100 dBm

-50 dBm

6
3 dB

6
3 dB

-113 dBm
-110 dBm

2.3G

6
3 dB

-60 dBm
-110 dBm

1. For GBR for transition from 2.3G to 1.8G, as 2.3G becomes than weaker than -105 for
GBR, UE starts measuring other band and transitions when 1.8G is stronger than
2.3G by A3 Offset which is 3 dBm
2. For Non GBR - A5 for transition from 1.8G to 2.3G (as 2.3G would be weaker than
1.8G in most cases ) i.e. when serving cell in 1.8G is weaker than -60 dBm and the
target cell in 2.3G is better than -110 dBm
3. A2 and A3 are used for GBR for intra & inter band handover and for intra band
handover for non GBR
4. A5 is used for inter band handover for non GBR

2016 Samsung Electronics

30

Active Mode Parameters Behavior


-75dBm

-80dBm

-85dBm

-90dBm

-95dBm

-100dBm

-105dBm

-110dBm

-115dBm

-70dBm

-75dBm

-80dBm

-85dBm

-90dBm

-95dBm

-100dBm

-105dBm

Band 40
(2300MHz)
-65dBm

Band 3
(1800MHz)

For GBR - For transition from 2300 to 1800,


as 2300 becomes than weaker than A2
threshold, UE starts measuring other band
and transitions when 1800 is stronger than
2300 by A3 Offset (which would always
happen except for Micro cells)

For Non GBR - A2 and A3 for transition intra


band
A5 for transition inter band especially from
1800 to 2300 (as 2300 would be weaker than
1800 in most cases )

-113 dBm
Example
Data Mode

Band 40
(2300MHz)

Band 3
(1800MHz)

VoLTE Mode
2016 Samsung Electronics

-105 dBm

31

Expected Relative Behavior in Connected Mode


Band 40
(2.3Ghz)

-90dBm

-95dBm

-100dBm

-105dBm

-110dBm

-113dBm

-120dBm

-125dBm

-130dBm

-80Bm

-85dBm

-90dBm

-95dBm

-100dBm

-103dBm

-110dBm

-115dBm

-120dBm

Band 3
(1.8GHz)
Continues Data
Services

Call
Start

Data Service
Data Service

Data Services to VoLTE


Before -113dBm
Data
Service

VoLTE Call
Start

VoLTE call end


Before 110 dBm

Data
Service

Data
Service

Data Service

VoLTE
Call END

Data Services to VoLTE


BEFORE -113dBm
Data
Service

VoLTE Call
Start

Data Service

VoLTE call end


After 110 dBm

VoLTE
Call END

Data Services to VoLTE


After -113dBm
Data
Service
VoLTE call end
After 110 dBm
2016 Samsung Electronics

Data
Service

Data
Service
Data
Service

VoLTE Call
Start

VoLTE
Call END

Data Service
32

Bhopal Learning : Why it Happened ?


A1 should always be higher than A2

Overview:

Parameter Template 1: Configuration with Multi-band Implementation

Parameter Template 2: Default Parameter implementation during Golden Parameter audit & corrections

Audit & corrections carried out on 24-Oct over-ride A2 value earlier implemented as a part of Multi-band Parameter list

Technical Analysis:
Default Parameter List
(Used in Work Stream 1)
A1: Not Set

vs

Multi-band Services Implementation


(Used in Work Stream 2)

Remarks

A1: -100dBm ( value: 40 )

If RSRP is better than A1, it


will stop all other measurements

A2: -90dBm ( value: 50 )

A2: -113dBm ( value: 27 )

If RSRP is lower than A2, UE has


to start measurement of neighbor band

Result:
A2 got configured as -90dBm in place of -113dBm, with A1 as -100dBm

When UE RSRP goes less than -90dBm (A2), inter-band measurements will trigger.

However since RSRP is better than A1 ( -100dBm ), UE will be instructed to stop the inter-band measurements.

Thus conflicting settings for A1 & A2, leads to high messaging & thus eNB going into Racing condition.
2016 Samsung Electronics

33

Learnings from incorrect EARFCN settings


CELL_NUM
0
0

FA_INDEX
0
1

DUPLEX_TYPE
TDD ( TDD )
FDD ( TDD )

1
1

0
1

TDD ( TDD )
FDD ( TDD )

2
2

0
1

TDD ( TDD )
FDD ( TDD )

3
3

0
1

FDD ( FDD )
TDD ( FDD )

4
4

0
1

FDD ( FDD )
TDD ( FDD )

5
5

0
1

FDD ( FDD )
TDD ( FDD )

STATUS
EQUIP ( EQUIP
EQUIP (
N_EQUIP )
EQUIP ( EQUIP
EQUIP (
N_EQUIP )
EQUIP ( EQUIP
EQUIP (
N_EQUIP )
EQUIP ( EQUIP
EQUIP (
N_EQUIP )
EQUIP ( EQUIP
EQUIP (
N_EQUIP )
EQUIP ( EQUIP
EQUIP (
N_EQUIP )

EARFCN_UL
38800 ( 38800 )
19294 ( 38800 )

EARFCN_DL
38800 ( 38800 )
1294 ( 38800 )

PRIORITY
7(7)
6(7)

38800 ( 38800 )
19294 ( 38800 )

38800 ( 38800 )
1294 ( 38800 )

7(7)
6(7)

38800 ( 38800 )
19294 ( 38800 )

38800 ( 38800 )
1294 ( 38800 )

7(7)
6(7)

19294 ( 19294 )
38800 ( 19356 )

1294 ( 1294 )
38800 ( 1356 )

6(6)
7(7)

19294 ( 19294 )
38800 ( 19356 )

1294 ( 1294 )
38800 ( 1356 )

6(6)
7(7)

19294 ( 19294 )
38800 ( 19356 )

1294 ( 1294 )
38800 ( 1356 )

6(6)
7(7)

EARFCN settings for eNode B and for Multiband should be the same
Else Handovers would get impacted

2016 Samsung Electronics

34

Active & Idle Mode All Three Bands


Service

Parameter

2.3G

Target
GBR Service (VoLTE)

2.3G
1.8G

850

1.8G

1.8G
2.3G

850

850

35

35

35

-105 dBm

-105 dBm

-105 dBm

-90 dBm

-90 dBm

-90 dBm

850M
2.3G

1.8G

QCI based A2 Threshold


QCI based A1 threshold (Meas Gap
Deactivation)

3 dB

3 dB

15+5 dB

3 dB

3 dB

3 dB

3 dB

3 dB

3 dB

QCI based A3 threshold


Non-GBR Service (FTP DL)

27

80

80

-113 dBm

-60 dBm

-60 dBm

-100 dBm

-50 dBm

-50 dBm

3 dB

3 dB

3 dB

A2 Threshold
A1 threshold(Meas Gap Deactivation)
A3 Threshold

Service
Target
Idle Mode

A5 Threshold (Th1)

-113 dBm

-113 dBm

-60 dBm

-113 dBm

-60 dBm

-113 dBm

A5 Threshold (Th2)

-110 dBm

-93 dBm

-110 dBm

-110 dBm

-110 dBm

-110 dBm

2.3G
1.8G

850

1.8G
2.3G

850

850M
2.3G

1.8G

14
-100
dBm

14
-100
dBm

Parameter

2.3G
Priority
QrxLevMin
S_NonIntraSearch

ThrehX_Low

ThrehX_High
2016 Samsung Electronics

850
5

64

64

64

-128

-128

-128

12

12
-104
dBm
12
-104
dBm

12
-104
dBm
12
-104
dBm

-104 dBm
12

Thresh_Serving Low

1.8G

-104 dBm

12
-104
dBm

12
-104
dBm

14
-104
dBm
14
-100
dBm

35

Neighbor Deletion Analysis

36

Neighbor Deletion Based on HO Success


Neighbor Table

Auto build by LSMR and UE based . Max NBR in table are 256

Key parameter are Neighbor cell PCI (Unique/Band), Cell ID , TAC, eNB ID

Wrong Neighbor impacting HO performance degradation

2016 Samsung Electronics

37

HO Performance Improvement
Radio Interface Failure are mainly happened by Poor RF after/Before HO Failures.

HO Failure Cases and Resolution

Wrong Neighbor Relation


2.UE Failed HO due to
wrong PRACH Sequence
1.HO Preparation done due to
wrong neighbor

UE
Real Target Cell
PCI : A

Source cell
Very far and signal is not detected

Wrong target
cell, PCI : A

Needs to turn on HO Success Rate Based NBR Deletion


The feature deletes Neighbor that shows HO failure count greater than Th1 if total HO Attempt is greather than Th2

PCI Confusion
Case1

Neighbor Cell
PCI : A

Source cell

Case2

Neighbor Cell
PCI : A

Source cell

Neighbor Cell
PCI : A

Very far but signal is detected

PCI Reallocation
SON PCI needs to be ON
Neighbor Cell
PCI : A
Overshooting, RF Optimization
needed

HO Command Reception Failure due to Poor RF


Too late HO, Needs to optimize RF condition

2016 Samsung Electronics

38

Neighbor Relation Table - Optimization


Methods for Wrong Neighbor deletions
MR Based This feature removes Neighbor Relation (NR) which cannot receive Measurement Report
(MR) messages among the NRs included when the network was initially created, so that only valid NRs
could be included in the Neighbor Table.
Deployed at Ahemdabad, Ujjain and Hyderabad

Neighbor Deletion based on HO Success This feature removes Neighbor Relation based on HO Success
Rate
Deployed in network
Neighbor Deletion based on NBRDEL-CAUSE This feature removes Neighbor Relation based on 2 cause
reported Cell not available & Unknown Target ID
Deployed in network
Manual Neighbor Deletion This is based on troubleshooting HO call failure observed in CSL/Truecall.

Not Applicable

2016 Samsung Electronics

39

Neighbor Deletion based on MR: Ahmedabad Plan


Day

eNB Configuration Parameter

Objective

PCI Conflict Notification

nrDelFlag = True (Enable


Feature)

Day 1

MR Based
Neighbor
Deletion

thNumMrNrDel = 3 (Count
of MR)

KPI & Performance Indicator Checks

Total NBR
Relation to
reduce after
Aggressive MR
based Deletion

thTimeNrDel = 1 (Days)

NBR Count
RRC Connection Setup Success Rate, ERAB Success
Rate, Attach Success Rate
Call Drop Rate, RRC Connection Re-establishment
Attempts and Success Rate
Handover Preparation Success Rate, Handover
Completion Success Rate
PCI Conflict Notification

thNumMrNrDel = 5 (Count
of MR)
Day 3

MR Based
Neighbor
Deletion
thTimeNrDel = 3 (Days)

NBR Count
Change in MR
Thresold to
reduce NBR
deletion count.

RRC Connection Setup Success Rate, ERAB Success


Rate, Attach Success Rate
Call Drop Rate, RRC Connection Re-establishment
Attempts and Success Rate

Handover Preparation Success Rate, Handover


Completion Success Rate

2016 Samsung Electronics

40

Neighbor Deletion based on MR: Ahmedabad


MR Based Deletion
Enabled MR based NR deletion on 13th Jan . Before enabling this feature the total NRs in Ahmedabad city are 407192.
on 14th Jan the MR based deletion algorithm deleted the most of the NBRs and the count on 15th Dec is 44K.

2016 Samsung Electronics

Date

NBR Addition

NBR deletion

1/12/2016

970

61

1/13/2016

1152

84

1/14/2016

28169

379783

1/15/2016

1187

12665
41

Handover Statistics

2016 Samsung Electronics

42

UE Based ANR
Overall Call Flow

#UE reports unknown PCI


#eNB orders UE to report eCGI by sending
RRCConnectionReconfig.
Point 1) Samsung eNB does not trigger
report CGI process for those UE with GBR
bearer. VoLTE UE does not report eCGI

#In order for UE to get eCGI information, UE


has to go to Idle mode with help of DRX mode
or Meausurement GAP (only applicable for
inter FA) to get eCGI information in SIB1 of the
target cell.
#Data Scheduling has higher priority than DRX
mode by 3GPP Standard.
Point2) If UE continuously receives or transmits
traffic, UE cannot go to DRX mode.
UE-based-ANR is triggered before LSM-based-ANR
2016 Samsung Electronics

43

MR Based Neighbor Deletion


Feature Description
This function removes Neighbor Relation (NR) which cannot receive Measurement Report (MR) messages among
the NRs included when the network was initially created, so that only valid NRs could be included in the Neighbor Table.

Feature Configuration
nrDelFlag = True
thNumMrNrDel = 5 in Ujjain (Range 0 to 1000K)
thTimeNrDel = 1 in Ujjain ( Range 1 to 7 days)
Ujjain Observations
Total eNB 84 , Neighbors 40,000
18Dec - Enabled MR based NR deletion
20Dec - MR based deletion algorithm deleted the most of the Neighbors.
24Dec 3413 Neighbor created mostly by UE based ANR.
Ujjain Analysis MR Based Neighbor Deletion
100.00%
99.80%

35000
30000

25000
20000
15000
5000
0

2100
400 1200

3413

3083

InterX2HOSuccRate

Number Of NBRs

45000 40000
40000

10000

InterX2 HO Attempts

HO Performance Stats
26

24

InterX2 HO Failure

15

18
70

91

99.60%

103

99.40%
99.20%
99.00%
98.80%
7557

5627

7107

12110

12880

9315

11888

22-Dec-15

23-Dec-15

24-Dec-15

25-Dec-15

26-Dec-15

27-Dec-15

28-Dec-15

98.60%

2016 Samsung Electronics

44

Neighbour Deletion based on HO Success Pre-Post Analysis 6 cities


In 6 cities analysis , 3K to 6K Neighbors having HO Attempt greater than 10 and very few neighbors having HO
Success less than equal to 5.
City

Total Neighbors
Daily Average

Ahmedabad
Bangalore
Hyderabad
Chennai
Kolkata
Bhopal

167225
380499
311600
191142
435667
92736

Neighbor Count HO Attempt Neighbors Count Attempt GT


Greater than 10
10 & HO Success less than 5
Daily Average
Daily Average
6600 (3.95%)
6000 (1.58%)
5500 (1.77%)
3200 (1.67%)
3700 (0.80%)
1250 (1.35%)

3
4
2
1.5
1.5
1

Neighbor Deletion threshold of HO Attempt GT 10 and HO Success less than 5 are not met in most of the cases
1 to 4% of neighbors having HO Attempt greater than 10 indicates less users.
Next Step Review the HO Attempts and HO Success every month for any optimization of threshold.

2016 Samsung Electronics

45

eNode B Prechecks

46

Pre-requisites/ Checklist (RAN)


1.

2.

3.

4.

5.

6.

..Contd.

SCFT: 100% completed and all sites to be radiating.


Run command RTRV-CELL-STS in all eNBs.
Cells with Status = DISABLE needs to be checked.
No Service affecting alarm in eNB
Run command RTRV-ALM-LIST in all eNBs.
Service impacting alarms needs to be rectified.
eNB on 5.0.0. all patches to be loaded , SW & FW Audits
Go to Configuration -> Software -> Verify. Select all eNBs and verify the Software and Firmware.
eNBs with NOK status, should be noted and packges should be upgraded in planned event.
Please refer Software/Firmware Verification in attached pre-requisite file.
OCNS to be removed
Run command MON-TEST in all eNBs.
eNBs with OCNS running should be noted. Run command TERM-TEST to remove OCNS, in all eNBs.
Please refer OCNS Check section for procedure in attached pre-requisite file.
Multiband/Golden Parameter Settings

Go to Configuration -> Parameter Manager -> Select appropriate parameter planner.


Golden Parameter: pkg_500new
Multiband Parameter : Multi_Carrier_23n18_CircleName (For eNBs having 2300 MHz and 1800 MHz bands)
Multi_Carrier_3band_CircleName (For eNBs having 2300 MHz and 1800 MHz bands)
Select all eNBs and compare the parameter manger.
eNBs with NOK status, should be noted and parameter planner should be applied in planned event.
Please refer Golden Parameter/Multiband Parameter Settings section for procedure in attached pre-requisite file.
PCI SON Notifications to be resolved

Go to SON -> Log Dump Management. Select the time period and get the dump.

From dump, check source and target eNBs and solve the PCI collision if both neighbors are required.

Please refer PCI SON Notifications section for procedure in attached pre-requisite file.

2016 Samsung Electronics

47

Pre-requisites/ Checklist (RAN)


7.

..Contd.

Inactivity Timer 30 Sec, All Sites on Same Power Option.


Run command RTRV-INACT-TIMER for all eNBs. Internal USER_INACTIVITY_TIMER should be 30 for all QCI.
Run command :CHG-INACT-TIMER for changing the USER_INACTIVITY_TIMER.

8.

Inter LSMR Neighbors to be added manually or using UE based ANR.


Inter LSMR required neighbor list would be provided by RF team.
Run command RTRV-NBR-EUTRA for provided cells.
In case, neighbor does not exist, it needs to be added using GUI.
Please refer Neighbour Addition/Deletion section for procedure in attached pre-requisite file.

9.

Inter Band Neighbors to be checked for 2300, 1800 & 850.


Run command RTRV-NBR-EUTRA for all cells and if inter band neighbor exists.
In case ,intra eNB inter band neighbor does not exist, neighbors need to be added using GUI.
Please refer Neighbour Addition/Deletion section for procedure in attached pre-requisite file.

10. PCI repetition within 2 km to be removed for all cases, Additional Audit for 4 km and first tier /second tier neighbor all PCI conflict cases to be plotted on map info and corrective actions identified.
Go to SON -> Log Dump Management. Select the time period and get the dump.
This dump needs to be provided to RF team, RF team will map conflict cases on map .
RF will recommend to change PCI of some sites.
PCI of sites can be changed using GUI.
Please refer PCI/EARFCN Change section for procedure in attached pre-requisite file.

2016 Samsung Electronics

48

Pre-requisites/ Checklist (RAN)


11.

..Contd.

Small Cells: Neighbor Definitions to be checked within Small cells (if in a cluster).
List neighboring sites of small cells and run RTRV_NBR_EUTRA on all cells.
If neighbors are found missing, add the neighbors using GUI.
Please refer Neighbour Addition/Deletion section for procedure in attached pre-requisite file.

12. RRH Log Analysis to ensure all sites radiating .


Run RTRV-RRH-STS for all eNBs.
Check if RRH are in operational state and NO OF FA =1.
13. EARFCN definitions to be audited for multiband operation for all elements (Macro & Small cells).
Run command RTRV-CELL-IDLE in all eNBs.
Check UL and DL EARFCN of all eNBs with EARFCN allocated to respective circle.
Please refer PCI/EARFCN Change section for procedure in attached pre-requisite file.

2016 Samsung Electronics

49

TrueCall Based Analysis

50

True Call Report Cell Wise RLF & X2 Fail (Sample)


eNB ID
RLF
1076
4147
2915
2166
2790
936
2918
318
1556
2919
4092
4242
1337
1418
1348

eNB ID
X2 Fail
1055
1386
2166
2176
2791
1266
2016 Samsung Electronics

Cell Name
5
13
109

16
14
1
147
78
4
24
40
38
98
5
2
4
21
3
25

17
13
11

123
5
13
1

5
23

16
3
14
7
3
266

17
56
11
80
104
19
21
56
70
19
79
15

18
112
96
8
35
124
86
53
2
8
61

30
4
5

24
27

35
22
3
66
24
41
12
29
1
28

4
9
7

17
1
168
76
447
5
43

18
1175
1108
18
67
26
50

36
20
4
2
26
1
48
13
114
28
22
125
2
7
3
50

Cell Name
35
1
4
980
19
2
32

37
104
76
10
16
61
13
17

48

15
14
23

49
12
39

1
3

7
4
1
55
1

36
328
6
11
5
3
21

1
21
17

37
17
22
25
37
20
3

9
20
18

48

Total

ACME Zone

353
339
313
283
268
235
233
225
213
174
146
134
127
125
124

New Mumbai JNPT


kalyan
New Mumbai JNPT
New Mumbai JNPT
New Mumbai JNPT
New Mumbai JNPT
New Mumbai JNPT
New Mumbai JNPT
kalyan
New Mumbai JNPT
thane central
south
New Mumbai JNPT
kalyan
west

49
1

469
4

1
1

Total

ACME Zone

1523
1336
1124
582
532
420

New Mumbai JNPT


thane central
New Mumbai JNPT
New Mumbai JNPT
kalyan
New Mumbai JNPT
51

Abnormal Release Causes


Abnormal Release Cause
GTP Pa th Fa i l ure

Abnormal Release Cause

RLC - Inva l i d Pa ra meter

eNB - Ra di o Li nk Fa i l ure

eNB - Rel ea s e Due To eNB Genera ted Rea s on

S1AP - Reduce Loa d In Servi ng Cel l

MAC - Inva l i d Cel l Id

RRM - UE S-TMSI Dupl i ca te


eNB Ti meout - Inter X2 Ha ndover Comma nd Compl ete
eNB Ti meout - RRC Connection Setup Compl ete Mes s a ge Not Recei ved
eNB Ti meout - RRC Connection Reconfi gura tion Compl ete Mes s a ge Not Recei ved
eNB Ti meout - Intra Ha ndover Comma nd Compl ete

eNB - Rees tabl i s hment Fa i l ure Due to Inva l i d State


S1AP - Ra di o Network Uns peci fi ed
eNB Ti meout - X2 SN Status Tra ns fer
eNB Ti meout - Inter S1 Ha ndover Comma nd Compl ete
RRM - MME Overl oa d
eNB Ti meout - S1Rel ocOvera l l Expi ry

eNB Ti meout - X2 Rel ocOvera l l Expi ry

eNB - Recei ved X2 Res et Reques t

eNB Ti meout - RRC Connection Rees tabl i s hment Compl ete Mes s a ge Not Recei ved

eNB Ti meout - Interna l Rees tabl i s h Control

eNB - ARQ Ma xi mum Retra ns mi s s i on

eNB Ti meout - RRC Securi ty Mode Compl ete Mes s a ge Not Recei ved

eNB - Ra di o Li nk Fa i l ure by RRC Connection Rees tabl i s hment

RLC - Ca l l Is Not Active

eNB - DSP Audi t RLC MAC Ca l l Rel ea s e

S1AP - Ca us e NAS Uns peci fi ed

S1AP - Authentica tion Fa i l ure

S1AP - HO Fa i l ure In Ta rget EPC eNB Or Ta rget Sys tem

eNB - ARQ Ma xi mum Retra ns mi s s i on by RRC Connection Rees tabl i s hment

eNB Ti meout - Interna l Da ta Tra ns fer Start

eNB - S1 SCTP Out of Servi ce

eNB Ti meout - S1 MME Status Tra ns fer Not Recei ved

X2AP - Ca us e Mi s c Uns peci fi ed

MAC - Inva l i d Pa ra meter

eNB Ti meout - RRC UE Ca pa bi l i ty Informa tion Mes s a ge Not Recei ved


eNB Ti meout - S1 Ini tia l Context Setup Reques t
eNB Ti meout - S1 Pa th Swi tch Reques t Ack Not Recei ved
eNB - Recei ved Cel l Rel ea s e Indi ca tion from eNB

RRM - MME Not Ava i l a bl e for PLMN


eNB Ti meout - Interna l Res ource Setup
GTP Setup Fa i l ure
PDCP - Inva l i d RB Id
X2AP - Cel l Not Ava i l a bl e
S1AP - Ra di o Connection Wi th UE Los t
S1AP - Loa d Ba l a nci ng TAU Requi red
S1AP - CS Fa l l ba ck Tri ggered
eNB - Recei ved Res et Reques t From eNB
RRM - CQI Res ource Fa i l ure
GTP Recei ve Error Indi ca tion From GW
X2AP - Trel ocprep Expi ry

2016 Samsung Electronics

52

True Call Sample Report Abnormal Release


Abnormal Release Cause

Mumbai

12474
S1AP - Reduce Loa d In Servi ng Cel l
12151
RRM - UE S-TMSI Dupl i ca te
8547
eNB Ti meout - Inter X2 Ha ndover Comma nd Compl ete
5796
eNB Ti meout - RRC Connection Setup Compl ete Mes s a ge Not Recei ved
4367
eNB Ti meout - RRC Connection Reconfi gura tion Compl ete Mes s a ge Not Recei ved 4214
eNB Ti meout - Intra Ha ndover Comma nd Compl ete
3020
eNB Ti meout - X2 Rel ocOvera l l Expi ry
2613
eNB Ti meout - RRC Connection Rees tabl i s hment Compl ete Mes s a ge Not Recei ved1625
eNB - ARQ Ma xi mum Retra ns mi s s i on
729
eNB - Ra di o Li nk Fa i l ure by RRC Connection Rees tabl i s hment
628
eNB - DSP Audi t RLC MAC Ca l l Rel ea s e
477
S1AP - Authentica tion Fa i l ure
215
eNB - ARQ Ma xi mum Retra ns mi s s i on by RRC Connection Rees tabl i s hment
196
eNB - S1 SCTP Out of Servi ce
187
X2AP - Ca us e Mi s c Uns peci fi ed
164
eNB Ti meout - RRC UE Ca pa bi l i ty Informa tion Mes s a ge Not Recei ved
99
eNB Ti meout - S1 Ini tia l Context Setup Reques t
63
eNB Ti meout - S1 Pa th Swi tch Reques t Ack Not Recei ved
36
eNB - Recei ved Cel l Rel ea s e Indi ca tion from eNB
32
GTP Pa th Fa i l ure
30
eNB - Ra di o Li nk Fa i l ure

2016 Samsung Electronics

21.57
21.01
14.78
10.02
7.55
7.29
5.22
4.52
2.81
1.26
1.09
0.83
0.37
0.34
0.32
0.28
0.17
0.11
0.06
0.06
0.05

Kolkata

8276
2235
5274
4117
2274
1408
2771
1248
2072
786
308
358
18
127
26
12
33
8
18
22
987

25.47
6.88
16.23
12.67
7.00
4.33
8.53
3.84
6.37
2.42
0.95
1.10
0.05
0.39
0.08
0.04
0.10
0.03
0.05
0.07
3.04

Delhi

6756
1242
8753
5809
4141
2053
2955
2728
2499
1050
283
440
55
162
417
77
43
114
65
32
127

16.88
3.10
21.86
14.51
10.34
5.13
7.38
6.81
6.24
2.62
0.71
1.10
0.14
0.41
1.04
0.19
0.11
0.28
0.16
0.08
0.32
53

Abnormal Release Trend


True Call Implemented
Only in Gujrat, Mumbai, Delhi, AP,
Karnataka, Kolkata, MP

5.00
4.80
4.60
4.40
4.20
4.00
3.80
3.60
3.40
3.20
3.00

51931704

4.50

50000000

44683001
4.24

39154166

60000000

53543901

4.22

40000000

3.98

30000000

Release Count

% Abnormal Failure

Total Abnormal Release (%) (PAN India)

20000000
2289216

1980235

1843142

2218316

10000000
0

Dec Week3

Dec Week4
Total Release

Jan Week1
Abnormal Release

Jan Week2
% Failure

800000

Absolute Failures

700000
600000
500000
Dec Week3

400000

Dec Week4

300000

Jan Week1

200000

Jan Week2

100000
0
Gujrat

Mumbai

Delhi

AP

Karnataka

Kolkata

MP

Circles With True Call Implemented


2016 Samsung Electronics

54

Abnormal Release Breakup (4th~10th Jan)


Sr No.

Failure Cause

Mumbai

AP

Delhi

Gujrat

Karnataka Kolkata

MP

eNB - Radio Link Failure

21.59%

25.98%

16.90%

31.32%

20.15%

25.48%

27.68%

S1AP - Reduce Load In Serving Cell

21.03%

2.89%

3.11%

5.45%

6.74%

6.88%

5.94%

RRM - UE S-TMSI Duplicate

14.80%

15.81%

21.90%

17.40%

17.09%

16.24%

18.63%

eNB Timeout - Inter X2 Handover Command Complete

10.03%

9.71%

14.53%

6.77%

9.35%

12.68%

8.09%

eNB Timeout - RRC Connection Setup Complete Message


Not Received

7.56%

10.90%

10.36%

7.04%

13.29%

7.00%

8.24%

eNB Timeout - RRC Connection Reconfiguration Complet


e Message Not Received

7.30%

4.40%

5.14%

4.79%

5.14%

4.33%

4.83%

eNB Timeout - Intra Handover Command Complete

5.23%

3.76%

7.39%

4.04%

5.27%

8.53%

4.90%

eNB Timeout - X2 RelocOverall Expiry

4.52%

5.38%

6.83%

5.09%

5.07%

3.84%

6.26%

eNB Timeout - RRC Connection Reestablishment Complet


e Message Not Received

2.81%

5.33%

6.25%

8.83%

7.14%

6.38%

6.83%

10

eNB - ARQ Maximum Retransmission

1.26%

1.81%

2.63%

2.00%

1.80%

2.42%

1.96%

11

X2AP - Cause Misc Unspecified

0.28%

7.93%

0.19%

0.66%

4.20%

0.04%

1.89%

12

GTP Path Failure

0.05%

1.41%

0.32%

0.17%

1.16%

3.04%

0.30%

2016 Samsung Electronics

55

Detail Analysis for Mumbai (7th Jan)

Total Release

Abnormal Release

% Fail

1781047

79516

4.46

Out of 79516 Abnormal release more than 95% failures contributed by below shown 10 Cause codes.
25.00

% Contribution to Total Failures

20.99

20.87

20.00

14.02

15.00

11.92
10.00

7.14

6.63
4.56

5.00

4.56

2.79

1.61

0.00
S1AP - Reduce
Load In Serving
Cell

2016 Samsung Electronics

eNB - Radio Link


Failure

RRM - UE S-TMSI eNB Timeout - eNB Timeout - RRC eNB Timeout - RRC eNB Timeout - X2
Duplicate
Inter X2 Handover
Connection
Connection Setup
RelocOverall
Command
Reconfiguration Complete Message
Expiry
Complete
Complete Message Not Received
Not Received

eNB Timeout - eNB Timeout - RRC


S1AP Intra Handover
Connection
Authentication
Command
Reestablishment
Failure
Complete
Complete Message
Not Received

56

Analysis on Reduce load in serving cell


Specific device model is contributing to this release cause
Sony, Motorola and Unknown device types
Make

Model

Total Failure

MOTOROLA MOBILITY LLC A LENOVO COMPANY

MOTOROLA LX12506245

19701

SONY MOBILE COMMUNICATIONS

TBD

7296

MOTOROLA MOBILITY LLC A LENOVO COMPANY

INDR001345

4414

MOTOROLA MOBILITY LLC A LENOVO COMPANY

INDR000945

3177

N/A

TAC: 35234207

2915

N/A

TAC: 35236007

1856

MOTOROLA MOBILITY LLC A LENOVO COMPANY

MOTO E (2ND GEN) WITH 4G LTE ST12424645

1818

N/A

TAC: 35548407

1265

N/A

TAC: 35235907

902

SONY MOBILE COMMUNICATIONS

XPERIA E4G DUAL

859

MOTOROLA MOBILITY LLC A LENOVO COMPANY

CA12459845

730

N/A

TAC: 35386307

488

N/A

TAC: 35235607

186

N/A

TAC: 35627407

117

N/A

TAC: 35627307

103

2016 Samsung Electronics

57

Analysis on Reduce load in serving cell


Device Capability Issue
If device does not support TDD-FDD HO or Inter FA HO, eNB sends the UE to other carrier with
Redirection with release cause of Reduce Load in Serving Cell in current eNB Pkg, instead of
Handover
Needs to check Feature Group Indicator 13, 25 and 30 in UE Capability Information for all devices

Call Flow
UE

MME

eNB 862

MR EVENT A2(LteHo)
RRC Conn ReConfiguration
RRC Conn Reconfiguration Comp
MR EVENT A5(IntraLteHandover)

RRC Connection Release (with redirectedCarrierInfo)

As UE does not support Inter frequency HO or


FDD TDD handover then eNB will initiate redirection

UE Context Release Request(Reduce load in serving cell(19))


UE Context Release Command
UE Context Release Complete
2016 Samsung Electronics

58

Radio Link Failure Analysis

Out of 16596 RLF observed in entire Mumbai, 11.6% fails happened in 11


cells.

Top 3 Cell Analysis

4294.16 is IBS of TC22, and supporting data for further analysis not available
from CSL for 05/01/2016.

Cell ID

Total Fail

4294.16

473

2761.50

217

2257.16

201

For 2761.5 & 2257.16,finding given below.

4294.17

171

In both cell, specific single IMSI is contributing to most of the fails

2257.17

154

820.37

143

4142.37

129

1076.18

114

2918.37

110

75.35

108

1076.37

104

Total

1924

eNB Id

2761.50
I-MU-KLYN-ENB-6008

2257.16
I-MU-MUMB-ENB-7334

2016 Samsung Electronics

IMSI Detail

Total
Failure

405874000046676

212

405874000025017

405874000047743

405874000026883

199

405874000062987

405874000016480

Remarks
Outskirt facing cell having no site within
2-3 km area. Failures from far location
suspected. Last CQI Reported is of Value
5 (SINR < 0).
Also fails observed in one specific IMSI
only.
No site upto 1 km and very high rise
clutter observed. Failures from far and
high-rise indoor location suspected. Last
CQI Reported is Of Value 3 (SINR < 0).
Also fails observed in one specific IMSI
only.

59

Radio Link Failure (2761.50 - I-MU-KLYN-ENB-6008)

Outskirt facing cell having no site within 2-3 km area. Failures from far location suspected.

Also fails observed in one specific IMSI only.

2761.50

> 1 km

2016 Samsung Electronics

60

Radio Link Failure (2217.56 - I-MU-MUMB-ENB-7334)

No site upto 1 km and very high rise clutter observed. Failures from far and high-rise indoor location suspected.

Also fails observed in one specific IMSI only.

Non-Integrated Site

> 1 km

2217.56

2016 Samsung Electronics

61

Inter X2 HO Fail Analysis

Out of 9481 fails observed in entire Mumbai, 21% fails happened in 1 cells only.

Cell ID

Total Fail

1350.18

1953

Top 1 Cell Analysis

When checked IMSI wise fails, only one IMSI contributing to 77% of the fails.

Need to trace that IMSI and do the further testing.

For remaining fails in this cell, wrong neighbor deletion/PCI conflict to be checked.

eNB Id

1350.18
I-MU-MUMB-ENB-1327

2016 Samsung Electronics

IMSI Detail

Total Failure

405874000050068

1498

405874000051815

61

405874000064118

53

Other IMSI

308

62

[Issue-1] Missing Events

TrueCall is not capturing all the events.

As seen below, in TrueCall Logs, there are no events between 01/06/16 1:26:56 to 1:28:41, whereas, as per CSL logs, 3
events occurred during same time.

TrueCall Logs

CSL Logs
eNBNAME

Call Release
Cause

eNB_1484
eNB_2166
eNB_4241
eNB_4103
eNB_3417
eNB_964
eNB_4091
eNB_1662
eNB_2686

0x00000FFF
0x00000FFF
0x00000FFF
0x00000FFF
0x00000308
0x00000324
0x00000236
0x00000343
0x00000308

2016 Samsung Electronics

Call Attempt Time

Call Setup Time

2016-01-06 10:55:48:650 10:55:48:744


2016-01-06 11:19:49:869 11:19:49:962
2016-01-06 12:04:18:839 12:04:18:944
2016-01-06 13:12:01:035 13:12:01:190
2016-01-06 13:27:25:845 255:255:255:65535
2016-01-06 13:26:57:707 13:26:57:801
2016-01-06 13:28:15:081 255:255:255:65535
2016-01-06 13:28:28:082 13:28:28:188
2016-01-06 13:30:44:682 255:255:255:65535

FreqBandI cgi.PLMN
ndicator
ID

40
3
40
40
40
40
40
40
40

405 874
405 874
405 874
405 874
405 874
405 874
405 874
405 874
405 874

cgi.CI

379921
554532
1E+06
1E+06
874768
246802
1E+06
425488
687633

PCI

Call Release Time

PCI cgi.PLMNID

76 2016-01-06 10:56:17:923 76
208 2016-01-06 11:20:21:337 208
332 2016-01-06 12:05:08:922 332
501 2016-01-06 13:12:27:062 501
252 2016-01-06 13:27:28:855 252
236 2016-01-06 13:27:30:859 236
501 2016-01-06 13:28:16:620 501
3 2016-01-06 13:28:41:234 3
181 2016-01-06 13:30:47:693 181

cgi.CI

405 874 379921


405 874 554532
405 874 1085714
405 874 1050384
405 874 874768
405 874 246802
405 874 1047313
405 874 425488
405 874 687633

IMSI

405874000017389
405874000017389
405874000017389
405874000017389
405874000017389
405874000017389
405874000017389
405874000017389
405874000017389
63

CSL Based
X2 Failure Analysis
For Wrong Neighbour

64

[0x0308]EccTmout_InterX2HandoverCmdComplete Case Study

[0x0308] Tags When A call is released when the target eNB cannot receive the RRC Connection Reconfiguration Complete message
during the X2 handover.

As Per UE Logs UE has sent MR For 118 PCI which It Found better so A3 Event Triggered. But After that UE has Started PRACH
Procedure but failed as done on wrong Cell.

Source cell
PCI 53

UE

Wrong Target cell


PCI 118

Actual Target
PCI 118

A3 Event Triggered
MR

HO Preparation

HO Command
PRACH FAIL MSG2

3000ms

EccTmout_InterX2HandoverCmdComplete
2016 Samsung Electronics

65

How To Identify Wrong Nbr Based Upon CSL (1/2)


Target CI

Call Release
Cause

Call Attempt Time

Call Setup Time

0x00000308
0x00000308
0x00000308
0x00000308
0x00000308
0x00000308
0x00000308

2016-01-04 00:05:04:044
2016-01-04 00:05:45:940
2016-01-04 00:05:46:505
2016-01-04 00:06:48:003
2016-01-04 00:06:50:607
2016-01-04 00:07:17:781
2016-01-04 00:07:20:331

255:255:255:65535
255:255:255:65535
255:255:255:65535
255:255:255:65535
255:255:255:65535
255:255:255:65535
255:255:255:65535

Source CI

cgi.CI[0] cgi.CI[1]
689445
354851
542225
887077
947728
557073
212259

89125
287762
936466
735781
869155
239633
541475

IMSI

PCI

405874000000000
405874000000000
405874000000000
405874000000000
405874000000000
405874000000000
405874000000000

203
147
31
323
45
4
105

By CSL we get Cause Code/ Cell Id From UE History, so that we can Find Source & Target eNB .

Please Check Call Release Cause 0x0000 308 is for A call is released when the target eNB cannot receive the RRC
Connection Reconfiguration Complete message during the X2 handover
By True call Please check Column Name Last HO Source Cell ID & Last HO Target Cell ID, For Call
Release Casue eNB Timeout eNB Timeout - Inter X2 Handover Command Complete
2016 Samsung Electronics

66

How To Identify Wrong Nbr Based Upon CSL (2/2)


eNB275

Wrong Id

eNB1231

Correct Id
eNB2155

eNB2155
Source Id

Wrong
Target Id

2155.37 275.17

Distance (Km)

Total
Failure

5.07

172

Source PCI Target PCI Nearst eNB Nearst Distance

53

118

1231

0.76

Remarks

DELETE

Ue has sent measurement report for eNB1231 PCI 118 but UE Does RACH With eNB275, Due to Wrong eNB
Defined in eNB2155 PCI 53.
Hence Handover Fail So we can Check Nearby Same PCI & Delete Wrong One To Reduce Drop Rate by Handover Fail

S1 AP Links Uptime Very Critical

21-Jan-2016

68

UE Temp IDs S1AP, GTP-C, GTP-U

2016 Samsung Electronics

69

SCTP Link Fluctuation MT UE in IDLE Mode


MT UE was connected to eNB-1 and moved to IDLE mode due to User Inactivity.
Incoming call received for the UE.
As the UE was in IDLE mode, MME has sent Paging Request to eNB-2.
As the SCTP link with the eNB-1 where UE is currently present, UE will not be able to recei
ve Paging Request message

2016 Samsung Electronics

70

SCTP Link Fluctuation MO Call


MO UE attached to MME-1 and GW-1 via eNB 1 and VoLTE call is on-going
SCTP link to MME-1 with eNB-1 and eNB-2 are down.
UE sent Service Request to MME-2 via eNB-2(as eNB2 -> MME-1 link is down)
MME-2 will send service Reject which results in UE sending Fresh Attach Request.
As UE sent new Attach, the previous call gets dropped

2016 Samsung Electronics

71

Network issue related to Backhaul

Introduction
GTP Loss
GTP Loss
Root Cause of GTP Loss
Statistic & Troubleshooting

MME&IND Communication Fail


MME&IND communication
Root Cause of MME&IND Communication
Statistic & Troubleshooting

X2 Communication Fail
X2 communication Fail
Root Cause of X2 Communication Fail
Statistic & Troubleshooting

2016 Samsung Electronics

73

GTP Loss GPRS Tunneling Protocol (GTP) is a group of IP-based communications protocols
used to carry general packet radio service (GPRS) within LTE networks
GTP Loss means packet drops between SAEGW and eNB. It is only calculated on
bearer down link(SAEGW --> eNB direction).
GTP Loss can happen only by user traffic but cannot occur by ping or traceroute
between SAEGW and eNB.
In case of Samsung SGW & PGW both are at same location (called SAEGW). So we
will always check GTP loss between SGW and eNB.

2016 Samsung Electronics

74

Root Cause of GTP Loss There are two reasons for GTP loss (packet drop) in backhaul Buffer Overflow (Backhaul Bandwidth & Buffer size defined)
Physical drops

1- Buffer OverflowWhen network element send data packets with high speed and receiving node
cannot forward it with the same speed due to bottleneck in network, And has small buffer t
o store packets so it will start discarding excess packets once its buffer gets
full. This situation is called Buffer overflow.

2- Physical drops The next common issue that can lead to drops would be a physical component
that is malfunctioning. Physical issues like SFP module problem, cable problem,
loosed connection, line card problem, duplex mismatch, etc. are the most
common reasons for packet drop in network.
These physical drops and backhaul media Buffer overflow results in low
throughput.

2016 Samsung Electronics

75

GTP Loss- Trouble-shooting


Check GTP loss counter(GtpSnEnbDeltaLossRate) in LSMR and if GTP loss more than 0.01%
observed then check for below 3 cases
1. Physical drops (drop rate requirement is less than 0.01%).
2. Small buffer in the bottleneck (Buffer size < 512KBytes and bottleneck bandwidth <= 25
0Mbps).
3. Low bandwidth in the bottleneck (bottleneck bandwidth < 100Mbps).
Even though low throughput is observed in the field, if the above 3 cases are
not in backhaul, definitely there's no backhaul issue causing low throughput.
Cisco team can only identify case # 1 in the above 3 cases. So Cisco team
should not declare that backhaul is clean without buffer size and rate limitation
or bottleneck bandwidth confirmation. Only by ping test, buffer size issue and
wrong rate limitation issue cannot be identified.

2016 Samsung Electronics

76

76

MME&IND Communication Fail MME is responsible for idle mode UE (User Equipment) paging and tagging
procedure including retransmissions. It is involved in the bearer activation/
deactivation process and is also responsible for choosing the SGW for a UE at
the initial attach and at time of intra-LTE handover involving Core Network (CN)
node relocation
MME communicates with eNB using S1-AP/S1-MME interface.
Initially SCTP connection is established between eNB and MME by exchanging SCTP messag
e and once SCTP connection is established then it is used for communication using S1-APset
up as shown below-

2016 Samsung Electronics

77

MME&IND Communication Fail


The trend of IND Fail with the worst sample circle (GJ, MH, MP, MU)
MME ID 11-Dec. 13-Dec. 15-Dec. 16-Dec. 21-Dec. 23-Dec. 04-Jen. 06-Jan. 15-Jan. 18-Jan. 20-Jan.
MME[4]
144
159
144
144
145
159
143
143
141
141
141
GJ
MME[5]
1
2
1
1
1
1
7
7
2
MME[0]
279
295
295
148
146
150
151
160
20
21
23
MME[3]
50
64
72
65
46
67
57
73
10
17
11
MH
MME[4]
4502 4546 4552 2344 2330 2360 2370
2374
50
57
44
MME[5]
40
21
20
26
22
25
19
21
5
11
5
MME[3]
2
1
2
1
4
22
6
7
1
1
MP MME[4]
4060 4502 4501 4504 4499 4510 4522
4518
55
109
3
MME[5]
2
3
3
3
4
9
3
4
2
2
2
MME[1]
237
253
234
242
205
222
24
18
10
8
46
MU MME[2]
228
247
227
237
208
267
114
86
82
51
95
MME[3]
223
243
221
226
203
260
118
83
80
51
94

In case of MH and MP Circle,

the IND Fail were cleared after CISCO do


in the SAR.

IND Communication Fail Post MME Pool split


Circle
Count

AP
12

AS

BR
3

21

DL
12

GJ
150

HR
20

JK
64

KA
11

KO
13

(Based on Site Count)

MH
76

MP
8

MU
185

PB
326

RJ

TN
3

10

UE
35

UW
31

WB
137

Total
1,117

Intensive inspections and improvements required


(GJ, MU, PB, WB)

1. Missing MME Interface for 1100 eNB added.


2. MME Interface not required in MME Pool deleted as Database cleanup activity.
3. eNB grown before MME Pool activity having Long time IND MME alarms since they were grown with old MME Pool.
Next Step Above step to continue for 1 week to clean IND communication alarm

2016 Samsung Electronics

78

Root Cause MME&IND Communication Fail There are two reasons for MME Communication Fail in backhaul SCTP Connection failure
S1-AP setup failure

Troubleshooting SCTP connection failure-IPv6 configuration(LTE_Signal_S1 should be true for 102 Vlan only)
-eNB to MME ping check.
Troubleshooting S1-AP setup failure-MME configuration(MME status should be EQUIP and Administrative state should be Unlock
ed).
-eNB must be defined in MME .
-Parameters related to eNB like MCC, MNC, Enb_ID, S1-AP_IP etc are defined correctly
in the MME.

2016 Samsung Electronics

79

X2 Communication Fail X2 handover is performed between a source eNB and a target eNB through the X2
interface.
In LTE networks allow eNBs to directly exchange status information with each other via the
X2 interface, and to independently perform handovers without any
intervention by EPC nodes.

2016 Samsung Electronics

80

X2 Link Status
The Trend of X2-Link disabled.
Observation from Samsung
- Missed configuration at eNB.
- Wrong IP in Neighbor list.
- Parameter set with neighbor
Unlocked
Observation from CISCO
- Missed configuration at AG1
- MTU Size different between eNB and

The count of X2-Link (Two-Way) disabled wit Circle basis.

BH
- Duplicate IP assigned
- Routing issues at CSS
- Physical Link issues
Observation from Juniper
- Neighbor discovery protocol issues.
- Routing issues at CSS
- Physical Link issues

2016 Samsung Electronics

81

Root Cause X2 Communication Fail There are two reasons for X2 Communication Fail in backhaul SCTP Connection failure
X2-AP setup failure

Troubleshooting SCTP connection failure-IPv6 configuration(LTE_Signal_X2 should be true for 101 Vlan only)
- Source eNB to Neighbour eNB ping check for Vlan 101.
Troubleshooting X2-AP setup failureSamsung-eNB IP Correction in eNB neighbor list.
-eNB Administrative State in both neighbors to be "Unlocked"
- "NO_X2State" should be "False" instead of "True" in both the neighbors.
Cisco-MTU Configuration Check & Correction .
-Two IP addresses for single eNB, removing the multiple IP's.
-AG1 Configuration change for boundary Cases
-Correction of routing issue at CSS and other nodes.
Juniper-Correction of routing issue at CSS.
2016 Samsung Electronics

82

GTP LOSS Troubleshooting

GTP Loss Resolution

2016 Samsung Electronics

84

Table of content
What is GTP loss
Why GTP loss happens
Where & How to see GTP loss
Troubleshooting steps
Case Study

Background
TCP Congestion : Packet loss has generated by network congestion as well as other things.
Speed of packets sending > network capacity Congestion
The packet loss happened intermittently, even if packets sending under network capacity.

TCP congestion control


HSS
S6a

MPLS Network

GTP-U

GTP-U
GTP-C
GTP-U

FTP

2016 Samsung Electronics

86

TCP Packet loss


Intermittently, the packet loss occurs by Rate limitation in the normal network
environment.
Even if the packet loss happened with an unspecified that it does not stop the DT test.

If constantly experiencing packet loss or find out the Rate limitation


then stop the DT test and have the action in order to fix it.
The MOP that is BH checking process already shared by RAN TF

If you find out the Low Throughput somewhere during test in the field.

As you well known, Please check the RF Condition including eNB status (Is it down or not ?)
Please check the current alarm (Related BH, Related RU, Related RF, Etc.)
Check the Rate limitation using shared MOP
If you do not find out the reason, then call the RAN BH TF

2016 Samsung Electronics

87

What is GTP Loss


GPRS Tunneling protocol is an important IP based protocol used in GSM, UMTS and LTE core networks. It
is used to encapsulate user data when passing through core network and also

carries bearer specific si

gnaling traffic between various core network entities.


It provides mobility. When UE is mobile, the IP address remains same and packets are still

forwarded

since tunneling is provided between PGW and eNB via SGW.


Multiple tunnels (bearers)can be used by same UE to obtain different network QoS
Main IP remains hidden so it provides security as well.

2016 Samsung Electronics

88

What is GTP Loss (Contd)


Note-

GTP Loss means packet drops between SAEGW and eNB.


It is only calculated on bearer down link(SAEGW --> eNB direction).
GTP Loss can happen only by user traffic but cannot occur by ping or traceroute

betwee

n SAEGW and eNB.


GTP tunnel is formed only between SAE-GW and eNB so GTP loss has no relation with UE
RF conditions.

2016 Samsung Electronics

89

Detail picture showing data flow inside GTP tunnel-

2016 Samsung Electronics

90

90

GTP OOS
When UE tries to download any data then TCP session is established between UE and server.
When this data comes to SAE-GW from server then It encapsulates this data using
P, IP, L2 and L1 headers and forwards this data to eNB through GTP tunnel.

GTP, UD

GTP header consists of Sequence no. & tunnel end point Id (TE ID) which is assigned to each
packet.
This encapsulated data transfers through GTP tunnel with Sequence no. 1,2,3,4,5.. and so
on and reaches to eNB.
If after 2nd packet eNB receives 4th packet instead of 3rd packet then this event would be cou
nted as loss in GTP loss counter but after receiving 3rd packet this event would be counted as
Out of Sequence packet(OOS) as shown below-

2016 Samsung Electronics

91

91

GTP Delta Loss

That means this packet was not lost but came after words (Out of sequence packet) so this will
not contribute to GTP loss. This can be seen in GtpSnEnbDeltaLoss column which is calculated as-

2016 Samsung Electronics

92

92

Why GTP Loss happens


There are basically two reasons for GTP loss (packet drop) in backhaul Buffer Overflow
Physical drops

Buffer OverflowWhat is Buffer in a network device?

-Small amount of memory used for temporary storage of data, usually to compensate for differe
nces in between receiving and transmitting speeds on a network device. It serves as a reservoir i
n which the higher speed ingress data to be dumped which is then 'trickled' to the slower one.
What is Buffer overflow?
-When server is sending data packets with very high speed and receiving node cannot
forwar
d it with the same speed due to bottleneck in network and it has small buffer to store packets
so it will start discarding excess packets once its buffer gets full .
This situation is cal
led Buffer overflow.

2016 Samsung Electronics

93

93

Buffer Overflow

Reason for Buffer overflowRate MismatchCase 1

2016 Samsung Electronics

Case2

94

94

Physical Drops
Physical dropsThe next common issue that can lead to drops would be a physical component that is

malfunct

ioning.
-Physical issues like SFP module problem, cable problem, loosed connection, line card
m, duplex mismatch, etc. are the most common reasons for packet drop in a

proble

network.

-If hardware is not working properly, it will usually lead to error messages being seen on the con
sole of the device or within system logs but there are other software issues also which can lead t
o packet drop and main concern is there is no alarm indication

mechanism for these issu

es in the system

2016 Samsung Electronics

95

95

GTP Loss As an Indication


GTP loss cant be avoided in commercial mode when multiple TCP session
establishes to access data ,making congestion in network bottleneck point.

GTP Loss cannot be used as any index or indicator which means backhaul

normal

ity because of speed mismatch in backhaul environment (10G -> 1G on AG1 to CSS
ring, 1G -> 470, 250, 150, 100, 45 on Microwave or Leased

ttleneck buffer overflow cannot be avoidable in


Multi TCP sessions keep trying to increase TCP

line), GTP Loss by bo

commercial mode where


window size so congestion happ

ens in a bottleneck point and so buffer overflow occurs.

2016 Samsung Electronics

96

96

Where and how to see GTP Loss


GTP loss can be seen in LSMR in performance statistics or On demand monitoring.
In GTP statistics, there are two items of GTP Loss and GTP OOS. The real packet loss can be calcul
ated by (GTP Loss Count - GTP OOS count) and it's shown in GtpSnEnbDeltaLoss. So GtpSnEnbDe
ltaLossRate shows the real loss rate.
How to check GTP loss ?
-Generate the traffic on the UE and check the GTP loss counter (GtpSnEnbDeltaLossRate) in on d
emand monitoring.

Login to LSMR.
Go to PERFORMANCE-On Demand.
Select target.
Register-In Statistics family, go to GTP & select GTP sequence number per eNB-period (30sec)duration (
as per testing requirement).Then click on register.
After this click on site which is registered then Monitor option will be highlighted. Click on monitor and select
GtpSnEnbDeltaLoss(count) and GtpSnEnbDeltaLossRate(%)-click Apply and then click
Start.

NOTE-GTP Loss must be less than equal to 0.01% for proper throughput. (>50Mbps)

2016 Samsung Electronics

97

97

Example Report
Example-

Note- Please note that the GTP loss counter is only valid for downlink throughput
analysis.
2016 Samsung Electronics

98

98

Troubleshooting Steps
Check GTP loss from LSMR as explained above and if GTP loss occurs perform IPERF
tween SGW and eNB to check

test be

Physical drop
Backhaul Bandwidth
Buffer size

Put iperf.octeon64 software in LENA and in eNB, use below command for Iperf test in
k.

Downlin

eNB Side:
iperf.octeon64 -s -V -B eNB_Bearer_IP -u -l 8000 -i 1 -p 8001
SGW(LENA) :

iperf.octeon64 -c eNB_Bearer_IP -u -V -i 1 -b 1m -l 8000 -t 9999 -p 8001


Note Change the mode of iperf file using command chmod777 iperf.octeon64
be
fore executing commands.
Where,
s->server, V->IP version6, B->binds eNB IP, u->UDP traffic, l 8000->length of datagram size(bytes),
p->port, c->client, t->time, b->B.W. to be pumped.
First step is to check if any physical drop is happening.

2016 Samsung Electronics

99

99

Identification
How to identify Physical issue ?
- Perform IPERF test between SGW and eNB on lower B.W.(less than equal to 10Mbps)
- If packet drop is observed while pumping lower B.W from SGW to eNB then that means there is
some physical issue in backhaul.
-Ask cisco team to check backhaul.
Note- Please note that on Lower B.W. (<=10Mbps) packet loss should be less the 0.01%.
If no drop observed on lower B.W, proceed further to check if any bottleneck present in backha
ul by performing iperf test on higher B.W.
How to find Backhaul bandwidth ?
- Initiate ping to eNB from SGW to check latency.
- Start sending traffic from SGW to eNB using Iperf. (Make server side ready before
g traffic i.e. command at eNB should be executed first in DL).

2016 Samsung Electronics

sendin

100

100

Examples
SAEGW IPERF O/P:
SGW is sending 100Mbps dummy traffic to eNB as shown below.

Now check at eNB how much traffic is being received by eNB .


eNB IPERF O/P:

2016 Samsung Electronics

101

101

Examples
-eNB is receiving 100Mbps traffic as shown above. If packet drop greater than 0.01%
observe
d then decrease B.W. (dummy traffic) and perform test or else increase the traffic and check the s
ame.
-Keep monitoring at eNB end, if continuous packet drop is observed on each instant.
-If continuous drop is observed then decrease B.W. (which is being sent from SGWeNB) or else
increase B.W. and monitor at eNB.
-Keep monitoring ping latency from SGWeNB if it is increasing. (Increasing latency
indicate
s buffer is getting full)
-Keep doing this process until you get a bottleneck point (B.W.) after which you get
CONTIN
UOUS packet drop at eNB.
This point would be your rate limit of backhaul.
ExampleSuppose while pumping 237Mbps traffic from SGW to eNB you are getting 0% packet
drop on
each instant and while pumping 238Mbps traffic you are getting continuous
packet drop at e
ach instant and ping latency is also increasing at 238Mbps then you can say that 237Mbps is bac
khaul BW or rate limit of backhaul.
NOTE-While performing iperf test you may find a situation where intermittent huge
packet
drop is observed. Even though packet drop is more than 0.01% but this point is not rate limit p
oint because loss is not continuous.
2016 Samsung Electronics

102

102

Examples
How to find Bottleneck Buffer size ?
-Once you start getting continuous packet drop at eNB, you will observe that your ping
eNB ping) latency is increasing.
-Keep observing it until you observe constant latency.
-Once you observe constant latency, stop (Ctrl+c) all three commands (traffic sending,
eceiving and ping) in all three windows.
-Now in ping window you will observe min/avg/max RTT as shown below-

2016 Samsung Electronics

(SGW

traffic r

103

103

Buffer Calculation

-Once you calculated the RTT, find Buffer size using Buffer estimation chart shown below-

2016 Samsung Electronics

104

104

Buffer Size
ExampleSuppose you got rate limit (Bottleneck B.W.) as 237Mbps and RTT as 0.271 sec then in

above c

hart search for RTT equal to or nearer to 0.271 in the row of 250Mbps and look corresponding
Buffer size vertically. Here Buffer size is 8192KBytes i.e. around 8MB.

2016 Samsung Electronics

105

105

Iperf Commands
TCP throughput can also be affected by uplink issue as TCP depends on
wledgements.
So we need to perform uplink iperf to check the same.

Ackno

Commands for Uplink iperfeNB Side :


Iperf.octeon64 -c SGW_IP -V -B eNB_bearer_IP -u -l 8000 -i 1 -t 999 -b 10m -p 8001
SGW(LENA) :
iperf.octeon64 -s -V -u -l 8000 -i 1 -p 8001
If no loss found till 10Mbps in uplink then we can say that uplink path is clean.

NOTEYou can use any port (-p 5001) for iperf test as per your wish but port number must be same
at both client and server end.
2016 Samsung Electronics

106

106

Iperf Command
These things can also be found using 103 VLAN i.e. by performing Iperf test between LSMR
and eNB in the same way as mentioned above.
Put iperf.octeon64 software in eNB and iperf software in LSMR, use below command for Iperf te
st.
DOWNLINK UDP (from LSMR to eNB)eNB side :
iperf.octeon64 -s -V -B eNB_OAM_IP -u -l 8000 -i 1 -p 5001
LSMR side :
./iperf -c eNB_OAM_IP -V -u -l 8000 -i 1 -b 10m -t 999 -p 5001
UPLINK UDP (from eNB to LSMR)eNB side :
iperf.octeon64 -c LSMR RS_IP -V -B eNB_OAM_IP -u -l 8000 -i 1 -t 999 -b 10m -p 5001
LSMR side :
./iperf -s -V -u -l 8000 -i 1 -p 5001
Note-LSMR to eNB iperf test is valid only if LSMR is at same location where SGW is located.
-No loss between LSMR to eNB (OAM VLAN) doesnt indicate that Bearer VLAN (SGW to eNB)
is also clean.
2016 Samsung Electronics

107

107

Iperf Commands
We can also perform Iperf test between FTP server and UE for troubleshooting
purpos
e.
Traffic comes from FTP server to UE using bearer (101) VLAN so we can also check GTP loss in
LSMR during this test.
(FTP---- > SAEGW ---- > MPLS network ---- > eNB ---- > UE)
-Put iperf software in FTP server and download iperf tool in UE from playstore to perform test.
DOWNLINK UDP (FTP server to UE)FTP server side:
iperf -c IP_of_UE -u -i 1 -t 90 -b 50m -p 5002
UE side:
iperf -s -u -i 1 -p 5002
UPLINK UDP (UE to FTP serve)UE side:
iperf -c Service_IP_of_FTP_server -u -i 1 -t 90 -b 10m -p 5002
FTP server side:
iperf -s -u -i 1 -p 5002

2016 Samsung Electronics

108

108

Iperf Commands
TCP IPERF test can also be performed between FTP and UE using below
ands-

comm

DOWNLINK TCP (FTP server to UE)FTP server side:


iperf -c IP_ of_UE -i 1 -t 90 -w 1m -p 8002
UE side:
iperf -s -i 1 -p 8002

UPLINK TCP (UE to FTP server)UE side:


iperf -c Service_IP_of_FTP_server -i 1 -t 90 -w 1m -p 8002
FTP server side :
iperf -s -i 1 -p 8002

2016 Samsung Electronics

109

109

Summary-Even though low throughput is observed in the field, if the below 3 cases are not in
aul, definitely there's no backhaul issue causing low throughput.

backh

-Physical drops (drop rate requirement is less than 0.01%)


-Small buffer in the bottleneck Buffer size < 512KBytes & bottleneck B.W. <= 250Mbps).
-Low bandwidth in the bottleneck (bottleneck bandwidth < 100Mbps).
- Cisco team can only identify case # 1 in the above 3 cases. So Cisco team should not
declar
e that backhaul is clean without buffer size and rate limitation or bottleneck
bandwidth co
nfirmation. Only by ping test, buffer size issue and wrong rate limitation issue cannot be iden
tified. Cisco team needs to find out a method to improve their
backhaul check.

2016 Samsung Electronics

110

110

CASE STUDY
PHYSICAL ISSUE on I-DL-FDBD-ENB-2685

Initial analysis using iperf-

Observed packet drop in iperf test and asked cisco team to check backhaul.

2016 Samsung Electronics

111

111

Case Study
Cisco Analysis

2016 Samsung Electronics

112

112

Case Study
Effect of Low power at AG1 & AG2

2016 Samsung Electronics

113

113

Case Study
IPERF logs after changes from Cisco end-

NOTEPacket loss got cleared after clearing physical issue in backhaul by cisco circle team.

2016 Samsung Electronics

114

114

X2 Study & Learnings

115

115

X2 Disabled Status Summary

Nexus
SAR
AG3

AG2

X2 Request to Target eNB

AG2

AG1

AG1
CSS

CSS

UE
Source eNB

X2

Target eNB

X2 Disabled cases decreased significantly due to X2 routing done through SAR instead of AG2 Router.
Out of 12,11,000 X2 links, only 4892 links (0.40%) are in disabled state.
These are primarily in Tamilnadu , UP(East), Karnataka which are deployed with juniper routers and Delhi,
Maharashtra with Cisco routers, Which we are jointly troubleshooting
X2 disabled cases is currently reduced to ~1%. Will further troubleshoot and resolve to make it nil
Confidential
2016 Samsung Electronics

Reliance Jio Infocomm Limited

116

116

Pre-checks for X2 Interface


From Cisco Side
MTU Configuration Check & Correction .
Two IP addresses for single eNB, removing the multiple IP's.
AG1 Config change for Boundary Cases
Cisco ASR 901 ( Old Cisco Router) Image Up-grade to handle Neighbor Discovery Protocol Issue.
Correction of routing issue at CSS and other nodes.
From Juniper Side
Juniper Router (ACX2200) Image Up-grade to handle Neighbor Discovery Protocol Issue.
Correction of routing issue at CSS

From eNB Side

"LTE_SIGNAL_X2 " needs to be true only for VLAN 101.


eNB IP Correction in eNB Neighbor list.
eNB Administrative State in both Neighbors to be "Unlocked"
"NO_X2State" should be "False" instead of "True" in both the neighbors.
eMBMS IP change in order to make subnet mask "/126" from "/64" as it was falling under same n/w
of neighbor eNB making wrong entries in eNB routing table.

Confidential
2016 Samsung Electronics

Reliance Jio Infocomm Limited

117

117

S1 AP Bearer Preservation

118

S1 AP Preservation To improve VoLTE performance


Call Drop Analysis of ACME drive Test
Sr. No.

Time

Samsung NEO RF/eNB

S1AP Cause code

Time taken to
recover (sec)

radioNetwork: ho-failure-in-target-EPCeNB-or-target-system (6)

0.22

radioNetwork: ho-failure-in-target-EPCeNB-or-target-system (6)

2.10

15:34:25.566

A call is released when the target eNB cannot receive the RRC
Connection Reconfiguration Complete message during the S1
handover.
RLF at MT due to poor SINR. MT send measurement report
continuously but no HO command received from eNB
RLF happened due to HO failure at MO, poor SINR

radioNetwork: failure-in-radio-interfaceprocedure (26)

0.30

16:27:08.406

MO logs are incomplete and no drop at MT found.

radioNetwork: ho-failure-in-target-EPCeNB-or-target-system (6)

0.10

16:37:09.774

radioNetwork: unspecified (0)

2.90

16:55:17.871

radioNetwork: ho-failure-in-target-EPCeNB-or-target-system (6)

0.30

18:21:42.528

No RF issue. No uplink RSSI issue found. Call is released when ARQ


ACK is not received from the UE after the eNB RLC performs
transmission as many as the Max Retransmission to the UE. UE issue
suspected.
HO Failure due to poor SINR at MT. A call is released when the target
cell cannot receive the RRC Connection Reconfiguration Complete
message during the intra-handover
RLF happened due to poor SINR at MT

radioNetwork: failure-in-radio-interfaceprocedure (26)

0.70

18:32:21.596

Call dropped because of HO failure. High DL BLER observed

No NW logs available

0.10

18:22:00.177

UE lost UL synch and trying to access eNB again. But RACH fails and
RLF happened at MO. Poor SINR.

radioNetwork: failure-in-radio-interfaceprocedure (26)

0.50

15:39:38.259

19:08:59.258

Average 800 msec


Average 317 msec (Excluding 2 & 5)
2016 Samsung Electronics

119

S1 AP Preservation To improve VoLTE performance


Observation

ACME drive reported 9 out of 600 VoLTE call were dropped after HO failure and RLF.

Analysis

MME deleting the QCI-1 bearer after receiving UE Context Release request with a cause of HO failure
and UE lost from eNB.

Solution

Since HO failure and UE lost due to RLF is being observed in the field and it is also observed that the

recovery process is also happening is less than 1 sec.

It is suggested to preserve the QCI-1 bearer in case of UEContextRelease with HO failure MME
deleting the QCI-1 bearer after receiving UE Context Release request with a cause of HO failure and
UE lost from eNB

Preservation ON is implemented in IDC on 1st Sep. Performance will be monitored for ACME .

2015 Samsung Electronics

120

Other Actions Taken to Avoid Call Drop

All eNBs upgraded to PKG 5.0.0


Multi-Band Implementation
Forward Hand Over enabled

Equal SAEGW Traffic distribution


TA/TAL/BTAL Configuration Audited
GBR Bearer preservation set to ON (i.e. In Test Area Serving 7 MMEs Only )

S1AP - Cause
Code Value
06
26
21

Cause Description
Handover Failure In Target EPC/eNB Or Target
System
Failure in the Radio Interface
Procedure
UE Connection Lost

Old Value

New Value

OFF

ON

OFF

ON

OFF

ON

Benefit

Reduction in Call Drops due to RLF


for fractions of seconds

Function Flag #9 (NOT_USE_PRSV_GBR_FOR_ACT_S1) to ON


To the preserve status of S1AP CAUSE (tx2 relocation timer expiry)

Clear Test Area Service Impacting Alarms


Check Test Area Availability Status

2016 Samsung Electronics

121

S1AP preservation option in MME


When eNB detects any RAN layer problems, eNB can send UE Context Release Request
message to MME to release the UE-associated S1 connection
As per 3GPP standard, MME should release the UE context as per the request

eNB

MME
UE Context Release Request (Cause)
UE Context Release Command (Cause)
UE Context Release Complete

However, eNBs local decision may be wrong in case of some causes, for example,
UE gets lost from an eNB, but the UE can re-attach to some other eNB

During HO, UE is not responding and X2 timer expires, but the UE can re-attach to some other eNB

For some cases, it can be better to preserve S1 connection rather than releasing UE context
Samsung MME provides such options (On/Off) for every S1AP cause code
2016 Samsung Electronics

122

Ref: S1AP cause code


3GPP TS 36.413 defines the following cause codes
Code

Cause

Code

Cause

Unspecified

20

User Inactivity

TX2RELOCOverall Expiry

21

Radio Connection With UE Lost

Successful Handover

22

Load Balancing TAU Required

Release due to E-UTRAN generated reason

23

CS Fallback triggered

Handover Cancelled

24

UE Not Available for PS Service

Partial Handover

25

Radio resources not available

Handover Failure In Target EPC/eNB Or Target System

26

Failure in the Radio Interface Procedure

Handover Target not allowed

27

Invalid QoS combination

TS1RELOCoverall Expiry

28

Inter-RAT redirection

TS1RELOCprep Expiry

29

Interaction with other procedure

10

Cell not available

30

Unknown E-RAB ID

11

Unknown Target ID

31

Multiple E-RAB ID instances

12

No radio resources available in target cell

32

Encryption and/or integrity protection algorithms not supported

13

Unknown or already allocated MME UE S1AP ID

33

S 1 intra system Handover triggered

14

Unknown or already allocated eNB UE S1AP ID

34

S1 inter system Handover triggered

15

Unknown or inconsistent pair of UE S1AP ID

35

X2 Handover triggered

16

Handover Desirable for Radio Reasons

36

Redirection towards 1xRTT

17

Time Critical Handover

37

Not supported QCI Value

18

Resource Optimization Handover

38

Invalid CSG Id

19

Reduce Load in Serving Cell

2016 Samsung Electronics

123

Preservation option case study


RJio network has the following S1AP preservation options = ON
The S1AP preservation options are compared with KT case in Korea
ID

DESC

KT

RJio

Tx2relocoverallExpiry

ON

ON

ReleaseDueToEutranGeneratedRsn

ON

HoFailInTargetEpcEnborTargetSy

ON

ON

TS1relocoverallExpiry

ON

16

HoDesirableForRadioReason

ON

20

UserInactivity

ON

21

RadioConnectionWithUeLost

ON

26

FailureInTheRadioInterfaceProcedure

ON

28

InterRatRedirection

ON

2016 Samsung Electronics

124

S 5 Interface Learning from Delhi

125

X2 HO - SGW change Between Noida/Vikaspuri

Possible Impacted Case:


For Single PLMN, two Sites
distributing traffic with S5.

Applicable Sites - Mumbai,


Delhi and Chennai.

Scenario:
On Registration Co-located
SAEGW get selected
Initial PGW provides IP to
user
Once HO happens
between SGW Group,
New SGW will get selected
but PGW remains same.
Signaling & Traffic moves
from new SGW to old PGW
on S5 interface.

In Delhi Case S5 GTP-C was ok hence bearer


was Modified correctly but S5 GTP-U was
down hence Voice/Data was not working.
2016 Samsung Electronics

126

X2 HO with SGW change in Delhi


S

2016 Samsung Electronics

127

S5 Interface GTP-U Packet Drop


GTPU packets on the S5 Interface getting dropped due to missing IP Route
Uplink packets are getting dropped in between SGW (Noida) -> PGW (Vikaspuri)
Downlink packets are getting dropped in between PGW(Vikaspuri) -> SGW(Noida)

2016 Samsung Electronics

128

MME Pool Breakdown - Update


Nagpur
Mumbai
Ahmedabad

Nagpur
Bhopal

Delhi
Jaipur

Ambala
Ludhiana
Shimla

Mumbai
Ahmedabad

Lucknow
Agra

Jammu
Srinagar

Chennai
Kochi

Bhubaneshwar
Patna

Kolkata
Asansol

Delhi
Lucknow
Ambala

Chennai
Hyderabad
Bangalore

Hyderabad
Bangalore

Guwahati
Shilong

Kolkata
Guwahati

MME pool break down activity completed.


-Balance re-arrangement after readiness of Jammu & Shilong

2016 Samsung Electronics

129

Core Network Learnings

130

ACME Related Issue Analysis Summary


Test Scope

Test Drive Location - Bhopal , Ahmedabad, Kolkata,


Mumbai, Delhi

Test Duration
Bhopal - 12th to 15th Nov
Ahmedabad 15th to 17th Nov
Kolkata 24th Nov to 4th Dec
Mumbai 3rd Dec to 8th Dec
Delhi 7th Dec to Ongoing
Test Devices
Bhopal - ACME
Ahmedabad Core Prime / ACME
Kolkata Core Prime / ACME
Mumbai ACME
Delhi - ACME
Total Calls Originated
Bhopal 1713
Ahmedabad 2000
Kolkata 2000+
Mumbai 1260
Delhi 3000+
Call Scenario
VoLTE to VoLTE , VoLTE to CS,
CS to VoLTE, VoLTE to PSTN

2016 Samsung Electronics

Analysis Summary
%Avg.
Failure

Call Drop

1.5%

Setup Fail

1.6%

DNS

DRA

eNB

EPC

IMS

MGCF

OCS

PCRF

RF

UE

IP

1%
11%

0%

3%
6%

18%

24%

2%
2%
15%

18%

131

Key Failures and Action Items (1/2)


Sr.
No.

Key Failures

Action Taken

Status

Owner Team

MGCF Issue resulting in SIP "480"

MGCF Faulty Card replaced.

Close

MCGF

MGCF Issue resulting in SIP 500"

MGCF getting release from Other Operator POI

Close

Other
Operator

eNB ERAB Setup Failures

A2 parameter Configuration Corrected PAN India to avoid


ERAB Setup Failure in RACE condition

Close

eNB

RTP Time Out while Intra eNB HO

PDCP layer Sequence number differences has caused


RTCP packet drops while Intra eNB HO.

Close

eNB

Close

OCS

Close

DNS

5
6

OCS Delayed response post X2 HO Solution applied by OCS to respond within 1 sec.
SIP 503
IMS to DNS interworking Failure DNS Configuration updated for missing IMS Node's
SIP "500"
FQDNs.

IMS internal Failure - SIP "500"

TAS/CSCF not forwarding 200 (OK) for PRACK and SIP


Invite. Patch applied to fix the same.

Close

IMS

IMS Failure SIP "480"

Patch applied to fix the same.

Close

IMS

2016 Samsung Electronics

132

Key Failures and Action Items (2/2) Conitnues..


Sr.
9

10

11

12

13

Key Failures
IMS Failure SIP 480

Action Taken

Status

Owner Team

1. PCSCF sends fragmented SIP Update message for packet


>1518 size on UDP instead of sending on TCP --- Issue is
with Nokia R&D team

Open

IMS

Open

IMS

Close

MME

Close

OCS/PCRF/
DRA

Close

Cisco

IMS NOT forwarding SIP INVITE, 183


Patch need to be applied to fix the multiple message
Session Progress, SIP UPDATE,
forwarding issue at TAS & SCSCF.
(200OK) for INVITE
UE find RLF during X2 Handover process due to X2
connectivity fails/X2 link not available.
X2 HO failure procedure fails due to
These causes QCI-1 bearer deactivation and hence call gets
Tx2 timer expiry or due to radiodropped.
procedure failure
Preservation mode is enabled in MME for SA1P cause code#
1, 6, 21,26 to make sure
Post successful X2 HO procedure, While requesting for S1U
Source change from old eNB to new eNB, (path switch
Post Successful X2 HO , Call drops
request) timer expires at eNB due to delay in response
due to Patch Switch request timer
( i.e. >3 sec) from OCS or PCRF or DRA to respond to CCR-U
expiry
message which is triggered after Modify bearer request
from MME to SAEGW.
While X2-HO between two eNB served by different MMEs,
S5 Interface issue - Packet loss btw SGW relocation get triggered while PGW remains same as
S-GW and P-GW due to SGW
earlier.
relocation process while X2-HO
- Because there was no IP routing configuration between
between two eNBs served by
SGW and PGW each located at different sites, GTPU packets
different MMEs
on the S5-interface were dropped.
- After IP routing configuration problem resolved.

2016 Samsung Electronics

133

[3] MO received SIP-503 service unavailable

eNB

Issue Description: QCI-1 bearer setup procedure failed in eNB


Resolution: Issue got fixed after changing A2 Parameter configuration change in eNB which was
generating race condition within eNB and were not responding to MME for ERAB Setup requests.

2016 Samsung Electronics

134

[5] MO Received SIP 503 Issue at OCS

OCS

Issue Description: Delay from OCS for CCA response resulted in VoLTE Call setup Failure with ASR error
Session released - service based local policy function aborted session".
Resolution: Temporarily OCS has been bypassed for ACME testing.

2016 Samsung Electronics

135

[7] Received 500 Server Internal Error for INVITE

Issue Description:

IMS

IMS is not forwarding 200 Ok for Invite towards MO UE.

Resolution: IMS patch applied at TAS.

2016 Samsung Electronics

136

[13] X2 HO - SGW change Btwn Noida/Vikaspuri


6 Pair of SAEG
Ws
(i.e. 36 combina
tions to check)

G
6

Possible Impacted Case: For S


ingle PLMN, two Sites distribu
ting traffic with S5.

Applicable Sites - Mumbai, D


elhi and Chennai.

Scenario:
On Registration Co-located SA
EGW get selected
Initial PGW provides IP to use
r
Once HO happens between S
GW Group, New SGW will get
selected but PGW remains sa
me.
Signaling & Traffic moves fro
m new SGW to old PGW on S
5 interface.

In Delhi Case S5 GTP-C was ok hence bearer was


Modified correctly but S5 GTP-U was down hence
Voice/Data was not working.
2016 Samsung Electronics

137

[13] S5 Failure (X2 HO with SGW change) (1/2)


S

2016 Samsung Electronics

Cisco

138

[13] S5 Failure (X2 HO with SGW change) (2/2)


GTPU packets on the S5 Interface getting dropped due to missing IP Route
Uplink packets are getting dropped in between SGW(Noida) -> PGW(Vikaspuri)
Downlink packets are getting dropped in between PGW(Vikaspuri) -> SGW(Noida)

2016 Samsung Electronics

139

No Response for SIP Bye(Backbone) Issue-1


Previous call was not completed. SIP Bye was sent for which response was not received.
Even after multiple retransmission(till 12:30:32), even TCP Ack was not received.
No SIP Invite seen at 12:30:14 in the network logs

City Mumbai
Date 06-Jan-2016

Similar issue is observe


d @ 13:10:02 also

2016 Samsung Electronics

140

UE Issue - No Response for SIP Invite


MT UE was in connected mode
MT UE was not responding the SIP Invite message sent.
Even after multiple retransmissions, there was no response from MT UE.

City Mumbai
Date 06-Jan-2016

2016 Samsung Electronics

141

Call Setup failure due to User Inactivity(30 sec)


MT UE has not answered the call within 30 sec.
As User Inactivity timer is 30 sec, eNB has sent UE Context Release with reason USER-INAC
TIVITY.
So MME deleted the QCI-1 bearer which resulted in VoLTE Call Setup failure.

City Mumbai
Date 07-Jan-2016

2016 Samsung Electronics

142

S-ar putea să vă placă și