Sunteți pe pagina 1din 210

..

..
..
..
..

Core RF Engineering
Dept. 5395

1xEV-DO RF Optimization Guide


.

1xEV DO Release 3.x

Product Line:
Version:
Issue Date:
Author:
Department:
Authorizing Manager:

1xEV-DO Release 3.0 ChR


Initial Release
March 3, 2006
Mike Woodley
Core RF
Frank Jager / Farhad Bassirat

2003 Nortel Networks


All rights reserved.
Information subject to change without notice.
The information disclosed herein is proprietary to Nortel Networks or others and is not to be used by or disclosed to unauthorized
persons without the written consent of Nortel Networks. The recipient of this document shall respect the security status of the
information.

Nortel Networks Confidential and Proprietary

Table of Contents
TABLE OF CONTENTS ............................................................................................................................. 2
1

INTRODUCTION ................................................................................................................................ 8
1.1
1.2
1.3
1.4
1.5

RELATED DOCUMENTS .................................................................................................................... 8


SCOPE.............................................................................................................................................. 9
REVISION HISTORY ......................................................................................................................... 9
CONTRIBUTORS ............................................................................................................................... 9
AUDIENCE ....................................................................................................................................... 9

1XEVDO OPTIMIZATION EVENTS FLOW CHART................................................................. 10

1XEV-DO PRE OPTIMIZATION OBJECTIVES.......................................................................... 11


3.1
ENTRANCE AND EXIT CRITERIA.................................................................................................... 11
3.1.1
RF Design Review ................................................................................................................ 11
3.1.2
Coverage Control ................................................................................................................. 11
3.1.3
RF Coverage Availability ..................................................................................................... 11
3.1.4
Inter-RNC Border Design..................................................................................................... 13
3.1.5
CIQ Review and Provisioning Verification .......................................................................... 15

1XEV-DO OPTIMIZATION EVENTS............................................................................................ 17


4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8

INITIAL RF PARAMETERS VERIFICATION ...................................................................................... 17


PRE OPTIMIZATION SYSTEM HEALTH CHECK ................................................................................ 17
FTAP AND RTAP TESTING ........................................................................................................... 19
TCP / IP PARAMETERS .................................................................................................................. 20
SHAKEDOWNS ............................................................................................................................... 24
GOLDEN VALUE (STATIONARY) TESTING...................................................................................... 24
CLUSTER DRIVE TESTING .............................................................................................................. 25
TROUBLESHOOTING....................................................................................................................... 25

DATA COLLECTION / POST PROCESSING TOOL SETUP .................................................... 26


5.1
LAPTOP SETUP............................................................................................................................... 26
5.2
AT SETUP ...................................................................................................................................... 27
5.3
DATA COLLECTION TOOL SETUP ................................................................................................... 27
5.3.1
RNC Logging Setup .............................................................................................................. 27
5.3.2
Optis-M Setup ....................................................................................................................... 32
5.3.3
Optis-M Log Masks .............................................................................................................. 35
5.3.4
Invex3G Chassis Setup ......................................................................................................... 38
5.3.5
InVex3G Log Masks.............................................................................................................. 40
5.4
AT LOG DATA POST-PROCESSING TOOL SETUP ............................................................................ 41
5.4.1
RF Optimizer EV-DO ........................................................................................................... 41
5.4.2
XCAP-DO ............................................................................................................................. 43
5.5
DOM - RNC LOG POST PROCESSING TOOLS ................................................................................. 45
5.5.1
Converting RNC Logs from Binary to Text........................................................................... 45
5.5.2
NEWTUN (Post Processing and Analysis of RNC Logs)...................................................... 45
5.6
OM DATA COLLECTION TEMPLATE CONFIGURATION SETUP ........................................................ 54
5.7
CELLDM DATA COLLECTION ...................................................................................................... 56
5.8
OM ANALYSIS TOOLS ................................................................................................................... 61
5.9
TCP PERFORMANCE LOGGING TOOLS ........................................................................................... 63

SHAKEDOWNS ................................................................................................................................. 65
6.1
6.2

OBJECTIVES ................................................................................................................................... 65
EVDO SITE SHAKEDOWN PROCESS .............................................................................................. 65

Nortel Networks Confidential and Proprietary

6.3
EVDO INTER SUBNET ACTIVE HANDOFF (ISSHO) SHAKEDOWN PROCESS .................................. 69
6.4
(A13) DORMANT SESSION HANDOFF SHAKEDOWN PROCESS ........................................................ 71
6.5
DATA PROCESSING AND ANALYSIS ............................................................................................... 73
6.5.1
EVDO Site Shakedown Analysis ........................................................................................... 73
6.5.2
Inter RNC Active Handoff Shakedown Analysis ................................................................... 77
6.5.3
A13 Dormant Session Handoff Analysis ............................................................................... 78
7

GOLDEN VALUE (STATIONARY) TESTING ............................................................................. 79


7.1
7.2
7.3
7.4

INTRODUCTION .............................................................................................................................. 79
FTP SERVER CONFIGURATION ...................................................................................................... 79
GOLDEN VALUE TESTING PROCESS ............................................................................................... 79
DATA PROCESSING AND ANALYSIS ............................................................................................... 82

OM COLLECTION AND ANALYSIS ............................................................................................ 84


8.1
CONNECTION ATTEMPTS ............................................................................................................... 84
8.2
FAILED CALL ATTEMPTS (FCA / ACCESS FAILURE)...................................................................... 84
8.3
DROPPED CONNECTIONS (ABNORMAL CONNECTION CLOSES)...................................................... 84
8.4
SESSION SETUP PERFORMANCE (INCLUDES A13 SESSION HANDOFFS) ........................................... 85
8.4.1
UATI Request Initiated A13 Dormant Handoff Performance............................................... 85
8.4.2
Prior Session Initiated Dormant Handoff Performance ....................................................... 85
8.4.3
A10 Connection Setup Performance..................................................................................... 85
8.4.4
Session Configuration Failures ............................................................................................ 86
8.5
SERVING SECTOR SWITCHING AND SOFT/SOFTER HANDOFF PERFORMANCE ................................ 86
8.6
INTER RNC ACTIVE HANDOFF TRAFFIC STATISTICS AND SECONDARY BORDER MEASUREMENT. 86
8.7
1XEVDO PAGING PERFORMANCE MEASUREMENTS ..................................................................... 86
8.8
PER SECTOR AIRLINK RESOURCES ALLOCATION (BLOCKING) ...................................................... 86

CLUSTER DRIVE TESTING........................................................................................................... 87


9.1
DATA COLLECTION PROCESS ........................................................................................................ 87
9.2
DATA PROCESSING AND ANALYSIS ............................................................................................... 92
9.2.1
Access Failure Rate .............................................................................................................. 92
9.2.2
Connection Drop Rate .......................................................................................................... 95
9.2.3
Throughput in Mobility Condition........................................................................................ 99
9.2.4
Connection Setup Time ....................................................................................................... 102

10

TROUBLESHOOTING............................................................................................................... 103

10.1 SESSION SETUP AND A13 HANDOFF SUCCESS RATE ................................................................... 103
10.1.1 Data Collection Methods.................................................................................................... 103
10.1.1.1
10.1.1.2
10.1.1.3

10.1.2
10.1.2.1
10.1.2.2
10.1.2.3
10.1.2.4

AT Logs..................................................................................................................................... 103
RNC Logs .................................................................................................................................. 103
OMs ........................................................................................................................................... 109

KPI Calculation.................................................................................................................. 113


KPI Value and Tolerance Factor................................................................................................ 113
Calculating KPI from AT Logs.................................................................................................. 114
Calculating KPI from RNC Logs............................................................................................... 114
Calculating KPI from OMs........................................................................................................ 114

10.1.3 Troubleshooting Guide ....................................................................................................... 115


10.2 SESSION SETUP TIME................................................................................................................... 118
10.2.1 Data Collection Methods.................................................................................................... 119
10.2.1.1
10.2.1.2
10.2.1.3

10.2.2
10.2.2.1
10.2.2.2
10.2.2.3
10.2.2.4

AT Logs..................................................................................................................................... 119
RNC Logs .................................................................................................................................. 119
OMs ........................................................................................................................................... 119

KPI Calculation.................................................................................................................. 119


KPI Value and Tolerance Factor................................................................................................ 119
Calculating KPI from AT Logs.................................................................................................. 119
Calculating KPI from RNC Logs............................................................................................... 120
Calculating KPI from OMs........................................................................................................ 120

Nortel Networks Confidential and Proprietary

10.2.3 Troubleshooting Guide ....................................................................................................... 120


10.3 CONNECTION SETUP TIME ........................................................................................................... 120
10.3.1 Data Collection Methods.................................................................................................... 120
10.3.1.1
10.3.1.2
10.3.1.3

10.3.2
10.3.2.1
10.3.2.2
10.3.2.3
10.3.2.4

AT Logs..................................................................................................................................... 120
RNC Logs .................................................................................................................................. 120
OMs ........................................................................................................................................... 120

KPI Calculation.................................................................................................................. 121


KPI Value and Tolerance Factor................................................................................................ 121
Calculating KPI from AT Logs.................................................................................................. 121
Calculating KPI from RNC Logs............................................................................................... 121
Calculating KPI from OMs........................................................................................................ 121

10.3.3 Troubleshooting Guide ....................................................................................................... 122


10.4 ACCESS FAILURES ....................................................................................................................... 122
10.4.1 Data Collection Methods.................................................................................................... 122
10.4.1.1
10.4.1.2
10.4.1.3

10.4.2
10.4.2.1
10.4.2.2
10.4.2.3
10.4.2.4

AT Logs..................................................................................................................................... 122
RNC Logs .................................................................................................................................. 124
OMs ........................................................................................................................................... 125

KPI Calculation.................................................................................................................. 125


KPI Value and Tolerance Factor................................................................................................ 125
Calculating KPI from AT Logs.................................................................................................. 125
Calculating KPI from RNC Logs............................................................................................... 126
Calculating KPI from OMs........................................................................................................ 126

10.4.3 Troubleshooting Guide ....................................................................................................... 128


10.5 CALL BLOCKS ............................................................................................................................. 133
10.5.1 Data Collection Methods.................................................................................................... 133
10.5.1.1
10.5.1.2
10.5.1.3

10.5.2
10.5.2.1
10.5.2.2
10.5.2.3
10.5.2.4

AT Logs..................................................................................................................................... 133
RNC Logs .................................................................................................................................. 133
Blocking OMs............................................................................................................................ 134

Blocking KPI Calculation................................................................................................... 134


KPI Value and Tolerance Factor................................................................................................ 134
Calculating KPI from AT Logs.................................................................................................. 134
Calculating KPI from RNC Logs............................................................................................... 134
Calculating KPI from OMs........................................................................................................ 134

10.5.3 Troubleshooting Guide ....................................................................................................... 135


10.6 CONNECTION DROPS ................................................................................................................... 138
10.6.1 Data Collection Methods.................................................................................................... 138
10.6.1.1
10.6.1.2
10.6.1.3

10.6.2
10.6.2.1
10.6.2.2
10.6.2.3
10.6.2.4

AT Logs..................................................................................................................................... 138
RNC Logs .................................................................................................................................. 139
Dropped Connection OMs ......................................................................................................... 140

Dropped Connection KPI Calculation ............................................................................... 141


KPI Value and Tolerance Factor................................................................................................ 141
Calculating KPI from AT Logs.................................................................................................. 141
Calculating Dropped Connections KPI from RNC Logs ........................................................... 142
Calculating Dropped Connection KPI from OMs...................................................................... 142

10.6.3 Dropped Connection Troubleshooting Guide..................................................................... 143


10.7 FL SECTOR SWITCHING SUCCESS RATE ...................................................................................... 154
10.7.1 Data Collection Methods.................................................................................................... 154
10.7.1.1
10.7.1.2
10.7.1.3

10.7.2
10.7.2.1
10.7.2.2
10.7.2.3
10.7.2.4

AT Logs..................................................................................................................................... 154
RNC Logs .................................................................................................................................. 154
OMs ........................................................................................................................................... 154

Forward Sector Switching Success Rate KPI Calculation ................................................. 154


KPI Value and Tolerance Factor................................................................................................ 154
Calculating KPI from AT Logs.................................................................................................. 154
Calculating KPI from RNC Logs............................................................................................... 154
Calculating KPI from OMs........................................................................................................ 154

10.7.3 Troubleshooting Guide ....................................................................................................... 155


10.8 FL SECTOR SWITCHING TIME ...................................................................................................... 155
10.8.1 Data Collection Methods.................................................................................................... 155
10.8.1.1
10.8.1.2

AT Logs..................................................................................................................................... 155
RNC Logs .................................................................................................................................. 155

Nortel Networks Confidential and Proprietary

10.8.1.3

10.8.2

OMs ........................................................................................................................................... 156

KPI Calculation.................................................................................................................. 156

10.8.2.1
10.8.2.2
10.8.2.3
10.8.2.4

KPI Value and Tolerance Factor................................................................................................ 156


Calculating KPI from AT Logs.................................................................................................. 156
Calculating KPI from RNC Logs............................................................................................... 156
Calculating KPI from OMs........................................................................................................ 157

10.8.3 Troubleshooting Guide ....................................................................................................... 157


10.9 RL SHO SUCCESS RATE.............................................................................................................. 157
10.9.1 Data Collection Methods.................................................................................................... 157
10.9.1.1
10.9.1.2
10.9.1.3

10.9.2

AT Logs..................................................................................................................................... 157
RNC Logs .................................................................................................................................. 157
OMs ........................................................................................................................................... 158

KPI Calculation.................................................................................................................. 158

10.9.2.1
10.9.2.2
10.9.2.3
10.9.2.4

KPI Value and Tolerance Factor................................................................................................ 158


Calculating KPI from AT Logs.................................................................................................. 158
Calculating KPI from RNC Logs............................................................................................... 158
Calculating KPI from OMs........................................................................................................ 158

10.9.3 Troubleshooting Guide ....................................................................................................... 159


10.10
FL PER-USER THROUGHPUT ................................................................................................... 160
10.10.1
Data Collection Methods................................................................................................ 160
10.10.1.1
10.10.1.2
10.10.1.3
10.10.1.4

10.10.2
10.10.2.1
10.10.2.2
10.10.2.3
10.10.2.4

AT Logs..................................................................................................................................... 160
TCP Trace Logs......................................................................................................................... 160
RNC Logs .................................................................................................................................. 162
OMs ........................................................................................................................................... 162

KPI Calculation.............................................................................................................. 162


KPI Value and Tolerance Factor................................................................................................ 162
Calculating KPI from AT Logs.................................................................................................. 162
Calculating KPI from RNC Logs............................................................................................... 163
Calculating KPI from OMs........................................................................................................ 163

10.10.3
Troubleshooting Guide ................................................................................................... 163
10.11
FL PER-SECTOR THROUGHPUT................................................................................................ 180
10.11.1
Data Collection Methods................................................................................................ 180
10.11.1.1
10.11.1.2
10.11.1.3

10.11.2
10.11.2.1
10.11.2.2
10.11.2.3
10.11.2.4

AT Logs..................................................................................................................................... 180
RNC Logs .................................................................................................................................. 180
OMs ........................................................................................................................................... 180

KPI Calculation.............................................................................................................. 181


KPI Value and Tolerance Factor................................................................................................ 181
Calculating KPI from AT Logs.................................................................................................. 181
Calculating KPI from RNC Logs............................................................................................... 181
Calculating KPI from OMs........................................................................................................ 181

10.11.3
Troubleshooting Guide ................................................................................................... 181
10.12
RL PER-USER THROUGHPUT ................................................................................................... 181
10.12.1
Data Collection Methods................................................................................................ 181
10.12.1.1
10.12.1.2
10.12.1.3

10.12.2
10.12.2.1
10.12.2.2
10.12.2.3
10.12.2.4

AT Logs..................................................................................................................................... 181
RNC Logs .................................................................................................................................. 182
OMs ........................................................................................................................................... 182

KPI Calculation.............................................................................................................. 182


KPI Value and Tolerance Factor................................................................................................ 182
Calculating KPI from AT Logs.................................................................................................. 182
Calculating KPI from RNC Logs............................................................................................... 182
Calculating KPI from OMs........................................................................................................ 182

10.12.3
Troubleshooting Guide ................................................................................................... 183
10.13
RL PER-SECTOR THROUGHPUT ............................................................................................... 185
10.13.1
Data Collection Methods................................................................................................ 185
10.13.1.1
10.13.1.2
10.13.1.3

10.13.2
10.13.2.1
10.13.2.2

AT Logs..................................................................................................................................... 185
RNC Logs .................................................................................................................................. 185
OMs ........................................................................................................................................... 185

KPI Calculation.............................................................................................................. 185


KPI Value and Tolerance Factor................................................................................................ 185
Calculating KPI from AT Logs.................................................................................................. 185

Nortel Networks Confidential and Proprietary

10.13.2.3
10.13.2.4

Calculating KPI from RNC Logs............................................................................................... 185


Calculating KPI from OMs........................................................................................................ 185

10.13.3
Troubleshooting Guide ................................................................................................... 186
10.14
FL ACHIEVABLE DATA RATE .................................................................................................. 186
10.14.1
Data Collection Methods................................................................................................ 186
10.14.1.1
10.14.1.2
10.14.1.3

10.14.2
10.14.2.1
10.14.2.2
10.14.2.3
10.14.2.4

AT Logs..................................................................................................................................... 186
RNC Logs .................................................................................................................................. 186
OMs ........................................................................................................................................... 186

KPI Calculation.............................................................................................................. 187


KPI Value and Tolerance Factor................................................................................................ 187
Calculating KPI from AT Logs.................................................................................................. 187
Calculating KPI from RNC Logs............................................................................................... 187
Calculating KPI from OMs........................................................................................................ 187

10.14.3
Troubleshooting Guide ................................................................................................... 187
10.15
RL ACHIEVABLE DATA RATE ................................................................................................. 191
10.15.1
Data Collection Methods................................................................................................ 191
10.15.1.1
10.15.1.2
10.15.1.3

10.15.2
10.15.2.1
10.15.2.2
10.15.2.3
10.15.2.4

AT Logs..................................................................................................................................... 191
RNC Logs .................................................................................................................................. 191
OMs ........................................................................................................................................... 191

KPI Calculation.............................................................................................................. 191


KPI Value and Tolerance Factor................................................................................................ 191
Calculating KPI from AT Logs.................................................................................................. 192
Calculating KPI from RNC Logs............................................................................................... 192
Calculating KPI from OMs........................................................................................................ 192

10.15.3
Troubleshooting Guide ................................................................................................... 192
10.16
PACKET LATENCY ................................................................................................................... 193
10.16.1
Data Collection Methods................................................................................................ 193
10.16.1.1
10.16.1.2
10.16.1.3

10.16.2
10.16.2.1
10.16.2.2
10.16.2.3
10.16.2.4

AT Logs..................................................................................................................................... 193
RNC Logs .................................................................................................................................. 193
OMs ........................................................................................................................................... 193

KPI Calculation.............................................................................................................. 194


KPI Value and Tolerance Factor................................................................................................ 194
Calculating KPI from AT Logs.................................................................................................. 194
Calculating KPI from RNC Logs............................................................................................... 194
Calculating KPI from OMs........................................................................................................ 194

10.16.3
Troubleshooting Guide ................................................................................................... 194
10.17
DO-TO-1X HANDOFF DELAY .................................................................................................. 194
10.17.1
Data Collection Methods................................................................................................ 194
10.17.1.1
10.17.1.2
10.17.1.3

10.17.2
10.17.2.1
10.17.2.2
10.17.2.3
10.17.2.4

AT Logs..................................................................................................................................... 194
RNC Logs .................................................................................................................................. 195
OMs ........................................................................................................................................... 195

KPI Calculation.............................................................................................................. 195


KPI Value and Tolerance Factor................................................................................................ 195
Calculating KPI from AT Logs.................................................................................................. 195
Calculating KPI from RNC Logs............................................................................................... 195
Calculating KPI from OMs........................................................................................................ 195

10.17.3
Troubleshooting Guide ................................................................................................... 195
10.18
DO-TO-1X SUCCESS RATE ...................................................................................................... 195
10.18.1
Data Collection Methods................................................................................................ 195
10.18.1.1
10.18.1.2
10.18.1.3

10.18.2
10.18.2.1
10.18.2.2
10.18.2.3
10.18.2.4

10.18.3

AT Logs..................................................................................................................................... 195
RNC Logs .................................................................................................................................. 195
OMs ........................................................................................................................................... 196

KPI Calculation.............................................................................................................. 196


KPI Value and Tolerance Factor................................................................................................ 196
Calculating KPI from AT Logs.................................................................................................. 196
Calculating KPI from RNC Logs............................................................................................... 196
Calculating KPI from OMs........................................................................................................ 196

Troubleshooting Guide ................................................................................................... 196

APPENDIX A: EVDO OPTIMIZATION AND TROUBLESHOOTING FLOW CHARTS............ 197


Nortel Networks Confidential and Proprietary

APPENDIX B: CLI SYSTEM READINESS CHECK.......................................................................... 202


APPENDIX C: SHAKEDOWN FORM ................................................................................................. 205
APPENDIX D: EXAMPLES OF PAST ISSUES................................................................................... 206

Nortel Networks Confidential and Proprietary

Introduction

This document provides a guideline for RF engineers to do the RF optimization of a 1xEV-DO system.
1xEV-DO system is optimized to carry non real-time packet data services and hence does not support voice
services. In the forward link, 1xEV-DO air interface is designed to provide up to 2.4 Mbps data rate with
only one 1.25 MHz carrier using TDM (Time Division Multiplexing) system. In the reverse link, 1xEV-DO
provides up to 153.6 kbps data rate similar to 1xRTT. Since the forward link physical layer implementation
of 1xEV-DO and CDMA/1xRTT are different, they cannot exist in the same frequency. 1xEV-DO has to
be deployed in a separate carrier.

1.1

Related Documents

[1] 1xEV-DO Coverage and Capacity Performance, v2.0, Muhieddin Najib, Core RF
Engineering, February 2004 (http://navigate.us.nortel.com/imds?pg=/eng/wne/cdma/sys/rf/lb)

[2] 1xEV-DO Provisioning Guideline, v2.1.03, Kenneth Ho, Core Network Engineering, August
2004 (http://navigate.us.nortel.com/imds?pg=/eng/wne/cdma/sys/dim/gl)

[3] 1xEV-DO RF Parameters and Datafill Guide, v3.01, Mike Garramone, Core RF
Engineering, October 2005.
(http://livelink.us.nortel.com/livelink/livelink.exe?func=ll&objId=9655009&objAction=browse&s
ort=name)

[4] DOM-RNC Logging, Wei Lou, Core RF Engineering, draft to be released.

[5] QTP-5500 Access Terminal Users Guide, 80-B1311-2, Rev A, Qualcomm, March 2004.
[6] 1xRTT and 1xEV-DO RF Link Budgets, Muhieddin Najib, Core RF Engineering, October
2004 (presentation to Orange).
[7] 1xEV-DO RF System Performance, Miro Budic, Core RF Engineering, October 2004
(presentation to Orange).
[8] 1xEV-DO RF Parameters, Brian Troup, Core RF Engineering, September 2004
(presentation in Verizon Users Group).

[9] Inter-RNC Active Handoff RF Engineering Guideline, Version 0.02, Brian Troup, October
2005.
(http://livelink.us.nortel.com/livelink/livelink.exe?func=ll&objId=9655009&objAction=browse&s
ort=name)
[10] Optis-M Quick Start Guide, v1.05, WirelessLogix, 2004 [11] OPTis-M User Guide,
v1.05, WirelessLogix, 2004
[11] Maximum 1xEV-DO Application Layer Throughput, Dept. 2N51, October 2002.
[12] Generic Network Acceptance Process 1xEVDO Network, v4.1, Mike Woodley, January
2005.
[13] Microsoft Windows 2000 TCP/IP Implementations Details, Dave MacDonald and Warren
Barkley, 2005 Microsoft Corp.
[14] RFC 3481 TCP over Second (2.5G) and Third (3G) Generation Wireless Networks,
www.faqs.org/rfcs/rfc3481.html
[15] TAP Users Guide, Rev. A, Airvana, April 22, 2001
[16] 1xEVDO Troubleshooting Guide, Std 2.05, NTP 411-2133-949, April 2005
[17] 1xEVDO Performance Measurements, Draft 4.07, NTP 411-2133-924, September 2005.
[18] Invex3G Quick Start Guide, version 4.3, Andrew Corporation, 2005.
[19] 1xEV-DO CELLDM Logging Feature Application Guide, Wei Lou, Version 0.1, October
20, 2005.
[20] A13 Dormant Handoff and UA10 Minimization Application Guide, Miroslav Budic, Draft,
September 23, 2005.

Nortel Networks Confidential and Proprietary

1.2

Scope

This document only covers the RF optimization aspect of 1xEV-DO Release 0.


RF optimization guideline document for IS-95 CDMA is available as NTP 411-2133-004. RF optimization
guideline document for IS-2000 (1xRTT) is available at
http://navigate.us.nortel.com/imds?pg=/eng/wne/cdma/sys/po/opt.

1.3

Revision History
Issue
1.0
1.1

Date
November 4, 2005
March 3, 2006

Reason
Initial release for EVDO rel. 3.0.
Updated to reflect EVDO 3.0 ChR.

The latest version of this document is available at


http://navigate.us.nortel.com/imds?pg=/eng/wne/cdma/sys/po/opt.

1.4

Contributors

The following are the key contributors to the contents of the document:

1.5

Rubianto Satrio
Wei Lou
Brian Troup
Miro Budic
Martin Kendall
Mike Woodley
Alexander Contreras
Saied Kazeminejad
Gordon Edwards
Remo Agostino

Audience

This document is intended for Nortel Networks and customer RF engineers that are involved in the 1xEVDO deployment and want to understand how to optimize a 1xEV-DO network. A basic understanding of
IS95 and IS2000, as well as general RF principles is expected of the reader.
THIS VERSION IS CURRENTLY FOR INTERNAL AUDIENCES ONLY

Nortel Networks Confidential and Proprietary

1xEVDO Optimization Events Flow Chart

Establish Network Optimization Entrance and


Exit Criteria (Section 3.1)

Pre Optimization Objectives

1xEVDO / CDMA 1xRTT network RF


Coverage Design Review
(Section 3.1.1 3.1.4)

Coverage design is sufficient to meet


optimization exit criteria?

Y
Review Customer CIQ responses and
current network provisioning
(Section 3.1.5)

Is network provisioning and call models


sufficient to support optimization exit
criteria?

Y
RF Datafill Parameters Review and
Verification (Section 4.1)

Pre Optimization System Health Check


(Section 4.2 and Appendix B)

In Market Optimization Activities

FTAP and RTAP Airlink Bandwidth


Verification (Section 4.3)

TCP / IP Parameter Review and


Verification
(Section 4.4)

Site and Border Shakedowns


(Section 6)

Golden Value (Stationary Testing)


(Section 7)

Cluster Drive Testing / Optimization


(Section 8 - 9)

Exit Market

Nortel Networks Confidential and Proprietary

Exit Criteria
Satisfied?

Perform Troubleshooting
(Section 10)

10

3 1xEV-DO Pre Optimization Objectives


This chapter gives an overview of the events and processes in the 1xEV-DO RF optimization work. The
subsequent chapters (Chapter 3 onward) will then discuss them in more detail.

3.1

Entrance and Exit Criteria

Entrance Criteria
In general there are four important steps that need to be done prior to the 1xEV-DO RF optimization:
Review the 1xEV-DO RF design (in iPlanner) to make sure that it meets the coverage and
capacity/throughput objectives.
Check that the 1xEV-DO network design matches the requirements listed in the CIQ (Customer Input
Questionnaire) response.
Ensure that the recommended datafill is used for the initial RF parameters.
1xEV-DO basic system readiness checkup is conducted.
The 1xEV-DO drive test routes are determined and agreed upon.
Exclusion criteria and warranty boundaries agreed upon by Nortel and the customer
Exit Criteria
The detailed exit criteria need to be negotiated between Nortel Networks and the customer. They have to be
agreed upon prior to the 1xEV-DO RF optimization effort. Metrics that can be used for this purpose
include:
Data-rate-to-DRC matching (i.e., showing that the decoded data rate received matches the DRC
requested).
Single-user application throughput under a certain RF and mobility condition.
Access failure rate.
Connection drop rate.
Connection setup time.

3.1.1

RF Design Review

Just like in CDMA, a good 1xEV-DO network starts with a good 1xEV-DO RF design. Therefore, the first
step in 1xEV-DO RF optimization should be to review the final RF design. Some important items to check
are discussed below.

3.1.2

Coverage Control

One of the most important principles of CDMA RF design, i.e. coverage control, still applies to 1xEV-DO.
In 1xEV-DO, the AT (Access Terminal) continuously estimates the SINR (signal-to-interference-plus-noise
ratio) in the forward link and requests a data rate based on the SINR. In general, the better the SINR, the
higher the data rate requested (the exact algorithm depends on the AT vendor implementation). Hence we
should check the RF design to ensure that each sector covers only the area it is intended to cover. Any
potential pilot pollution area should be cleaned up as much as possible, e.g., by using different antenna
configuration. The final RF design result should show that:

The forward link SINR or Ec/Nt (C/I) is -2 dB throughout the network.

3.1.3

RF Coverage Availability

As in CDMA, the 1xEV-DO RF design must satisfy the path loss requirement for the designed data rate
from the link budget. This can be shown using the pilot composite coverage plot or the required AT
Transmit power plot.
Nortel Networks Confidential and Proprietary

11

The 1xEV-DO cell radius depends, among other things, on the designed data rate. If the 1xEV-DO network
is overlaid 1-to-1 on an IS-95 network, then it is good for the forward link data rate of 307 kbps (at 1900
MHz and with AT receive diversity) and reverse link data rate of 19.2 kbps as the chart below shows [1]. If
higher data rate is needed, then additional cell sites are needed.

1900
MHz

850
MHz

450
MHz

38.4
kbps

76.8
kbps

153.6
kbps

307.2
kbps

614.4
kbps

921.6
kbps

1228.8
kbps

1843.2
kbps

2457.6
kbps

w/o
Diversity

136%

111%

99%

88%

73%

61%

55%

45%

40%

w/
Diversity
w/o
Diversity
w/
Diversity

161%

131%

116%

104%

87%

72%

65%

53%

47%

158%

129%

114%

102%

85%

71%

64%

52%

46%

185%

151%

134%

120%

100%

83%

75%

62%

54%

w/o
Diversity
w/
Diversity

145%

118%

105%

93%

78%

65%

59%

48%

42%

169%

137%

122%

109%

91%

76%

68%

56%

49%

1xEV-DO forward link cell radius relative to IS-95 EVRC.

All
Frequencies

9.6 kbps

19.2 kbps

38.4 kbps

76.8 kbps

153.6 kbps

111%

101%

89%

75%

57%

1xEV-DO reverse link cell radius relative to IS-95 EVRC.


Another important thing to consider is that a certain data rate in the reverse link is required to support the
high data rate in the forward link. Therefore the RF design has to ensure the reverse link data rate is enough
to support the forward link data rate. The chart below shows the measurement results from LAV network in
Ottawa [6], [7]. To support the TCP ACKs, the reverse link data rate has to be at least 1/40 of the forward
link data rate. So, for example, a forward link data rate of 2 Mbps requires a reverse link data rate of 50
kbps or higher.

Normalized Fwd Rate [%]

Max Fwd Rate vs Rev Rate


100
98
96
94
92
90
88
86
84
82
80
19.2

38.4

76.8

Reverse Rate [kbps]

Reverse link data rate impact on forward link throughput.


Nortel Networks Confidential and Proprietary

12

3.1.4

Inter-RNC Border Design

In Nortels EVDO release 3.0 there is a new feature called Inter RNC Active Handoff (IRAHO) that will
allow ATs to add pilots to its Active set from DOMs primarily homed on adjacent RNCs. The intent of the
feature is to:

Prevent dropped connections and access failures in border areas


Enhance sector and AT throughput in border areas

This feature allows the AT to add pilots from a DOM in another subnet through secondary homing.
Secondary homing allows an operator to home DOMs to up to 8 secondary RNCs in addition to its Primary
RNC (i.e. data fill the routing table in the DOM with the primary RNC and up to 8 secondary RNCs).
Through secondary homing an AT can add pilots from DOMs in an adjacent subnet without having any
knowledge of that subnets Color Code, or Subnet ID. The figure below shows an example of the physical
connectivity of DOMs and RNCs in a multi homed scenario.

Figure EVDO Multi Homing Configuration example


The limitations of this feature are that it is only functional when the AT is already in an Active data call and
crosses a subnet boundary. If the call continues long enough such that the AT requests a pilot from a DOM
in the adjacent subnet that is not secondarily homed to the source or anchor RNC then the call will drop
because the Resource Lookup on the anchor RNC will fail.
Implementation of the inter RNC active soft handoff feature requires the secondary sector border to be
planned such that coverage between the primary sectors and the secondary sectors is contiguous to enable
soft handoffs to occur. To prevent dropped connections the secondary sectors should be in areas of low
mobility such as secondary roadways or through areas where there are not high concentrations of fixed
Nortel Networks Confidential and Proprietary

13

users. Care should be taken in border planning such that there will be a dominant PN in the coverage
overlap region between border sectors. The absence of a dominant server in these areas may lead to Ping
Pong. Excess ping pong can lead to excessive session setups, access failures and dropped connections. The
following figure illustrates secondary sector deployment in an area of high mobility. The secondary border
can be defined as Subnet 1+ in the figure below. The shape of the secondary border for Subnet 1 will allow
ATs originating calls in Subnet- 1 to move to sectors deep within Subnet-3 and Subnet -7 without
dropping the active call and would be ideal for extending coverage in Subnet-1 along the highway or any
other high speed road [9].

(Subnet-2)
RNC-2

(Subnet-3)
RNC-3

(Subnet-7)
RNC-7

Highway or other
high speed road

RNC-1
(Subnet-1)

Subnet-1+

RNC-4
(Subnet-4)

RNC-6
(Subnet-6)

RNC-5
(Subnet-5)

Figure: IRAHO Example Showing the Ability to Maintain an Active Connection from RNC-7 -> RNC-1
-> RNC-3
The level of mobility can be seen as the percentage of calls entering or leaving a cell at any particular
period of time. It could be expressed by the following ratio:

Cell
Coverage

A
ATs entering cell

C
ATs leaving cell

ATs staying in cell

Nortel Networks Confidential and Proprietary

14

Level of mobility = (A + C) / (A + B + C)
One way to get an idea of low mobility areas would be to look at the 3G data mobility behavior of the
underlying 1xRTT network. There is no formula that can provide an exact mobility ratio, but the BTS OMs
FCHOriginationNonBlocking3GData and the FCHHandoffNonBlocking3GData can help to provide an
educated guess. They can be used in the following ratio which is similar to the mobility ratio above:
Level of mobility (FCHHandoffNonBlocking3GData) /
(FCHHandoffNonBlocking3GData + FCHOriginationNonBlocking3GData)
Similarly, one way to get an idea of the level of data activity in a 1xEV-DO network is to analyze the data
activity in the underlying 1xRTT network. This can be done by comparing, on a per sector basis, the data
Minutes of Use (MOU) within the 1xRTT network. On a Nortel Networks 1xRTT data network, the MOU
can be determined using the 3G Data MOU, i.e. PrimaryFrameCntFCH[3,5,7] from the BTS performance
OMs. Lower amount of 3G Data MOU would indicate a lower data activity area.
After initial deployment, RNC logs should be collected and analyzed using the NEWTUN tool. The
NEWTUN tool will provide a list of DOMs that should be homed as secondary DOMs to the RNC under
consideration. The use of NEWTUN to refine the DOM homing configuration will be iterative but should
be highly refined after about the 3rd round of analysis.
Setting and managing the number of primary and secondary DOMs per RNSM is done at the DO RNCs
topology manager. In addition to managing the number of primary and secondary DOMs per RNSM, each
DOM has a primary selection table used for data filling the IP address of the primary RNC. Likewise there
is a secondary RNC homing table which contains fields for adding the IP addresses of secondary RNCs
and a field the indicates whether secondary homing has been enabled or not [9].

Example DOM Menu Config Primary / Candidate and Secondary RNCs

3.1.5

CIQ Review and Provisioning Verification

Prior to the 1xEV-DO RF optimization, it is a good practice to review the 1xEV-DO CIQ that the customer
filled out and check that the 1xEV-DO network design matches the requirements listed in the CIQ
response. Several important lists/information to check are as follows:

IP address design for RNCs, nodal elements, and DOMs


Inter Subnet borders and assignment of secondary RNCs to border sites up to 8 secondary RNCs
homed per sector.

Nortel Networks Confidential and Proprietary

15

Unique Color Code value for each RNC


EVDO Neighbor List maximum of 20 neighbors per sector
Traffic Engineering data application call model, distribution of services, and subscriber counts

Reference [3] provides the complete provisioning rules and procedures for the DOM, DO-RNC, DO-EMS,
and the backhaul networks. Of particular interest to RF engineers is probably the DOM provisioning.
Currently each Metrocell can house three DOMs. However, there can be only one DOM per carrier (for 3
sectors). Therefore, if 1xEV-DO is deployed in only one carrier, the maximum number of DOMs is the
same as the number of Metrocells. If the required number of DOMs is bigger than the number of
Metrocells, then cell splitting may be considered.
The required number of DOMs depends on the DOM capacity limitation, reverse link Erlang requirement,
forward link throughput requirement, and the Metrocell limitation mentioned above [2]. A DOM has 96
channel elements and can serve about 90 connections in the reverse link (across 3 sectors). Assuming an
average CE/user of 1.5, the number of primary users that can be served by a sector in a tri-sector site is 20
[1].

Nortel Networks Confidential and Proprietary

16

4 1xEV-DO Optimization Events


4.1

Initial RF Parameters Verification

Since 1xEV-DO introduces many new RF parameters, it is important to verify that the correct RF
parameters have been configured. The default RF datafill parameters are the recommended starting point
for optimizing a network. Please refer to [3] and [8] to get detailed descriptions these parameters and their
recommended values.
EVDO Editor is a PC based application for viewing and editing specific RF parameters at the DOM level
(i.e. RF Gain, Cell Radius, RAB Offsets, and Neighborlists). It also will allow users to view parameters at
the RNC level. Currently, EVDO Editor has business rules checking capabilities for DOM level
parameters. Future releases will have business rules checking capabilities for RNC level parameters.
EVDO Editor can be found at http://navigate.us.nortel.com/imds?pg=/eng/wne/core/tool/adm/cdma/doed.

EVDO Editor

4.2

Pre Optimization System Health Check

The basic system readiness checkup is done to verify that the 1xEV-DO system is ready to carry traffic. In
summary, the following attributes should be checked:

Verify status of all Nodes (DOM and RNC) using NORTEL-XX# show node have an administrative
and operational status of Up
Verify status of all Modules (BIO Cards, RNSM, SC, DOM, etc) using NORTEL-XX# show module
has a status of Active.
Verify status of all the interfaces (i.e. Ethernet, T1/E1, PPP, node 1/0/1) using NORTEL-07# show
interface command. All interfaces must have an administrative and operational status of UP.

Nortel Networks Confidential and Proprietary

17

Ethernet interfaces:
DOM Backhual If a T1 is connected to a DOM along with 100/10 base T ethernt
connection, the DOM will prompt traffic from the RNC over T1.
2 Ethernet ports per BIO card for support of A10/A11 interface communication,
Abis, A12, and OA&M paclets. On CLI, BIO Ethernet interfaces are listed as
Ethernet 1/x/1, 1/x/2 where x denotes the slot of the card.
1 Ethernet port per SC control module - these are optional interfaces for carrying
OAM traffic. On CLI, the active SC port is listed as Ethernet 1/0/1.
Management IP Interface an optional interface that is used in conjunction with the
SC aux. Ethernet if operators want to forward management IP traffic to a virtual
management interface. On CLI the management IP interface is listed as mgmt1/0/1.
o All Used T1/E1:
DOM Backhaul used to backhaul DOM physical and link layer traffic to the
aggregation router.
o PPP Interface for each T1/E1 interface
DOM Backhaul - a PPP link is associated with each T1 between the DOM and the
aggregation router to provide duplexed, simultaneous, bidirectional, sequential,
packet transfer of encapsulated Abis and OAM IP packets between two dedicated
network peers. On CLI, the DOM T1 / PPP interface is denoted as: ppp1/0/1,
ppp1/0/2, ppp1/0/3, and ppp1/0/4 interfaces.
o Node 1/0/1 interface
Abis and OAM data - provides a loop back or virtual interface for the transfer of
Abis data from the DOM to the primary RNC and OAM data to the EMS. On CLI it
is shown as : node 1/0/1 interface.
Status of the Abis links between the DOM and the RNC using NORTEL-07# show abis peer. Every
RNC should have an Abis status of Connected to every DOM that is homed to it.
Status of the traffic and control channels established between the DOM and the RNC.
Ping the Node IP address of DO RNC, IP Addresses on the aggregation router associated with DOM
backhaul links, and the DO EMS IP address from the DOM. NORTEL-07> ping 10.0.0.0
Ping the IP address of all DOMs, IP addresses of DOM PPP Links, all PDSNs, and the DO EMS from
the RNC.
Status of GPS for all DOMs. NORTEL-07> show GPS health
o

The basic system readiness checkup should be done from the EMS. An example of how to do it from the
command line interface can be found in Appendix B. In the event any of the system readiness checks fail
refer to the troubleshooting or recovery sections of reference [16].

Nortel Networks Confidential and Proprietary

18

4.3

FTAP and RTAP Testing

FTAP and RTAP tests should be performed to network end to end performance capabilities inclusive of the
EVDO airlink. FTAP and RTAP are built in test EVDO testing utilities that send test frames to determine
the available bandwidth of the network.
Prior to testing, a location where RF is known to be good should be selected. In addition, AT logs should be
captured during TAP testing to measure the PER% seen at the AT.
Data rates given by FTAP tests should be equivalent to the average DRC request rate and RTAP tests
should show equivalence to the average Reverse link Rate limit of the sector.
The following is an example of how to configure TAP tests from CLI: For more information on TAP tests
see, reference [15].

Nortel Networks Confidential and Proprietary

19

4.4

TCP / IP Parameters

Prior to starting optimization a review of configured TCP / IP parameters should be performed. Due to the
fact the FTP protocol, used extensively in optimization activities, is based on TCP, a thorough review of
configurable TCP and PPP layer parameters should be performed to insure accurate performance reporting.
See Reference [13] for more information on TCP/IP parameter descriptions.
Recommended TCP/IP Configuration:

Data Collection Laptop configuration Insure that the following TCP / IP settings are
optimized. TCP send and receive buffers should be optimized according to the bandwidth delay
product (i.e. Buffer Size = Bandwidth x delay). For EVDO, ideal Buffer Size =
(2450kbps/8bits/byte * 150msecs) = 45.9kBytes.
o

Recommended TCP RX Window Size = 64240

TcpWindowSize (Configured from Registry Editor)


Key: Tcpip\Parameters, Tcpip\Parameters\Interface\interface

Value Type: REG_DWORDnumber of bytes


Valid Range: 00x3FFFFFFF (1073741823 decimal). In practice the TCP/IP stack will
round the number set to the nearest multiple of maximum segment size (MSS). Values
greater than 64 KB can be achieved only when connecting to other systems that support
RFC 1323 Window Scaling.
Description: This parameter determines the maximum TCP receive window size
offered. The receive window specifies the number of bytes that a sender can transmit
without receiving an acknowledgment. In general, larger receive windows improve
performance over high-delay, high-bandwidth networks. For greatest efficiency, the
receive window should be an even multiple of the TCP Maximum Segment Size (MSS).
This parameter is both a per-interface parameter and a global parameter, depending
upon where the registry key is located. If there is a value for a specific interface, that
value overrides the system-wide value. See also GobalMaxTcpWindowSize.

Recommended MTU Size = 1500


MTU (Configured from Registry Editor)
Key: Tcpip\Parameters\Interfaces\interface
Value Type: REG_DWORDnumber
Valid Range: 88the MTU of the underlying network
Default: 0xFFFFFFFF
Description: This parameter overrides the default Maximum Transmission Unit (MTU)
for a network interface. The MTU is the maximum packet size, in bytes, that the
transport can transmit over the underlying network. The size includes the transport
header. An IP datagram can span multiple packets. Values larger than the default for the
underlying network cause the transport to use the network default MTU. Values smaller
than 88 cause the transport to use an MTU of 88.

Note: Windows 2000 TCP/IP uses PMTU detection by default and queries the NIC
driver to find out what local MTU is supported. Altering the MTU parameter is
generally not necessary and may result in reduced performance.

PMTU Discovery (Enabled)

Nortel Networks Confidential and Proprietary

20

When a connection is established, the two hosts involved exchange their TCP maximum segment
size (MSS) values. The smaller of the two MSS values is used for the connection. Historically, the
MSS for a host has been the MTU at the link layer minus 40 bytes for the IP and TCP headers.
However, support for additional TCP options, such as time stamps, has increased the typical
TCP+IP header to 52 or more bytes.

EnablePMTUDiscovery
Key: Tcpip\Parameters
Value Type: REG_DWORDBoolean
Valid Range: 0, 1 (false, true)
Default: 1 (true)

IP Header Compression (Disabled)


It is well known (and has been shown with experimental data) that TCP header compression does
not perform well in the presence of packet losses. If a wireless link error is not recovered, it will
cause TCP segment loss between the compressor and decompressor, and then header compression
does not allow TCP to take advantage of Fast Retransmit Fast Recovery mechanism. The header
compression algorithm does not transmit the entire TCP/IP headers, but only the changes in the
headers of consecutive segments. Therefore, loss of a single TCP segment on the link causes the
transmitting and receiving TCP sequence numbers to fall out of synchronization. Hence, when a
TCP segment is lost after the compressor, the decompressor will generate false TCP headers.
Consequently, the TCP receiver will discard all remaining packets in the current window because
of a checksum error. This continues until the compressor receives the first retransmission which is
forwarded uncompressed to synchronize the decompressor.
As previously recommended, header compression SHOULD NOT be enabled unless the packet
loss probability between the compressor and decompressor is very low. Actually, enabling the
Timestamps Option effectively accomplishes the same thing. Other header compression schemes
like RFC 2507 and Robust Header Compression are meant to address deficiencies in RFC 1144
header compression.
Figure 4 Disabling IP Header Compression in Windows 2000
In order to disable IP header compression, go to Start -> Settings->Network and Dial Up
Connections, and then select the appropriate device and then Properties. After selecting
properties, then go to the TCP/IP component under the Networking tab and select properties again.

Nortel Networks Confidential and Proprietary

21

Software data compression (Disabled)

Data compression enables information to be transmitted beyond the actual connection speed. Data,
particularly text and graphics, usually contain repeated sequences of identical information. Data
compression works by replacing many characters of repeated information with a few characters
and transmitting only one copy of repeated sequences of data.
Communication software, such as Network and Dial-up Connections, may support data
compression. For example, using a 14.4 Kbps V.32bis modem, you can enable software
compression, and experience an average throughput of 28.8 Kbps. Tests show that software
compression can result in higher data transfer rates than hardware compression.
To insure accurate reporting of application or physical layer throughput over the access network
the Enable Software Compression setting in the Windows Dialup Networking Properties ->>
PPP Settings should be disabled.
Figure Disable Software Compression

Nortel Networks Confidential and Proprietary

22

LCP Extensions (Disabled)


LCP is part of the PPP protocol that provides for the establishment, configuration, and testing of
the peer to peer data link connection. The enable LCP extensions option within the PPP settings of
Windows Dial Up Networking allows the software to take advantage of additional LCP features.
Some extensions for LCP and there functions are as follows and more detail provided in
http://ietfreport.isoc.org/idref/rfc1570/#ref-2
1.
2.
3.
4.
5.
6.

Identification [12] allows the application to identify itself to its peer (Link Maintenance
Packet)
Time Remaining [13] notifies the peer of the time remaining in the session
Frame Check Sequence (FCS) Alternatives [9] provides a method to specify another FCS
format to be sent by the peer or to negotiate it away altogether.
Self Describing Padding [10] Indicates to the peer that the use of padding is understood.
Used when some protocols, such as network layer or compression protocols, are configured
which require detection and removal of any trailing padding.
Callback [13] provides a method for the implementation to request a dial up peer to call
back.
Compound Frames [15] allows the implementation to send multiple PPP encapsulated
packets within the same frame. (self describing padding must be used in conjunction with this
option)

TCP Timestamps (Enabled)


Another RFC 1323 feature introduced in Windows 2000 is support for TCP time stamps. Like
SACK, time stamps are important for connections using large window sizes. Time stamps were
conceived to assist TCP in accurately measuring round-trip time (RTT) to adjust retransmission
time-outs. The use of time stamps is disabled by default. It can be enabled using theTcp1323Opts
registry parameter.
Tcp1323Opts
Key: Tcpip\Parameters
Value Type: REG_DWORDnumber (flags)
Valid Range: 0, 1, 2, 3
0
1
2
3

(disable RFC 1323 options)


(window scale enabled only)
(timestamps enabled only)
(both options enabled)

Default: No value; the default behavior is as follows: do not initiate options but if requested
provide them.
Description: This parameter controls RFC 1323 time stamps and window-scaling options. Time
stamps and window scaling are enabled by default, but can be manipulated with flag bits. Bit 0
controls window scaling, and bit 1 controls time stamps.

Selective Acknowledgment (SACK) - (Enabled)


SACK is especially important for connections using large TCP window sizes. Prior to SACK, a
receiver could only acknowledge the latest sequence number of contiguous data that had been
received, or the left edge of the receive window. When SACK is enabled, the receiver continues to
use the ACK number to acknowledge the left edge of the receive window, but it can also
acknowledge other non-contiguous blocks of received data individually.
When SACK is enabled (the default), a packet or series of packets can be dropped, and the
receiver can inform the sender of exactly which data has been received, and where the holes in the
data are. The sender can then selectively retransmit the missing data without needing to retransmit
blocks of data that have already been received successfully.

Nortel Networks Confidential and Proprietary

23

SACK is controlled by the SackOpts registry parameter. The Network Monitor capture below
illustrates a host acknowledging all data up to sequence number 54857341, plus the data from
sequence number 54858789-54861685. SACK is enabled by default in Windows
+ FRAME: Base frame properties
+ ETHERNET: ETYPE = 0x0800 : Protocol = IP: DOD Internet Protocol
+ IP: ID = 0x1A0D; Proto = TCP; Len: 64
TCP: .A...., len:0, seq:925104-925104, ack:54857341, win:32722,
src:1242 dst:139
TCP: Source Port = 0x04DA
TCP: Destination Port = NETBIOS Session Service
TCP: Sequence Number = 925104 (0xE1DB0)
TCP: Acknowledgement Number = 54857341 (0x3450E7D)
TCP: Data Offset = 44 (0x2C)
TCP: Reserved = 0 (0x0000)
+ TCP: Flags = 0x10 : .A....
TCP: Window = 32722 (0x7FD2)
TCP: Checksum = 0x4A72
TCP: Urgent Pointer = 0 (0x0)
TCP: Options
TCP: Option Nop = 1 (0x1)
TCP: Option Nop = 1 (0x1)
+ TCP: Timestamps Option
TCP: Option Nop = 1 (0x1)
TCP: Option Nop = 1 (0x1)
TCP: SACK Option
TCP: Option Type = 0x05
TCP: Option Length = 10 (0xA)
TCP: Left Edge of Block = 54858789 (0x3451425)
TCP: Right Edge of Block = 54861685 (0x3451F75)

FTP Server (Sun) XMIT Hi Water mark


o If the FTP server is a SUN Solaris workstation, insure that the TCP XMIT HIWAT
setting is configured for 64k.
o Windows based FTP servers have no configurable TCP TX buffer setting.
o ndd -set /dev/tcp tcp_xmit_hiwat 65536

The tcp_xmit_hiwat parameter is effectively the size of the send buffer. The send
buffer high watermark tunes the transport layer socket buffer size on a kernel wide
basis. The socket buffer can also be changed on a per-socket basis by using the
SO_SNDBUF socket option within an application. Mind that for UDP the size of the
output buffer represents the maximum datagram size.

4.5

Shakedowns

The first step in 1xEV-DO RF optimization is to perform a shakedown of each cell site and all inter subnet
borders. Like in CDMA, the objective of the shakedown is to verify that we can get a connection on each
sector and switch to the adjacent sector in the forward link (and softer-handoff to it in the reverse link), and
to ensure that the sector RF parameters (like PN offset and neighbor list) are correct. Chapter 6 presents the
detailed procedure to do the shakedown.

4.6

Golden Value (Stationary) Testing

The second step in 1xEV-DO optimization is to perform a stationary data testing to measure the maximum
single-user application throughput in the forward and reverse link. During this exercise, the TCP, RLP,
and/or other RF parameter setting can be optimized to improve the throughput. Once those settings are
optimized and finalized, the measured single user application throughput becomes the golden value, i.e.,
Nortel Networks Confidential and Proprietary

24

the upper bound of what we can achieve in the network. Chapter 7 presents the detailed processes involved
in performing stationary (Golden Value) data testing.

4.7

Cluster Drive Testing

Once the stationary data testing is done, we can do drive tests along the important roads to measure various
RF performance in a cluster. We can also do more stationary data testing in different locations (that are
deemed important) throughout the cluster. Based on these test results, the datafill (i.e. Power and resources
related parameters, etc) can be adjusted to improve the 1xEV-DO performance. Chapter 9 details the
process involved in performing typical 1xEV-DO cluster drive testing.

4.8

Troubleshooting

If RF performance problems are encountered during the stationary and drive testing (or anytime during the
deployment period), engineers will have to analyze various performance attributes in efforts to diagnose
where the root cause lies. Depending on the problems, troubleshooting may require more than the regular
mobile logs obtained from the cluster drive testing. Chapter 10 presents various tips and examples to help
engineers troubleshoot various 1xEV-DO RF performance problems.

Nortel Networks Confidential and Proprietary

25

5 Data Collection / Post Processing Tool Setup


This chapter will discuss briefly the requirement and the setup procedure for 1xEV-DO RF data collection
and post-processing tools. More detailed discussion can be found in the reference documents mentioned in
each section.

5.1 Laptop Setup


The preferred method of 1xEV-DO data collection/testing is using a laptop connected to WirelessLogix
Optis-M tool (EV-DO version) or Andrew Invex3G. If only one AT is used (like in the shakedown), then
XCAL-DO or Invex3G PC can be used.

Laptop connected to Optis-M data collection tool.

Laptop connected to Invex3G data collection tool


The laptop used for 1xEV-DO testing has to have the following:
Windows 2000 or XP operating system
Intel Pentium4 processor 1.6 GHz equivalent or above
512MB RAM or above
Please refer to [12] for more detailed specification requirements.
The following software needs to be installed in the laptop:

Nortel Networks Confidential and Proprietary

26

Drive Test Data Collection Tools:


WirelessLogix Optis-M / XCAL -DO version 2.2.0 or later.
o Available from Couei website: http://pms.couei.co.jp
o Login with user id = guestna@nortelnetworks.com and password = WLsupportNortel
Andrew Invex3G / PC version 4.3 or later.
o http://www.andrew.com/products/measurement_sys/Invex/
Drive Test log Post Processing Tools:

EV-DO RF Optimizer version 2.2.5.30 or later.


o Available at http://navigate.us.nortel.com/imds?pg=/eng/wne/core/tool/cdma/rfoevdo)
XCAP-DO ((from WirelessLogix) version 3.77 or later can also be used.
o Available from Couei website: http://pms.couei.co.jp
o Login with user id = guestna@nortelnetworks.com and password = WLsupportNortel
DOM-RNC logs post-processing tools:

NEWTUN Available at: http://navigate.us.nortel.com/imds?pg=/eng/wne/core/tool/adm/cdma/ntn


LogFileConvert (Java tool) (Available on RNC)
LogConvertDOS (Available on RNC)
Applications:

FTP Client application such as WS-FTP Pro. Chariot may also be used.
Optional tools:
TCP performance logging tools: Windump, Snoop, Ethereal and TCPTrace.
OM analysis tool: OMAX (Nortel OM analysis tool available through Mike Anderson)
There are several TCP parameters that need to be set correctly. See Section 4.4 above.
If either Optis-M or Invex3G are used as the data collection tool, then the TCP parameters need to be set
inside the tool. However, if either XCAL-DO or Invex3G - PC are to be used, then we need to change these
parameters in the laptop. One easy way to do it is by using DrTCP utility software (available in the internet,
for example from http://www.dslreports.com/drtcp).
Section Appendix 4.4 contains a detailed description of settable TCP / IP paremeters on how to optimize
them for the data collection tools (i.e. laptop, etc) and the FTP server.

5.2 AT Setup
The ATs (mobiles) that are recommended to be used for 1xEV-DO RF optimization along with data
collection platform support are given below:

Qualcomm QTP-6500 (ZRF 6500)


Sierra Wireless 580
Sierra Wireless /Audiovox 5220
Samsung A890
LG Vx8000
Novatel Merlin V620

Optis / XCAL / Invex3G


Optis / XCAL / Invex3G
Optis / XACL / Invex3G
Optis / XCAL
Optis / XCAL
Optis / XCAL / Invex3G

If XCAL-DO or Invex3G - PC is used as the data collection tool then the right driver for the AT needs to
be installed in the laptop.

5.3 Data Collection Tool Setup


5.3.1

RNC Logging Setup

RNC logs should be collected for the following tests:


Nortel Networks Confidential and Proprietary

27

A13 Dormant Session Handoff Shakedowns


Cluster Drive Tests

Note: The timestamps given in RNC logs are based on GMT time. In order to correlate the RNC logs with
AT logs it will be necessary to adjust the RNC log timestamps according to the difference in local time and
GMT time.
For a fully loaded DO-RNC, there is one active System Controller on slot 7, one redundant SC on slot 9,
one hot swap SC on slot 10 and one hot swap redundant SC on slot 8. A single DO-RNC can have up to
four RNC-BIO modules and up to eight RNC-RNSM modules.
For a DOM, there is one BIO/SC module and one 1xEVDO modem. The 1xEVDO-Modem contains two
modules, Forward Link Processor Module (FLM) and Reverse Link Processor Module (RLM). Given that
call processing resources could be assigned to any cards in the RNC, logging has to be enabled on all the
cards (RNSM or SC) in the system.
Logging can be configured through the DO EMS or CLI to configure the logging system of each individual
card.
Logging Trap Severity
Within each hardware module that has a CPU, various software components generate all kinds of events.
The verbal description of these events is considered as event messages. Those event messages can be
selectively captured as a log by logging system of the module. The selective capturing functionality is
implemented by setting up a set of input traps which is also called Severity.
So in the end, the captured logging messages by the logging system are the subset of all event messages
generated by those software components within the hardware module. The EVDO system supports severity
levels from 1-32. For most events a severity level of 8 is all that is required. Other events like Forward
Sector Switching may require logging severity levels of 22.
The following is shows the components of an RNC log message:

Time Stamp
of the Date

Message
Severity Level

Component ID

Message
Body

6-10-04 10:19:47.241 S=16 C=010301 F=0009 ID=0397 [0x3403ef (uati) SCSM


SCSM_Open] : Received Connection Opened Indication event

Time Stamp of
hh:mm:ss.sss

CPU ID

Message ID

Configure the Log File


Both the DO-RNC have there own logging manager where the logging manager is the means of
configuring the properties of the Log File. The log file itself is the actual captured log saved on the RNC
SC, RNSM or the DOM BIO/ SC cards.

Nortel Networks Confidential and Proprietary

28

To configure the log files using the DO-EMS go to Network Database DO-RNC RNC
Menu Show Cards as shown below.
Select the SC Card from the Cards Menu
From the SC Card Menu select Show logfacilityMgr Modify Logfacility 5 (Call Control)
The following boxes need to be configured in the Call Control Log facility Manager on the SC Card
o Logging Enabled = True
o Maximum Severity = 5, 8, 16, 22, 32, etc
o Output to File = On
o Maximum Severity on File = Maximum Severity entered above

DO-EMS RNC Menu

Nortel Networks Confidential and Proprietary

29

RNC Cards Menu

SC Card Menu

Log Facility Manager

Nortel Networks Confidential and Proprietary

30

Call Control Log Facility


Once all changes are made to the Call Control Log Facility Manager select Submit
To perform a severity level range selection, for example from 1 to 16 on call-control component do
the following from the CLI prompt:
NORTEL-07>en
NORTEL-07#config
NORTEL-07(config)#logging trap severity 16 call-control
To perform a set of individual severity levels selection, for example severity level 5, 9 and 16
on call-control component.
NORTEL-07>en
NORTEL-07#config
NORTEL-07(config)#logging trap severity fatal call-control
NORTEL-07(config)#logging trap severity 5 call-control +
NORTEL-07(config)#logging trap severity 9 call-control +
NORTEL-07(config)#logging trap severity 16 call-control +
NORTEL-07(config)#logging trap severity fatal call-control NOTE: To configure a set of individual severity level selection, users need to first specify a
range of severity setting then add each individual severity level one at a time with +. Finally,
users need to remove the first range severity setting with - sign.
To access those non-SC modules, users need first login into those SC modules. From there use
following procedure to access and configure the input traps on those modules.
Nortel Networks Confidential and Proprietary

31

NORTEL-07>en
NORTEL-07#config
NORTEL-07(config)#module-logging bio1/11/1
NORTEL-07(module-logging-bio1/11/1)#trap severity 16 resource-control
NORTEL-07>en
NORTEL-07#config
NORTEL-07(config)#module-logging rnsm1/13/1
NORTEL-07(module-logging-rnsm1/13/1)#trap severity 16 call-control
To Enable or disable logging from the CLI prompt do the following:
On RNC SC card for call-control component:
NORTEL-07>en
NORTEL-07#config
NORTEL-07(config)#logging start call-control
(To enable log capture)
NORTEL-07(config)#no logging start call-control (To disable log capture)
Parsing and Retrieving Log Files
To parse the logs under RNC: (This step is not necessary if using NEWTUN)
NORTEL-07(config)#logging convert file <rnc_binary_log.bin> <rnc_output_file.txt>
No wild card supported under command line, so log need to be parsed one by one.
To check the log files under RNC:
NORTRL-07>shell
NORTRL-07(shell)(disk0)>cd logs
NORTRL-07(shell)(disk0)>ls

(To view all the files under the logs directory)

To get the log the log files, users can use various FTP programs to access the log directory and then extract
those logs. Once those logs are download to users local hard drive, they can be viewed with any kind of
text file editor program.
For more detailed information on RNC logging see reference [4].

5.3.2

Optis-M Setup

Once the Optis-M GUI is installed, the user needs to do the following:
Connect AC/DC Converters input connector to power source.
Connect AC/DC Converters output connector to the Power supply connector on the front panel of
Optis-M.
Connect Ethernet cable to Ethernet connector on the front panel of Optis-M and laptop.
Turn on the Optis-M unit using the power switch on the front panel.
Connect Mobile data cable to the USB port of each slot on the front of Optis-M.
Connect GPS Antenna to the GPS Antenna connector on the front panel of Optis-M.
Set the laptop IP address to the appropriate one (usually 1.1.1.1) so that it can communicate with
Optis-M box.
Start the Optis-M software. Ensure that the client hardware status light (at the bottom right corner of
Optis-M window) is green.
Configure the Optis-M ports. Below is an example for two Chesters connected to Optis-M.

Nortel Networks Confidential and Proprietary

32

Port Setting window in OPTis-M.

Phone Type setting under Port Settings window.

Nortel Networks Confidential and Proprietary

33

Data Port Configuration under the Port Settings window

Set the log masks.

For more details on setting up Optis-M, please refer to [10].

Nortel Networks Confidential and Proprietary

34

5.3.3

Optis-M Log Masks

The log mask settings for the general 1xEV-DO data collection (e.g. cluster drive testing) are shown in the
pictures below. The 1xEV-DO log masks are always needed. The CDMA/1xRTT log mask is needed for
measurement with hybrid AT (e.g. for 1xEV-DO to 1xRTT handoff analysis).
For a specific testing or troubleshooting, different log masks may be needed.

Nortel Networks Confidential and Proprietary

35

OPTis-M 1xEV-DO log mask.

Nortel Networks Confidential and Proprietary

36

OPTis-M
CDMA/1xRTT
log mask setting.

Nortel Networks Confidential and Proprietary

37

5.3.4

Invex3G Chassis Setup

Note: Invex3G Chassis, Release 4.3 / 4.4 should be provisioned with at least 2 DI (digital
communications interface) boards for EVDO optimization testing. The legacy CI boards are
processor limited which will have negative consequences when measuring data throughput
performance.
DI Card Slot

Note: If Invex3G DI boards are not available at the commencement of optimization testing,
Invex3GPC or Wirelesslogix tools should be used to insure success.

Invex3G Chassis - Setup

Connect the RJ45 Ethernet crossover cable from the System Controller Module (GWSC-0100) of the
Invex3G mainframe to the Network Interface Card of the Personal Computer (laptop) that will be used
to run the Invex3G software.

Connect the GPS antenna to the GPS ANT connector on the mainframe.

Insert the AT (PCMCIA) into the Digital Card Interface (DCI) Module or attach the AT to the
Communications Interface Card (CI) using the appropriate connector cable to the CI ports (STS-A and
STS-B).

Invex3G mainframe also charges all ATs that use the proprietary cable interface to the CI board.

The upper port (A) of the CI module supports Data and Voice application, and the lower port (B) is
reserved for Voice ONLY. Therefore if doing a data call, use ONLY the upper ports in the
Communication Interface (CI) Card.

Each CI port and DI card slot has dedicated LED Indicators (STS-A and STS-B) for status check:

Green: Phone Attached - Normal

Red: Fault

Amber: Waiting

Connect the power supply to the Invex3G unit.

Nortel Networks Confidential and Proprietary

38

Open the start menu on the laptop and go to My Network Places or Network and Dial up
Connections and select the Local Area Connections icon and then select properties.
On the General tab go the Internet Protocol (TCP / IP) component and select Properties
On the General tab select Use the Following IP Address and enter the following:
o IP Address 192.168.3.100
o Subnet mask 255.255.255.0
Click OK and Close the Networking tab
Power the laptop and the Invex3G unit. The unit can be powered either by connecting directly to the
cigarette lighter or using an inverter in the van.
Once the software has been launched it is necessary to create new device connections if they were not
created beforehand.
o Click on the Connections tab in the workspace

o Click on the Connections folder icon and select New connection

o Next, the add new connection dialog appears. There are three options available for
connection configuration:
o Invex3G Chassis specifies connection to the hardware mainframe
o Invex3G Serial Device specifies connection to a device attached to
the PC (Invex3G-PC option)

o Select Invex3G default chassis option and then OK.


o Select Connect All from the workspace connections menu to open the connection to the
Invex3G chassis.

o After selecting connect all the Open Connections dialog begins. Click the Andrew
chassis name to highlight it and then click OK.

Nortel Networks Confidential and Proprietary

39

o The Invex connection status dialog box will open up to display the connection progress.
At the initial start up it may take several minutes for the chassis flash memory to update
so, be patient.
Once the connection process is complete a list of devices attached to the chassis will be displayed in
the devices tree

5.3.5

InVex3G Log Masks

Configure Invex3G logs masks as follows for EVDO and CDMA log collection.

Go to the Devices tab in the Invex3G workspace and right click on the collection device and
select properties. The following window will appear.

Nortel Networks Confidential and Proprietary

40

Select the CDMA L3 and QCP 1xEVDO L3 tabs and click Collect 1xEVDO Layer 3 Messages
and Collect CDMA Layer 3 Messages boxes.
Select the QCP 1xEVDO tab and then select all messages.
Select the QCP1X tab and select all messages.
For more information on Invex3G configuration see reference [18].

5.4 AT Log Data Post-Processing Tool Setup


5.4.1

RF Optimizer EV-DO

RF Optimizer EV-DO software can be downloaded from the link below:


http://navigate.us.nortel.com/imds?pg=/eng/wne/core/tool/cdma/rfoevdo
Then follow the installation procedure carefully. Once RFO is installed, attach a RFO dongle key and
launch RFO. Go to Tools Options to configure the tool:

Nortel Networks Confidential and Proprietary

41

In the "File Locations" tab, be sure to select a location for "Extraction Logs" that you can easily locate
later. The extraction log accumulates many informative messages during raw processing. Since new
messages always get appended to the existing file, the file can become quite large. It's a good idea to
periodically delete the extraction log once you have reviewed its contents.
When selecting locations for "Database Data Files", "Extraction Logs" or "Temporary Files", be sure
to select only folders to which you have full access rights, and on the drive that has plenty of available
space. These files, created by RF Optimizer EVDO during processing, can consume a significant
amount of disk space.

In the "Binning" tab, set temporal and spatial bin sizes (the default setting of 1s and 100m is a good
starting point). Keep in mind that the smaller the bin sizes, the longer it will take RF Optimizer EVDO
to process the data, and the larger the resulting databases will be.

In the "Processing Options" tab, it is strongly recommended to keep the "Delay..." option checked.
This will delay the Data Synchronization analysis for any log file, which can be very time consuming
due to the amount of data processed, until the first time you have a need to open the DRC Information
Viewer. Keeping this option checked will significantly save upfront processing time, compared to not
checking this option.
You are ready to process data.
Please keep in mind that by Microsoft's design, each of your database (MSDE) is limited to 2GB. Due
to the varying nature of raw data, it is not possible to predict how much database space each raw file

Nortel Networks Confidential and Proprietary

42

will take up after it is processed, so exercise caution and avoid processing too many files into one
single database. You are free to create as many databases as your hard drive can hold.

5.4.2

XCAP-DO

XCAP DO software also needs a dongle key to run. Once launched, click on Tools Options to configure
the tool. There are six tabs in the Options window, and most of the default values are good for typical use.
General: contain settings for distance unit, message viewer option, sync option, etc.
Path: select the directories for the log data (raw data), model (processed data), and exported data.

Map: contains settings for map display.


Map2: contains additional settings for map display (like shift offset).
Graph: contains settings for graph display.
CDF/PDF: contains options for CDF and PDF graph type, i.e. line or bar and 2D or 3D.

To process a log file (XCAP Do calls it creating a new model), click on File New:

Nortel Networks Confidential and Proprietary

43

Sampling Interval: As in RFO, it is typically set to 1000 ms.


Merge/Separate: If you select more than one file, and want to create one model for them (e.g. to get an
aggregate statistics from a cluster), then choose Merge. If you want to create a model for each file
separately, then choose Separate.
Model Name: If you merge several files, you can choose a name for the merged model. If you choose
Separate then the model name is the same as the individual file name.

Selective Parsing: If you want to process only certain messages, you can select them from here. For
example, if you are measuring RF coverage and not concerned about throughput, then you can uncheck
PPP Data and Throughput Info box.

Nortel Networks Confidential and Proprietary

44

5.5 DOM - RNC Log Post Processing Tools


Currently, RNC logs can be analyzed one of two ways; converting the logs to text and opening the resulting
logs with Excel, or to use NEWTUN to post process and analyze the logs in binary format.

5.5.1

Converting RNC Logs from Binary to Text

The binary to text file parser resides in the RNC SC and DOM card. To parse a DOM or RNC log, you
need to telnet into the DOM or the RNC. The following steps should be followed once logged into the
DOM or RNC to convert the binary data to text:
Nortel-07> en
Nortel-07# shell
Nortel-07#(shell)(disk0:/)# cd logs
Copy the filename of the log
Nortel-07(config)# logging convert file <bin filename> <new text file name>
After converting the file ftp it down to a PC for analysis.

5.5.2

NEWTUN (Post Processing and Analysis of RNC Logs)

NEWTUN (Nortel Engineering Wireless Tuning Tool) is a Windows based tool developed by Nortels
Wireless Tools group for processing and analyzing EVDO call statistics according to events captured in
RNC logs. NEWTUN is currently only a Beta release and is only available to select groups. To obtain a
license key please contact Tag Support. The processing of RNC logs with NEWTUN requires the input
files from the RNC to be in the raw *.bin or *.gz format (format prior to text conversion). During post
processing of the logs NEWTUN will convert the logs to a viewable format. NEWTUN will not read RNC
logs that are in *.txt format.
NEWTUN supports the following functionality in the current release:
Call Performance statistics such as dropped connection and access failure rates,
Call flow analysis in various states
Call radius configuration tuner
In the future it will support Neighborlist tuning and integration with EVDO editor output files

NEWTUN Setup Guidelines


When first starting NEWTUN, the user should ensure that the tool settings are configured as required.

File Locations

Nortel Networks Confidential and Proprietary

45

It is recommended that the User save the Database files in a location where there is a lot of free
disk space available when processing large amounts of data.
NEWTUN creates a number of temporary files during processing that may be large depending
upon the amount of data processed. These temporary files are cleaned up upon completion of
processing activities.
To save browsing time, the user can setup default location of where the program should search for
raw binary logs.

Process Filtering

Nortel Networks Confidential and Proprietary

46

Filter Messages

When checked the tool will only process Call Control messages
Certain useless CC messages will also be excluded

Skip Files

Currently does nothing but will allow automatic removal of files with Severity levels lower than
required by call model analyzer.

Max Severity

When calculating fail / drop statistics it is important that the log files are collected at the
appropriate severity level. Should the level be too low the tool will warn the user accordingly.

Database Creation
Raw data must be processed and stored into a database for post-analysis. To create a new database:
1.

Open DataManager from Main Window Toolbar

2.

Click New Database Tool button on Data Manager

Nortel Networks Confidential and Proprietary

47

3.

Enter Name of new database

File Processing
NEWTUN processes raw binary Airvana RNC log files. For space reasons, the tool will accept gzipped
binary log files. To process Files
1.
2.
3.
4.
5.
6.

Create database for storing data


Highlight database to store processed data
Select files for processing
Add files to highlighted Database
Repeat steps 1 4 to batch up data as desired
Click Process Button

Nortel Networks Confidential and Proprietary

48

Once all files have been successfully processed, select the database of interest and click the Select
button.
By default, the statistics view will automatically be populated for post-analysis.
Notes
When selecting data, all files within a database are automatically selected
Files cannot be removed from a database.
Clicking the delete button will remove the highlighted database
Tool does NOT process RNC converted text files

Statistics Display
The statistics view is the main apparatus for displaying performance results and navigating throughout the
call messaging for debugging purposes.

Nortel Networks Confidential and Proprietary

49

The statistics displayed are based upon all the calls identified within the database selected from the data
manager.

Connection types
Two types of connections are defined within NEWTUN:
Session Connections assisting with the configuration of a session startup
Data Connections associated with the transfer of user data
How these connection types are determined is beyond the scope of this document. As mentioned in the
Known Issues section, break-up of connection types is not accurate and is undergoing improvements.

Call Classification Types


Name
Call Attempts
Failed Access
Dropped Calls
Border Dropped
Calls
Good Calls
Incomplete Access
Incomplete Traffic
Unknown Failures

Description
Total number of Connection Attempts
Connections that failed to acquire traffic channel
Connections that dropped once on traffic channel
Dropped calls that may be close to a border RNC region
Calls that closed normally
Calls that had no more messaging to indicate outcome of Access Attempt
Call that had no more messaging to indicate whether call would terminate
successfully or not
Calls that failed but had insufficient information to determine whether traffic was

Nortel Networks Confidential and Proprietary

50

Unknown
Access Failure Rate
(%)
Drop Call Rate (%)

acquired or not
Calls that were found (End of calls) but no information to classify was available

100 *

Failed Access
Attempts Access Incomplete Unknown Failures Unknown

100 *

Drops + Border Drops


Good + Drops + Border Drops

Determination of how the calls are classified is beyond the scope of this document.

Notes

All the classification types mentioned above apply to both Session and Data Connection types.
Where possible, call drops / failures are pegged against a DOM / PN.
If no DOM / PN information is available the call will be pegged under Unknown DOM entry.

Debugging Calls
To review the cause of a drop or failure the user must Double-Click on the calls of interest within the
Statistics View

Tip
Double-click a cell containing connections of interest to investigate will bring up message view

Nortel Networks Confidential and Proprietary

51

Message View
The message view will display all decoded messages for the UATI relating to the call selected by the user.

The green color coded messages highlight the connection of interest (Not exact but provides an
approximation)

Note

The Route Update Detail message has been re-formatted to accommodate later NEWTUN tuning
module development.
This message resolves all the rows of a Route Update providing one line per PN

Nortel Networks Confidential and Proprietary

52

Active List
When the user selects calls to analyze from the Statistics view, this pane will show the list of calls for
examination.

Double-clicking on any of the calls will update the message view with the messaging relating to that
connection.

Note
It is recommended to not select a Statistics View Cell with too many calls (100 or more) to
minimize time delays during subsequent analysis.

Nortel Networks Confidential and Proprietary

53

Peg Selection
To choose which metrics to display within the statistics, click the edit tool button.

Select the metrics to show by checking / un-checking as required.

Saving Results
Later versions of NEWTUN will generate reports of results. Until that time users can save results to Excel
for custom report generation.

5.6

OM Data Collection Template Configuration Setup

The Nortel 1xEVDO system is very flexible for collecting operational measurements. The system includes
many pre configured data collection templates that allow users to collection measurements with various
degrees of granularity. In addition the system allows users to build customized data collection templates.
Creating a data collection configuration allows the user to tailor the metrics to meet specific needs. The
following instructions explain how to create a data collection configuration on the DO-EMS.

Log into the EMS client as a user with PerformanceAdminGroup privileges.


Open the statistics administration window

Nortel Networks Confidential and Proprietary

54

Click Add Data Collection Configuration from the menu. The DO-EMS will open the Data
Collection Configuration Form

Configure the data collection parameters as follows:


o Configuration Name Name of the data collection configuration
o Start Time - Used for OM synchronization. The Start Time parameter specifies the
starting time of the first OM sampling, in date-time format: mm/dd/yyyy HH:MM. If the
Start Time has already passed when the DC Configuration is pushed to the NE, then the
first sampling will be taken at the next sampling time (using Start Time as the reference).
o End Time - Used for OM synchronization.The End Time parameter specifies the ending
time of the first OM sampling, in date-time format: mm/dd/yyyy HH:MM. The end time
validation is at the hour granularity.

Nortel Networks Confidential and Proprietary

55

Sample Rate - The period at which the network elements in the configuration read and
store the performance statistics. Select a sampling rate within the range 1 - 3600 seconds.
It is not recommended to select rates below 900 s because it may degrade network
performance, result in data collection errors, or impact OM synch time.
o Push Interval - The collected data is pushed from the NE to EMS at this time frequency.
Select a push interval within the range 60 - 7200 seconds. It is not recommended to use
values less than 1800s because it may result in data collection errors.
o Template - Select a data collection template from the list. The template determines which
object IDs are sampled at the specified sample rate. Click the link to browse all existing
templates.
o Instance IDs to collect data on a specific instance list the instance ID here. If it is
desired to collect data on all instances leave this field blank. For table based data
collection templates it is not recommended to collect data on all instances as this may
degrade NE performance.
Select the node group to collect date from and submit the new template.
Data collection will start and end at the times specified.
Data collection results can be viewed on a per node basis in DO-EMS by selecting Data Collection
Files from the statistics view.
Alternatively, engineers can FTP the results down from the EMS using an FTP client.
o

5.7

CELLDM Data Collection

CELLDM or Cellular Diagnostics and monitoring logs provide a means to retrieve airlink performance
information for each sector from the baseband module. CELLDM has the following components:

CELLDM SC runs on the BIO/SC


CELLDM FL runs on the FLM
CELLDM RL runs on the RLM

The airlink performance statistics comprise per sector and per user statistics. The per sector statistics are
part of the OMs but the per users stats can be determined from CELLDM logging
CELLDM provides the user two interfaces, NE CLI and GUI DO-EMS to access the airlink performance
statistics and perform diagnostic logging of the baseband module. There are three different configuration
options for CELLDM. These options are given in the table below.

Nortel Networks Confidential and Proprietary

56

CELLDM Configuration Options

Statistics managed by CELLDM are sector based, user based, and broadcast channel based. These statistics
are always collected by different components. The sector and user based statistics are of two types: traffic
statistics and signal quality statistics. Traffic statistics can be divided in to cumulative statistics and
throughput statistics.
Cumulative statistics
These statistics are the sum total of various traffic-related variables associated with a sector or connection.
These statistics are monotonically increasing as a function of time. Examples are:
The total number of bytes transmitted for a connection.
The total number of bytes received from the AT for a connection.
The number of slots for which a specific DRC value was requested by the AT.
Throughput statistics
These statistics are the time-average of the cumulative statistics. Examples are:
Forward link sector throughput (the rate at which bytes are transmitted out of a specific sector).
Average requested DRC (the sum total of all the DRC values requested by an AT averaged over time).
Collection of statistics
An operator can choose to collect any of the statistics using the method of data collection. Data collection is
a feature of the software release and provides the ability to sample any MIB OID at a configurable rate. The
CELLDM component also provides a mechanism that allows the operator to sample the statistics associated
with a sector, connection or broadcast channel at a configurable interval and log these statistics to disk.
Sector-based traffic statistics
Sector-based signal-quality statistics
User-based traffic statistics
Broadcast channel statistics
User-based signal-quality statistics
CELLDM Loggable Attributes
The following operator provisionable attributes can be configured through the DO-EMS GUI or CLI. All
attributes are configured on a per modem card basis
Nortel Networks Confidential and Proprietary

57

sectorStatsLoggingStatus
sectorThruputStatsLogMask
sectorOverheadStatsLogMask

Whether sector statistics are logged periodically to disk.


This mask should be set to log the sector throughput statistics to disk.
This mask should be set to log the overhead (access and control)
channel statistics to disk.
sectorBroadcastStatsLogMask This mask should be set to log the broadcast channel statistics to disk.
sectorTCStatsLogMask
This mask should be set to log the sector-based traffic-channel stats to
disk.
sectorDrcChannelStatsLogMask This mask should be set to log the sector-based DRC channel stats to
disk.
sectorRxQualityStatsLogMask This mask should be set to log the sector-based signal quality stats to
disk.
connectionStatsLoggingStatus Whether sector statistics are logged periodically to disk.
connectionThruputStatsLogMask This mask has to be set to enable the connection throughput statistics
to be logged to disk.
connectionDrcStatsLogMask
This mask has to be set to enable the connections DRC statistics to be
logged to disk.
connectionTrafficStatsLogMask This mask has to be set to enable the traffic channel packet count
statistics to be logged to disk.
connectionFlowStatsLogMask This mask has to be set to enable the connections flow statistics to be
logged to disk.
connectionMiscStatsLogMask This mask has to be set to enable some miscellaneous ( ACK and PRC
channel) connection statistics to be logged to disk.
broadcastStatsLoggingStatus
Whether broadcast statistics are logged periodically to disk.
broadcastGeneralStatsLogMask This mask has to be set to enable the general packet-count stats
associated with the broadcast channel to be logged to disk.
statsSamplingInterval
The period (in milliseconds) of how often the sector, connection and
broadcast statistics are logged to disk.
throughputSamplingInterval
The period (in milliseconds) of the duration of which instantaneous
throughputs are calculated.
ftcSchedulerLogStatus
Forward Traffic Channel Scheduler Logging status for this modem.
cchSchedulerLogStatus
Control Channel Scheduler Logging status for this modem.
broadcastSchedulerLogStatus Broadcast Scheduler Logging status for this modem.
AchLogStatus
Access Channel Log status for this modem
AchFingerLogMask
Access Channel Finger Logging enable status for this modem.
AchSearcherLogMask
Access Channel Searcher Logging enable status for this modem.
RtcLogStatus
Reverse traffic channel logging enable status for this modem
Reverse traffic channel finger logging enable status for this modem.
RtcFingerLogMask
RtcSearcherLogMask
Reverse traffic channel searcher logging enable status for this modem.
RtcHandoffLogMask
Reverse traffic channel handoff logging enable status for this modem.
RtcDopplerLogMask
Reverse traffic channel doppler logging enable status for this modem.
RtcDrcLogMask
Reverse traffic DRC channel logging enable status for this modem
RtcAckLogMask
Reverse traffic ACK channel logging enable status for this modem
RtcRriLogMask
Reverse traffic RRI channel logging enable status for this modem
RtcRpcLogMask
Inner loop power control logging enable status for this modem
csm5500DiagMask<Component> The Mask for controlling the diagnostics information of the specified
component for CSM5500-based baseband module units. A value of 1
means enable, 0 means disable.
csm5500VerboseMask<Component> Mask for controlling the verbosity of the diagnostics information
of the specified component for CSM5500-based baseband module
units. A value of 0 means terse, a value of 1 means verbose.
CELLDM Logging Setup
1.
2.
3.

From the DO-EMS GUI, expand the Network Database DOM folders.
Select the DOM that where CELLDM logs are to be collected.
From the DOM Menu, select Config CELLDM.

Nortel Networks Confidential and Proprietary

58

4.

There are three configurations for CELLDM logging available: CSM550CELLDMDiags,


GlobalCELLDMLogging, and ModemCELLDMLogging.

CSM5500 CELLDM Diagnostics Configuration


Nortel Networks Confidential and Proprietary

59

CELLDM Global Logging Configuration

CELLDM Modem Logging Configuration


Nortel Networks Confidential and Proprietary

60

5.

After the logging attributes have been configured select Submit to begin logging.

Accessing Statistics
In release 3.0, there is a new (Connections) pull down menu for accessing per user CELLDM statistics data
6.
7.

From the DOM Menu select Show Connections to access the per user statistics.
The available user Based statistics are as follows:
FL Rate Statistics Information
RL Rate Statistics Information
Connection Sector Statistics Information
Flow Statistics Information
Flow Queue Statistics Information
Rx Signal Quality Information

8.
9.

In order to access per sector Statistics, from the DOM Menu select ConfigSectorElements.
Once on the Sector Elements page, go to the IS856 Sector Element Menu and select SectorPerfStats

10.

The following Sector Performance stats can be viewed from this page.
Access Channel Statistics Information
Broadcast Sector Statistics Information
Control Channel Statistics Information
FTC Information
RTC Information
Rx Signal Quality Information
General Sector Statistics Information

For More detailed information on CELLDM logging see reference [19].

5.8

OM Analysis Tools

The core RF engineering team has developed OMAX (OM Analysis for 1xEVDO and 1xRTT) for
retrieving and calculating OM performance metrics for 1xEVDO and 1xRTT. The process of creating the
OM statistics comprises; retrieving data from EMS, reading data into MySQL Control Center, and
extracting performance data via SQL Queries.
RAW OM data from EMS is organized into standard data collection templates (dc files). The data collected
in the templates should be transferred to a local server where they can be FTP ed to a local PC with
MySQL control center installed.
Perl scripts are used on the PC analysis machine to parse the RAW OM data into a database (mySQL)
MySQL can be used to run default performance queries or to build specialized queries for custom
performance reports.
The following is needed to install and configure MySQL:
Files Needed
o mysql-4.0.18-win.zipsly
o mysql-4.1.1a-alpha-win.zip
o mysqlcc-0.9.4-win32.zip
The files can be found at the following locations:
o http://dev.mysql.com/downloads/
o \\mikej-2\MySQLDownloads\rirs
Unzip mysql-4.0.18-win.zip in temporary directory:
o Run the Setup.exe
o Follow setup instructions
o Remove temporary directory contents when installation is complete
Nortel Networks Confidential and Proprietary

61

Unzip mysql-4.1.1a-alpha-win.zip in temporary directory:


o Copy the contents of the mysql-4.1.1a-alpha directory (including sub-directories) into
the installed mysql directory (i.e. c:\mysql) from previous step.
o Remove temporary directory contents when complete
Rarins un winmysqladmin.exe located in the mysql\bin directory
o If prompted for user id and password, select cancel
o Should see a stop light (with green light) in the task bar
o Server should be running in background and should automatically start after reboots.
Unzip mysqlcc-0.9.4-win32.zip in temporary directory
o Run Setup.exe
o Follow setup instructions
o Remove temporary directory contents when complete
Obtain the following tar files:
o mysql-4.1.1-alpha.tar
o DBI-1.42.tarnalin
DBD-mysql-2.9003.tara
o \\mikej-2\MySQLDownloads\
To compile MySQL client for Cygwin:
o unpack the mysql-4.1.1-alpha.tar file
o Dont use windows to unpack the file; use tar -xvf
o cd into the unpacked dir (i.e. cd mysql-4.1.1-alpha)
>./configure --prefix=/usr/local/mysql --without-server
This prepares the Makefile with the installed Cygwin features. It takes some time, but
should finish without error. The prefix, as given, installs the whole Cygwin/MySQL
thingy into a location not normally in your PATH, so that you continue to use already
installed Windows binaries. The --without-server parameter tells configure to only build
the clients.
>make
This builds all MySQL client parts be patient. It should finish finally without any
error.

>make install
o This installs the compiled client files under /usr/local/mysql.
o Remove unpacked dir
To compile DBI for Cygwin:
o unpack the DBI-1.42.tar file
>cd into the unpacked directory
>make
>make test

>make install
o Remove unpacked dir
To compile DBD-mysql for Cygwin:
o Unpack the DBD mysql-2.9003.tar file
o cd into the unpacked dir
>perl Makefile.PL -libs=-L/usr/local/mysql/lib/mysql lmysqlclient lz
-cflags=-I/usr/local/mysql/include/mysql -testhost=127.0.0.1
>make

Nortel Networks Confidential and Proprietary

62

>make test # Some minor error message can be ignored here


>make install

5.9

o Remove unpacked dir


Documentation for MySQL can be found under the installed mysql/Docs directory (i.e.
c:\mysql\Docs)

TCP Performance Logging Tools

Windump and Snoop logs are generated to analyze the TCP performance between client (data collection
computer receiver) and server (FTP server sender). Typically Windump would be installed on the data
collection laptop connected to the AT and the Snoop logs would be installed on the FTP server such that
the end to end TCP performance of the network can be analyzed during testing.

Windump
Windump is a Windows portable version of TCPdump that runs on top of a low level packet capture
utility called winpcap. Windump is freeware and can be downloaded from the following location:
http://windump.polito.it/install/default.htm

Snoop
Snoop is a Unix command that is used to capture network packets in a readable format Snoop captures
packets from the network and displays their contents. snoop uses both the network packet filter and
streams buffer modules to provide efficient capture of packets from the network. Captured packets can be
displayed as they are received, or saved to a file (which is RFC 1761- compliant) for later inspection.
Command Options:
-C List the code generated from the filter expression for either the kernel packet filter, or
snoop's own filter.
-D Display number of packets dropped during capture on the summary line.
-N Create an IP address-to-name file from a capture file. This must be set together with the -i
option that names a capture file. The address-to-name file has the same name as the capture file
with .names appended. This file records the IP address to hostname mapping at the capture site
and increases the portability of the capture file. Generate a .names file if the capture file is to be
analyzed elsewhere. Packets are not displayed when this flag is used.
-P Capture packets in non-promiscuous mode. Only broadcast, multicast, or packets
addressed to the host machine will be seen.
-S Display size of the entire ethernet frame in bytes on the summary line.
Procedure:
1.
2.
3.
4.

Telnet into the server.


Login as administrator
Go to the directory where you want to save the log.
Type >/usr/sbin/snoop o filename AT IP address <ENTER>
This will start the Snoop log. o here implies Output to

5.

To stop and save the log Ctrl C

Nortel Networks Confidential and Proprietary

63

TCPTrace
TCPTrace can take as input the files produced by several popular packet-capture programs, including
tcpdump, snoop, etherpeek, HP Net Metrix, and WinDump. tcptrace can produce several different types of
output containing information on each connection seen, such as elapsed time, bytes and segments sent and
recieved, retransmissions, round trip times, window advertisements, throughput, and more. It can also
produce a number of graphs for further analysis. Follow the link to download a copy of TCPTRACE:
http://www.tcptrace.org/download.html

ETHEREAL
Ethereal can read capture files from tcpdump (libpcap), NAI's Sniffer (compressed and uncompressed),
Sniffer Pro, NetXray, Sun snoop and atmsnoop, Shomiti/Finisar Surveyor, AIX's iptrace, Microsoft's
Network Monitor, Novell's LANalyzer, RADCOM's WAN/LAN Analyzer, HP-UX nettl, i4btrace from the
ISDN4BSD project, Cisco Secure IDS iplog, the pppd log (pppdump-format), the AG
Group's/WildPacket's EtherPeek/TokenPeek/AiroPeek, or Visual Networks' Visual UpTime. It can also
read traces made from Lucent/Ascend WAN routers and Toshiba ISDN routers, as well as the text output
from VMS's TCPIPtrace utility and the DBS Etherwatch utility for VMS. Any of these files can be
compressed with gzip and Ethereal will decompress them on the fly. Follow the link to download a copy of
Ethereal: http://www.ethereal.com/download.html

Nortel Networks Confidential and Proprietary

64

Shakedowns

The first step of the 1xEV-DO RF optimization once RF datafill, FTAP / RTAP, and TCP/IP parameter
verifications have all been performed is the shakedown of each BTS and RNC Border areas to verify call
processing functionality and RF parameters/datafill.

6.1 Objectives
The objectives of the shakedown are as follows:
To verify that we can get a connection on each sector.
To verify that we can switch to the adjacent sector in the forward link and softer-handoff to the
adjacent sector in the reverse link.
To verify that the following parameters have been datafilled correctly for each sector:
o PN offset
o Neighbor list
To verify that PER (Packet Error Rate) is 1%0.2%.
To verify that the forward frame rates match the requested DRC.
To Verify that Inter Subnet Active handoffs (ISSHO) are successful
To verify that A13 dormant handoffs are successful

6.2 EVDO Site Shakedown Process


The shakedown process is as follows:
Setup the laptop, the AT, GPS, and data collection tool (Optis-M or Invex3G) in the drive test van.
Drive to near (within 1 km) the cell site.
Park the van in a location that is covered only by the alpha sector.
Start the laptop, AT, Optis-M, or Invex3G. Setup the appropriate logmask.
Click on the Auto Call button on Optis-M or right click on the AT device in the workspace of Invex3G
and select Properties.
In Optis-M, click on the Create New Scenario button. Create an FTP Get scenario (please see the
example below). Then click OK.
In the device properties window of Invex3G, select the Data Connection tab and click on the Add
button. Select FTP Get from the Task Type drop down menu. Configure the FTP server IP address
in the Destination Address box as well as the path and filename, username and password in the
appropriate boxes.

Invex3G

Optis-M
Invex3G and Optis-M FTP Get call scenario for shakedowns.
Nortel Networks Confidential and Proprietary

65

In Optis-M select the call scenario you just created for the autocall. Select All scenario for the
AutoCall Logging Option.

Optis-M
Scenario Select

Start the log by clicking OK.


Invex3G
In Invex3G, all that is required to start logging after the Data Connection configuration is completed is
to go to the Connections menu select Connect All.

Invex3G Connect
All Devices

Optis-M - Verification
Check the OPTis-M EVDO AT Status window to make sure that:
o The AT state changes from idle to connected.
o The session state is open.
Invex3G - Verification
In Invex3G verify that:
o The Device color changes from Red to Green
o

The State or History window shows:


Connection State = Connected
Connection Type = 1xEV-DO
Session State = Open
Neighborlist of the sector is correct
RF Channel is correct
The Active set PN is correct for the sector under test.
Receive diversity is enabled

Nortel Networks Confidential and Proprietary

66

Invex3G State and History Windows

Optis-M - Verifications
Check the 1xEV-DO Pilot Set Graph window in Optis-M and verify that:
o Alpha sector is the serving sector PN.
o The neighbor list of alpha sector is correct.
o The channel number is correct.
Check the Network Status window in Optis-M and verify that:
o The AT receive diversity is on.

Optis-M
Network Settings

Nortel Networks Confidential and Proprietary

67

Examples of OPTis-M display to check during shakedown.

From the EVDO Rate Statistics window, verify that the reverse link is in variable rate.

Nortel Networks Confidential and Proprietary

68

Start driving towards beta sector. Monitor the Pilot Pn[ ] and verify the AT does the softer handoff
with alpha and beta sector (both PNs are in the active set). Ensure that sector switching to beta sector
occurs without a connection drop. Stop the van at a location that is covered only by beta sector. Verify
that now only beta sector PN is in the active set.
From the EVDO Rate Statistics window, verify that PER = Bad CRC/Total CRC = 1%0.2%.
Stop the log.
Repeat the steps above for:
o Connection origination at beta sector and softer handoff to gamma sector.
o Connection origination at gamma sector and softer handoff to alpha sector.
Fill out the shakedown form (an example is given in Appendix B).

6.3 EVDO Inter Subnet Active Handoff (ISSHO) Shakedown Process

Setup the GPS, AT, Laptop, and Optis-M or Invez3G inside the test van
Drive to the subnet boundary on the source side
Start the laptop, AT and the data collection tool. Select all the EVDO attributes for the log mask.
Optis-M
Click on the Auto Call button in Optis-M and then, click on the Create New Scenario button. Create
an FTP Get scenario (please see the example below). Then click OK.
Select the call scenario you just created for the autocall. Select All scenario for the AutoCall
Logging Option.
Start the log by clicking OK.
Invex3G
Right Click on the device in Invex3G and select properties followed by the data connection tab.
Select Add from the Data Connection tab and configure an FTP Get task then select OK.
Select the Connections menu object then Connect All. Logging will start.
Optis-M
Check the OPTis-M EVDO AT Status window to make sure that:
o The AT state changes from idle to connected.
o The session state is open.
Check the 1xEV-DO Pilot Set Graph window and verify that:
o The serving PN is on the source side of the subnet boundary
Begin driving across the subnet boundary toward a site on the target subnet
o Verify that a PN from a site on the target subnet is added to the Active set by looking at the
Best ASP SINR, EVDO Finger Information window, and EVDO Airlink Summary window.
EVDO Finger Information Optis-M

Nortel Networks Confidential and Proprietary

69

EVDO AT Status

Invex3G
Verify from the Invex3G state window that:
o Connected State = Open
o Session State = Open
Open a bar Chart or History window and verify that:
o Active set pilot PN is from a DOM on the source RNC
Begin driving across the subnet or RNC border and verify:
o Active set has a PN from the target subnet or RNC is visible in the bar chart or History
window.

Nortel Networks Confidential and Proprietary

70

Invex3G

Repeat the process coming from the target subnet side.


Insure that the call does not drop as the subnet boundary is crossed.

6.4 (A13) Dormant Session Handoff Shakedown process

After completing the Inter RNC active handoff (IRAHO) test stop AT logs and proceed to one side of
the subnet / RNC boundary.
RNC logs will need to be collected to verify dormant session handoffs
Process to setup logs:
o RNC logs can be configured using CLI or DO-EMS GUI. The GUI method is given
below.
o Open the EMS GUI and from the EMS elements tree, select Network Databas DORNC
o Click on the RNC(s) involved in the A13 shakedown process
o From the RNC Menu, select Show Cards
o Select each RNSM and SC Card followed by RNSM or SC Card Menu Show Log
FacilityMgr
o Select LogFacility Modify

Nortel Networks Confidential and Proprietary

71

Insure that :
Logging is enabled for the Call Control Component
Logging Enabled = True
Maximum Severity = 25
Output to File = True
Maximum Severity on File = 25
Select Submit
Check logging status
o When Logging is complete ftp the log files in bin or GZ format down to a PC for
analysis.
NORTEL -07>shell
NORTEL-07(shell)(disk0)>cd logs
NORTEL-07(shell)(disk0)>ls
Configure the data collection tool to have the AT perform an FTP GET which will insure an Active
session. After one download allow the AT to transition to a dormant state (i.e. No active packet data
session).
Before crossing the border, verify from the EVDO AT Status window that the AT State goes from
Idle Accessing Connected Idle and the Session State is Open.
Start driving towards the subnet border and verify that the Session stays Open but the Subnet ID
changes
o

Optis-M EVDO AT Status

Nortel Networks Confidential and Proprietary

72

Invex3G State

Verify that UATI, Subnet Mask, and Color Code change when the subnet border is crossed and the
session stays open as the subnet border is crossed.
o Note: There are two ways in which the AT can trigger an A13 dormant session handoff:
1. UATI Request using its present UATI - occurs when the AT has continuous
reception of the network broadcast messages as it moves from one subnet to the
other.
2. UATI Request using RATI (also referred to as Prior Session-initiated dormant
handoff), this typically occurs when the AT loses CCH supervision as it crosses
from one subnet to the other (e.g. AT powers down in one subnet and powers up in
another).
Repeat the test coming from the opposite side of the subnet boundary.

6.5 Data Processing and Analysis


After shakedowns are complete the collected data should be analyzed to insure that call processing
functionality and configured data fill parameters are accurate.

6.5.1

EVDO Site Shakedown Analysis

After the shakedowns have been completed, process the shakedown files with RF Optimizer. Do the
following:
Verify that there is only one connection (i.e., there is no connection drop) by looking at the Statistics
Tool Calls Statistics.

Nortel Networks Confidential and Proprietary

73

Example of shakedown call statistics showing access success and no connection drop.

Verify that softer handoff occurs. Look at the PN1 and PN2 columns in Grid View and check that the
PNs change from alpha sector PN to alpha and beta sector PNs to beta sector PN only (similarly for
beta to gamma and gamma to alpha).
Verify that average PER = 1%0.2%. Right click on red bar below the PER column in Grid View and
click on Average.
Verify that the Rx Power is appropriate for the shakedown locations.
See the illustration below for an example of softer handoff occurrence, average PER, and Rx Power
display.

Example shakedown file analysis showing softer handoff and average PER.

The following RF parameters can be verified from the messages:


Message & Parameters

Definition

Quick Configuration or Sync


CHAN_NUM
Pilot_PN
COLOR_CODE

Channel Number for this carrier


Pilot PN offset for this sector
Unique Color Code value for the RNC

Access Parameters
ACC_CYCLE_DURATION
ACCESS_SIG
OPEN_LOOP_ADJ

Time instances at which AT may start an access probe


?
Nominal power used by AT in open loop pwr estimate

Nortel Networks Confidential and Proprietary

Recomm.
Value
-

32
-86

74

PROBE_INITIAL_ADJ
PROBE_NUM_STEP
POWER_STEP
PREAMBLE_LENGTH
CAPSULE_LENGTH_MAX
PERSIST_COUNT
Sector Parameters
COUNTRY_CODE
SUBNET_MASK
LATITUDE
LONGITUDE

Correction factor used by AT for the initial tx on Access Ch.


Max number of probes for each access probe sequence
Power step size applied to successive probes
Length of preamble in an access probe
Length of capsule in an access probes
A-Persistence vector for controlling contention on the access ch?

Mobile Country Code for this sector


Length of the subnet mask
Latitude of the sector, in thousandths of a degree
Longitude of the sector, in thousandths of a degree

ROUTE_UPDATE_RADIUS

Distance beyond which the AT must send Route Update Message

REV_LINK_SILENCE_DURATION

Duration of the silence interval to be maintained on the reverse link

REV_LINK_SILENCE_PERIOD
NEIGHBOR_COUNT
NEIGHBOR PNs
NEIGHBOR_SRCH_WIN_SIZE_INC
NEIGHBOR_SRCH_WIN_SIZE
NEIGHBOR_SRCH_WIN_OFFSET_INC
NEIGHBOR_SRCH_WIN_OFFSET

Pperiod of the silence interval


Number of neighbors in the NL of this sector
PN list of neighbors
"True" if there are neighbor PNs with different window size set
Search window size of a neighbor.
Whether specific search win. offset for neighbors is included in NL
Search window offset of a neighbor.

Traffic Channel Assignment


DRC_LENGTH
DRC_CHANNEL_GAIN
ACK_CHANNEL_GAIN
RAB_LENGTH
RAB_OFFSET

Number of consecutive slots in which the DRC value is repeated


Power of DRC Channel relative to Pilot
Power of Ack Channel relative to RL Pilot
Length of Reverse Activity Bits for this sector
For sectors in same cell should be >= (RABLength 8) slots if
possible

Pilot Sets v2
ASET_WINDOW
RSET_WINDOW
(Neighbor) WindowSize

Search window size for active set


Search window size for remaining set
Search window size for neighbor set

State Information
AT State
Session State
Idle State
Connected State
Route Update State

Tells if the AT is connected to the access network


Tells whether the EVDO session is open or closed
Tells if the AT is active or inactive in an idle state
Tells if an AT connection to the access network is open or closed
Tells if the mobile is connected to the traffic channel

HDR_Hybrid_Mode

Hybrid or EVDO (HDR) is active

Nortel Networks Confidential and Proprietary

0
8
6
2
4
0

104
0
(disabled)
0
(disabled)
0
(disabled)
0
0
-

4
58 (-6)
6
32
-

60 chips
100 chips
100 chips

On =
(hybrid)

75

Sector Parameters 42201501


BAND : 1
CHAN_NUM : 1125
Pilot_PN : 112
HSTR : 0
SEQ_VALID : 0
ACK_SEQ_VALID : 0
FRGMENTED : 0
RELIABLE : 0
SEQ_NO : 255
ACK_SEQ_NO : 255
InConfPro : 0
Type : 15
COUNTRY_CODE : 1
SECTOR_ID_1 : 0
SECTOR_ID_2 : 16777292
SUBNET_MASK : 104
SECTOR_SIG : 1
LATITUDE : 653025
LONGITUDE : 7297592
ROUTE_UPDATE_RADIUS : 0
LEAP_SECONDS : 13
LOCAL_TIME_OFFSET : 1808
REV_LINK_SILENCE_DURATION : 0
REV_LINK_SILENCE_PERIOD : 3
CHANNEL_COUNT : 1
[ 1 ]
SYSTEM_TYPE2 : 0
BAND_CLASS2 : 1
CHANNEL_NUMBER2 : 1125
NEIGHBOR_COUNT : 9
[ 1 ]
PILOT_PN : 104
[ 2 ]
PILOT_PN : 108
[ 3 ]
PILOT_PN : 116
[ 4 ]
PILOT_PN : 120
[ 5 ]
PILOT_PN : 124
[ 6 ]
PILOT_PN : 132
[ 7 ]
PILOT_PN : 148
[ 8 ]
PILOT_PN : 152
[ 9 ]
PILOT_PN : 156
[ 1 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 2 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 3 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 4 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 5 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 6 ]
CHANNEL_INC : 1

Nortel Networks Confidential and Proprietary

76

SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 7 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 8 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
[ 9 ]
CHANNEL_INC : 1
SYSTEM_TYPE : 0
BAND_CLASS : 1
CHANNEL_NUMBER : 1125
NEIGHBOR_SRCH_WIN_SIZE_INC : 0
[ 1 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 2 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 3 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 4 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 5 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 6 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 7 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 8 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
[ 9 ]
NEIGHBOR_SRCH_WIN_SIZE : Not Found
NEIGHBOR_SRCH_WIN_OFFSET_INC : 0
[ 1 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 2 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 3 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 4 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 5 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 6 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 7 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 8 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found
[ 9 ]
NEIGHBOR_SRCH_WIN_OFFSET : Not Found

6.5.2

Inter RNC Active Handoff Shakedown Analysis

After completing the Inter subnet handoff runs an analysis of the collected logs should be performed to
determine rate of success.

Post process collected logs with EVDO RFO

Verify from the statistics view there was only one connection for each run.

If there were any dropped connections, a determination of the cause should be made.

Open the Grid view and verify that the AT was in soft handoff with PN(1) from sector with the source
RNC as a primary and PN(2) from sector with the source RNC as a secondary.

Nortel Networks Confidential and Proprietary

77

PN 116 is Primary
and PN 164 was
homed as
secondary

Figure Example of Grid View showing ISSHO

6.5.3

A13 Dormant Session Handoff Analysis

Process RNC logs from A13 session handoff shakedowns with NEWTUN.
Verify from the logs that a UATI Request initiated handoff triggers a A13 Request and finally that an
A13 Confirm is sent indicating that the session is successfully handed off from the source suibnet to
the target subnet. (See Figure Below).

Example: RNC Logs showing A13 handoff (Foreign UATI)

Nortel Networks Confidential and Proprietary

78

Golden Value (Stationary) Testing

7.1 Introduction
The objective of the Golden Value Testing (i.e. stationary throughput testing) is to measure the maximum
user application throughput in the forward link using the network components (PDSN, DO-RNC, DOM,
FTP server, AT, and laptop) that will be used in the cluster optimization stage. During this exercise, the
TCP and other settings can be optimized to improve the application performance. Once those settings are
optimized and finalized, the measured user application throughput becomes the golden (or baseline)
value, i.e., the upper bound of what we can achieve in the network.
This measurement should be done in a stationary location with a good RF link (close to the cell site, SINR
6 dB or Ec/Io -1 dB) and covered by a single sector. The user application throughput is measured with
FTP test using a single AT. Since we want to measure the maximum user application throughput, we need
to ensure that the network is unloaded (no other users in the network).
For the stationary golden value testing it is recommended to perform 10 downloads of a 5MB test file. The
test file should be a zip file or MP3 to insure that any compression algorithms do not skew the results..

7.2 FTP Server Configuration


To maintain a controlled test environment, it is recommended to connect the FTP server to the same subnet
as the PDSN. It is desirable to avoid having the test data traffic going through routers, firewalls, etc., to
avoid unpredicted problems like delay that will degrade the performance.

7.3 Golden Value Testing Process


The Golden Value Testing process is as follows:
Setup the laptop, the AT and the data collection tool (Optis-M or Invex3G) in a stationary location
with a good RF link.
Start the laptop, AT and the data collection tool. Setup the appropriate logmask.
Optis-M FTP Setup
Click on the Auto Call button, then click on the Create New Scenario button. Create a FTP Get
scenario similar to the one used for shakedown (see section 4.2).
Select the call scenario you just created for the autocall. Select All scenario for the AutoCall
Logging Option.
Start the log.
Invex3G FTP Setup
If an FTP GET scenario was already configured for shakedown process just select Connections and
then Connect All to start the logs
Connection and FTP State Verification
Check the OPTis-M EVDO AT Status or the Invex3G state window to make sure that:
o The AT state changes from idle to connected.
o Verify that in Optis-M that the Current State in the CDMA Statistics window = FTP DN
Start
o Verify in Invex3G state window that the FTP Get task state = Running
o The session state is open.
Check the Optis-M Network Status window or Invex3G State window and verify that:
o The AT receive diversity is on / Enabled.
Optis-M Throughput and C/I Verification
Check the EVDO Signal Graph display in Optis-M and verify that:
o Best ASP SINR is good (mostly above 6 dB).
o DRC Requested are mostly 1.2288 Mbps or higher.
Nortel Networks Confidential and Proprietary

79

o Slot utilization is high (in 80% - 90% range) due to single user and full-queue condition.
Check the reverse traffic statistics in EVDO Rate Statistics window and verify that the average reverse
data rate is sufficient to support the forward link data rate [2].
Verify that the Application Throughput Rx is consistent with the data rate distribution shown in the
Forward Traffic Statistics in EVDO Rate Statistics window, and also with the average DRC and Slot
Utilization. Since the SINR in this location is good, RLP retransmission should be minimal, and an
Application Throughput Rx should not be much less than 80% of the average physical layer
throughput (the overhead being about 20%; please see [11]). Lower throughput may indicate TCP
problem. See Section 4.4 (TCP/IP Performance)

Optis-M EVDO Signal Graph display.


Invex3G Throughput and C/I Verification
Check the Invex3G State window and verify the following:
o Composite C/I is > 6dB
o DRC requests DRC Value are >= 1.2288Mbps
o Task RX Throughput (kbps) (Cumulative) is consistent with the DRC request Rate
o Packet Error rate <1%
o Reverse link throughput is sufficient to sustain forward link (34.9kbps RL for 2.4Mbps FL).
o Slot Utilization is between 80 and 90%
o RPC Channels in use (Users connected to sector) = 1
Nortel Networks Confidential and Proprietary

80

Invex3G State Window

Stop the logs.


This test should be repeated five to ten times (or as much as time and resources permit) to ensure that
the result is statistically valid.

Nortel Networks Confidential and Proprietary

81

7.4 Data Processing and Analysis


After the Golden Value Testing is done, process the files with RF Optimizer. Do the following
Open the Graph View. Display the DRC Requested and Traffic C2I. Verify that DRC Requested
follows the trend of C2I1.
Also display App Rx Rate, Actual Fwd Data Rate, and Fwd Serving Rate in another window. Verify
that Fwd Serving Rate follows DRC Requested. This proves that the network follows the DRC
requests from the AT.

Example of the golden value testing result.


(blue = 1st parameter, green = 2nd parameter, red = 3rd parameter)

Click on the Statistics Tool Data Statistics Application. Calculate:


Average user application throughput = Rx Data (Bytes) x 8 / Elapsed Time
In the example below:
Average user application throughput = 15,000,000 x 8 / 62.683 = 1.91 Mbps

Nortel Networks Confidential and Proprietary

82

Verify that the average user application throughput obtained in this test is appropriate for the SINR
(C2I) condition. The following plot can be used as a guide. Please refer to [1] to more information on
the throughput performance number.
2400

Throughput (kb/s

2000

0.826 km/h - 1 Ant.

0.826 km/h - 2 Ant.

3 km/h - 1 Ant.

3 km/h - 2 Ant.

120 km/h - 1 Ant.

120 km/h - 2 Ant.

1600

1200

800

400

0
-8

-6

-4

-2

10

12

14

Median Ec/Nt (dB)

Single user throughput vs. Ec/Nt.

Nortel Networks Confidential and Proprietary

83

OM Collection and Analysis

This chapter discusses methods for collecting and analyzing operational metrics for purposes of
performance benchmarking. 1xEVDO system operation measurements have the following granularity:

Per RNC
Per Slot (slot corresponds to the physical RNSM slot in the RNC chassis)
Per DOM (RN)
Per Sector Carrier
Per Connection
Per Interface (physical or logical)
Per Channel

Operational measurements should be collected during the baseline drive test as well as during subsequent
optimization cluster drive rounds of the network to get a system level view of the performance attributes
given below.

8.1 Connection Attempts


There are a few varieties of connection setup attempts. These attempts can occur during various stages of
connection and are denoted as Total Served Connections. The following events will result in a successful
connection attempt. These OMs are per RNC level metrics.

DO-RNC has a connection open occurs when a connection request is received from the AT while a
current connection is already open. In this case the Connection request is cached until the current
connection can be closed. The following OMs are pegged in the case:
numConnectionReqsWhileOpen and numConnectionRequestsfromAT
DO RNC is in the process of tearing down a connection in this case the connection request is cached
and the previous connection is allowed to close normally before the new request is processed. The
following OMs are pegged in this case: numConnectionReqsWhiletearingdown and
numConnectionRequestsfrom AT
DO RNC is in the process of setting up a connection a connection request is received by the RNC
while the network is waiting for a TCC from the AT. In this case the previous TCA is retransmitted to
the AT. The following OMs are pegged in this event: numConnectionReqsWhileSettingUp and
numConnectionRequestsfrom AT

Total Connection Setup Performance OMs can be found in section 7-4 of [17]

8.2 Failed Call Attempts (FCA / Access Failure)


Failed connection attempts are defined as events where the AT does not successfully arrive on the traffic
channel. Metrics are defined to measure the rate of failed call attempts both before and after a demarcation
point. Performance measured before the demarcation point is defined as; connections attempted after
session configuration has taken place but the AT has no data to send, therefore A10 is not opened (no data
to send). Performance after the demarcation point is defined as: connection setups attempted after session
configuration is complete and the A10 connection is opened.
Failed Call Attempts OMs are given in section 7-5 of [17].

8.3 Dropped Connections (Abnormal Connection Closes)


A connection drop refers to an abnormal connection close. These abnormal connection closes can occur as
a result of RF link loss, Reverse Link Handoff Failure, or a variety of other reasons.
Connection Drop or Abnormal Connection Close metrics are given in section 8-4 of [17].

Nortel Networks Confidential and Proprietary

84

8.4 Session Setup Performance (includes A13 session handoffs)


Session setup is the first stage of communication between the AT and the AN, and is usually initiated
by a UATIRequest message from the AT. The EVDO session setup usually involves the following
stages:

UATI Request
UATI Assignment
A13 dormant session Handoff
Session configuration
Terminal authentication
Hardware ID retrieval
AT ID validation

Total 1xEV-DO Session Setup Success Rate OMs are defined in section 2-8 of [17]

8.4.1

UATI Request Initiated A13 Dormant Handoff Performance

Regular A13 handoff is triggered when an AT crosses the subnet border without losing supervision of the
DO carrier.
UATIRequest Initiated A13 Dormant Handoff OMs are defined in section 2-11 in [17]

8.4.2

Prior Session Initiated Dormant Handoff Performance

A prior session initiated dormant handoff occurs when the AT, during the session configuration phase
requests to retrieve a prior session from the RNC or when the AT looses supervision of the source RNC and
regains it on the target RNC. In this situation the target RNC will attempt to retrieve the prior session
information from the source RNC over the A13 interface. For session transfer to proceed, A13 dormant
handoff must be enabled on the target RNC.
Prior Session Initiated Dormant Handoff OMs can be found in section 2-13 of [17].

8.4.3

A10 Connection Setup Performance

In the previous releases, an A10 connection was setup every time the AT completed a location notification
or after every session configuration regardless of the need to send data or not. In release 3.0 we now have a
UA10 minimization feature that prevents the setting up needless A10 connections. The minimization
feature looks for the following events to trigger the opening of an A10 connection. Otherwise A10 remains
closed.
ULN Mode
New Session
Inter RNC Dormant Handoff (Regular)
Inter RNC Dormant Handoff (Prior Session)
Inter Tehnology handoff intra RNC
Inter Tehnology handoff inter RNC

Open A10 if
ULN received
Non-0 PDSN IP in A13 Response OR ULN Received
ULN Received
ULN Received
ULN Received

MIP Mode
New Session
Inter RNC Dormant Handoff (Regular)
Inter RNC Dormant Handoff (Prior Session)
Inter Tehnology handoff intra RNC
Inter Tehnology handoff inter RNC

Open A10 if
First RLP Packet received from AT (a.k.a PPP resync)
Non-0 PDSN IP in A13 Response OR PPP Resync
PPP resync
PPP resync
PPP resync

Refer to section 6-1 of [18] for A10 Connection Setup Performance OMs
Nortel Networks Confidential and Proprietary

85

8.4.4

Session Configuration Failures

There are two phase of session configuration; the AT initiated portion where the AT initiates negotiation of
protocol attributes, and the AN initiated portion where the AN initiates negotiation of protocol attributes.
The session configuration process is not complete until both the AT and the AN have committed to the
recommended protocol attributes. Session configuration is unsuccessful if: the protocol negotiation fails
between the AT and the AN, the session configuration timer expires (60secs), or the DO Airlink connection
drops during the process.
Session Configuration Failure metrics can be found in section 5.1 of [17].

8.5 Serving Sector Switching and Soft/Softer Handoff Performance


In 1xEV-DO there is no soft handoff on the forward link. The AT indicates its desire to switch sectors to the AN
by sending a null DRC value and thus the AN handles the sector switching. On the reverse link there is soft and
softer handoff just like in 1xRTT.
Refer to section 11-1 of [17] for Serving Sector Switching and Soft/Softer Handoff Performance OMs.

8.6 Inter RNC Active Handoff Traffic Statistics and Secondary Border Measurement
Prior to release 3.0 there was no mechanism for preventing call drops at subnet (RNC) boundaries. In prior
releases, when the AT would send a route update containing a Pilot from another subnet, the Pilot Lookup
would fail (RNC has no knowledge of pilots in other subnets) and the connection could drop as a
consequence.
With the introduction of release 3.0, the Inter RNC Active Handoff feature will be added. This feature will
allow DOMs from adjacent RNC to be secondarily homed to the anchor or source RNC in the immediate
subnet. This capability will allow active calls to continue on DOMs in adjacent subnets. New OMs are
available to help answer the following questions:

Should the DOM be homed to other secondary RNCs to extend the border region?
Should the DOM not home with any secondary RNCs it is currently configured with?
Is the DOM homed to the optimum Primary RNC?

Refer to section 13-8 of [17] for description of Inter RNC Active Handoff Traffic Statistics and Secondary
Border Measurement OMs.

8.7 1xEVDO Paging Performance Measurements


Prior to release 3.0 the system would only send a single page to the AT. Current with release 3.0 is the DO
repage feature. This feature can be configured to repage an AT up to 2 times (total of 3 pages to AT) before
declaring an airlink paging failure. Additional OMs have been added in release 3.0 to account for the
repage feature.
When the DO RNC attempts to set up a connection with the AT it can do a normal setup involving paging
the AT or by Fast Connect which does not involve paging the AT. The OMs presented for this section
involve pages that are actually sent to the AT from the AN.
Refer to Section 10.1 of [17] for definitions of Paging Performance OMs.

8.8 Per Sector Airlink Resources Allocation (Blocking)


Airlink resource allocations refer to the resources required for the AT to add pilots to its Active Set at
connection setup or during Soft / Softer Handoff. These OMs help during network deployment to determine
where resources may limit connection setup success. For example, if during connection setup or Soft /
Softer handoff the Route update message requests the addition of pilot P1, P2, and P3. The RNC will send
allocation requests for the requested pilots. If DOMs are able to successfully allocate resources for P1 and
Nortel Networks Confidential and Proprietary

86

P3 but a traffic channel can not be assigned for P2, a block due to Reason X will be pegged against Pilot P2
only and success counts will be pegged against pilots P1 and P3. In the end the connection setup will fail.
These OMs measure Airlink resources allocation for primary and secondary DOMs during connection
setup and soft / softer handoff.
Refer to Section 13.1 of [17] for more details on Per Sector Airlink Resources Allocation OMs.

9 Cluster Drive Testing


This chapter discusses the typical cluster optimization drive testing that is done prior to acceptance testing.
The objective of cluster drive testing is to measure the 1xEV-DO network performance in mobility
conditions. Typical KPIs measured are connection drops, access failures, connection setup time, and
mobility throughput. Prior to the optimization drive rounds a baseline drive test should be performed to
determine the initial performance capability of the system. The baseline will provide an initial look at
where problematic areas are such that optimization priorities can be established.
For this test, the network should be divided into clusters; each containing 10-15 cell sites. The same setting
(laptop, AT, FTP server) used in the stationary data testing should be used for cluster optimization.
The cluster optimization is done using one laptop, data collection tool (Optis-M or Invex3G) and two ATs
in a drive test van. AT #1 will perform FTP download of a small file (100 KB) to measure the access
failures and connection setup time. AT #2 will perform FTP download of a large file to measure the
connection drops, forward-link sector switching time, and throughput.
There are two options for the file size for AT #2, 5 MB or 100 MB. 5 MB file will yield about 80 s calls at
500 kbps average throughput. In this case, to calculate the connection drop rate, we just take the number of
connection drops divided by the number of calls. In the second option (100 MB), divide the total call time
by the average CHT (call holding time) to get the number of calls. Then, to calculate the connection drop
rate, take the number of connection drops divided by the number of calls.

9.1 Data Collection Process


The data collection process for cluster optimization is as follows:
Setup the laptop, the two ATs and Optis-M or Invex3G at the drive test route starting point.
Start the laptop, AT and data collection tool. Setup the logmask to collect all EVDO attributes
Optis-M Scenario Setup
Click on the Auto Call button then click on the Create New Scenario button. Create an FTP Get
scenario for Phone 1 (AT #1) and Phone 3 (AT #2) according to the example below. Then click OK.
Verify the following settings:
o Allow Dormant State = Checked
o Keep PPP = Checked. This will insure that the dialer is not disconnected between data calls.
o Dormant Mode timer (Not Shown Below) = at least 15sec. This will insure that at the end of
each data call the AT will transition to a dormant state before the next data connection
attempt.
o Phone Reset = checked. This if the phone fails to connect after the defined number (None
Success) the Optis Client will reset the phone automatically before attempting to connect
again.
None Success = 2
o Retry FTP in Server Drop = Checked If an FTP server failure is detected, Optis-M will
attempt to re establish the FTP connection

Nortel Networks Confidential and Proprietary

87

Optis-M Auto Call setup for cluster drive testing.

Select the call scenario you just created for the autocall. Select All scenario for the AutoCall
Logging Option. This will allow data from both scenarios (ATs) to be captured in one log file.
Start the log by clicking OK.
Invex3G Scenario Setup
For EVDO data collection it is imperative that the Invex3G Chassis be configured with at least 2 DI
boards. The legacy CI boards are processor limited and will result in a reduction in throughput
performance.
Right Click on the device (AT#1) and select Properties. Select the Data Connection tab to configure
the data connection properties.
Follow the same process to configure (AT#2). See figures below for Reference.
o Disconnect Timer = 15 seconds For Disconnect tasks, enter a Hang up Timeout value as
required. If Invex3G does not see data from the network for a certain length of time, it
assumes that the data task is dead and tries to hang up the call (session). Invex3G waits for an
acknowledgement from the network (LCP TERM ACK); however, at some point the software
gives up waiting for the acknowledgement and starts another task loop. If the
acknowledgement is not received, it is possible that the network will keep the original data
session open. When a new task loop is started by Invex3G, the network may not allow the
new session since it is keeping the previous session open. The Hang up Timeout determines
the time that Invex3G will keep trying to get a hang up acknowledgment from the network to
confirm that the previous data session was cleared. If you are experiencing call backs from the
network when a session is ended, or the network will not allow new sessions, try increasing
the Hang up Timeout value to a larger number.
o Wait Timer = 5 seconds time between calls where phone will be inactive.
o Initial Connect - Parameters required for connection to the EVDO network. See the Data
Connection Task example below.
Connection Type = EVDO
Authentication = Chap / PAP
Compression = None
TCP Input (RX) window = default

Nortel Networks Confidential and Proprietary

88

Serial Layer MTU = default


If network is simple IP, set the Username and Password
If network is Mobile IP Username and Password should be left blank.
After the initial EVDO connection parameters are configured select the Add button at the bottom of
the Data Connection window to add the FTP GET task
o Task Name For AT#1 should be Access Failure Performance for AT#2 Dropped Call
Performance
o Task Type FTP GET
o Destination Address IP address of FTP server
o Path and Filename The path on the FTP Server, starting with the root directory, where the
test file is loaded.
A 5MByte file should be used fort AT#2 doing dropped connection and throughput
testing
A 100kbyte file should be used for AT#1 doing access failure testing.
o Username and Password As required to access the Applications Server

Invex Initial
Connection
Config.

Nortel Networks Confidential and Proprietary

89

Invex Data
Connection
Configuration

After all data connection tasks have been defined click ok. Go to file and save workspace as Cluster
Drive.IWF
Select Connect All from the connections menu then Record logfile from the file menu to start logs.
Drive Testing Process
Start driving along the cluster drive test route.
Check the following display in Optis-M throughout the drive test:
o GPS Status window: to ensure that GPS is working.
o 1xEV-DO Pilot Set Graph: to monitor that the AT does soft handoff as necessary with other
sites.
o EVDO Signal Graph and Call Statistics window: to monitor the data call progress and the call
statistics.
Check the following displays in Invex3G throughout the drive test:
o Go to displays and select New and add a State display and a 1D bar Chart.
o In the state display drag the Current Task Status from the data connection Codec of each
device listed in the device tree.
o The Current Task Status will list the following attributes:
Connection State - Verify Connected
Connection Type Verify 1xEV-DO
Task State Verify Running
Task Name Verify FTP GET 5MB or 100kbyte
Task RX Throughput (kbits/s) (Cumulative) Verify > 0 kbits/s
o In the Bar Chart drag the Satellites icon from the GPS Codec. Verify
GPS lock to at least 7 satellites
Verify from the Reports icon in the device tree that Alerts configured to show:
o Dropped Calls
o Call Set up Failures
o Data Connection Failures
o Data Task Faliures
If any of these events occur there will be an audible alarm as well as a recoding of the event in the
Alerts tab of the Invex3G workspace. (See example Below)

Nortel Networks Confidential and Proprietary

90

Call Statistics window in Optis-M.

Invex3G State Window

Note down the areas where access failures, connection drops, and/or handoff failures occur.
At the end of the drive test, stop the logs. Try to keep each log to be no more than 30-minute long.

Nortel Networks Confidential and Proprietary

91

9.2 Data Processing and Analysis


After the data collection for the cluster optimization is done, process the AT log files with RFO and the
RNC logs files with NEWTUN.

9.2.1

Access Failure Rate

An access failure is defined as the number of times (during the cluster drive test) the AT failed to arrive at
the traffic channel. It should measure the RF-related failures excluding the ones caused by resource
blocking and other non-RF related failures.

AccessFail % = # AT _ Fails _ to _ Arrive _ on _ TC


# Valid _ Connection _ Attempts

A successful access attempt is marked by Connection Request message. AT success on arriving on the
traffic channel is marked by Traffic Channel Complete message.

Connection
Request (Access)

AF

AC Ack (Control)

Traffic Channel
Assignment
(Control)

Traffic Channel
Complete
(Rev Traffic)

RTC Ack
(Fwd Traffic)

Access Failure Rate from RNC Logs

After processing the RNC log files in NEWTUN select the files from the Data Manager window.
The call statistics from the RNC logs will be displayed in Statistics view.

Call statistics are grouped as follows in NEWTUN:


o Network Statistics statistics for calls displayed by RNC, IP, and PN.
o Connection Details Call Summary for every connection identified in the logs
o Access Terminal call statistics that are displayed according to ESN, IMSI, or Hardware
type.

For each group of statistics there are groupings of statistics as follows:


o Data = statistics for Active Data Calls
o Session = statistics for session configuration related connections
o All = (Data + Session)

The statistics shown in these views will be for all users on the network. There are utilities in
NEWTUN that will allow engineers to calculate failure statistics for the AT under test.

Select the Access Terminal tab then click on the data icon.

Nortel Networks Confidential and Proprietary

92

Sort the list by IMSI

To determine the access failure rate, look for the IMSI of AT#1 (AT used for Access Failure
Testing.

The Access Failure Rate for the AT inclusive of only Data or Session Configuration related
connections is defined as:

X 100
AF % = failures
Attempts (Blocks + IncompleteAccesses )

Note: Incomplete Access connections that reached the access phase, but had no more messaging to
indicate the outcome.
Click on the Sessions icon and sort by IMSI to determine the Session related Access Failures.

The get the total access failure rate click on the All: icon.

The Total Access Failure Rate is defined as:

X 100
AF % = TotalFailedCalls
TotalCallAttempts (TotalBlocks + TotalIncompleteAccess )

For more information on which calls failed, click on the Total Failed Calls in the statistics view.
The actual calls will be listed in the Active List.

Nortel Networks Confidential and Proprietary

93

Active Call
List

By clicking on the calls in the Active list the engineer will open the message view where the actual
RNC level messages will be highlighted in green in the message view.

Connection was closed because there was an unknown UATI (Local Connection Close, Reason
Code 2).

Local Close Event Reasons (RNC Logs):


Reason
Code

Reason Description

Session Close

Session Close Disable

SSM to close connection and


return to disabled state

HDR Slow Path Close

When HDR slow path


encounters errors

HDR Fast Path Close

When HDR Fast path


encounters errors.

Call Control Inactivity


Close

Sent by Call Control when


the connection has not
passed any data on the FWD
or RVS link for a
configurable inactivity period.

BTS Requested Close

Not currently used

Session Configuration
Close

Called by SCSM when


Session configuration is
complete

Connection Request
Error Close

Generated by CSM when a


connection request is
received when the AN thinks
it already has a connection
open. ( No longer used)

DLSM All RTC Lost

When all RTC comms. are


lost

10

DLSM No FTC

Forward Traffic Channel lost

11

DLSM HFP Error

Unable to request HDR Fast


Path to Stop or re Start
transmission of data

Nortel Networks Confidential and Proprietary

Comments
Called by SSM to close
connection.

94

Access Failure Rate from AT logs

Post Process the AT logs from the Cluster Drive using EVDO RFO
Open the Statistics View and select Call Statistics.

Example of Access Failure statistics in EVDO RFO

Example using the numbers above:


Access Failure Rate
o = DO Access Failures / (DO Attempts DO Blocks Handoff to 1x DO EOF Partial
Access)
o = DO Access Failures / (DO Access Failures + DO Access Successes)
o = 3 / 25 = 12%
The access failure rate should meet the optimization criteria. Please see troubleshooting section for
guidance on the typical access failure rate value and methods for troubleshooting if it is not achieved.

RFO Message View window showing one complete call.

9.2.2

Connection Drop Rate

A normal call is ended by a Connection Close message from the AT with the reason = Normal Close or a
Connection Close from the AN with the reason = Normal Close. Any other call termination is considered a
Nortel Networks Confidential and Proprietary

95

connection drop (e.g. forward Connection Close message with reason = Connection Error or QCP
Connection Release packet with reason = System Lost).
Dropped Connection Rate from RNC logs

After processing the logs with NEWTUN, open the Data Manager window and select the database
the files where processed to.

After opening the data base the statistics view will open. As stated above, call statistics are divided
into three categories:
o Network Statistics statistics for calls displayed by RNC, IP, and PN.
o Connection Details Call Summary for every connection identified in the logs
o Access Terminal call statistics that are displayed according to ESN, IMSI, or Hardware
type.

Each of the above categories are further grouped by call type for more granularity:
o All includes all calls
o Data connections setup for an active data session
o Session connections setup for EVDO session configuration

Select the Access Terminal tab from the call statistics view and sort the statistics by IMSI.

NEWTUN Access Terminal Statistics View Sorted by IMSI

The data presented in the file will be representative of all ATs in the network at the time of testing.
Find the IMSI of AT#2. The statistics on this line will be used to calculate the dropped connection
rate.
Dropped connection statistics should be calculated for individually for Data, Session, and All
connection types according to the following equation.

X 100
DroppedConnection% = Drops
SuccessfulAccesses IncompleteTraffic

Note: Incomplete Traffic is defined as a connection that made it to the traffic channel but there
were no additional messages to indicate the outcome (i.e. if there was a normal or abnormal
connection close).
For more information on why connections were dropped double click on Total Dropped Calls in
the statistics view. This will bring up all of the calls in the Active list.
As above, by clicking on each call in the Active List, RNC logging messages for that call will be
displayed in the message view.

Nortel Networks Confidential and Proprietary

96

NEWTUN Message View

Call dropped due to loss of the Reverse Traffic Channel (Local Connection Close, Reason Code
9).
Another useful method for diagnosing reasons for a call failure is by clicking on the Connection

Details Connection List icon in the statistics window.


From the Connection list, drag the IMSI column header to the black space and then sort the list by
failure reason (i.e. Unclassified Drop). This will list all connections that failed for the selected
reason and will be listed in order of UATI.

Nortel Networks Confidential and Proprietary

97

NEWTUN Connection List

Once the particular failure is isolated, find the IMSI of the AT used for testing.
Expand the view for that IMSI. All the failures meeting the criteria used to sort the list are listed
under the selected IMSI.
Various Attributes are available in this view that may give clues as to why the failure occurred.
The available attributes are selectable by using the attribute selector.
By double clicking on each connection under the IMSI the RNC message details will be shown
message view.

Dropped Connections From AT logs

In RFO, open the Statistics Tool Calls to find the number of connection drops. Example:

Example of Connection Drop statistics in RFO

To calculate the connection drop rate:


Method 1: File Size = 5 MB
Connection Drop Rate = DO Drops / (DO Access Success DO EOF Release Handoff to 1x)
= DO Drops / (DO Drops + DO Completions)
Example using the numbers shown above:
Connection Drop Rate = 7 / (7 + 15) = 31.8%
Method 2: File Size 100 MB

Nortel Networks Confidential and Proprietary

98

Connection Drop Rate


= DO Drops/Number of Effective Calls
= DO Drops/(Total Rx Time/Call Holding Time)
The CHT (Call Holding Time) should be mutually agreed by Nortel Networks and the customer.
The Total Rx Time can be found in RFO from Call Statistics Data Statistics RLP.

The connection drop rate should meet the optimization criteria. Please see troubleshooting section
for guidance on the typical connection drop rate value and ways to troubleshoot if it is not
achieved.
The illustration below shows a connection drop that happened due to poor C2I.

Example of a connection drop in RFO.

9.2.3

Throughput in Mobility Condition

Only files/data from AT#2 are used to calculate throughput. There are no throughput statistics available
from the RNC logs therefore, all per user mobility statistics will come from AT logs.

After post processing the AT log files with EVDO RFO, click on the Statistics Tool Data
Statistics Application.
Look for Rx Active Throughput in the statistics list. This is the average user application
throughput.
EVDO RFO displays the application throughput for each FTP download as well as the average
throughput for each drive test file.

Nortel Networks Confidential and Proprietary

99

Application throughput numbers in RFO.


As in the stationary data testing analysis, check the following to verify that the throughput result is valid:
Use the Graph View to display Traffic C2I, PER, DRC Requested, and Fwd Serving Rate.
Verify that DRC Requested follows C2I 1 trend.
Verify that PER is 1%0.2% most of the time, except when C2I 1 is bad.
Verify that Fwd Physical Data RX Rate follows the DRC Requested trend.
Verify that the user application throughput obtained in this test is appropriate for the SINR (C2I)
condition. Use the plot in section 7.4 as a guide. Please refer to [1] to more information on the
throughput performance number.
Display the Reverse Data Info in Grid View or Graph View. Verify that the reverse link data rate is
sufficient to support the forward link data rate.

Example of DRC Request, App. RX Rate, and Forward Physical RX Rate

Nortel Networks Confidential and Proprietary

100

Example RFO Grid View of Traffic C21, DRC Request Rate, Forward Physical RX Rate, etc

The user application throughput under mobility condition here has to meet the optimization
criteria.

The throughput number for cluster drive testing will generally be lower than the one obtained in the
stationary data testing due to the mobility condition (as shown in the picture in section 7.4). The areas with
access failures, connection drops, handoff problems, and coverage problems (low SINR) will also
contribute to the decrease in throughput number. Therefore, fixing all of those problems and improving the
SINR condition will improve the throughput number. Please see section 10 for guidance on the typical peruser mobility throughput value and ways to troubleshoot if it is not achieved.

Nortel Networks Confidential and Proprietary

101

9.2.4

Connection Setup Time

Only files/data from AT#1 are used to calculate the connection setup time.
Connection setup time is calculated from the time AT sends a Connection Request message to the time it
sends a Traffic Channel Complete message.
In RFO, click on the Statistics Tool Connection Statistics Connections. Export the table to Excel and
calculate the average Time Delta to get the average Connection Setup Time throughout the drive test.

Example of Connection Statistics in RFO


The connection setup time should meet the optimization criteria. Please see section 10 for guidance on the
typical connection setup time value and ways to troubleshoot if it is not achieved.

Nortel Networks Confidential and Proprietary

102

10 Troubleshooting
10.1 Session Setup and A13 Handoff Success Rate
EVDO Session setup is simply defined as the initial interaction between the AT and the AN. An EVDO
session setup is analogous to a registration in CDMA. Session setup is always an AT initiated action. Only
a UATI Request message from the AT with either RATI or foreign UATI can initiate a valid EVDO
session setup attempt. In release 3.0 A13 dormant session transfer protocol is introduced and will impact
the session setup success. EVDO session setup can be classified as follows:

Regular 1xEV-DO session setup initiated with an RATI (also known as regular session setup)
UATIRequest initiated dormant handoff (also known as Regular A13-dormant handoff)
Prior session-initiated dormant handoff

EVDO Session setup consists of the following Stages:

Session instance creation


UATI assignment
A13 dormant session handoff (for dormant handoff only)
Session configuration
Terminal authentication, if enabled
Hardware ID retrieval
AT ID (IMSI and Hardware ID) validation

10.1.1 Data Collection Methods


10.1.1.1 AT Logs
Method of Collection
1. Set AT to download 1MB file wait 15s.
2. Close AT session from RNC every 60 s.
3. To close the session, at the CLI prompt type > no open 1xevdo session <UATI>
4. Set up Optis-M or Invex3G Log masks to collect all EVDO log attributes.
5. To test A13, setup a short data call and allow it to transition to a dormant state.
6. Drive across the RNC border.

10.1.1.2 RNC Logs


Method of Collection
1. RNC logs should be collected in conjunction with the AT logs to capture Session setup events and
the reason codes associated with any failures.
2. The following shows session setup events with and without A13 dormant handoff.

Nortel Networks Confidential and Proprietary

103

AT

AN

PDSN

UATI Request
UATI Assignment
UATI Complete
Open Traffic Channel
Session Configuration
Close Traffic Channel
Page
Open Traffic Channel
Term Auth, HID, LUP
Open A10 (If data to send)
PPP Traffic
Regular Session Setup without A13 Handoff

Nortel Networks Confidential and Proprietary

104

AT

Target
AN

UATI Request
(foreign)

Source
AN

PDSN

A13-Request
A13-Response

UATI Assignment

(with PDSN IP and Hdw Id)

UATI Complete

Skipping Hdw Id
exchange
Source
closes DO
session

Location Assignment
Location complete
Open A10

A13-Confirm

Close A10

PPP Traffic

A13 Dormant Handoff Session Setup


3. Session setup success rate is defined as:
Session Setup Successes = Normal Session Setup Successes + Dormant Handoff Successes +
Dormant Handoff Successes Prior Session
Session Setup Success / Session Setup Attempts = Transitioned from SSMWaitForAtIdRsp state
to SSMWaitForLocationNotify state / Transitioned from SSMWaitForUATIReq state to
SSMWaitForUATIRspFromRC state
4. Set up the RNC logs to capture Call Control information at severity level 16. The events shown in
yellow are what is required to determine the Session Success Rate.
5. The other events shown are useful for determining why a Session Setup may have failed.
Severity
Level
16

Component

Debug

DO-RNC CPUs

Call Control

011

RNSM and SC

Message
ID
81

16

Call Control

011

RNSM and SC

151

19

Call Control

014

RNSM and SC

156

13

Call Control

008

RNSM and SC

150

Nortel Networks Confidential and Proprietary

Description
Session State Transitions
(UATI Req - HwId Resp)
Session Configuration
Event Succeeded
Terminal Authentication
Success event
Session Configuration
Failed Event
105

5-info

Call Control

N/A

RNSM and SC

164

13

Call Control

008

RNSM and SC

157

5-info

Call Control

N/A

RNSM and SC

174

5-info

Call Control

N/A

RNSM and SC

167

5-info

Call Control

N/A

RNSM and SC

414

Session Configuration
Failed
Terminal Authentication
Failure
Terminal Authentication
Failure
Closing Session
Unknown UATI
Session did not Receive
HW ID Resp. from the AT

Sample RNC Log Event Outputs:


Session State Transitions

SCSM Event - (Attempt)

Session Configuration Success

Terminal Authentication Success

Session Configuration Failed Event

Session Configuration Failed Event

Terminal Authentication Failed

Nortel Networks Confidential and Proprietary

106

Session Failed (Unknown UATI)

Session Failed (No Hardware ID Response)

Sample from RNC Logs (Session Config Failed):


SSM SSMWaitForConfigDone:
SSM SSMWaitForConfigDone:
SCSM SCSM_ANInitiated:
SCSM:
CSM CSMOpen:
CSM CSMOpen:
CSM CSMOpen:
CSM CSMAwaitATClose:
Session:
SCSM:
SCSM SCSM Inactive:
SCSM SCSM Inactive:

RX from SCM: Session Configuration Failed Event


Closing Session Session Configuration Failed
Received Deactivate Event
Transitioning from SCSM AN_Initiated to SCSM SCSM_Inactive
Rx: Local Connection Close Event Reason 2
Tx to AT: Connection Close Event Reason 2
Exiting State
Entering State
Transitioned from SSMWaitForConfigDone state to
SSMWaitForAppsToDisable state
Transitioning from SCSM SCSM_Inactive to SCSM SCSM Inactive
Processing ProtocolSubtypeConfig Configuration Response Timeout
Received invalid Configuration Done Indication Event

Note: SSM = Session State Machine, CSM = Connection State Machine, and SCSM = Session
Configuration State Machine
A13 Dormant Handoff Sessions
With release 3.0, the A13 dormant session handoff feature is added. This feature forces a session handoff
for dormant ATs based on the following:

UATI request with foreign UATI (UATI from session on source RNC)
UATI request with RATI (prior session handoff, usually occurs when the AT has lost supervision
prior to A13 handoff request

In most cases A13 handoff should be successful. The following are some typical failures that may be a
result of the process not completing:
Possible A13 Session Failures

Target RNC fails to locate the source. This typically means that the source is not configured in the
targets A13 configuration table

Session does not exist at the source RNC and consequently the source sends an A13 reject.

Configuration parameters are not acceptable to the target. This may happen if there is some delta
in session configuration between the RNCs

Failure can also occur in some fast inter-RNC ping pong cases.

Nortel Networks Confidential and Proprietary

107

The following RNC log sample shows an A13 (foreign UATI) handoff as seen from the target RNC:

A13 Session Ping Pong Scenarios


The purpose of this section is to discuss potential performance issues related to fast ping-pong at the RNC
boundaries. By fast ping-pong, we assume that AT switches between subnets faster than the speed of
session transfer process itself. Here we will analyze different scenarios and point out the problems and
impact on performance.
CASE 1: Target (b) receives A13 Response from the source (a), but AT ping-pongs back to the source (a),
before UATI is assigned.
In this case there will be no problem since when AT bounces back to the source (a) RNC it will use its old
UATI. Target (b) will assign new UATI, so there will be temporary waste of session memory at the target
(b), but this will clear after a time-out. Target (b) will not send A-13 confirm to the source (a).
CASE 2: Target (b) receives A13 Response from the source (a), assigns new UATI, but AT ping-pongs back
to the source (a), before UATI assignment procedure is completed.
In this case AT may have received new UATI from the target (b), but the target does not know this. So
when AT bounces back to the source (a) it will attempt new dormant handoff using UATI it got from (b).
When (a) sends A-13 request to (b) it will be rejected since there is no record of this session at (b). Upon
receiving A13-Reject, (a) will close session and proceed with new session establishment.
Therefore the consequence of this scenario is that session is closed and needs to be re-established. This is
slow and it does require set-up of traffic channel. Also, in this scenario numDormantHandoffAttempts is
pegged, but not numDormantHandoffSuccesses. Therefore this constitutes session setup failure.
Such fast ping-pong scenarios can occur in challenging RF situations, with no clear dominant server, where
UATI procedures take long to complete.
CASE 3: Target (b) receives A13 Response from the source (a), and UATI complete from the AT, but AT
ping-pongs back to (a) while re-configuration of session protocols is under way.
Same as in case 3, when AT bounces back to (a) RNC will send A-13 request to (b), which will be rejected.
Nortel Networks Confidential and Proprietary

108

Upon receiving A13-Reject, (a) will close session and proceed with new session establishment.
Therefore the consequence of this scenario is that session is closed and needs to be re-established. This is
slow and it does require set-up of traffic channel. Also, in this scenario numDormantHandoffAttempts is
pegged, but not numDormantHandoffSuccesses. Therefore this constitutes session setup failure.
Such fast ping-pong scenarios can occur in challenging RF situations, with no clear dominant server, where
UATI procedures take long to complete.
CASE 4: Target (b) receives A13 Response from the source (a), and UATI complete from the AT;
completes the re-configuration of session protocols, but AT ping-pongs back to (a) before Hardware Id
exchange is completed.
Consequence would be the same as in cases 2 and 3, but this should not occur frequently, because A-13
should be configured to pass hardware id in A-13 Response message and therefore hardware id exchange is
skipped at the target (b).
CASE 5: Target (b) receives A13 Response from the source (a), and UATI complete from the AT;
completes the re-configuration of session protocols and hardware Id is retrieved, but AT ping-pongs back
to (a) before (b) receives Location Notification.
In this case A10 between (b) and PDSN is not opened. Also A13 Confirm is not sent to (a), so old session
remains opened at (a). When AT bounces back to (a), that RNC will request session info from (b). This will
complete and (a) will open new session. However at hardware id stage (a) will realize that it already ahs
[old] session for that AT and will proceed closing it. This closes Associated A10 as well, which in turn
clears PPP at the PDSN. Note that AT still keeps the PPP session so there is a state mismatch.

10.1.1.3 OMs
The following OMs should be observed for Session Setup Performance:

Regular Session Setup Failures:


1xEV-DO Session Setup Failures = totalSessionSetupsFailed
where:
totalSessionSetupsFailed =
numSessionSetupsFailedToUATICmpltTimeout+
numSessionSetupsFailedToInvalidUATICmpltMsgSeq +
numSessionSetupsFailedToSessionConfig + numSessionSetupsFailedToTermAuth +
numSessionsTerminatedToHwIdRspFailure +
numSessionSetupsFailedToInvalidHwIdValue +
numSessionSetupsFailedToInvalidHwIdType +
numSessionsTerminatedToAtIdRspTimeout +
numSessionSetupsFailedToATInitiatedSessionClose +
numSessionSetupsFailedToRNCInitiatedSessionClose+
numSessionSetupsFailedToSessionInfoConfirm +
numSessionSetupsFailedToOtherCauses
numSessionSetupsFailedToSessionConfig - Total number of Session Setups which failed on this
DO-RNC due to the Session Configuration failure. It includes the scenario when the connection was
failed or dropped during session setup.
numSessionSetupsFailedToTermAuth - Total number of Session Setups which failed on this DORNC
due to Terminal Authentication failure.
Nortel Networks Confidential and Proprietary

109

numSessionsTerminatedToUnknownLocalUati Total number of sessions which were terminated


by the DO-RNC due to the received local UATI being unknown. It happens when the color code in
the UATI matches the color code of the DO-RNC but the session instance does not exist on the DORNC. This attribute is pegged when any message with an unknown local UATI is received, not just
the UATI Request message. This is a state mismatch when the AT thinks that it has a 1xEV-DO
session with the AN, but the AN does not have a session with the AT.
numSessionsTerminatedToHwIdRspFailure - Total number of sessions which terminated on this
DO-RNC due to Hardware Id Response Failure. Note that it is a session setup failure, not a session
termination. Session setup is not completed before the HardwareIdResponse message is received.
numSessionsTerminatedToAtIdRspTimeout - Total number of sessions which terminated on this
DO-RNC due to AT Id Response Timeout. Note that it is a session setup failure, not a session
termination. Session setup is not completed before receiving AT Id Response with a result of OK.
The AT Id Response is an internal message from the SC card. It is not a standard airlink message.
This attribute IS supported.
numSessionSetupsFailedToOtherCauses - Total number of Session Setups which failed on this
DO-RNC
due to other causes such as the DO-RNC not receiving UATI Complete or UATI Request message
from the AT.

A13 Dormant Handoff Failure OMs:


UATI Request initiated A13 handoff OMs
Percentage of Regular A13-Dormant Handoff Failures due to A13REJECT from the Source RNC =
100% *
numA13TotalRejectRNC /
(numA13TotalRejectRNC + numTotalDormantHandoffFailureRNC)
Percentage of Regular A13-Dormant Handoff Failures due to Possible
Source RNC Table Datafill Error =
100% *
(numDormantHandoffFailureToSourceLookupFailureRNC +
numDormantHandoffFailureUati104RNC) /
(numA13TotalRejectRNC + numTotalDormantHandoffFailureRNC)
The following could be caused by a connectivity problem between the RNCs.
If A13 is not enabled on the source RNC, it can also cause this failure.
Percentage of Regular A13-Dormant Handoff Failures due to A13
Message Timeout =
100% *
numA13ReqTimeoutRNC /
(numA13TotalRejectRNC + numTotalDormantHandoffFailureRNC)
Percentage of Regular A13-Dormant Handoff Failures due to Socket
Issue =
100% *
numDormantHandoffFailureSourceUnreachableRNC /
(numA13TotalRejectRNC + numTotalDormantHandoffFailureRNC)
Percentage of Regular A13-Dormant Handoff Failures during the
Nortel Networks Confidential and Proprietary

110

UATIAssignment Stage =
100% *
(numDormantHandoffFailureNoUatiReqRNC +
numDormantHandoffFailureNoUatiCmpltRNC +
numDormantHandoffFailureInvalidUatiCmpltRNC +
numDormantHandoffFailureNoRncResourceRNC) /
(numA13TotalRejectRNC + numTotalDormantHandoffFailureRNC)
One possible cause of the following failure is the different setting for terminal
authentication between the source and target RNCs.
Percentage of Regular A13-Dormant Handoff Failures due to Retrieved
Session Configuration Unacceptable =
100% *
numDormantHandoffFailureRetrievedConfigUnacceptableRNC /
(numA13TotalRejectRNC +
numTotalDormantHandoffFailureRNC)
The session configuration failure is usually caused by the connection setup
failure or connection drop during the session configuration phase. Such
events are usually due to the poor RF condition along the DO-RNC subnet
border.
Percentage of Regular A13-Dormant Handoff Failures during the
Session Configuration Stage =
100% *
numDormantHandoffFailureSessionConfigDuringReconfiguration
RNC /
(numA13TotalRejectRNC +
numTotalDormantHandoffFailureRNC)
Percentage of Regular A13-Dormant Handoff Failures during the
Terminal Authentication Stage =
100% *
numDormantHandoffFailureTAAfterA13RspRNC/
(numA13TotalRejectRNC + numTotalDormantHandoffFailureRNC)
The event pegged by numDormantHandoffFailureHdwIdTimeoutRNC is
frequently caused by poor RF condition. The airlink messages associated with
hardware ID retrieval could be lost in many cases.
Percentage of Regular A13-Dormant Handoff Failure during the
Hardware ID Acquisition Stage =
100% *
(numDormantHandoffFailureHdwIdTimeoutRNC+
numDormantHandoffFailureInvalidHdwIdValueRNC+
numDormantHandoffFailureInvalidHdwIdTypeRNC) /
(numA13TotalRejectRNC +
numTotalDormantHandoffFailureRNC)
The AT ID validation happens within the DO-RNC. Such failure is caused by
the DO-RNC and not related to any external environment.
Percentage of Regular A13-Dormant Handoff Failures during the AT ID
Validation Stage =
100% *
Nortel Networks Confidential and Proprietary

111

numDormantHandoffFailureAtIdTimeoutRNC/
(numA13TotalRejectRNC +
numTotalDormantHandoffFailureRNC)
Prior Session Initiated Handoff OMs
Similar OMs as those given for regular A13 initiated dormant handoff can be developed for prior
session initiated dormant handoffs. For more information on A13 handoff see reference [21].
Using OMs to diagnose common problems (See [21])
OM
Attempts
Successes
Rejection Messages

Failure Source RNC not Found


Failure No A13 Response

Common Diagnosis Scenario


Number of times a dormant handoff was attempted
When the number of successes is close to the number of attempts
the system is operating in a healthy manner.
Increments because either, the wrong source RNC was chosen, or
else the AT did not complete setting up its session on the RNC prior
to moving to another RNC.
However, if there are a large number of rejections due to Session
not Found then another possibility is that the subnet ID was
misconfigured in the Source RNC lookup table.
A13 is disabled at the target RNC
At attempted a dormant handoff from a Source RNC not listed in
the Source RNC lookup table.
Either the source RNC is out of service or there is a connectivity
problem. Try to ping the source RNC.
The following may be reasons for no A13 response from the
Source RNC:
A13 is disabled on the source RNC,
The IP address on the source RNC was incorrectly configured in
the target RNCs Source RNC lookup table.
Source RNC is down
IP connectivity issue.

Failure Session Configuration


Unacceptable

Failure Hardware ID

Failure Missing UATI


Complete

Failure UATI -104 Mismatch

Error while parsing configuration attributes received within the A13


response. Could indicate that either, the message was not formed
properly, or else an error occurred in the transmission.
Error while trying to reconfigure the session parameters at the target
after a successful A13 retrieval. This could occur to the airlink
connection being lost.
After a successful A13 retrieval at the target, there was no
hardwareID response during session setup. This could occur
due to the airlink connection being lost or the AT moved away
before sending the response, and does not necessarily indicate any
problem.
After a successful A13 retrieval at the target, there was no
UATIComplete response when assigning a new UATI. This
could occur due to the airlink connection being lost or the AT
moved away before sending the response, and does not necessarily
indicate any problem.
After a successful A13 retrieval at the target, it was determined
that the AT UATI-104 did not match the source RNC.
The most likely reason is that the wrong source RNC was
selected (due to colorcode reuse). This does not indicate any

Nortel Networks Confidential and Proprietary

112

Failure UATI-104 Matches


Local Subnet

Failure Source RNC


unreachable
Failure Missing UATI Request

Failure internal errors

Post Dormant Handoff


reconfiguration needed
A13 Message Count To
Remote RNCs
A13 Message Count From
Remote RNCs

operational problem.
This is a very rare error that can occur during a prior session
handoff. This requires the AT to ping-pong between subnets
rapidly. If the AT attempts a prior session on the target and if the
prior session attribute matches the local subnet, then this error is
pegged.
A socket problem occurred on the target RNC. If this counter
constantly increases, it could indicate a problem with the
system. Determine which slot has the problem (use the per-slot
OMs) and reboot that slot.
UATI Request was not received after an initial foreign UATI
message was received at the target. This may be due to the AT
moving away after the initial message. This does not indicate an
operation problem.
This may indicate that the RNC is either temporarily overloaded,
or has encountered internal resource problems. If this OM
continues to increment over a long period of time, it may be prudent
to reboot the RNC.
Indicates that the source and target RNC has different operator
settings. Does not necessarily indicate any operational
problems.
If this counter does not increment during dormant handoffs, then
A13 handoffs might be disabled.
If this counter does not increment during dormant handoffs then
A13 might be disabled or there may be a connectivity problem.

10.1.2 KPI Calculation


10.1.2.1 KPI Value and Tolerance Factor
Session Setups should be a rare event in that Sessions are long-lived. The Session Setup Success rate
shall be > 95% and should be > 98%. Because of the current design of the product the success rate is
~ 50% as measured via OMs. There is no A13 support and each Subnet Id is identified to a single
RNC. This creates many borders in a normal network causing a large number of session setup
attempts in these border areas. Because these are border areas, the RF channel is by definition less
than optimal, increasing the failure rate.

Nortel Networks Confidential and Proprietary

113

10.1.2.2 Calculating KPI from AT Logs


Session Setup Success Rate = Session Setup Successes / Session Setup Attempts
= # of UATI Request Messages / # of Hardware ID Response Messages
Example From AT logs:

10.1.2.3 Calculating KPI from RNC Logs


After downloading the RNC logs, use a macro to extract all Transitioned from
SSMWaitForAtIdRsp state to SSMWaitForLocationNotify state and Transitioned from
SSMWaitForUATIReq state to SSMWaitForUATIRspFromRC state. Use the following formula to
calculate the Session Setup Success Rate:
Formula: Session Setup Success / Session Setup Attempts = Transitioned from
SSMWaitForAtIdRsp state to SSMWaitForLocationNotify state / Transitioned from
SSMWaitForUATIReq state to SSMWaitForUATIRspFromRC state
Currently NEWTUN does not provide specific statistics related to A13 success. However, the
success rate can be determined by:
A13RNCSuccess% =

# A13Confirms

#UATI Re quests _ w / ForeignUATI / RATI

10.1.2.4 Calculating KPI from OMs


The OMs required to determine the session setup and A13 handoff success rate are calculated as
follows:
Regular EVDO Session Setup Success Rate
1xEV-DO Session Setup Successes = numSessionSetupSuccessful
Therefore,
Regular 1xEV-DO Session Setup Success Rate =
numSessionSetupSuccessful /
(numSessionSetupAttempts
numSessionSetupsFailedToATInitiatedSessionClose
numSessionSetupsFailedToRNCInitiatedSessionClose)
UATI Request Initiated Regular A13 Dormant Handoff Success Rate

Nortel Networks Confidential and Proprietary

114

The UATIRequest Initiated Dormant Handoff (also known as Regular A13-dormant handoff) is
triggered when an AT crosses the subnet border without losing supervision of the 1xEV-DO carrier.
However in order for the A13-dormant handoff to proceed, the A13-dormant handoff must be
enabled on the target RNC.
Regular A13-Dormant Handoff Success Rate =
Regular A13-Dormant Handoff Successes /
Valid Regular A13-Dormant Handoff Attempts
Where:
Valid Regular A13-Dormant Handoff Attempts =
(numDormantHandoffAttemptsRNC
numDormantHandoffFailureATInitiatedCloseRNC
numDormantHandoffFailureRNCInitiatedCloseRNC)
Prior Session Initiated Dormant Handoff Performance
A prior session initiated dormant handoff is triggered when the AT, during the session configuration
phase, requests to retrieve a prior session from the source RNC. When such request is received, the
target RNC will attempt to retrieve the session information from the source RNC via the A13
interface. For the prior session retrieval to proceed, the A13-dormant handoff must be enabled on the
target RNC.
Prior Session Initiated Dormant Handoff Success Rate =
numDormantHandoffSuccessesPriorSessionRNC /
(numDormantHandoffAttemptsPriorSessionRNC
numDormantHandoffFailureATInitiatedClosePriorSessionRNC
numDormantHandoffFailureRNCInitiatedClosePriorSessionRNC)
Nortel- XX# show 1xEVDO counters all
The following equation should be used to determine the Session Setup Success Rate from OMs:
Formula: numSessionSetupSuccessful/(numSessionSetupAttempts-numSessionsTerminated
ToReceivingUatiReq )

10.1.3 Troubleshooting Guide


DOM Health
First, perform basic readiness check-up on the DOM with session setup failures (see Appendix A). Log on
to the DOM and check:
Are the modules up and running?
Is DOM node up?
Are sector-elements up?
Is Abis Peer up?
Is there an IP connectivity to the RNC ?
Check the GPS timing source
Check the SNTP Time from GPS
Check if there are traffic channels between the DOM and RNC
Check if there are sessions and connections in this DOM
Events related to Session Setup Failure:
1. Repeat Offenders: Often Session setup success rate can be adversely impacted by stationary terminals
close to an RNC border that periodically tune to a sector that is on the adjacent RNC. When these periodic
Nortel Networks Confidential and Proprietary

115

session setup events occur between RNCs, the failures that result skew the overall network performance
trend although the impact as a whole is only to a handful of users.
If the AT does not complete Terminal Authentication or reach the Hardware ID retrieval phase of session
setup IMSI information is unavailable to identify the user.
A method for troubleshooting Repeat offenders will be presented in a later release of this document.
2. Session Configuration Failure: Look for Session Configuration Failed Events in the RNC logs. From
the OMs look for numSessionSetupFailedtoSessionConfig
3. Terminal Authentication Failure: Look for TerminalAuthenticationFailureEvents in the RNC logs. In
the OMs look for numSessionSetupsFailedtoTermAuth.
4. Follow the following steps to troubleshoot Terminal Authentication or Session Configuration problems:
a. For terminal authentication to work the AT, DO-RNC, and AN-AAA servers all should be
configured properly.
b. Required AT Configuration:
i. NAI must be equal to the username configured for the AT in the AAA server(s)
ii. Password - must equal the password setup for the AT in the AAA server(s).
iii. Compare the ATs Network Access Identifier (NAI) and Password to the Username and
Password configured for the AT in the AAA server, as follows:
a. On the AT, use QPST (or another provisioning tool) to display the NAI and Password for
the AT, typically found on the 1xEV tab.
b. On the access network AAA server, check the user profile for this AT and display the
username and the password for the particular user.
If the AAAs User name for the AT does not match the NAI setting configured in the AT, or if
the AAAs Password for the AT does not match the Password configured in the AT, there is a
misconfiguration of the authentication parameters for this AT.
To recover, use QPST to configure the ATs:
NAI to equal the Username configured for the AT in the AAA server
Password to equal the Password configured for the AT in the AAA server
c.

Required DO-RNC configurations:


i. IP address of the AAA server
ii. Display the IP address the DO-RNC uses for A12 terminal authentication communications
with the AAA server with the Enable mode show termauth configuration command,
as follows:
NORTEL-XX#show termauth configuration
===============================
TA Scalar Configuration Record
===============================
Terminal Authentication : Disabled
A12 Interface Source IpAddress : 10.12.0.241
A12 Maximum Retransmit Attempts : 6
A12 Access Timeout Value : 1
A12 Load Balancing Mode : NAI realm based
Aaa Routing Table Max Size : 64
Aaa Routing Table Default Route : 1
Default AnPPP Auth Method : CHAP
Default ACCM value : 0x00000000
AnPpp IdleTimeout value : 180
===============================
Ensure that the A12 Interface Source IpAddress is the IP address the AAA server(s) uses to
communicate with DO-RNC clients. If the IP address is incorrect, configure the AAA server with
the A12 Interface Source.
iii. Ensure the secret key that the AAA server and the DO RNC use to validate A12 communications

Nortel Networks Confidential and Proprietary

116

iv. Examine and verify the Authport for each AAA server listed.
The Authport is the IP port number on which the AAA server is configured to listen for
terminal authentication requests. This is typically 1812.
If the AAA is configured to listen on a different IP port number, modify the DO-RNC
configuration of the AAA to match the correct port number with the Configure Termauth
Server mode server <name> <secret> <A.B.C.D> [authorization port number] command, as
follows, assuming a name of AAA1, a secret of xyz, and a AAA server IP address of
10.11.12.13, and an authorization port number of 1812:
NORTEL-XX(config-termauth-server)#server AAA1 xyz 10.11.12.13 1812
vi. Ping each listed AAA server IP address from the DO-RNC with the Enable mode ping
<Ip address> command, as follows:
NORTEL-XX#ping <AAA server ip address>
If the ping fails, check the AAAs actual server IP address. If it is not the same as configured
in the DO-RNC, change the DO-RNCs configuration with the Configure Termauth Server
mode server <name> <secret> <A.B.C.D> command, as follows, assuming a name
of AAA1, a secret of xyz, and an AAA server IP address of 10.11.12.13:
NORTEL-XX(config-termauth-server)#server AAA1 xyz 10.11.12.13
vii. To check current authentication activity use the following command while in Enable mode at the
RNC: Nortel-XX# show termauth statistics scalar
Useful OMs used to diagnose A13 HO problems

Nortel Networks Confidential and Proprietary

117

10.2 Session Setup Time

Nortel Networks Confidential and Proprietary

118

10.2.1 Data Collection Methods


10.2.1.1 AT Logs
Same as for session setup success rate.

10.2.1.2 RNC Logs


There are currently no good means for measuring session setup time from the RNC logs but may be
added in future releases of NEWTUN.

10.2.1.3 OMs
There are no system OMs currently available that track session setup time. However, the engineer can
go through CLI to view the Max, Min, and Ave Session setup time.
Nortel-XX# >show 1xEVDO counters all
Session Setup Time :Minimum :

00:00:02.320

Maximum : 00:00:02.320
Average :

00:00:02.320

10.2.2 KPI Calculation


10.2.2.1 KPI Value and Tolerance Factor
From [14]: EVDO <= 8s

EVDO + PPP <= 14 s

Because this should be a rare event, we can afford more time for a setup without end-user performance
degradation. But since Session Setup is a normal system event using normal system messages and
signaling, the time for setup should be defined. Session Setup is affected by the amount of
configuration data that must be negotiated.
Session Setup Time: 11-12 seconds for unloaded network and 14-15 seconds for loaded network (with
TA). {We need to define start/stop points and test method. This will have to be measured with drive
testing}

10.2.2.2 Calculating KPI from AT Logs


EV-DO session setup time is measured from UATI Request message to Hardware ID Response
message.
In RFO, open Statistics Tool Session Statistics EVDO Sessions. The statistics can be exported to
Excel.

Nortel Networks Confidential and Proprietary

119

Example: RFO Statistics View EVDO Session Statistics

10.2.2.3 Calculating KPI from RNC Logs


10.2.2.4 Calculating KPI from OMs
10.2.3 Troubleshooting Guide

10.3 Connection Setup Time


10.3.1 Data Collection Methods
10.3.1.1 AT Logs
Method of Collection:
1. Configure Autocall in Optis-M or XCAL-DO to perform an FTP download of a 100kByte file with a
15second interval between data calls.
2. Insure that the AT is in HDR only mode and receiver diversity is enabled prior to beginning the test.
3. Insure that all EVDO log mask attributes are selected.
4. Perform a drive test over the cluster golden route to collect AT logs
5. At the end of each cluster drive stop logs and save for post processing.

10.3.1.2 RNC Logs


RNC logs can be collected simultaneously with the mobile logs for the purpose of measuring
connection setup time. However, the present set of RNC log analysis tools does not calculate
connection setup time, therefore the engineer will have to develop scripts that will filter out the AT
UATI and calculate the time deltas between Connection Request and Traffic Channel Complete.

10.3.1.3 OMs
Method of Collection:
1. There are currently no specific OMs for measuring connection setup time per UATI or overall.
2. Per UATI connection setup statistics can be collected from the CLI prompt but the statistics are only
for the last or current AT connection.
NORTEL-XX# show 1xEVDO connection <UATI> counters all
Connection Setup Time : 00:00:00.220
Nortel Networks Confidential and Proprietary

120

3. The average connection setup time for the entire network can be determined from CLI using the
following command:
NORTEL-XX# show 1xEVDO counters all
Connection Setup Time:Maximum: 00:00:00.280
Minimum: 00:00:00.150
Average: 00:00:00.220
4. At the end of the cluster drive test log into the CLI prompt

10.3.2 KPI Calculation


10.3.2.1 KPI Value and Tolerance Factor
From [14]: 2s for 95% of events (AT initiated)

10.3.2.2 Calculating KPI from AT Logs


Please see section 6.2.4.
Example: RFO Connection Statistics

10.3.2.3 Calculating KPI from RNC Logs


Will have to be done manually at this time

10.3.2.4 Calculating KPI from OMs


As shown is section 7.3.1.3

Nortel Networks Confidential and Proprietary

121

10.3.3 Troubleshooting Guide


Perform a drive test of the cluster
Golden Route.
FTP downloads of 100kB file
Post Process AT logs with RFO

From Connection Statistics in RFO,


is the average connection time
exceed 2 seconds

Y
From the Access Attempt Message is
the Probe Sequence Count greater
than 1?

N
Identify top ten sectors with longest
connection setup times. Is ROT
excessive?
NORTEL-XX#show rate-control
rev-link-stats <sector> e1=alpha

END

- Reduce the Num Steps for Probes to 4.


- Increase the Power Step Size for Probes to 12
(6dB) per probe.
- Increase the Probe Initial Adjust from 0 to 6dB.
- Verify the Access Cycle Duration = 64 slots
- Verify new settings from Access Parameters

10.4 Access Failures


10.4.1 Data Collection Methods
10.4.1.1 AT Logs
Method of Collection:
Configure the Autocall script in Optis or Invex3G for download of a 500 KB test file with the idle /
wait time set for 15s. Configure the number of calls for 1000 calls such that a significant sample of data
can be collected. (See Below):

Nortel Networks Confidential and Proprietary

122

Example: Optis-M AutoCall Setup for Access Failures

Nortel Networks Confidential and Proprietary

123

Example: Invex3G Data Connection Task Setup for Access Failures

1. In Optis-M Insure that the Allow Dormant State checkbox is not checked unless it is desirable to
have the AT wait for the data inactivity timer to expire such that the system closes the RF connection
between downloads.
2. For data collection select all EVDO attributes for the log mask
3. Drive the golden route to collect AT logs.

10.4.1.2 RNC Logs


1. Log into the DO EMS and within the Log Facility manager configure the SC cards to collect RNC
logs at severity level 16. Logging can also be enabled on the RNSM cards individually as given in
section 5.3.1 for setting up RNC logging
2. RNC log events associated with Call Setup (AF) Failures are shown in the table below:
RNC State
CSM
CSMAwaitCloseSHOL
SSM
SSMAwaitforConfigDone
CSM CSMAwaitTCC

RNC Log Message


TX to AT: Connection Deny Message (This message indicates that
resources were unavailable to support call setup at this time. Usually
associated with a state mismatch in the RNC)
Closing Session Unexpected UATI Request (This message
indicates that the RNC was waiting for the AT to respond to a
previous message from the AN)
Traffic Channel Complete Message not Received from AT in time
(This describes an exception in the Connection State Machine
indicating the RNC (while in the process of opening a connection)

Nortel Networks Confidential and Proprietary

124

CSM CSMAwaitTCC

did not receive the TCC in time)


No pilots in active set at time of sending
Traffic Channel Assignment message (Exception
message from the Connection State Machine
indicating that the RNC had a problem sending
out the TCA to the AT.)

10.4.1.3 OMs
1. In order to setup data collection, follow the guidelines in section 5.4.4 to setup the data collection
configuration with the RNCPerfBySectorCarrier_R3.0 template selected.
2. The following OMs are available from the RNCIS856 Sector Carrier Performance Table:
numATConnectionSetupsFailedTccTimeoutSC
numANConnectionSetupsFailedTccTimeoutSC
numFCConnectionSetupsFailedTccTimeoutSC
o The overall per RNC FCA rate is given by the following level 1 formula:
Percentage of Overall Connection Setup Failure rate =
((numConnectionSetupsBlockedByRncResources +
umConnectionSetupsFailedByRncResources + numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources + numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources + numConnSetupsFailedRncTimeout +
numConnSetupsFailedRuTimeout + numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError + numConnSetupsAborted) /
(numConnectionRequestsFromAT + numFastConnectsAttempted
numConnReqsWhileSettingUp numConnReqsWhileTearingDown
numConnReqsWhileOpen)) * 100
3. The following level 2 and 3 OMs show the percentage of connection setup failures broken out
according to the following categories:
a. Unsuccessful DO-RNC resource allocations numConnectionSetupsBlocked +
FailedBy_[RNCResources / RN]
o Includes events such related to RNC and DOM resources
b. RF Failures numConnSetupsFailedRU + TCCTimeout
o RF failure metrics include events related to TCC timeouts, and RU timeouts etc, that
contribute to the overall connection setup failure rate.
c. Misc Failures cnumConnSetupsFailedSWError
o Usually includes events like software errors
o FCA OMs are available for Session related FCA and are denoted by (sNum***) whereas regular data
oriented connection setup failures are denoted by (cNum***).

10.4.2 KPI Calculation


10.4.2.1 KPI Value and Tolerance Factor
From [12] Access Failure% <= 4%?

10.4.2.2 Calculating KPI from AT Logs


1. Post process the AT logs with RFO
2. Open the Call Statistics window in RFO and look at the number of Access Attempts and Access
Failures.

Nortel Networks Confidential and Proprietary

125

Example: EVDO RFO Statistics Window

3. The Access Failure Rate from the AT logs is defined as follows:


For HDR only data collection = DO Access Failure Rate = DO Access Failures / DO Attempts
For Hybrid data collection = Access Failure Rate = {(DO Access Failures + 1x Access Failures)/
(Do Attempts + 1x Originations + 1x Page Reponses)}

10.4.2.3 Calculating KPI from RNC Logs


1. To determine the number of access failures from RNC logs the use of the NEWTUN post processing
tool is recommended.
2. From NEWTUN, open the database containing the post processed RNC log files.
3. The default screen is the statistics view. From the statistics view select Access Terminals All.
4. Sort the list by IMSI.
5. Look for Total Access Failure Rate in the statistics.

10.4.2.4 Calculating KPI from OMs


1.To determine the access failure rate from the OMs the following: equation should be used:
2.Total Failed Connection Attempts can be broken down between session connection setup attempts and
data connection setup attempts. OMs are available to collect both cases. Session OMs are prefixed
with an [sNum] and data connection attempt / failure OMs are prefixed by [cNum]
Failed Connection Attempts (FCA) =
(cNumRFRelatedFCA +
cNumResourceRelatedFCA +
cNumMiscFCA) /
cNumConnectionSetupAttempts
3.Unsuccessful DO-RNC resource allocations
Percentage of Connection Setup Failures due to Unsuccessful DO-RNC
Resources Allocation =
100% *
(numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRncResources) /
(numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources +
numConnSetupsFailedRncTimeout +
Nortel Networks Confidential and Proprietary

126

numConnSetupsFailedRuTimeout +
numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted)
4.Unsuccessful DOM Resources allocations
Percentage of Connection Setup Failures due to Unsuccessful DOM
Resources Allocation =
100% *
(numConnectionSetupsBlockedByRn +
numConnectionSetupsFailedByRn) /
(numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources +
numConnSetupsFailedRncTimeout +
numConnSetupsFailedRuTimeout +
numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted)
5. Connection Setup Failures due to RF Failures
Percentage of Connection Setup Failures due to RF Failures =
100% *
(numConnSetupsFailedRuTimeout +
numConnSetupsFailedTccTimeout) /
(numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources +
numConnSetupsFailedRncTimeout +
numConnSetupsFailedRuTimeout +
numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted)
6. Connection Setup failures due to Misc. Reasons
Percentage of Connection Setup Failures due to Misc Failures =
100% *
(numConnSetupsFailedRncTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted) /
(numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources +
numConnSetupsFailedRncTimeout +
numConnSetupsFailedRuTimeout +
numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted

Nortel Networks Confidential and Proprietary

127

10.4.3 Troubleshooting Guide


Call Failure Reasons from RNC logs
From the NEWTUN Statistics view, select Network Statistics. From the network statistics it is possible to
determine the DOM(s) associated with each access failure.

1.
2.

3.
4.

More detailed information regarding each access failure can be determined by selecting the
Connection Details tab from the statistics view and selecting the Connections List.
Form the Connections List sort by Failure Classification Access Failure. This will pull up all the
access failures and the connection / RF attributes seen just prior to the failure.

If a failure reason cannot be readily inferred from the data in the connections list, click in the access
failures in the Network Statistics which will add the connection attempts that ended in a setup failure
to the Active List.
Select the call in question from the Active list to view the call sequence in the Message View.

Nortel Networks Confidential and Proprietary

128

Multiple TCAs
without an ACK
from the AT.

New CR
NEWTUN Message View
5.

Information about the distance the AT was from the site can be obtained from the Cell Radius Tool in

6.

NEWTUN.
By matching the DOM IP and sector PN where the call setup failed from the Network Statistics view
in the Cell Radius Tool the distance and relative Ec/Io of the originating PN can be determined.

Nortel Networks Confidential and Proprietary

129

Calls originate far (>3.4km)


from the site. This could be
an overshooting site.

NEWTUN Cell Radius Tool


Operational Measurements
7. Operational measurements can be used further to determine the top contributors to connection setup
failures. In the case of Failed Connection attempts, these are typically considered RF or equipment
error related
o Percentage of Connection setup failures due to RF Failures (Level 2, per RNC)
o This level 2 metric includes events which are related to TCC timeouts and RU Timeouts and
contribute to the overall connection setup failures.
o When this metric evaluated with other level 2 metrics, it can be easily determined that which
level 2 metric is HIGHER contributor to the overall connection setup failures.
8. Percentage Contribution of TCC Timeouts to Connection setup failures due to RF Failures (Level 3,
per RNC)
o If this metric is the highest contributor among level 3 metrics, it indicates the issues related to
TCC timeouts.
o TCC timeout happens when Call Control on RNC times out waiting for Traffic Channel
Completes message from the AT.
o Higher TCC timeouts indicates that the RF conditions were not good enough for the TCC
message to reach the DOM or TCA message reach the AT or TCC Wait Timer parameter value is
not good enough. Link budget need to be verified for the forward and reverse link coverage
balance. Power used (configurable parameter) for sending TCA need to be verified

Nortel Networks Confidential and Proprietary

130

9.

Percentage Contribution of Route Update Timeouts to Connection setup failures due to RF Failures
(Level 3, per RNC)
o If this metric is the highest contributor among level 3 metrics, it indicates the issues related to RU
timeouts.
o RU timeout happens when Route Update message is not received by the RNC from AT within a
predefined time interval. The timer used for this event is not configurable.
o Higher RU timeouts indicates that the RF conditions were not good enough for the RU message
to reach the DOM. Other contributing factors to this event exists but could not be fixed by us
(needs to have design take care of the software). Link budget need to be verified for the forward
and reverse link coverage balance

Verification of System Readiness


Are the modules (DO Network Elements) up and running?
o Module Status in EMS must be Active NORTEL-XX>show module
o If not all of the NEs are physically present or do not have an Active status see NE Outage
Troubleshooting Flowchart on page 3-10 of NTP 411-2133-949.
Insure all Interfaces are operational (i.e. T1/E1, PPP interface for each deployed T1, node 1/0/1
interface).
o From CLI use the following commands to check the interface: NORTEL-XX#show
interface
o If any of the interfaces have an operational status of Down proceed according to the IP RAN
troubleshooting flow chart in section 3-11 of NTP 411-2133-949:
Is DOM node up?
o From the RNC use NORTEL-XX#show node to verify the operational state
of the DOMs
o If the operational state is Down follow the troubleshooting flowchart on page 3-10 of NTP 4112133-949
Are sector-elements up?
o On the DOM check the administrative and operational status of sector elements NORTELXX#show sector-element
o If the operational status is down then there is a problem with Abis or topology manager
o Follow recovery procedure 4-16 in NTP 411-2133-949 to re enable Abis or topology manager.
Is Abis Peer up?
o ABIS is a TCP protocol used to carrier user signaling and data traffic between the DOM and the
DO-RNC
o IF the ABIS peer is down it is common for the ATs to fail session setup and connection setups.
o To verify connectivity to the ABIS peer, Ping the RNSM module from the DO-RNC
NORTEL-XX>ping 127.1.(RNSM slot).1
o Another way to verify connectivity is to check if there is an ABIS peer by performing the
following CLI command: NORTEL-XX#show abis peer. If there are no
active connections listed there could be a problem with the
Topology manager or Abis. (See NTP 411-2133-949 for more info)
Is there an IP connectivity to the RNC ?
o A route to every DOM PPP interface IP address, and routes to PDSNs, the DO- EMS server, and
the AAA server.
o A standard Ping test can be used to determine if any of the required routes between a source and /
or destination route is missing.
o Use recovery procedure 4-14 - 15 in NTP 411-2133-949 to configure missing routes between NEs.
Check the GPS timing source
o Use the following to check for the presence and lock status of DOM GPS NORTELXX>show gps health
o If GPS health is not correct check GPS hardware
Check the SNTP Time from GPS
o All network elements require a timing source GPS etc.

Nortel Networks Confidential and Proprietary

131

For DO-RNCs the timing source must be the IP address of a DOM serving as an SNTP server.
For DOMs the timing source must be GPS
o Use the following command to show the SNTP time NORTEL-XX>show sntp time
o If the NE is the DO-RNC and the SNTP time is incorrect see TroubleshootDO-RNC Time
Reference on page 4-31 in NTP 411-2133-949
Check if there are traffic channels between the DOM and RNC
o Check for Abis Peer NORTEL-XX#show abis peer
Check if there are sessions and connections in this DOM
o Check for Sector Carrier connections NORTEL-XX#show 1xevdo rn 10.10.10.10
counters
o Verify that Airlink Resources Currently Allocated is not equal to zero.
o

Other Events Associated with DO Access Failures


To accurately see where problems occur that result in Access Failures it is necessary to collect and
correlate AT and RNC logs.
RNC Time Stamps Note: It should be noted that the RNC log time stamps are given in GMT time,
therefore the timestamps will have to be adjusted for the local time zone in order to correlate the RNC
and AT logs.

Access Fails following Connection Drops Troubleshoot connection drop symptoms first.

Access Fails due to insufficient cell coverage radius Data connection access failures due to ATs
being in the fringe of coverage.

Coverage radius statistics can be determined using the Cell Radius Tool in NEWTUN.

Poor C/I and high


TX Power

In the event that per sector carrier access failures are occurring due to poor RF coverage at an average
distance, steps should be taken to deny access to the site beyond the pre determined coverage radius.
o Collect RNC logs with severity set to 16.
o The distance from the serving cell can be determined by reviewing the last Route Update Detail
prior to an Access Failure.
o Within the Route Update Detail there is a parameter called EPNO that gives the RTD value
which can be used to determine the distance from the site prior to failure.

Nortel Networks Confidential and Proprietary

132

o
o
o
o

Calculate the average RTD for all Access Failures on a per DOM basis. This value will be used to
set the Cell Radii for Cells with a high AF rate.
To modify a carrier associated with a particular DOM open the DOEMS and go to the DOM to
be modified by opening the Network Database in the left hand pane under DOEMS.
Expand the Network Database and select Carriers.
In the Right-hand pane select the appropriate DOM according to IP address.

From the IS856 Carrier Menu select Modify Configuration


Change the Cell Radius parameter to the proper number of chips to limit access to the sector
from outside a specific radii.
o Where (Meters = # Chips x 244m/chip) to specify access radius
o Once the table has been modified, Click Submit to save the changes.

o
o

10.5 Call Blocks


10.5.1 Data Collection Methods
10.5.1.1 AT Logs
Method of Collection:
1. Tests of Call Blocking rate should be performed in a loaded network under busy hour conditions and
should generally encompass all users on the system.
2. To get average per user Blocking statistics, configure autocall in Optis-M or the data connection tab
in Invex3G to perform FTP downloads of a 100KB test file (small files so number of connection
attempts will be statistically significant). Insure that the idle / Wait timers in the data collection setup
are set for 15seconds.
3. Select all EVDO attributes in the log mask.
4. Drive test the cluster to collect the AT logs.

10.5.1.2 RNC Logs


Method of Collection:
1. Go to the DO-EMS and configure the log facility manager to collect Control Logs as follows:
Severity
Level
13

Component

ID

DO-RNC CPUs

Call Control

009

RNSM and SC

Nortel Networks Confidential and Proprietary

Message
ID
231

Description
No RC Resources to
Allocate
133

2. RNC logs should be collected simultaneously with AT logs. The RNC logs can be post processed
with NEWTUN on a per RNC basis to determine the number of RC (Resource Control) blocks that
occurred.
3. Drive the cluster route and stop the RNC logs at the conclusion of drive testing.

10.5.1.3 Blocking OMs


Method of Collection:
1. The following OMs measure the airlink resource allocation performance for connection setups and
soft / Softer handoff only. With release 3.0 call admission control is moved from the DO-RNC to the
DOM therefore, setup failure OMs due to resource allocation issues should be from DOM specific
metrics.
2. In order to setup data collection, follow the guidelines in section 5.4.4 to setup the data collection
configuration with the RNCPerfBySectorCarrier_R3.0 template selected.

10.5.2 Blocking KPI Calculation


10.5.2.1 KPI Value and Tolerance Factor
This is operator specific and what the System was designed to meet. Need to define different Blocking
rates for different entities within the System. Channel Elements, CPU, Connection Limits, Session
Limits. Can this be used for troubleshooting as well as provisioning? There are datafill parameters
{connection limits} than can affect blocking. FDD for remote sectors also affects this KPI.

10.5.2.2 Calculating KPI from AT Logs


1.From EVDO RFO:
Blocking% = DOBlocks

(DOAtts DOAccessFails Handoffto1x DOEOF_ParialAcces EOF Re l )

10.5.2.3 Calculating KPI from RNC Logs


1.Process the RNC logs with NEWTUN and open the statistics view.
2.Click on the Access Terminals Data and Session tabs and note the number of blocks total for each.
3.The following equation can be used to calculate the blocking percentage during drive testing.

X 100
Blocking % = DataBlocks + SessionBlocks
TotalCallAttempts

10.5.2.4 Calculating KPI from OMs


1.The following formulas can be used to determine the blocking rate at the RNC or the Sector Carrier
level.
2.Connection Setup Failures due to unsuccessful DO-RNC resource allocations can be determined from
the OMs as follows:
Percentage of Connection Setup Failures due to Unsuccessful DO-RNC
Resources Allocation =
100% *
(numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRncResources) /
(numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources +
numConnSetupsFailedRncTimeout +
numConnSetupsFailedRuTimeout +
Nortel Networks Confidential and Proprietary

134

numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted)
3.Connection Setup failures due to unsuccessful DOM resource allocations can be determined from the
following OMs:
Percentage of Connection Setup Failures due to Unsuccessful DOM
Resources Allocation =
100% *
(numConnectionSetupsBlockedByRn +
numConnectionSetupsFailedByRn) /
(numConnectionSetupsBlockedByRn +
numConnectionSetupsBlockedByRncResources +
numConnectionSetupsFailedByRn +
numConnectionSetupsFailedByRncResources +
numConnSetupsFailedRncTimeout +
numConnSetupsFailedRuTimeout +
numConnSetupsFailedTccTimeout +
numConnSetupsFailedSWError +
numConnSetupsAborted)
4.Airlink Resources allocation blocks during connection setup and soft / softer handoff on Primary
DOMs can be determined from the following OMs:
Airlink Resource Blocks (Connection Setup +
Soft/Softer Handoff) on Primary DOM =
numAllocationBlockRnDriverResourceSC +
numAllocationBlockRnConnectionLimitSC
5.Airlink Resource allocation blocks during connection setup and soft / softer handoff on Secondary
DOMs can be determined from the following OMs:
Airlink Resource Blocks (Connection Setup +
Soft/Softer Handoff) on Secondary DOM =
numAllocationBlockRnDriverResourceSSC +
numAllocationBlockRnConnectionLimitSSC

10.5.3 Troubleshooting Guide


1.If the per RNC or Sector Carrier OMs indicate that blocking exceeds performance thresholds set forth in
the exit criteria, determine the top ten sector carriers for blocking.
2.The top DOM / sector carrier contributors to blocking can be determined from the RNC logs by
analyzing the Network Statistics in NEWTUN.

Nortel Networks Confidential and Proprietary

135

3.To determine the top contributors to blocking from the OMs, evaluate sector performance and determine
Sectors which contribute highly to DOM related resource allocation failures. Following Origination
Sector (ACH of which used for sending/receiving messages) OMs need to be utilized to isolate high
contributors.
sum of OMs:
numATConnectionBlockedByRnSC, numANConnectionBlockedByRnSC and
numFCConnectionBlockedByRnSC
need to be evaluated to identify sectors which experience higher blockings. As OMs are pegged against
these sectors for connection blocks even-if other sectors may be blocking when additional links were
setup during the connection setup, determine the neighboring sectors of these high running sectors.
4.Once neighboring sectors are identified, numAllocationBlockDriverResourceSC OR
numAllocationBlockDriverResourceSSC (for lack of Mac Index and CE),
numAllocationRnConnectionLimitSC OR numAllocationRnConnectionLimitSSC (for Max. allowed
airlinks configuration), numallocationBlockRnSectorCarrierDownSC OR
numallocationBlockRnSectorCarrierDownSSC, numAllocationBlockRnModemTimeoutSC OR
numAllocationBlockRnModemTimeoutSSC ( for internal timeout between SC and Modem), and
numAllocationBlockRnNoConnectionSC OR numAllocationBlockRnNoConnectionSSC (for not being
able to find existing connection during softer handoff) OMs need to be evaluated and appropriate actions
need to be taken to solve the performance issues.
5.Percentage Contribution of TCC Timeouts to Connection setup failures due to RF Failures (Level 3, per
RNC). If this metric is the highest contributor among level 3 metrics, it indicates the issues related to TCC
timeouts.
a.

TCC timeout happens when Call Control on RNC times out waiting for Traffic Channel
Completes message from the AT.

b.

Higher TCC timeouts indicates that the RF conditions were not good enough for the TCC
message to reach the DOM or TCA message reach the AT or TCC Wait Timer parameter value
is not good enough. Link budget need to be verified for the forward and reverse link coverage
balance. Power used (configurable parameter) for sending TCA need to be verified

6.Percentage Contribution of Route Update Timeouts to Connection setup failures due to RF Failures
(Level 3, per RNC). If this metric is the highest contributor among level 3 metrics, it indicates the issues
related to RU timeouts.

Nortel Networks Confidential and Proprietary

136

a.

RU timeout happens when Route Update message is not received by the RNC from AT within a
predefined time interval. The timer used for this event is not configurable.

b.

Higher RU timeouts indicates that the RF conditions were not good enough for the RU message
to reach the DOM. Other contributing factors to this event exists but could not be fixed by us
(needs to have design take care of the software). Link budget need to be verified for the forward
and reverse link coverage balance

7.Percentage Contribution of Software errors to Connection setup failures due to Miscellaneous Failures
(Level 3, per RNC)
a.

If this metric is the highest contributor among level 3 metrics, it indicates the issues related to
software errors. Software errors occur when the memory pool usage reaches the maximum
1500 connections and failure to create HDR instance occur

8.Percentage Contribution of Software errors to Connection setup failures due to Miscellaneous Failures
(Level 3, per RNC)
a.
b.
c.
d.
e.

If this metric is the highest contributor among level 3 metrics, it indicates the issues related to
aborted connection setups due to following
Route update message with Keep = No for all pilots received and Enable Access when last RU
has all keep = No is set to false
Failure to allocate memory from Heap
Local close request is generated by operator
Page timer expires and AT initiated connection setup was not in progress.

9.From CLI determine if the blocking is related to the maximum number of connections being exceeded:
NORTEL-XX>enable NORTEL-XX# show 1xevdo sector carrier <IP> < channel number> <pn
offset> counters all
RN Resource Allocation Counters
Number of Requests Sent: 2
Successful Allocations: 2
Blocks:
Driver: 0
Connection Limit: 0
Sector Down: 0
Modem Timeout: 0
No Connection: 0
Malformed message: 0
10. The connection limit in the DOM is determined from two parameters: Maximum Forward Distribution
Delay and Connection Limit per sector element. The DOM logic sets the connection limit to the
minimum of the two of these parameters. In addition, the RNC Maximum Airlink Connnections per
Sector Carrier overrides these two parameters [3].
11.The following is an overview of how the DOM connection limit algorithm works.
12.While setting up a connection, the configured values for Connection Limit per sector at the DOM and
Maximum Airlinks per SectorCarrier are checked separately, in the following order:
Step 1: Check against RNC Maximum Airlinks Per SectorCarrier setting. If there is a block,
totalBlockedAirlinkRsrcAllocationsSectorCarrier is pegged and setup is aborted.
Step 2: Check against DOM per-sector Connection Limit setting. If there is a block,
numAllocationBlockRnConnectionLimitSC is pegged and setup is aborted.

Nortel Networks Confidential and Proprietary

137

Step 3: Send allocation request to modems (driver). It will check for number of channel elements
available. If is a block, numAllocationBlockRnDriverResourceSC is pegged and setup aborted
(setup aborted meaning the next step is not executed).
Here are some examples to illustrate how this algorithm works:
With RNC Maximum Airlinks per SectorCarrier = 40, the DOM Connection Limit per sector =
48, and Number of Channel Elements = 30, a block at 3rd step of the algorithm will occur for 31st
connection
With RNC Maximum Airlinks per SectorCarrier = 30, the DOM Connection Limit per sector =
48, and Number of Channel Elements = 30, a block at 1st step of the algorithm will occur for 31st
connection
With RNC Maximum Airlinks per SectorCarrier = 48, the DOM Connection Limit per sector =
30, and Number of Channel Elements = 30, a block at 2nd step of the algorithm will occur for 31st
connection
13. Verify from the RNC RF datafill that the Maximum Airlinks per sector carrier is according to the Nortel
recommended value of 48.
NORTEL-XX>enable
NORTEL-XX#show 1xevdo config
Maximum Airlinks per Sector-Carrier 48
14. If the value is below recommended it can be reset through CLI as follows:
NORTEL-XX(config)#1xevdo max-airlinks-per-sector-carrier 48
15. Verify from the DOM RF datafill that the connection limit is at the recommended value of 48 for the
sector carrier being considered. If the value is less than the recommended value use the following CLI
command to adjust the value:
NORTEL-XX>enable
NORTEL-XX# configure
NORTEL-XX(config)# sector-element element1
NORTEL-XX(config-element)# connection limit 48
16. If Blocking persists there may be provisioning issues. Contact network engineering for a verification
provisioning of the network elements.

10.6 Connection Drops


10.6.1 Data Collection Methods
10.6.1.1 AT Logs
Method of Collection:
The data collection process to troubleshoot connection drop problem is as follows:

Use one AT and OPTis-M or Invex3G to collect the AT logs.

Setup the log masks to collect all EVDO attributes. If the hybrid mode testing is to be performed
also collect all CDMA log attributes.

Following the process given in the Cluster drive testing section , configure the AT to download a
500 MB test file wait for a period greater than the data inactivity timer (datafilled in the system
to allow the connection to transition to a dormant state between tasks. Repeat the download over
and over until at least 30 minutes worth of AT logs have been collected. Please see the example
below.

Nortel Networks Confidential and Proprietary

138

Drive the cluster where drops are occurring to collect AT logs. Keep the drive test route within one
RNC boundary.

10.6.1.2 RNC Logs


Method of Collection:
1.
2.

Follow the RNC logging setup procedure in section 5.3.1 to configure logging.
The most important log to collect is:

Severity
Level
5
3.

Component

ID

DO-RNC CPUs

Call Control

009

RNSM and SC

Message
ID
423

Description
Connection Drop

This message contains useful information such as: drop reason (FTC lost/RTC Lost), frame error
rate, and the last 2 sets of TCA and Route Update messages. An example of Message 423 is shown
below:

Connection Drop : Drop Reason : No Reverse Traffic Channel


Connection Drop : Frame Error Rate 0.9401 (%)
Connection Drop : Dump of Last 100 Eb/No Values (0.125 dB)
Connection Drop : Last TCA message Details : TCA Seq# 1 sent in response to RouteUpdate Seq# 29
Connection Drop : Details of TCA Seq# 1
Connection Drop : TCA 1 : Channel Included N : Channel 0x0 : Frame Offset 0x8
Connection Drop : TCA 1 : DRCLength 4 : DRC Channel Gain -6 : Ack Channel Gain 6 : NumPilots 1
Connection Drop : TCA 1 : PN Offset 36 : Softer Handoff 0 : MAC Index 63 : DRC Cover 1 : RAB Length 32 : RAB Offset 2
Connection Drop : Details of RouteUpdate Seq# 29
Connection Drop : RU 29 : PN Offset 36 : PN Phase 0 : Strength 2 : Keep Y
Connection Drop : RU 29 : Channel Included N : Channel 0|0|73 : RN IP 10.140.96.2
Connection Drop : RU 29 : PN Offset 32 : PN Phase 2048 : Strength 63 : Keep N
Connection Drop : RU 29 : Channel Included N : Channel 0|0|73 : RN IP 10.140.96.2
Connection Drop : Previous TCA message Details : TCA Seq# 0 sent in response to RouteUpdate Seq# 27
Connection Drop : Details of TCA Seq# 0
Connection Drop : TCA 0 : Channel Included N : Channel 0x0 : Frame Offset 0x8
Connection Drop : TCA 0 : DRCLength 4 : DRC Channel Gain -6 : Ack Channel Gain 6 : NumPilots 2
Connection Drop : TCA 0 : PN Offset 36 : Softer Handoff 0 : MAC Index 63 : DRC Cover 1 : RAB Length 32 : RAB Offset 2
Connection Drop : TCA 0 : PN Offset 32 : Softer Handoff 1 : MAC Index 62 : DRC Cover 2 : RAB Length 32 : RAB Offset 1
Connection Drop : Details of RouteUpdate Seq# 27
Connection Drop : RU 27 : PN Offset 36 : PN Phase 0 : Strength 6 : Keep Y
Connection Drop : RU 27 : Channel Included N : Channel 0|0|73 : RN IP 10.140.96.2
Connection Drop : RU 27 : PN Offset 32 : PN Phase 2048 : Strength 11 : Keep Y
Connection Drop : RU 27 : Channel Included N : Channel 0|0|73 : RN IP 10.140.96.2

4. Some other logs that may be useful are:


Severity
Level

Component

ID

5
9
9
16

Call Control
Call Control
Call Control
Call Control
Call Control

9
9
9
9
9

DO-RNC
CPUs
RNSM and SC
RNSM and SC
RNSM and SC
RNSM and SC
RNSM and SC

9
Nortel Networks Confidential and Proprietary

Message
ID

Description

213
215
216
220

Connection Closed
Connection Request
Traffic Channel Complete Message
Local Connection Close Event : Reason #
Rx from AT : Connection Close Message :
Reason #

221

139

Call Control
9
9
9

Call Control
Call Control
Call Control

9
9
9
9

RNSM and SC
RNSM and SC
RNSM and SC
RNSM and SC

266
269
326
429

Tx to AT : Connection Close Message :


Reason Code #
Traffic Channel Assignment Message
Add/remove SHO leg
Last N Eb/No values after a connection
drop occurs

10.6.1.3 Dropped Connection OMs


Method of Collection:
1.Setup the OM data collection as outlined in section 5.4.4.1 using the RNCPerfByRNC_R3.0
template.
2.To get a statistically significant trend of performance, particularly in a market that is already
commercial it will be useful to collect OMs over an entire week.
3.Dropped connection OMs can be classified by:
RF Related Drops No FTC or RTC Lost
Handoff Related Drops SHO Blocked by RNC Resources, SHO Blocked by RN
(DOM), SHO Failed By RNC Resources, SHO Failed by RN, SHO Failed due to TCC
Timeout, or SHO Failed to RNC Timeout
Miscellaneous Drops Close Internal Error, Close RLP, State Mismatch, Sector Down,
or SSM Disable
4.The Connection Drop OMs are defined below:
numConnectionCloseNoFTC: The number of connections that were closed by DO-RNC
because of indications that there is no active Forward Traffic Channel.
numConnectionCloseRTClost: The number of connections that were closed by DORNC because of indications that the reverse link(s) were lost.
NumRevLinkSHOFailedTccTimeout: The number of reverse link soft handoffs that
failed because the Traffic Channel Complete message was not received from the Access
Terminal in time.
NumConnectionCloseStateMismatch: The number of connections that were closed by
DO-RNC due to state mismatch. For example, receiving a Connection Request message
from the Access Terminal when the DORNC is in Connected State.
NumRevLinkSHOfailRncTimeout: The number of reverse link soft handoffs that failed
because resource allocation/release on the DO-RNC did not complete in time.
NumRevLinkSHOFailedByRncResources: The number of reverse link soft handoffs
that were failed by DO-RNC resource allocation.
NumRevLinkSHOFailedByRn: The number of reverse link soft handoffs that were
failed by RN resource allocation.
NumRevLinkSHOBlockedByRn: The number of reverse link soft handoffs that were
blocked by RN resource allocation. The resources considered are the channel elements,
number of links per sector, and the MACIndices.
NumRevLinkSHOBlockedByRncResources: The number of reverse link soft handoffs
that were blocked by DO-RNC resource allocation. The only resource checked is the
number of airlinks per sector.
NumConnectionCloseFromATError: The number of Connection Close messages from
the AT that had a reason code of Error. It is up to the AT implementation.
NumConnectionCloseSectorDown: The number of connections that were closed by DORNC because of indications that a sector associated with the connection has changed state
to down.
NumConnectionCloseRlp: The number of connections that were closed by DO-RNC at
the request of the Radio Link Protocol due to errors..
NumConnectionCloseSlp: The number of connections that were closed by DO-RNC at
the request of the Signaling Link Protocol due to errors.

Nortel Networks Confidential and Proprietary

140

NumConnectionsOpened: The number of connections opened successfully on this DORNC (the AT arrives on the Traffic Channel).

10.6.2 Dropped Connection KPI Calculation


10.6.2.1 KPI Value and Tolerance Factor
The typical Connection Drop Rate value from drive testing is <= 3% CI
The typical Connection Drop Rate value from OMs is 4% 1%

10.6.2.2 Calculating KPI from AT Logs


1.Normal calls are ended by a Connection Close message from the AT or the AN with the reason =
Normal Close. Connection Closes with Reason other than Normal Close are considered connection
drops (e.g. forward Connection Close message with reason = Connection Error or QCP Connection
Release packet with reason = System Lost).
2.To calculate the connection drop rate, process the AT logs with EVDO RFO, then look at the Statistics
Tool Calls to find the number of connection drops.
Connection Drop Rate

= (Method #1) DO Drops / Number of DO Connections


= (Method #1) DO Drops / (DO Drops + DO Completions) or
= (Method #2) DO Drops / Effective Calls
where:

Effective Calls = Total Elapsed Active Time / Average Data Call Holding Time
Example: Method #1 from EVDO RFO Statistics

Method #1 - Connection Drop Rate = 11 / (11 + 45) = 19.6%

Example: Method #2 Using Data Call Holding time

Nortel Networks Confidential and Proprietary

141

o
o
o

Method #2 Connection Drop Rate


Take the Time value provided in the RLP statistics window and divide it by a data call folding
time (typical length of time a user has an active connection) DCHT = 30s.
o 4717s/30s = 157.2 effective calls
To determine the drop call rate, take the number of drops from the call statistics window (DO
Drops) divided by the number of effective calls:
o 11 / 157.2 = 7%

10.6.2.3 Calculating Dropped Connections KPI from RNC Logs


1.After the .bin files are downloaded from the RNC post process the files with NEWTUN.
2.To view the statistics, open the database where the processed results were saved.
3.Open the Access Terminal statistics and sort by the test ATs IMSI or ESN.
4.The Connection Drop Rate is calculated individually for Data and Session related connections and is
given as a Total = Data + Session drops. The following equations show the dropped connection rate
from the RNC logs:

x100
DroppedConnection% = Drops
Successful
Accesses

Incomplete
Traffic

5. The Dropped Connection Rate should be within the exit criteria specifications

10.6.2.4 Calculating Dropped Connection KPI from OMs


Connection drops typically referred to as abnormal connection closes. These typically occur as a result of
loosing the RF link or other error conditions
(sNumRFReleatedDrops +
cNumRFRelatedDrops) =
numConnectionCloseRtcLost +
numConnectionCloseNoFtc

Nortel Networks Confidential and Proprietary

142

(sNumSoftHandoffReleatedDrops +
cNumSoftHandoffRelatedDrops) =
numRevLinkSHOBlockedByRncResources +
numRevLinkSHOBlockedByRn +
numRevLinkSHOFailedByRncResources +
numRevLinkSHOFailedByRn +
numRevLinkSHOFailedTccTimeout +
numRevLinkSHOFailRncTimeout
(sNumMiscDrops +
cNumMiscDrops) =
numConnectionCloseInternalError +
numConnectionCloseRlp +
numConnectionCloseStateMismatch +
numConnectionCloseSectorDown +
numConnectionCloseSsmDisable
Total Dropped Connection Rate (DC) =
(cNumRFRelatedDrops +
cNumSoftHandoffRelatedDrops +
cNumMiscDrops) /
(cNumConnectionSetupSuccess +
numSToCCrossovers)
Where: numSToCCrossovers number of connections opened before the demarcation point, and were
closed normally after the demarcation point.

10.6.3 Dropped Connection Troubleshooting Guide


The typical causes of connection drops are as follows:
Description
Symptoms
in Data
(AT
Logs/RNC
Logs/OMs)

Poor RF coverage (probably want to sub-divide into fwd/rvs link, edge of network vs.
internal)
AT LogsMobile Rx Power < -85 dBm
Mobile Tx Power > -10dBm
Traffic C/I <= -2dB

Drop

RNC LogsUse EVDO remote optimization scripts to process and analyze RNC logs. The scripts will
be available as a module in the Nortel version of EMS parser due for release in June 2005.
Currently, the scripts are stand alone and are available on a case by case basis for
Nortel Networks Confidential and Proprietary

143

optimization. The following output from the CROS scripts shows a drop due to coverage
related issues:
RNC :
Drop
File

Drop
Num

HMS

23

49

####
#

Fail
Trigger
Connection
Drop Log

23

50

####
#

Connection
Drop Log

Drop
or AF
FCA
Drop

Estimated
Root Cause
Whole call not logged so
unclassified
Fwd and Rvs coverage both
bad, genuine coverage issue
Connection Suicide

Poor Forward link coverage typically will result in a dropped connection when the Ec/Io
(Strength) of all pilots is less than -9dB.
Example from RNC Logs
MOB CSMOpen] : Rx from AT : Route Update Message : Seq# 7
MOB CSMOpen] : Route Update Detail : Processing Seq# 7 : Connected :
RefPN 228 : RefStrength 37 : RefKeep Y RefStrength x -0.5 = -18.5dB
MOB CSMOpen] : Route Update Detail : RefPN IP Address 10.245.36.39
MOB CSMOpen] : Route Update Detail : Other Pilot# 1 : PNPhase 18942 :
Strength 63 : Keep Y : ChanIncluded N : Channel 0|00 Strength x -0.5 = 31.5dB
MOB CSMOpen] : Route Update Detail : PNPhase 18942 : PNOffset 296 : RN
IP 10.245.37.2 : EPNOin4Chips 0
FlowControl 2 Open] : Rx : Route Update Event event
MOB CSMOpen] : Counts after formMobility: Add (0) : Keep (2) : Delete (0) :
Keep(2) : Delete (0)
SHO SHOSM_Open] : [IP:10.245.36.39, CR:0x00000aee, RNC Connection
ID:0x00103965] : Rx from RN : FTC Stopped Indication Message : Success
SHO SHOSM_Open] : [IP:10.245.36.39, CR:0x00000aee, RNC Connection
ID:0x00103965] : Rx from RN : FTC Desired Indication Message : Success
SDU CSMOpen] : Setting Active soft handoff leg [IP:10.245.36.39,
CR:0x00000aee, RNC Connection ID:0x00103965] : DRC Mask 0x00000004
SHO SHOSM_Open] : [IP:10.245.36.39, CR:0x00000aee, RNC Connection
ID:0x00103965] : Rx from RN : FTC Stopped Indication Message : Success
SHO SHOSM_Open] : [IP:10.245.37.2, CR:0x00000aee, RNC Connection
ID:0x000f2c54] : Rx from RN : RTC Lost Status Indication Message :
DRCMask:0x00000002
SHO SHOSM_Open] : [IP:10.245.36.39, CR:0x00000aee, RNC Connection
ID:0x00103965] : Rx from RN : RTC Lost Status Indication Message :
DRCMask:0x00000004
CSM CSMOpen] : Rx : Local Connection Close Event : Reason 9
CSM CSMOpen] : Tx to AT : Connection Close Message : Reason Code 2
Solution

For areas where dropped connections result from poor RF coverage changes to the
underlying 1xRTT network may be required. Any changes to the underlying network will
have to be coordinated with the customer.

Comments

Nortel Networks Confidential and Proprietary

144

Description
Symptoms
in Data
(AT
Logs/RNC
Logs/OMs)

Slow handoff
AT Logs
N/A
RNC Logs

Observe the timestamps between last TCA and Traffic Channel Complete Message from AT not received
in time. If the delta between the two is greater than the TCC wait timer it may be necessary to increase this
value especially if the TCC is received after the timeout.
OMs
Percentage of Reverse Link Soft Handoff due to RF Failure =
100% *
numRevLinkSHOFailedTccTimeout /
(numRevLinkSHOBlockedByRn +
numRevLinkSHOBlockedByRncResources +
numRevLinkSHOFailedByRn +
numRevLinkSHOFailedByRncResources +
numRevLinkSHOFailRncTimeout +
numRevLinkSHOFailedTccTimeout +
numRevLinkSHOAborted)
Solution
Comments

Verify Neighborlists, Increase TCC wait Timer.


Slow Handoff When a TCA is sent but no TCC is received.

Description
Symptoms in Data
(AT Logs/RNC
Logs/OMs)

Full active set - no mechanism to select best 6 if 7 or more requested ( Fixed in 3.0)
AT Logs
RNC Logs
Look for Route update messages in the logs where the number of other pilots listed
is six and the SHO SHOSM Failed.

Nortel Networks Confidential and Proprietary

145

: Route Update Detail : Processing Seq# 142 : Connected : RefPN 476 : RefStrength 7 : RefKeep Y
: Route Update Detail : RefPN IP Address 10.245.37.59
: Route Update Detail : Other Pilot# 1 : PNPhase 28433 : Strength 42 : Keep Y : ChanIncluded N : Channel 0
: Route Update Detail : Other Pilot# 2 : PNPhase 23523 : Strength 38 : Keep Y : ChanIncluded N : Channel 0
: Route Update Detail : Other Pilot# 3 : PNPhase 20468 : Strength 47 : Keep Y : ChanIncluded N : Channel 0
: Route Update Detail : Other Pilot# 4 : PNPhase 26139 : Strength 63 : Keep Y : ChanIncluded N : Channel 0
: Route Update Detail : Other Pilot# 5 : PNPhase 27183 : Strength 63 : Keep Y : ChanIncluded N : Channel 0
: Route Update Detail : Other Pilot# 6 : PNPhase 27904 : Strength 63 : Keep Y : ChanIncluded N : Channel 0
: Route Update Detail : PNOffset 444 : PNPhase 28433 : RN IP 10.245.37.61 : EPNOin4Chips 7
: Route Update Detail : PNOffset 368 : PNPhase 23523 : RN IP 10.245.37.72 : EPNOin4Chips 0
: Route Update Detail : PNOffset 320 : PNPhase 20468 : RN IP 10.245.37.30 : EPNOin4Chips 0
: Route Update Detail : PNOffset 408 : PNPhase 26139 : RN IP 10.245.37.67 : EPNOin4Chips 9
: Route Update Detail : PNOffset 424 : PNPhase 27183 : RN IP 10.245.37.71 : EPNOin4Chips 14
: Route Update Detail : PNOffset 436 : PNPhase 27904 : RN IP 10.245.37.61 : EPNOin4Chips 3
: Rx : Route Update Event event
: Created PBTSCM node [IP:10.245.37.72, CR:0x00000aee
: Created PBTSCM node [IP:10.245.37.61, CR:0x00000aee
: Created PBTSCM node [IP:10.245.37.30, CR:0x00000aee
: Created PBTSCM node [IP:10.245.37.67, CR:0x00000aee
: Created PBTSCM node [IP:10.245.37.71, CR:0x00000aee
: Counts after formMobility: Add (6) : Keep (1) : Delete (0)

Solution

OMs
numRevLinkSHOFailedByRn
This problem is not readily recognized in the OMs as it will be aggregated with
other SHO failure reasons.
Patch 2.1.0.25 (Performance Patch) will fix the 7th Pilot issue by limiting the number
of pilots to be added to the active set such that it never exceeds 6. At most (6
(Current Active Set Size)) pilots may be added to the Active Set. Pilots will be
chosen according to signal strength.

Comments
Description
Symptoms in Data
(AT Logs/RNC
Logs/OMs)

InterRNCActiveHandoff Optimization Unhomed or Unknown Pilot


Unknown Pilot - If the received RouteUpdate message contains a pilot that does not
appear in the neighbor-list of any of the pilots in the ATs current active-set or in the
neighbor-list of any of the pilots in the RouteUpdate message being processed, the
pilot will be ignored. The soft handoff, however, will not fail just because of the
presence of the unknown pilot in the RouteUpdate message.
Unhomed Pilot - If the received RouteUpdate message contains a pilot that is in the
neighborlist of any of the pilots in the ATs current active-set or in the neighbor-list
of any of the pilots in the RouteUpdate message being processed but the DOM
owning the sector is not homed to the DO-RNC, the pilot will be ignored. The soft
handoff, however, will not fail just because of the presence of the unhomed pilot in
the RouteUpdate message.
RNC Logs

In the logs above the Route update is for PNs 93, 45, and 426. The source has no
knowledge of the PNs 93 and 426. It is likely that these PNs live on another RNC.

Nortel Networks Confidential and Proprietary

146

OMs
Number of Times an Unknown Pilot in RU when a Primary Pilot Is
Strongest =
numPilotLookupFailuresUnknownPilotSC
Number of Times an Unknown Pilot in RU when a Primary Pilot Is
Strongest =
numPilotLookupFailuresUnknownPilotSSC

When this OM is pegged, the neighbor list needs to be examined and


optimized. The pilot lookup failure logs from the DO-RNC should then be
used for detailed information.
Number of Times an Unhomed Pilot in RU when a Primary Pilot Is
Strongest =
numPilotLookupFailuresRNNotHomedSC
Number of Times an Unhomed Pilot in RU when a Primary Pilot Is
Strongest =
numPilotLookupFailuresRNNotHomedSSC
When this OM is pegged, the secondary homing configuration needs to be
examined and optimized. The pilot lookup failure logs from the DO-RNC
should then be used for detailed information.
numAllocationAttemptsTxRnSC and
numRevLinkSHOAddAttemptsSC
numPilotLookupFailuresRNNotHomedSC
numAllocationAttemptsTxRnSSC and
numRevLinkSHOAddAttemptsSSC
numPilotLookupFailuresRNNotHomedSSC
Solution
Comments

Descriptio
n
Symptoms
in Data
(AT
Logs/RNC
Logs/OMs)

Add source RNC as secondary RNC in interfering DOMs secondary Homing table.
Add adjacent (interfering) DOM to the neighborlist of source sector carrier.
In 3.0, the existence of alien pilots in the route update message will not cause the
call to fail but will contribute to interference which can cause drops and impede
maximum sustained data rates.
Hybrid AT non-returning tune away
AT Logs
In a long tune away the AT will release the EVDO connection after being tuned away for
longer than 5 seconds. To recognize long tuneaways from the message view in EVDO set the
message filter for the following:
1. CDMA 2000 f-csch (Paging Messages) - All
2. CDMA 2000 forward Traffic Channel Neighborlist Update
3. IS856 Access (Connection Request, Route Update)
4. IS856 Control Channel Directed (Traffic Channel Assignment, ACAck,
5. IS856 Control Channel Broadcast All overhead messages
6. IS856 Forward Traffic (Connection Close, Traffic Channel Assignment,
7. IS856 Reverse Traffic (Connection Close, Route Update, Traffic Channel Complete
8. QCP EVDO Connection Release, ARQ Effective Receive Rate, Connection Attempt
From RFO Map View:

Nortel Networks Confidential and Proprietary

147

AT has lost supervisory


communication with the
EVDO network

Example from Grid View:

No EVDO RF statistics for 5 seconds tuned away

Example from Message View below:

Nortel Networks Confidential and Proprietary

148

Nortel Networks Confidential and Proprietary

149

RNC LogsIn the RNC logs long tuneaways can typically be identified by an FTC Stopped Indication
Message (Null DRC from AT) followed by at least 5.12 seconds of inactivity in the RNC
messaging. In the example below it can seen that the stopped indication is sent and roughly 5
seconds later the AT communication is re-established with the DO network but the RTC is
lost as a result.
Example from RNC logs:
16 03-10-05
03:45:10.972
51 03-10-05
03:45:11.002
ca 03-10-05
03:45:11.002
(No activity)
57 03-10-05
03:45:15.882
50 03-10-05
03:45:15.882
08 03-10-05
03:45:15.882

Solution

SHO SHOSM_Open] : [IP:10.245.37.59, CR:0x00000aee, RNC


Connection ID:0x000d3650] : Rx from RN : FTC Stopped
Indication Message : Success
CSM CSMOpen] : Periodic Counters : RLP Tx 105.0 : SLP Tx
21.0 : RLP Rx 154.0 : SLP Rx 32.0 : RLP to A10 152.0
CSM CSMOpen] : Periodic Counters : Connection Duration :
00:00:22.579
(Tuned Away to 1x)
SHO SHOSM_Open] : [IP:10.245.37.59, CR:0x00000aee, RNC
Connection ID:0x000d3650] : Rx from RN : RTC Lost Status
Indication Message : DRCMask:0x00000002
CSM CSMOpen] : Rx : Local Connection Close Event : Reason
9 (Reason Code 9 = RTC Lost)
CSM CSMOpen] : Tx to AT : Connection Close Message :
Reason Code 2

OMs
numConnectionClosenoFTC Pegged when an FTC Stopped is received and an FTC
Desired message is not received within 5 seconds
Currently there is no complete fix for a drop due to a long tune away. Some theories on
insuring that the 1x Channel lists are consistent from BTS to BTS.

Comments
Description
Symptoms
in Data
(AT
Logs/RNC
Logs/OMs)

SHO block
RNC LogsIn the RNC logs a drop due to a SHO block may be identified by a RC lookup fail.
Example from RNC Logs:

MOB CSMOpen] : Rx from AT : Route Update Message : Seq# 68


MOB CSMOpen] : Route Update Detail : Processing Seq# 68 : Connected : RefPN 72 : RefStrength 27
MOB CSMOpen] : Route Update Detail : RefPN IP Address 10.245.37.41
MOB CSMOpen] : Route Update Detail : Other Pilot# 1 : PNPhase 2299 : Strength 19 : Keep Y : ChanI
MOB CSMOpen] : RC lookup failed : Dropping pilot from Route Update (PNOffset: 36, Phase: 2299
OMsnumRevLinkSHOFailedByRN number of reverse link soft handoffs failed due to resource
allocation.
numRevLinkSHOBloakedByRN - The number of reverse link softhandoffs blocked by RN
resource allocation.
numRevLinkSHOBlockedbyRNCResources The number of reverse link softhandoffs that
were blocked by DO-RNC resource allocation.
Solution
Comments

Nortel Networks Confidential and Proprietary

150

Hand down algorithm to 1xRTT kicks in and AT doesn't manage to send


connection close

Description
Symptoms in Data
(AT Logs/RNC Logs/OMs)
Solution
Comments
Description
Symptoms in
Data
(AT Logs/RNC
Logs/OMs)

Loss of Reverse Traffic Channel


A reverse traffic channel is determined lost when the RL Fade timer expires on the
DOM (there is one RL Fade timer for each of the soft/softer handoff leg). The RL
Fade timer is triggered when the DOM is unable to successfully demodulate nonerased DRC decisions from the AT. This timer is reset when non-erased DRCs are
received from the AT (the DRC cover for the DOM is not matched for this
indication). When RTCLost indication has been received from all the Down
Legs associated with the connection, it is treated as a loss of the reverse link
to the AN and the connection is closed.
AT Logs
Loss of the Reverse Traffic Channel is hard to be sure of when looking at the AT logs.
The best way to determine that this has occurred is to examine the Power Log
message and see that the ATs transceiver has turned off.
RNC Logs

OMs
numConnectionCloseRtcLost
numConnectionCloseRtcLostSlot
Solution

Increase the Reverse Link Fade Timer. Care must be taken when doing this as it may
waste resources when adequate RF does not exist to overcome a fading.

Comments

Nortel Networks Confidential and Proprietary

151

Description
Symptoms
in Data
(AT
Logs/RNC
Logs/OMs)

Missing neighbor
AT LogsN/A
RNC Logs

In the logs it can be seen from the Route Update Detail that the IP lookup failed for PN 348
and PN 351.
Solution

If the drop is in fact due to a missing neighbor the neighborlists for the reference site and its
neighboring sectors should be evaluated to insure all the required PNs are included and
that the proper IPs are also included.
Alternatively, if the requested PNs are from sites in an adjacent RNC insure that those
DOMs have the source RNC datafilled as a secondary RNC and that this DOM is in the
neighborlist of the source DOM.

Comments

Nortel Networks Confidential and Proprietary

152

Description
Symptoms in Data
(AT Logs/RNC
Logs/OMs)

PDSN Release
PDSN release A10 de registration
AT Logs
RNC Logs

When a connection to the AT is opened LCP requests are sent over the open
channel, but are not responded to since there is no PPP end point at the AT. The
connection will stay up until the PDSN times out (typically about 30 seconds +).
This condition will lead to an increase in dropped calls.
OMs
Percentage of A10 Connection Close due to PDSN Initiated Close =
100% *
numA10ClosedPDSNInitiatedRelease /
totalA10Closed
Solution

Comments

Insure that the UA10 minimization feature is enabled. The UA10 feature will
prevent unnecessary setup of A10 and thus PDSN timeouts. With UA10 enabled
the AT will only establish a PPP session with the PDSN when the following
events occur:

ULN unsolicited Location Notification after session parameters are


negotiated

Dormant RNC handoff when IP of PDSN that has the PPP session is passed
in the A13 Response message

In Mobile IP terminal when PPP re synch (Connection Request on dormant


or inter technology handoffs.
Characterized by A10 de registration after 30+ seconds.

Nortel Networks Confidential and Proprietary

153

Description
Symptoms
in Data
(AT
Logs/RNC
Logs/OMs)

Solution
Comments

GBU (Montana drop-off)


The delta in PN Ec/Io from best to worst is >=14dB.
RNC Logs

Max PN Ec/Io = -7.5dB Worst PN Ec/Io = -31.5dB delta = 24dB


No Current Solution
GBU (the good, the bad, and the ugly) usually results when the power control bits are not
read accurately.

10.7 FL Sector Switching Success Rate


10.7.1 Data Collection Methods
10.7.1.1 AT Logs
Not currently available from AT logs

10.7.1.2 RNC Logs


Not Currently Available from Statistics in NEWTUN

10.7.1.3 OMs
Method of Collection
1.Setup the OM data collection as outlined in section 5.4.4.1 using the RNCPerfByRNC_R3.0 template.
2.To get a statistically significant trend of performance, particularly in a market that is already
commercial it will be useful to collect OMs over an entire week.
3.If the market is not commercial perform drive test of each cluster on the golden route.

10.7.2 Forward Sector Switching Success Rate KPI Calculation


10.7.2.1 KPI Value and Tolerance Factor

1.Forward Sector Switching Success Rate should be in >=95% for the 90th percentile of all attempts.

10.7.2.2 Calculating KPI from AT Logs


10.7.2.3 Calculating KPI from RNC Logs
10.7.2.4 Calculating KPI from OMs
1.The following metrics only measure the forward sector switches between BTSs. Forward Sector
Switching between adjacent sectors is not visible to the DO-RNC.
Fwd Serving Sector Switching Successes = numTotalSuccessSHO
Nortel Networks Confidential and Proprietary

154

Fwd Serving Sector Switching Failures = numConnectionCloseNoFtc


Therefore,
Fwd Serving Sector Switching Success Rate =
numTotalSuccessSHO /
(numTotalSuccessSHO +
numConnectionCloseNoFtc)

10.7.3 Troubleshooting Guide


Description
Symptoms in Data
(AT Logs/RNC Logs/OMs)

FTC Desired Not Received in Time


OMs
numConnectionCloseNoFTC
When the currently serving DOM either receives a null rate DRC, null
DRC cover, DRC cover from another DOM or fails to demodulate a nonerased DRC from the AT (the QCOM ASIC driver on the DOM uses a
filter to avoid being excessively reactive to these situations while at the
same time ensuring that a legitimate DRC switch indication is handled in
time), the Forward Traffic Channel is stopped and an FTCStopped
indication is sent to RNC. On receiving the FTCStopped indication, the
RNC triggers the FTCDesiredWaitTimer (this would apply in the case of
the connection drop in normal DRC switching scenarios, the FtcStopped
indication could be paired with a pending FtcDesired indication from
another leg). If no FTCDesired indication (triggered by DOM receiving a
valid non-null or nonerased DRC from the AT, either on the same leg or a
different leg) is received by the time the FTCDesiredWaitTimer expires,
the RNC closes the connection and pegs this OM.
The same scenarios that cause this OM to peg could also cause
(numConnectionCloseRtcLost and numConnectionCloseRtcLostSlot) to
peg, however only one reason will be pegged for a single connection close
depending on whichever timer expires first. If an open connection needs
to be closed, the attributes (numConnectionCloseToAtError and
numConnectionCloseToAtErrorSlot) will also be incremented.
AT Logs

Solution

RNC Logs
Look for FTC Desired Not Received in Time.
Increase the FTC Desired Wait Timer. Insure neighborlist integrity and
reciprocity between BTS.

Comments

10.8 FL Sector Switching Time


10.8.1 Data Collection Methods
10.8.1.1 AT Logs
Method of Collection
1. Set AT to download a large file (>= 100 MB) over and over.
2. Drive test and collect AT log.
3. Collect all EVDO logging attributes

10.8.1.2 RNC Logs


Method of Collection

Nortel Networks Confidential and Proprietary

155

N/A

10.8.1.3 OMs
Forward Sector Switching performance metrics are only available for DOM to DOM (inter-DOM) sector
switching. Intra DOM sector switching is transparent to the DO-RNC. The following OMs are available
to calculate the switching success or failures between BTSs.
CLI can be used to determine the DRC switching successes between DOMs on a network or Session
specific basis as follows:
NORTEL-XX#show 1xevdo counters all (Network wide)
NORTEL-XX#show 1xevdo session <UATI> counters all (UATI / Session specific)

10.8.2 KPI Calculation


10.8.2.1 KPI Value and Tolerance Factor
From [12]: <200 ms for 90th percentile?

10.8.2.2 Calculating KPI from AT Logs


Currently, the forward link sector switching time can not be determined accurately from the AT logs. It
has to be done using RNC logs.

10.8.2.3 Calculating KPI from RNC Logs


From [12]:
1. In order to determine the amount of time required for the AT to switch between DOMs in a soft
switchover event, it is necessary to determine when one DOM stops sending data and the new DOM
begins.
2. The key indicators of a soft forward sector switch are as follows:
a. FTC Stopped or FTC Desired It should be noted that the initiation of a sector switching
event can be demarcated by either of the above messages appearing first, although FTC Stopped
and FTC Desired messages should have different RN IP addresses to denote two distinct DOMs.
b. The completion of the forward sector switching event concludes with the Setting Active soft
handoff leg [IP (same as that in the corresponding FTC Desired Message)]
3. Using a Unix emulator such as Cygwin perform:
a. Egrep (FTC Stopped |FTC Desired \Setting Active soft handoff leg) This will parse out
only the commands in the RNC logs where commands are sent by the AT telling the DOM to stop
sending data and the address of the new DOM.
b. Soft events are denoted by the UATI of the AT followed in the same line by the IP address of
the DOM sending the FTC Stopped or Desired message. Soft events are identified by seeing
different IP addresses for the FTC Stopped and FTC Desired DOMs.
4. Calculate the time delta between (FTC Stopped or the FTC Desired message, whichever comes
first) and the Setting Active soft handoff leg message for the UATI of the AT under test for all soft
switchover events and calculate the 90% percentile value
Message Router Consumer
[IP
[UATI
Message Router Consumer
Message Router Consumer
[IP
Setting Active soft handoff leg [IP
[IP
[UATI
[UATI

Rx from RN IP Address 47.135.201.245


47.135.201.245, CR
0xfc03e9]
Received TLV 18 fromRN IP Address
Rx from RN IP Address 47.135.201.249
47.135.201.249, CR
47.135.201.245, CR
47.135.201.249, CR
0xfc03e9]
0xfc03e9]

FTC Desired Indication Message


0xc65, RNC Connection ID
Received Stopped Indication
47.135.201.249
FTC Stopped Indication Message
0xc65, RNC Connection ID
0xc65, RNC Connection ID
0xc65, RNC Connection ID
Received a Set Desired indication, qId
Received a Start Transmission indication

0x300ca] Rx from R FTC Desired Indication Message


Destinatio127.1.3.1 on SF
0x300c8] Rx from R FTC Stopped Indication Message
0x300ca] DRC Mask 0x4
0x300c8] Tx to RN Flush FTCQueue Request Message
2, Frames
500

RNC Log Example for Forward Switching

Nortel Networks Confidential and Proprietary

156

10.8.2.4 Calculating KPI from OMs


Forward Sector Switching performance metrics are only available for DOM to DOM (inter-DOM)
sector switching. Intra DOM sector switching is transparent to the DO-RNC. The following OMs are
available to calculate the switching success or failures between BTSs.
Fwd Serving Sector Switching Successes = numTotalSuccessSHO
Fwd Serving Sector Switching Failures = numConnectionCloseNoFtc

10.8.3 Troubleshooting Guide


1. Cases where forward sector switches fail are most likely due to an RC resource allocation failure, TCC
(from AT) not received in time, or a tuneaway.
2. If the RNC logs show a high rate of TCC not received in time messages during forward sector
switchovers trying increasing the TCC wait timer.
TCC Wait Timer
Range

Unit

Recommended

1000 30000

1/1000
seconds

5000

NORTEL-07>enable
NORTEL-07#configure
Then the TCC Wait Timer can be changed through the following command
NORTEL-07(config)#1xevdo timer tcc-wait <TCC Wait Timer setting>
To verify the timer was changed, go back to the enable state and enter the following,
NORTEL-07#show 1xevdo config
The parameter is listed as CSM Traffic Channel Complete Wait Timer.

10.9 RL SHO Success Rate


In EVDO, reverse link soft handoff is initiated by the AT by sending a Route Update message requesting
the addition or removal of pilots from the Active set. Each route update is counted as a handoff attempt
regardless of the number of pilots in the message.

10.9.1 Data Collection Methods


10.9.1.1 AT Logs
Method of Collection:
1.Configure the AT log data collection tools to perform an FTP download of a large file (100Mbytes).
2.Perform a drive of the Cluster golden route
3.Post Process the collected data with EVDO RFO.
4.There are currently not any statistics in EVDO RFO for determining the RL SHO Success rate but
the number of attempts and successes are defined as follows:
Attempts = number of Route Updates on Reverse Traffic Channel with Num Pilots > 0
Successes = number of Traffic Channel Assignments on Forward Traffic channel (with pilots
from previous Route update) followed by the corresponding Traffic Channel Complete
message sent on the Reverse Traffic Channel.

10.9.1.2 RNC Logs


There is currently no easy way to determine the RL SHO success rate from RNC logs. However RNC
logs should be collected such that timing issues or resource allocation issues between the AT and the
RNC can be diagnosed.
Nortel Networks Confidential and Proprietary

157

Method of Collection:
1. Follow RNC logging setup given in section 5.3.1 of this document.
2. Collect logs at Severity level : 8

10.9.1.3 OMs
Method of Collection:
1. Follow the procedure in section 5.4.4.1 for configuring OM collection
2. The following OMs are available to measure the RL SHO attempts, successes and Success Rate:
Rev Link SHO Attempts = numRevLinkSHOAttempts
Rev Link SHO Successes = numRevLinkSHOSuccess
Successful Reverse Link SHO Rate= numRevLinkSHOSuccess / numRevLinkSHOAttempts
3. Reverse Link Soft handoff OMs can be determined from the CLI prompt using the following
commands:
NORTEL-XX# enable
NORTEL-XX# show 1xevdo counters all
NORTEL-XX# show 1xevdo session <UATI> counter all (per UATI based for specific ATs)

10.9.2 KPI Calculation


10.9.2.1 KPI Value and Tolerance Factor
Shall be >= 99% as measured via OMs.

10.9.2.2 Calculating KPI from AT Logs


The following procedure can be used to estimate the RL SHO Success Rate from the AT logs:
1. Open the Message View in EVDO RFO followed by the Message Set Editor.
2. From the Message Set editor select the following IS856 messages:
a. Access
i. Connection Request
b. Forward Traffic
i. Traffic Channel Assignment
ii. Connection Close
iii. RTC Ack
c. Reverse Traffic
i. Route Update
ii. Traffic Channel Complete
iii. Connection Close
d. QCP EVDO
i. Connection Release
3. Export the Message Set to Excel
4. Use Auto Filter to filter on Reverse Traffic Channel
5. Under MessageName use the Countif worksheet function to determine the number of Route
Updates and the number of Traffic Channel Completes and the number of Connection Requests.
6. There is typically a route update sent after the mobile is on traffic that is not necessarily for SHO. In
order to make the Rate estimate as accurate as possible count the number of CR to account for it and
subtract it from the total RUs.

RL _ SHO _ Success% = Successes(# TCC )


[Atts(# RU ) Connection Re q(# CR )]

10.9.2.3 Calculating KPI from RNC Logs


N/A

10.9.2.4 Calculating KPI from OMs


Successful Reverse Link SHO Rate= numRevLinkSHOSuccess / numRevLinkSHOAttempts
The RL SHO Success rate can be determined from CLI using:
Nortel Networks Confidential and Proprietary

158

NORTEL-XX# show 1xevdo counters all (Network specific)


NORTEL-XX# show 1xevdo session <UATI> counters all (AT / Session specific statistics)
RL Soft Handoff Success Rate = Successes / Attempts

10.9.3 Troubleshooting Guide


1. If the RL SHO success rate is less than 99% according to the OMs, the following events may have
caused the failure:
Resource Blocks:
RL Soft Handoffs Blocked by RN (Links per sector, MAC indices, or channel elements)
RL Soft Handoffs Blocked by RNC Resources (Number of Airlinks per sector)
Allocation Failures:
RL Soft Handoffs Failed by RN (DOM Resource allocation failure)
RL Soft Handoffs Failed by RNC Resources (RNC Resource allocation failure)
RL Soft Handoffs Fail RNC Timeout (TCC was not received from the AT in time)
2. From the OMs there are failure measurements that will help pin point failure reasons
There is no individual OM identifying the blocking or failure reasons such as
no MAC Index, no reverse link demodulator, or no forward link queue.
Percentage of Reverse Link Soft Handoff due to Unsuccessful DOM
Resources Allocation =
100% *
(numRevLinkSHOBlockedByRn + numRevLinkSHOFailedByRn) /
(numRevLinkSHOBlockedByRn +
numRevLinkSHOBlockedByRncResources +
numRevLinkSHOFailedByRn +
numRevLinkSHOFailedByRncResources +
numRevLinkSHOFailRncTimeout +
numRevLinkSHOFailedTccTimeout +
numRevLinkSHOAborted)
Percentage of Reverse Link Soft Handoff due to RF Failure =
100% *
numRevLinkSHOFailedTccTimeout /
(numRevLinkSHOBlockedByRn +
numRevLinkSHOBlockedByRncResources +
numRevLinkSHOFailedByRn +
numRevLinkSHOFailedByRncResources +
numRevLinkSHOFailRncTimeout +
numRevLinkSHOFailedTccTimeout +
numRevLinkSHOAborted)
3. From the CLI prompt perform the following commands to determine the root cause of handoff failures:
NORTEL-XX# show 1xevdo counters all
Reverse Link Soft Handoffs :Attempts :
Successes :
Blocked by RN :
Blocked by RNC :
Failed by RN :
Nortel Networks Confidential and Proprietary

0
0
0
0
0
159

Failed by RNC :
0
Failed because of no TCC :
0
Failed because of RNC Resource Timeout :
0
4. If any of the RL SHO failures are due to blocking refer to the troubleshooting section on call blocking.
5. The unsuccessful DOM resources allocation could be the result of improper setting of the maximum
airlinks per sector attribute in the DOM. See Per Sector Airlink Resources Allocation on page 13-1 and
DOM Resources Usage on page 16-1 of reference [18].
6. Any RL SHO failures due to Failed RNC Timeouts can only be due to the DOM being on a software
load prior to Release 3.0.
7. TCC timeouts may also be enhanced by increasing the TCC Wait Timer using CLI. The timer can be
configured in increments of 1 millisecond between 1000 to 30000 milliseconds. (default 5000)
NORTEL-XX> enable
NORTEL-XX#(configure)
NORTEL-XX(config) #1xevdo timer tcc-wait <time>

10.10 FL Per-User Throughput


10.10.1 Data Collection Methods
10.10.1.1 AT Logs
Per user FL throughput is determined by the RF characteristics reported by the AT. Prior to beginning
any throughput tests of the EVDO network, any pilot pollution and coverage issues associated with the
underlying CDMA network should be addressed.
Forward Link per user throughput data can be captured for the physical layer as well as the application
layer using Optis-M or Invex3G and post processing the log files with RFO. Tests of forward link per
user throughput can be performed in either mobility or stationary conditions. Stationary testing is
typically done to determine the maximum achievable rate per user.
Method of Collection
1. Set up the data collection tool (Optis-M or Invex3G) to perform ftp downloads of a 5MB
(5000000Bytes) test file.
2. Refer to [11], [12], and chapter 5.3 of this document for specific details regarding setting up the test
equipment to test throughput.

10.10.1.2 TCP Trace Logs


1.Ethereal or WinDump should be used to capture traces of TCP layer performance data and protocols
being negotiated during PPP session establishment.
2.Capture of TCP trace logs at PPP session setup will allow engineers to determine if IP header
compression is enabled on the PDSN.
Method of Collection
3. Download and install Ethereal (v 0.10.10) http://www.ethereal.com/distribution/win32/ on the
collection computer.
4. Start a new live capture prior to connecting the AT to the network.

Nortel Networks Confidential and Proprietary

160

5.

Connect the AT to the network and perform an FTP download.

6. At the end of the download stop the TCP trace capture. Ethereal will process the trace logs and
display the protocol messages in the workspace.
7. Insure that within the PPP IPCP protocol configuration request message options field, that IP
Header compression is not a requested option or that it is being rejected. (see below)
Example: No Request for Header Compression Option

Nortel Networks Confidential and Proprietary

161

Example: Request for Header Compression Option Rejected by PDSN

10.10.1.3 RNC Logs


Data available in the RNC logs does not contain any useful information about user throughput.

10.10.1.4 OMs
There are no OMs currently available for measuring per user throughput

10.10.2 KPI Calculation


10.10.2.1 KPI Value and Tolerance Factor
The KPI for FL per user throughput is measured at the application layer in order to determine the end
user experience. However, requested data rate is based on the DRC request rate and is determined by
the AT pursuant to local RF conditions.
In general, data rates of 1.5Mbps or greater should be achievable where the average DRC rate is
between 2.457Mbps and 1.2288kbps. The table shown below shows estimated values of equivalent
application layer throughput based on a percentage of DRC rate requests and other factors.
% Rate Requested

DRC Error Rate

DRC Rate (Mbps)

100
80
20
80
10
10

10%
10%
10%
10%
10%
10%

2.457
2.457
1.2288
2.457
1.843
1.2288

Application
Throughput (Mbps)
1.83
1.464
0.183
1.464
0.1372
0.0915

Delta from
Requested Rate
23.75%
25.5%
21.7%

10.10.2.2 Calculating KPI from AT Logs


1. After AT log data collection is complete, use EVDO RFO to post process the log files.
2. For application layer throughput statistics, in RFO select, Statistics tool followed by Data
Statistics then Application. (see below)
3. For physical layer throughput statistics and slot utilization rate, in RFO select the Statistics Tool
followed by Data Statistics and Forward Rate. (see below)

Nortel Networks Confidential and Proprietary

162

4. The Graph view in RFO can also be used to show Application, and Physical layer throughput versus
time.
Example: Application Throughput Statistics

Formula:

Application throughput (kbps) = (RX Data (Bytes) x 8) / 1000) / RX Time (sec)

Example: Physical Layer Throughput Statistics

Average Physical Layer Throughput => Avg Actual Data Rate (kbps)

10.10.2.3 Calculating KPI from RNC Logs


Not Applicable RNC logs contain no useable information for per user throughput at this time.

10.10.2.4 Calculating KPI from OMs


There are no specific OMs available for calculating per user throughput however, it can be estimated
using the FTC scheduler OM perfFLthruput. This OM is averaged over a sliding 10 second window.
The reported throughput is the average over the last 10 seconds of the data call.
To estimate the per user throughput perform the following:
1. From the data collected remove data from the sectors that only have a few active airlinks
(IS856SecElActiveConnections <=2 This OM is not currently collected but can be enabled by
adding a new data collection template) It is necessary to remove this information because the
estimated throughput will be artificially low because of the ratio of data to be sent to total number of
slots.
2. Use the remaining data to estimate the FL user throughput:
Formula: Average forward link per user Physical Layer throughput = Average of [Forward link
sector Physical Layer throughput (perfFLThruput) / Number of Airlinks on the sector] calculated
from the remaining data from the previous step.
3. The value obtained using this method is an estimate over all users. The individual estimate could
deviate from the estimate significantly.

10.10.3 Troubleshooting Guide


1. Follow the testing and troubleshooting flow chart given in Appendix A.

Nortel Networks Confidential and Proprietary

163

2. It should be expected that user throughput will suffer in areas at the fringe of coverage, inter RNC
borders, or where user traffic is high (scheduling contention).
3. Verify that he DRC requested rate and the Application Rate track within about 15% of one another. In
addition insure that the C/I and PER are valid for the data rates being requested. Also verify from the
EVDO Rate Statistics in XCAL, that the majority of DRC requests are between 1.2288 and
2.457Mbps.
Example: Avg. Effective DRC, Application RX Bytes, C/I1, and TC_PER

Nortel Networks Confidential and Proprietary

164

Example: Grid View App. RX Rate, Avg. Effective DRC, C/I, and TC_PER

Formula % difference DRC and App rate: ((Avg. Effective DRC App Rx Rate)/Avg. Effective DRC)
4. If the C/I and PER values are ideal (i.e. between 8 12dB) and the DRC rate is low, verify the port
settings in the data collection tool are correct. If using XCAL-DO go to My Computer Properties
Hardware Device Manager Modems and Ports. Verify the COM ports for the modem are those
configured in XCAL COM Port and Data Port Settings screens. For Optis-M, port configuration is
automatically detected.

Nortel Networks Confidential and Proprietary

165

Example: Device Manager Modem Port Configurations

Data Port

Control Port /
Diagnostics Port

Example: Port Configuration in XCAL-DO

5. If DRC and C/I are within ideal limits, verify Application throughput is between 1.5 and 2.1Mbps.

Nortel Networks Confidential and Proprietary

166

Example: Application Throughput from XCAL-DO

6. If the application throughput seen with the data collection tool is lower than expected. Perform a DOS
FTP download to confirm the achievable throughput on the FL. If the DOS throughput agrees with the
tool, verify that IP header compression is disabled. Go to: Network and Dialup Connections PPP
Adapter Advanced TCP/IP properties.
Example: IP Header Compression

7. To determine if there are per user throughput problems where coverage is generally expected to be
good, post process collected AT logs with RFO and verify that the C2I and DRC request rates track and
that the PER does not exceed 1%. Remember that the mobile will decrease the DRC request rate to
maintain PER at 1%. (see below)
8. Verify that the application throughput is approximately 15% less than that of the Forward physical rate.
An inspection of the graph view in RFO should indicate that application throughput trends less than that
of the physical layer. In addition to the graph view, an estimate of the difference in throughput can be
determined from the grid view by filtering out all the blank values in the Application RX Rate column
(see below).

Nortel Networks Confidential and Proprietary

167

Example: Graph View App RX Rate vs. Fwd Physical Data Rate

9. If the physical layer throughput is less than that of the application layer throughput and slot utilization is
low, software compression is probably enabled in the network and dial up settings on the collection tool
/ laptop. Software compression will also exaggerate the actual application layer throughput. Software
compression should be disabled prior to performing further tests.

Nortel Networks Confidential and Proprietary

168

Example: Grid View of Application layer vs. Physical Layer Statistics

Expected
Example Calculation: ((Fwd Physical Data RX Rate Application RX Rate) / Forward Physical Data
Rate) = 13.5%
10. An initial review of the application statistics in RFO may NOT indicate an immediate problem with
achievable data rate. However, a review of the physical layer statistics may show disparity in
throughput between the physical and application layer. Application layer throughput should always be
less than the Physical Layer rate.
Example: Data Statistics RX Active Throughput (App. Data Rate)

Nortel Networks Confidential and Proprietary

169

Example: Data Statistics Fwd Physical Data RX Rate

11. Slot utilization indicates if the application is making efficient use of the physical resources (time slots).
In ideal RF conditions slot utilization should be between 65 90%. (See below)
Example: Graph View of Application, Physical layer Throughput, and Avg. Effective DRC

Application RX Rate
is higher than the
Physical layer
throughput rate.
Compression could be
enabled.
Example: Graph View of Poor Fwd Slot Utilization %

Nortel Networks Confidential and Proprietary

170

Example: Graph View of Good Fwd Slot Utilization %

App RX Rate (kbps) = Average Application Layer throughput per reporting period
Forward Physical Data RX Rate (kbps) = Instantaneous Physical Layer throughput from Forward Rate
Statistics V2 message. Indicative of how many packets are sent at a given rate every second. Packets per
rate are accumulative on a per second basis.
Average Effective DRC Rate = Data rate requested by the AT based on measured C2I (RF conditions).
Fwd Slot Utilization % = Slot utilization is low due to inefficient use of available slots. In ideal RF
conditions where 1 user has the full queue of the scheduler, slot utilization should be in the 80 -90% range.
Example: Fwd Rate Statistics V2 Message
Poor Slot Utilization ~ 2%

Nortel Networks Confidential and Proprietary

Better Slot Utilization ~37%

171

12. Low slot utilization with relatively good application layer throughput may be indicative of software
compression. Otherwise poor slot utilization is likely due to poor RF conditions (i.e. PER, C/I, TX poor,
etc), AT problem, or many users accessing the sector.
13. If RF problems (i.e. Dropped connections and Access Failures) are excessive, refer to: Section 7.4 and
7.6 for tips on how to recognize and address these problems.
14. It is important to remember that the proportional fairness scheduler will impact throughput as more
users access the network (scheduling contention). The scheduler divides time on the system amongst
users according to current RF conditions, data rate requests, and past data rate history. The following
equation describes how the scheduler determines the throughput per slot per user. .

1
1
Tk (n + 1) = 1 Tk (n ) + Rk (n), n = 0,1,2,...
tc
tc
Where:
TK(n) = mobile #ks throughput (T) history for time slot n
TC is a time constant
RK(n) = the rate at which the user is being served in slot n
15. To determine the number of users / links in use at a particular time in the AT log file, open the
Message View in RFO and look for the Quick Config message. This message gives an indication of
how many links are attached to a sector while data throughput testing is being performed.
16. The RPC count indicates the number of users attached to the sector at the time. According to the
example below, there are four links in use.
Example: Quick Config Message

Nortel Networks Confidential and Proprietary

172

17. Given that the maximum forward data rate is contingent upon the reverse link data rate capability, a
verification of Forward Physical Rate to reverse data rate ratio should be made. A ratio of no more than
63:1 (optimally 40:1)is sufficient to support data rate requests in excess of 2.0Mbps.
Example: Fwd Physical Rate to reverse data rate ratio

Formula: Ratio = Fwd Physical Rate/Reverse Data Rate = 44.9%


18. If the Forward Physical to Reverse Data rate is greater than approximately 63:1, verify the following RF
data fill:
RL Max Rate Table
Num-AT-From
1

Recommended
Num-AT-To
59

Rate
153.6 Kbps

RL Transition Probabilities Table


Attribute
Transition009k6_019k2
Transition019k2_038k4
Transition038k4_076k8
Transition076k8_153k6
Transition153k6_076k8
Transition076k8_038k4
Transition038k4_019k2
Transition019k2_009k6

Recommended
128
64
32
8
255
32
16
16

19. RL SHO failures can impact FL throughput by limiting the ACK rate on the RL. To determine the
per UATI RL SHO failure rate, from CLI go to enable mode and view the per UATI connection
counters:
1xEVDO session <UATI> counters <All>:
Reverse Link Soft Handoffs :Attempts : 0
Success : 0
Nortel Networks Confidential and Proprietary

173

Blocked by RN : 0
Blocked by RNC : 0
Failed by RN : 0
Failed by RNC : 0
Failed because of no TCC : 0
In EVDO soft or softer handoffs are initiated by the AT sending a Route update message to request
the addition or removal of a pilot to its Active Set. Each Route update is counted as a handoff attempt.
Soft handoff failures generally occur as a result of resource blocking or failures at the RNC or RN.
Currently there are no reason codes for specific resource blocks.
Other failures may have occurred as a result of configuration errors. In the event of failures verify the
following RF data fill parameters:
TCC Wait Timer
The amount of time the RNC waits for a traffic channel complete (TCC) message from the AT. A
Traffic Channel Assignment message is typically sent by the RNC to the AT as part of a reverse link
soft handoff. If the TCC is not received in time the connection will be closed.
Range

Unit

Recommended

1000 30000

1/1000
seconds

5000

IS856Neighbor
Defines the neighbors surrounding the sector an AT may be connected to. Inconsistencies in the
neighborlist may result in missed handoffs. Insure that reciprocal neighbors are not missing in
adjacent sites neighborlists.
19. If the Fwd Physical Rate to reverse data rate ratio is acceptable, a verification of the RLP Timeout rate
should be performed. Based on an access layer PER of 1% the system will inherently encounter NAK
timeouts (i.e. RLP packet retransmission is not received in .5ms or received in error) but the percentage
should be small. If the percentage is large then there will be failures or retransmissions at the TCP layer.
It should be noted that the relationship between PER of the initial packet and the PER of a retransmitted
packet in effect makes the worst case PER .01% (i.e. PER initial =1% x PER for retransmitted =1% =
.01%)
By example, assume that a 5Mbyte file (40Mbits) is being downloaded. If this is application layer we
need to assume roughly 15% overhead (40Mbits *1.15 = 46Mbits). Each FTC MAC packet is 1024 bits
so the system will send ~ 45,922 packets. Then the number of NAK timeouts that should be expected =
45,922x.01% = ~5NAK timeouts (i.e. packets not received or received in error on retransmission).

Nortel Networks Confidential and Proprietary

174

Example: RLP Packet Error Rate

Formula: Equivalent MAC packets = ((29724213Bytes x 8)/1024 = 232220 packets


Formula: RLP Packet Error Rate = SQRT (29 Packet Errors (NAK Timeouts) / 232220 packets) x
100% = 1.12%
20. In the event that RLP Packet Error Rate is >5% DOM logs should be collected to verify whether or not
packets are being delayed at the DOM for resequencing. The following script can be used to log the
GRE / ABIS packets seen at the DOM. Be careful as these logs are intrusive.

DOM_Logging.txt
(1 KB)

21. If the celldm logs show packets are being received out of sequence at the DOM and there are excessive
RLP NAKS as a result, the SDU Resequencing Wait Timer may be too short. The default wait period is
25milliseconds and is adjustable in increments of 5 milliseconds. The time may be updated with
following CLI command:
NORTEL-XX(config)#1xevdo sdu-resequencing-wait <period>
22. To verify of further impact on throughput, TCP trace data should be collected using Windump or
Ethereal in parallel with AT logging. TCP traces will show whether or not there were delays in
application throughput due to TCP segment losses / retransmissions.
23. TCP segment retransmissions are common when IP header compression is utilized in wireless
networks. It is known that TCP has a slow recovery time when a TCP segment is lost or corrupted as
the decompressor becomes unsynchronized. The decompressor only knows the next packet sequence it
is supposed to receive by validating the checksum value calculated based on difference values from one
header to the next. In the case of high speed wireless links, the contents of the TX buffer window (64k)
will be exhausted before TCP recovers. All TCP packets will be discarded without Acknowledgment by
the receiver before TCP sends a packet with an uncompressed header. The packet with the
uncompressed header will allow TCP to recover by sending retransmissions of the discarded packets.

Nortel Networks Confidential and Proprietary

175

Example: TCP segment loss


This segment is really E, not D

Compressor

E D C B A

RF Link

E D B A

Packet loss or corruption over lossy medium

Decompressor

D C B A
This segment is really D not C

Example: Time Sequence Graph (TCP packet throughput)

Nortel Networks Confidential and Proprietary

176

Example: Packet Loss

Packets are
discarded.

24. If many instances of stalled throughput or packet discards are seen in TCP traces, process the logs with
TCPTrace http://tcptrace.org/. Use the (tcptrace-lr) command to determine what node is most affecting
packet loss and / or latency.
Example: TCPTrace Output
================================
TCP connection 6:
host k:

h226.13.39.162.ip.alltel.net:20

host l:

216.96.112.225:1112

complete conn: yes


first packet: Tue May 24 13:15:15.380032 2005
last packet: Tue May 24 13:23:39.204496 2005
elapsed time: 0:08:23.824464
total packets: 64180
filename:

may24_16_eth

k->l:

l->k:

total packets:

38734

total packets:

25446

ack pkts sent:

38733

ack pkts sent:

25446

pure acks sent:

pure acks sent:

25444

sack pkts sent:

sack pkts sent:

236

dsack pkts sent:

dsack pkts sent:

max sack blks/ack:

max sack blks/ack:

Nortel Networks Confidential and Proprietary

0
1

177

unique bytes sent: 54888890


actual data pkts:

unique bytes sent:

38731

actual data pkts:

actual data bytes: 55305445

actual data bytes:

rexmt data pkts:

293

rexmt data bytes:

416556

zwnd probe pkts:

zwnd probe pkts:

zwnd probe bytes:

zwnd probe bytes:

outoforder pkts:

pushed data pkts:

843

SYN/FIN pkts sent:

0
0

rexmt data pkts:

rexmt data bytes:

0
0

outoforder pkts:

0
0

pushed data pkts:

1/2

SYN/FIN pkts sent:

req sack:

req sack:

sacks sent:

sacks sent:

236

1/1

urgent data pkts:

0 pkts

urgent data bytes:

0 bytes

urgent data bytes:

mss requested:

1428 bytes

mss requested:

max segm size:

1428 bytes

max segm size:

min segm size:

452 bytes

min segm size:

0 bytes

avg segm size:

1427 bytes

avg segm size:

0 bytes

max win adv:

17136 bytes

max win adv:

65535 bytes

min win adv:

16384 bytes

min win adv:

2703 bytes

zero win adv:

0 times

avg win adv:

17135 bytes

initial window:

2856 bytes

initial window:

2 pkts

urgent data pkts:

1460 bytes
0 bytes

0 times

avg win adv:

65472 bytes

initial window:

0 bytes

initial window:

0 pkts

ttl stream length:

missed data:

0 bytes

missed data:

truncated data:

0 bytes

truncated data:

0 pkts

0 bytes

zero win adv:

ttl stream length: 54888890 bytes

truncated packets:

0 pkts

0 bytes

0 bytes
0 bytes

truncated packets:

0 pkts

data xmit time:

503.634 secs

data xmit time:

0.000 secs

idletime max:

18076.0 ms

idletime max:

17915.8 ms

throughput:

108944 Bps

throughput:

24962

RTT samples:

RTT samples:
RTT min:

10.0 ms

RTT max:

200.3 ms

RTT avg:

5.8 ms

RTT stdev:

12.1 ms

RTT from 3WHS:

0.0 ms

RTT min:
RTT max:
RTT avg:
RTT stdev:

0 Bps
2
130.2 ms
140.2 ms
135.2 ms
0.0 ms

RTT from 3WHS:

Nortel Networks Confidential and Proprietary

130.2 ms
178

RTT full_sz smpls:

24959

RTT full_sz smpls:

RTT full_sz min:

10.0 ms

RTT full_sz min:

RTT full_sz max:

200.3 ms

RTT full_sz avg:

5.8 ms

RTT full_sz stdev:

12.1 ms

post-loss acks: 9

1
140.2 ms

RTT full_sz max:

140.2 ms

RTT full_sz avg:

140.2 ms

RTT full_sz stdev:

0.0 ms

post-loss acks: 0

For the following 5 RTT statistics, only ACKs for


multiply-transmitted segments (ambiguous ACKs) were
considered. Times are taken from the last instance
of a segment.
ambiguous acks:

48

ambiguous acks:

RTT min (last):

0.0 ms

RTT min (last):

RTT max (last):

190.3 ms

RTT avg (last):

11.5 ms

RTT avg (last):

0.0 ms

RTT sdv (last):

35.0 ms

RTT sdv (last):

0.0 ms

segs cum acked:

13420

segs cum acked:

duplicate acks:

408

triple dupacks:

18

triple dupacks:

max # retrans:

max # retrans:

0.0 ms

RTT max (last):

0.0 ms

duplicate acks:

min retr time:

190.3 ms

min retr time:

max retr time:

3014.3 ms

max retr time:

avg retr time:

1258.5 ms

avg retr time:

sdv retr time:

856.2 ms

sdv retr time:

0.0 ms
0.0 ms
0.0 ms
0.0 ms

25. Throughput problems related to packet loss issues are often related to T1 connectivity of timing issues
in the backhaul that result in path or line code violations. To see whether path or line code violations are
consistently occurring, monitor the link status alarms in the DO EMS
27. Verify if the T1 timing source is incorrect or faulty T1 Timing Source. Set the interface to the correct
timing source. Insure interfaces at both ends are the same.
NORTEL-XX>enable
NORTEL-XX#configure
NORTEL-XX(config)#controller t1 1/0/1
NORTEL-XX(config-controller)#clock source line-loop
Confirm timing configuration change:
NORTEL-XX#clear counters
NORTEL-XX#show controller t1 1/0/1
26. If packet loss is found from the TCP trace logs to be greater than 1%, a system health check should be
performed as given in [17] page 30.

Nortel Networks Confidential and Proprietary

179

10.11 FL Per-Sector Throughput


10.11.1 Data Collection Methods
10.11.1.1 AT Logs
The FL per sector throughput can only be determined from the AT logs when collecting logs from a
stationary location. There is no good way to collect the per sector throughput from cluster drive data.
Method of Collection:
1. To collect data to measure FL per sector throughput from AT logs will require at least 2 ATs.
2. Configure all the ATs for HDR only operation with receive diversity enabled.
3. Insure IP header and Software Compression algorithms are disabled.
4.Testing should be performed when there is no other traffic on the sector.
5.Configure Autocall in Optis-M or Invex3G for the ATs to perform a single download of a 50MByte
test file.
6.Apply a full EVDO log mask prior to starting collection.
7.Go to a location within the sector where average DRC request rate is between (1.2288 and
2.457Mbps) and C/I is consistently between 10 and 12dB.
8.Start download. When download is complete, stop logs and post process logs using EVDO RFO.

10.11.1.2 RNC Logs


There is currently no useful information in the RNC logs that can be used to determine per Sector
throughput.

10.11.1.3 OMs
The aggregated sector throughput at the physical layer can be estimated using the OM PerfFLThruput.
This OM only gives a time averaged estimate of the sector throughput based on a sliding 10 second
window.
Method of Collection:
1. Telnet into the DOM and enable the FTC Scheduler. It is necessary to enter configuration mode to
enable FTC scheduler NORTEL-03> config NORTEL-03 celldm scheduler ftc on
2. Log out of the DOM NORTEL-03> exit.
3. Repeat this process for all DOMs
4. The Average FTC Physical Layer throughput loading = perfFLThruput / 1000 (Unit: kbps)
5. Note: The throughput value given will only be for the last reported sampling window
6. A histogram of slot utilization and throughput can be generated using CLI show celldm flthruput
<sector>
Sample: flthruput histogram from celldm
NORTEL-XX#show celldm flthruput alpha
RATE: 38k4 76k8 153k6 307k2 614k4 921k6 1228k8
1843k2 2457k6
SLOTS: 0 0 0 0 0 0 0
0 159235
FL data throughput for sector alpha: 534749 bps

Nortel Networks Confidential and Proprietary

180

10.11.2 KPI Calculation


10.11.2.1 KPI Value and Tolerance Factor
Forward link sector throughput is dependent upon the data rates being requested and the number of
users on the sector.

10.11.2.2 Calculating KPI from AT Logs


To determine the aggregated sector throughput from the AT logs go to RFO Statistics View, Forward
Rate Statistics Avg. ActualData Rate this will be the average physical layer data rate of the sector.
Example: Average Actual Data Rate for all users (Physical layer sector throughput)

10.11.2.3 Calculating KPI from RNC Logs


There are no sector throughput attributes currently available from RNC logs

10.11.2.4 Calculating KPI from OMs


The aggregate sector throughput is as given in the celldm logs as shown in 10.11.1.3 above.

10.11.3 Troubleshooting Guide


1. In the event that there are problems with per sector data rate capability follow the steps in
Stationary Troubleshooting in Appendix A. .
2. Refer to section 10.10.3 for tips on troubleshooting sector throughput.
For a non commercial network sector throughput will not be a key indicator of performance problems due
to low traffic density. In this case per user throughput will be a better indicator of potential problems.
For an in service network, sector throughput should be trended over time and records kept of the variability
in performance. In the event that the average sector throughput dips below a typical value, based on trends,
then the troubleshooting guidelines in Appendix A should be followed to resolve.

10.12 RL Per-User Throughput


In the EVDO system the AN reports the status of the reverse link loading to the ATs by setting the
Reverse Activity Bit (RAB). The AT uses the RAB and several other attributes to determine its reverse
link data rate. Simply stated the AT reduces its data rate on the reverse link when loading is heavy and
increases it when loading is light.

10.12.1 Data Collection Methods


10.12.1.1 AT Logs
Method of Collection:
1. Use Optis-M or Invex3G to collect reverse link user throughput logs.
2. Configure the AutoCall or Data Connection tabs to perform an FTP PUT of a 5MByte file to a server
in close proximity to the PDSN.
3. Configure the AT for HDR only operation with received diversity enabled.
4. Insure that all compression algorithms (S/W, IP header, etc) are disabled prior to collecting any AT
log data.
5. Begin stationary tests or cluster drive test.
Nortel Networks Confidential and Proprietary

181

6. At the end of the test stop AT logs and post process using EVDO RFO.

10.12.1.2 RNC Logs


There are no RL per user throughput attributes available in the RNC logs.

10.12.1.3 OMs
The Reverse link physical layer throughput is estimated over a 3 second sliding window and is
available from the per sector RL throughput OM perfRLThruput. The per user throughput can be
estimated as follows:
Method of Collection:
1. Collect the Reverse Link histogram from the celldm logs show celldm rlthruput <sector>.
NORTEL-XX#show celldm rlthruput alpha
RATE: 9k6 19k2 38k4 76k8 153k6
BYTES: 139648 533632 428928 38656
512
RL data throughput for sector alpha: 26112 bps

10.12.2 KPI Calculation


10.12.2.1 KPI Value and Tolerance Factor
RL - ~ 80 kbps {depends on TP values and 1xRTT tune-away times (tune-away times should be less
than 200 ms 95% of the time)}

10.12.2.2 Calculating KPI from AT Logs

1. After post processing the AT logs with RFO, open Statistics View Data Statistics Application
TX Active Throughput.
Example: RFO RL throughput statistics

10.12.2.3 Calculating KPI from RNC Logs


There are no RL throughput attributes available in the RNC logs.

10.12.2.4 Calculating KPI from OMs


1. Calculate the weighted average reverse rate from the celldm data as follows:

( Bytes * 8)
TotatTransmissionTime = (( Bytes * 8) / Rate)
TotalBytes Re ceived =

AllRates

AllRates

WeightedAverage Re verseRate = TotalBytes Re ceived / TotalTransmissionTime


Re verseLinkPhysicalLayerBurstRate = Averageof (WeightedAverage Re verseRate)
fromall sec tors
2. The calculated average given above reflects the average burst rate per user. However, this is only an
estimate and does not reflect the user experience exactly.

Nortel Networks Confidential and Proprietary

182

10.12.3 Troubleshooting Guide


1.For a point by point procedure for testing and troubleshooting, refer to Appendix A.
2.If a problem with Reverse Link Throughput is suspected, process the AT logs and verify that the Graph
View that the Reverse Data Rate and the App TX Rate track closely. Also verify that the AT transmit
power is not excessive.
Example: Graph View of Reverse Data Rate, App TX Rate, and TX Total Power

3.Verify that the laptop / collection tool TCP/IP settings are in accordance with Appendix E TCP/IP
Configuration.
4.Verify if the problem is isolated to a specific DOM or DOMs. If the problem is over several DOMs
troubleshoot Ethernet Performance or GRE / A10 resequencing. See page 4-12 and 4-75 in [18].
5.If the problem is isolated to 1 DOM then troubleshoot the site for Radio malfunction, antenna problems,
feeder problems, DOM chassis malfunctions, or interference. See page 5-126 [18].
6.If the problem is random throughout the network troubleshoot the backhaul timing source or connectivity.
See page 4.34 36 [18]
7.Verify the following RF datafill parameters are within recommended values established by Nortel and the
customer:
Reverse Link Max Rate Table
This table provides a per sector datafill method to control the maximum data rate granted to the AT. CLI
must be used to make changes to this table. Changes made to this table are not persistent (i.e. any restart
of the DOM will cause the operator to have to go in and reset the values.
Num-AT-From
1

Recommended
Num-AT-To
59

Rate
153.6 Kbps

RAB Threshold
Range

Unit

Recommended

0 16384

1/16384 %
of Load

9830 (60%)

Nortel Networks Confidential and Proprietary

183

Reverse Traffic Channel Transition Probabilities Table


Whenever in a connection, the AT will always be transmitting at a particular data rate on the reverse
link. Initially, the AT will always begin with a 9.6 Kbps data rate. But the next and successive frames
will transmit at a rate determined by the Reverse Link Transition Probabilities listed above. The AT
will increase, decrease or not change the data rate of the next transmitted frame depending on the data
rate of the current frame, the state of the combined busy bit, and an internal random number generator
ranging between 0 and 1.
Attribute
Transition009k6_019k2
Transition019k2_038k4
Transition038k4_076k8
Transition076k8_153k6
Transition153k6_076k8
Transition076k8_038k4
Transition038k4_019k2
Transition019k2_009k6

Recommended
128
64
32
8
255
32
16
16

8.RL SHO failures can impact the RL throughput. From CLI go to enable mode and view the per UATI
connection counters to determine the number RL SHO failures:
1xEVDO session <UATI> counters <All>:
Reverse Link Soft Handoffs :Attempts : 0
Success : 0
Blocked by RN : 0
Blocked by RNC : 0
Failed by RN : 0
Failed by RNC : 0
Failed because of no TCC : 0
In EVDO soft or softer handoffs are initiated by the AT sending a Route update message to request the
addition or removal of a pilot to its Active Set. Each Route update is counted as a handoff attempt. Soft
handoff failures generally occur as a result of resource blocking or failures at the RNC or RN. Currently
there are no reason codes for resource blocking. Other failures may have occurred as a result of
configuration errors. In the event of failures verify the following RF data fill parameters:
TCC Wait Timer
The amount of time the RNC waits for a traffic channel complete (TCC) message from the AT. A Traffic
Channel Assignment message is typically sent by the RNC to the AT as part of a reverse link soft
handoff. If the TCC is not received in time the connection will be closed.
Range

Unit

Recommended

1000 30000

1/1000
seconds

5000

IS856Neighbor
Defines the neighbors surrounding the sector an AT may be connected to. Inconsistencies in the
neighborlist may result in missed handoffs. Insure that reciprocal neighbors are not missing in adjacent
sites neighborlists.
9.If problems persist perform a FTAP and RTAP tests.

Nortel Networks Confidential and Proprietary

184

10.13 RL Per-Sector Throughput


10.13.1 Data Collection Methods
10.13.1.1 AT Logs
Method of Collection:
1. To measure sector throughput from AT logs testing will have to be performed in stationary locations
where RL sector throughput is known to be compromised.
2. At least 2 ATs will be required to measure the sector throughput from AT logs.
3. Configure the ATs for HDR only operation with receive diversity enabled.
4. Insure all compression algorithms are disabled prior to beginning the test.
5. Drive to a predetermined location (sector) within the RNC where the C/I is between (10 12dB).
6. Insure that the AutoCall setup in Optis-M is set for synchronous logging to insure all ATs begin
uploads at the same time.
7. Start AT data collection logs in each drive van.
8. Perform 5 consecutive FTP uploads of a 1MByte file for all ATs.
9. When logging is complete stop logs and use EVDO RFO to post process.

10.13.1.2 RNC Logs


There are no sector throughput attributes available from RNC logs.

10.13.1.3 OMs
The Reverse link physical layer throughput is estimated over a 3 second sliding window and is
available from the per sector RL throughput OM perfRLThruput. The per sector reverse link physical
layer throughput is available as a per sector OM.
Method of Collection:
1. The reverse link physical layer throughput can also be collected using the celldm logs.
2. Collect the Reverse Link histogram from the celldm logs show celldm rlthruput <sector>.
NORTEL-XX#show celldm rlthruput alpha
RATE: 9k6 19k2 38k4 76k8 153k6
BYTES: 139648 533632 428928 38656
512
RL data throughput for sector alpha: 26112 bps

10.13.2 KPI Calculation


10.13.2.1 KPI Value and Tolerance Factor
Aggregated reverse link sector throughput is approximately 200 kbps per Qualcomm.

10.13.2.2 Calculating KPI from AT Logs


1. The aggregated reverse link throughput will be the summation of the average application or physical
layer throughput of each AT averaged over the number of ATs used.

(1

# ofAT

AggregatRLSectorThroughput =

TXActiveThroughput )

TotalNumberofATs

10.13.2.3 Calculating KPI from RNC Logs


Not Applicable

10.13.2.4 Calculating KPI from OMs


1. The aggregate RL physical layer sector throughput can be determined directly from the celldm logs.
Nortel Networks Confidential and Proprietary

185

2. Collect the Reverse Link histogram from the celldm logs show celldm rlthruput <sector>.
NORTEL-XX#show celldm rlthruput alpha
RATE: 9k6 19k2 38k4 76k8 153k6
BYTES: 139648 533632 428928 38656
512
RL data throughput for sector alpha: 26112 bps

10.13.3 Troubleshooting Guide


Follow the troubleshooting guidelines for Reverse Link User throughput (7.12.3).

10.14 FL Achievable Data Rate


Forward Link Achievable Date Rate is the maximum data rate that can be obtained by a user in ideal RF
conditions anywhere within the network. The maximum achievable rate is basically the golden value for
per user forward link throughput.

10.14.1 Data Collection Methods


10.14.1.1 AT Logs
Method of Collection:
1. Follow the system integrity pre check guidelines given in the flowchart shown in Appendix A. prior
to starting testing.
2. Insure that the AT is in HDR only mode and that receiver diversity is enabled.
3. Verify all compression algorithms are disabled.
4. Insure that there is a server located in close proximity to the PDSN for testing and troubleshooting
purposes.
5. Configure Optis-M or Invex3G to perform 10 consecutive FTP downloads of a 5MByte test file.
6. Go to a location where C/I is between (10 12dB) and the DRC requests are between (1.2288
2.457Mbps).
7. Verify from the User Size attribute in Optis-M or XCAL that there are no other users attached to
the sector.
8. Begin downloads. When completed, stop logs and post process with EVDO RFO.

10.14.1.2 RNC Logs


Currently, there are not throughput attributes available from the RNC logs.

10.14.1.3 OMs
There are currently no per user throughput OMs that can be used for determining the achievable data
rate on the forward link. Achievable FL throughput can be estimated from the celldm logs show celldm
flthruput <sector>
Method of Collection:
1. Telnet into the DOM and enable the FTC Scheduler. It is necessary to enter configuration mode to
enable FTC scheduler NORTEL-03> config NORTEL-03 celldm scheduler ftc on
2. Log out of the DOM NORTEL-03> exit.
3. Repeat this process for all DOMs
4. The Average FTC Physical Layer throughput loading = perfFLThruput / 1000 (Unit: kbps)
5. Note: The throughput value given will only be for the last reported sampling window

Nortel Networks Confidential and Proprietary

186

6. A histogram of slot utilization and throughput can be generated using CLI show celldm flthruput
<sector>
Sample: flthruput histogram from celldm
NORTEL-XX#show celldm flthruput alpha
RATE: 38k4 76k8 153k6 307k2 614k4 921k6 1228k8
1843k2 2457k6
SLOTS: 0 0 0 0 0 0 0
0 159235
FL data throughput for sector alpha: 534749 bps

10.14.2 KPI Calculation


10.14.2.1 KPI Value and Tolerance Factor
Maximum Achievable rate depends on the RF channel conditions in which the AT is embedded. The
data rate is proportional to the DRC request rate ADR = DRC, where 0<<1.

10.14.2.2 Calculating KPI from AT Logs


1. After the data is post processed with EVDO RFO, open the Statistics View, Data Statistics,
Application RX Active Throughput.
2. For maximum FL physical layer throughput go to, Statistics, Data Statistics, Forward Rate Avg.
Actual Data Rate

10.14.2.3 Calculating KPI from RNC Logs


No FL throughput data is available from RNC logs.

10.14.2.4 Calculating KPI from OMs


1. To approximate the Fl achievable throughput based on user experience, use the following method
based on the data in the celldm logs.

WeightedAverageForwardRate =

( RatexSlots)

AllRates

Slots

AllRates

10.14.3 Troubleshooting Guide


1.Refer to the troubleshooting flowcharts in Appendix A for a point by point process for testing a problem
resolution.
2.From the AT logs plot in the Graph View of RFO the RF characteristics (C/I, Avg. Effective DRC Rate,
and Traffic Channel (TC) PER, seen at the location or locations where data was collected.

Nortel Networks Confidential and Proprietary

187

Example: Graph View of RF Parameters from RFO

2.If the average application layer throughput is low (e.g. < 1.5Mbps) where RF conditions are ideal (e.g. C/I
is in the 10-12dB range) perform a sanity check by performing a DOS FTP of the file. If the throughput is
good (i.e. 1.5Mbps or greater) the check to insure that all compression algorithms that may be selectable
in the collection tool are disabled.
3.Verify from the Sector Parameters Message in the AT logs that the correct Pilot PN, Band Class,
Channel Number and Subnet Mask for the sector are correct.
4.Us DrTCP to insure that the TCP RX Window size is = 64240 (TCP Window Size in Windows Registry)
and the MTU size = 1500Bytes (ProtocolMTU in Windows registry). Refer to Appendix E for more tips
on TCP IP configuration.
5.If the throughput from the collection tool and DOS FTP are low verify that there are no other users on the
site. In Optis or XCAL view the User Size in the EVDO Signal Graph or from the Quick Config
message in the Layer 3 messaging available in RFO.
Example: User Size in Optis-M EVDO Signal Graph

6. If the User Size is equal to 1 verify that the DRC request rate is between 1.2288 and 2.457Mbps.
7. If the DRC request rate is low verify from the AT logs that the Fwd Physical rate to Reverse data rate
ratio is approximately 40:1.

Nortel Networks Confidential and Proprietary

188

8. In the event that the Fwd Physical to Rvs Data rate is much greater than ~65:1 perform the following
checks:
o

Do the AT logs indicate drops and access failures?

Is the RL SHO factor > 2? Show 1xEVDO connection <UATI> counters <all>

Verify that there are no failures due to resource allocation failures? show 1xevdo connection
<UATI> counters <all>

Verify the RF datafill for RL Max Rate Table, and the Reverse Transition probabilities table See [8]
for Nortel recommended RF datafill values and methods for updating datafill tables.

Perform FTAP and RTAP tests to verify the physical connection between the AT and RNC.
Example: Fwd Physical to Reverse Data rate ratio

Formula: Fwd Physical Rate / Rvs Data Rate = 45.6


9. If the Forward to Reverse data rate ratio is acceptable, perform a health check of the backhaul links to
verify that there are not physical errors being reported (i.e. path line code violations, etc). Perform a
system health check (T1 interface status, Ethernet, etc) according to the User performance
troubleshooting flow chart in [18].
10. T1 errors will result in reduced throughput, Abis failure, link status alarms, or wilted radios. Ethernet
problems are characterized by discards or collisions and may be a result of duplex mode or speed
configuration issues. Follow recovery steps in [18].
11. Collect TCP trace logs using Ethereal or Windump while performing a DOS or XCAL-DO FTP
download. Use the Ethereal TCP sequence graph or TCPTrace extended output to determine is there is
excessive packet loss. A TCPtrace analysis (tcptrace-lr) will indicate at what node packet loss or
retransmissions are occurring as well as the average RTT time between nodes. This information can aid
engineers in determining the source of problems.

Nortel Networks Confidential and Proprietary

189

Example: Ethereal TCP Sequence Graph:

Dropped TCP
Packets

TCP Packet
Retransmission

12. An inspection of the RLP statistics in the AT logs will indicate NAK timeout (NAK timeout occurs
when an RLP packet is retransmitted in error or not received by the AT within the .5msec wait time)
rate. If the NAK timeout rate exceeds 5 %, the SDU Resequencing Wait Timer may be too short.
13. To adjust the SDU Resequence Wait Timer use the following CLI command: NORTEL-XX#
(config)#1xevdo sdu-resequencing-wait <period>
Example: RLP NAK Timeout Rate

Nortel Networks Confidential and Proprietary

190

Formula: ((RX Total Bytes x 8) x 1.15)/1024 = RX Total packets (SQRT (NAK Timeouts / RX
Total Packets)) x100 = NAK Timeout Rate
14. Perform FTAP and RTAP tests to validate the integrity of the physical link between the RNC and
the AT [17].

10.15 RL Achievable Data Rate


The Achievable rate for the reverse link represents the maximum data rate achievable from the network /
sector given that all equipment is configured and operating as designed and the RF conditions are ideal. In
ideal RF conditions the achievable rate will represent a golden value for the system.

10.15.1 Data Collection Methods


10.15.1.1 AT Logs
Method of Collection:
1. Follow the system integrity pre check guidelines given in the flowchart shown in Appendix A prior
to starting testing.
2. Insure that the AT is in HDR only mode and that receiver diversity is enabled.
3. Verify all compression algorithms are disabled.
4. Insure that there is a server located in close proximity to the PDSN for testing and troubleshooting
purposes.
5. Configure Optis-M or Invex3G to perform 10 consecutive FTP uploads (PUT) of a 1MByte test file.
6. Go to a location where C/I is between (10 12dB) and the DRC requests are between (1.2288
2.457Mbps).
7. Verify from the User Size or RPC Size attribute in Optis-M or Invex3G that there are no other
users attached to the sector.
8. Begin uploads. When completed stop logs and post process with EVDO RFO.

10.15.1.2 RNC Logs


There are currently no useable attributes in the RNC logs that can be used to determine RL Achievable
data rate.

10.15.1.3 OMs
The Reverse link physical layer throughput is estimated over a 3 second sliding window and is
available from the per sector RL throughput OM perfRLThruput. The per user throughput can be
estimated as follows:
Method of Collection:
1. Collect the Reverse Link histogram from the celldm logs show celldm rlthruput <sector>.
NORTEL-XX#show celldm rlthruput alpha
RATE: 9k6 19k2 38k4 76k8 153k6
BYTES: 139648 533632 428928 38656
512
RL data throughput for sector alpha: 26112 bps

10.15.2 KPI Calculation


10.15.2.1 KPI Value and Tolerance Factor
Maximum achievable throughput on the reverse link is contingent upon current sector loading
conditions as well as the RF conditions seen at the collection location. In ideal RF conditions the
maximum achievable rate should be in the 100 140kbps range.

Nortel Networks Confidential and Proprietary

191

10.15.2.2 Calculating KPI from AT Logs


1. After post processing the AT logs with RFO, open Statistics View Data Statistics Application
TX Active Throughput.
Example: RFO RL throughput statistics

2. To determine the maximum RL throughput at the physical layer in RFO, Statistics View Data
Statistics Reverse Rate Avg. Data Rate.
Example: RFO RL Physical Layer Statistics

10.15.2.3 Calculating KPI from RNC Logs


There are no RL throughput attributes available from the RNC logs.

10.15.2.4 Calculating KPI from OMs


1. To determine the maximum achievable rate from OMs, calculate the weighted average reverse rate
from the celldm data as follows:

TotalBytes Re ceived =

( Bytes * 8)

AllRates

TotatTransmissionTime =

(( Bytes * 8) / Rate)

AllRates

WeightedAverage Re verseRate = TotalBytes Re ceived / TotalTransmissionTime

Re verseLinkPhysicalLayerBurstRate = Averageof (WeightedAverage Re verseRate)


fromall sec tors
2. The calculated average given above reflects the average burst rate per user. However, this is only an
estimate and may not reflect the maximum achievable RL data rate of the sector.

10.15.3 Troubleshooting Guide


1. Refer to Appendix A for the point by point testing and troubleshooting process.
2. For more detailed troubleshooting examples and references for RL performance, refer to section
7.12.3 RL per User Throughput troubleshooting.
Nortel Networks Confidential and Proprietary

192

10.16 Packet Latency


10.16.1 Data Collection Methods
10.16.1.1 AT Logs
1. Insure that the AT is in HDR only mode and receiver diversity enabled.
2. Insure all compression algorithms are disabled and TCP/IP settings are configured as per Appendix
D.
3. The Ping utility in Optis can be used to send multiple pings with a user defined payload for the
purpose of measuring packet latency across the network.
4. Pings should be sent to the PDSN IP address.
5. Configure Optis-M or Invex3G for Ping and Trace tests as shown below:
Example: Optis-M Autocall for Ping

6. An alternative to using Ping to measure packet latency is to collect TCP trace logs with Ethereal or
Windump.
7. Since XCAL and Invex3G-PC uses the PCs protocol stack for data collection, Ethereal or Windump
can be used concurrently to capture TCP trace data. In the event XCAL or Invex3G-PC are not
available for use, the engineer can open an Ethereal or Windump log and subsequently perform a
DOS FTP download or upload to the test server.
8. After the Ping test and trace logs are collected stop logs and use TCPTrace or the playback function
in XCAL to determine the average packet latency.

10.16.1.2 RNC Logs


RNC logs do not contain any useable attributes for measuring packet latency.

10.16.1.3 OMs
There are no packet latency OMs available
Nortel Networks Confidential and Proprietary

193

10.16.2 KPI Calculation


10.16.2.1 KPI Value and Tolerance Factor
For 1xEV-do system only {using PDSN to ping AT Latency shall be less than 200 ms, should be < 150
ms}. Systems Engineering times are 150 ms (mean), 172 ms (90th percentile).

10.16.2.2 Calculating KPI from AT Logs


1. To determine the KPI from the Ping test conducted with XCAL or Optis, use the replay function and
from the Status menu select Ping Status to display the latency over time. Also from the Status
menu the TraceRT Status can be selected to show the average RTT per node.
2. Alternatively, the TCP trace logs collected can be processed using TCPTrace (tcptrace-lr) which
produce packet latency statistics per node as shown in the example below:
Example: TCP Trace output
RTT
RTT
RTT
RTT
RTT

samples:
min:
max:
avg:
stdev:

24962
10.0 ms
200.3 ms
5.8 ms
12.1 ms

RTT
RTT
RTT
RTT
RTT

samples:
min:
max:
avg:
stdev:

2
130.2
140.2
135.2
0.0

ms
ms
ms
ms

10.16.2.3 Calculating KPI from RNC Logs


10.16.2.4 Calculating KPI from OMs
10.16.3 Troubleshooting Guide
1. Ping, Pathping, TCPTrace, Ethereal, or other protocol analysis tools can be used to identify packet loss
or latency problems between the AT, PDSN or the application server.
2. Perform several FTP downloads while capturing TCP trace or Ethereal logs
3. Process the logs using TCP Trace with the lr option to determine the node or nodes where packet
latency is excessive RTT Ave.
4. If the packet latency is >>300ms on any intermediate nodes follow the troubleshooting flowchart shown
in Appendix A.
5. Packet latency can often be a result of: T1 path or line code errors or faults, DOM PPP interface
configuration, Ethernet configuration, T1 aggregator problems port issues, TCP configuration, or server
issues.
6. Methods to troubleshoot these issues are covered in [18].

10.17 DO-To-1x Handoff Delay


10.17.1 Data Collection Methods
10.17.1.1 AT Logs
Method of Collection
1. Configure autocall in Optis to allow the AT to perform an FTP download of 50MB file.
2. Insure that keep PPP and allow dormant mode options are selected in the autocall setup.
3. Go to a location on the outskirts of the EVDO coverage area and start the AT logs.
4. Drive out of EVDO coverage but within the coverage of the underlying 1xRTT coverage area.
5. Perform multiple drives from within EVDO coverage into areas where EVDO coverage ends and
where there is only 1xRTT coverage.
6. Stop and save logs after each pass.
Nortel Networks Confidential and Proprietary

194

7. Make note that the EVDO to 1xRTT handoff is actually an EVDO connection drop followed by an
origination on 1xRTT. The FTP download application started on EVDO should continue on 1xRTT
from where it stopped on the EVDO network.

10.17.1.2 RNC Logs


There is currently no useful information regarding EVDO to 1xRTT handoffs in the RNC logs.

10.17.1.3 OMs
There are no current OMs to show EVDO to 1xRTT handoff delay.

10.17.2 KPI Calculation


10.17.2.1 KPI Value and Tolerance Factor
From [14]: 15s for SIP

25s for MIP

10.17.2.2 Calculating KPI from AT Logs


1. Using EVDO RFO, post process the log files collected for the EVDO to 1xRTT handoff test.
2. DO to 1x handoff delay is measured as the time delta between the Connection Release on EVDO and
the Service Connection Complete message on 1xRTT.
3. Determine 90th percentile delay for all valid EVDO to 1x handoffs.

10.17.2.3 Calculating KPI from RNC Logs


N/A

10.17.2.4 Calculating KPI from OMs


N/A

10.17.3 Troubleshooting Guide


EVDO to 1xRTT handoff is governed by the aggregate value of Ec/Io as determined from all EVDO pilots
in the ATs Active and Neighbor sets. Once the aggregate value of all of these drops below a vendor
specific value, the AT will declare the EVDO system lost and attempt to originate on the 1x network. The
time required to make a seamless transition is as given above. However, other factors may influence the
time delay:
1. Intermittent recovery of EVDO coverage as influenced by terrain, multipath etc. In these cases the
EVDO connection may be released by the AT due to poor coverage thus, loss of supervision,
However, if supervision is recovered the AT may attempt to reconnect to EVDO prior to
originating on 1x. In these cases little can be done to improve the transition performance.
2. Excessive delay can also be caused by the AT having to search the PRL for the 1x system. In this
case the PRL should be optimized. The AT PRL can be optimized using the Qualcomm utility
QPST.
3. Insure that the AT has been datafilled in the CDMA system HLR.

10.18 DO-To-1x Success Rate


10.18.1 Data Collection Methods
10.18.1.1 AT Logs
Method of Collection:
Same as EVDO to 1x handoff delay

10.18.1.2 RNC Logs


N/A
Nortel Networks Confidential and Proprietary

195

10.18.1.3 OMs
N/A

10.18.2 KPI Calculation


10.18.2.1 KPI Value and Tolerance Factor
10.18.2.2 Calculating KPI from AT Logs
1. Using EVDO RFO post process the collected AT logs collected for EVDO to 1x delay / success rate
testing.
2. Do to 1x handoff success rate is based on the number or 1x originations that result in a continuation
of the application process.

10.18.2.3 Calculating KPI from RNC Logs


N/A

10.18.2.4 Calculating KPI from OMs


N/A

10.18.3 Troubleshooting Guide


Same as above

Nortel Networks Confidential and Proprietary

196

Appendix A: EVDO Optimization and Troubleshooting Flow Charts


Optimization and Troubleshooting Flow Chart

Perform Network pre checks


and preliminary parameter
audits Page 2

Is the network in a
commercial or launched
state?

Perform site Shakedowns

Do specific sites or areas


have performance issues?

Perform Baseline Cluster


Drive Tests
N

Do specific sites or areas


have performance issues?

Perform Stationary
troubleshooting tests
Page 3 (#2)

Are stationary perf and


optimization objectives
complete?

Perform mobility
troubleshooting tests
Page 4 (#3)
Y
N

Are mobility performance


and optimization objectives
complete?
Y
Complete Final Cluster
Drives

Nortel Networks Confidential and Proprietary

197

RF Datafill
-Verify RF Datafill is in accordance
with Nortel recommended values or
what has been mutually agreed to with

Refer to Nortel 1xEVDO


RF Datafill Guide v 2.6
(March 2 2005)

TCP/IP settings are according to recommended


values (e.g. IP header Comp. off, MTU, etc)?

Y
NE operational status is Active?
NORTEL-XX# show node

Y
All modules have Active status?
NORTEL-XX# show modules

System Integrity
Verification Checklist

Refer to Appendix E 1xEVDO


Optimization Guide

N
Go to troubleshooting sections of
NTP 411-2133-949

Y
Administrative & operational status of all
interfaces is Active? (PPP, LCP,
Ethernet) NORTEL-XX# show interfaces

Y
Can Ping the following from DOM /
AT?
- Node IP of DO-RNC
- IP of Agg Router interfaces for
backhaul
- IP of DO -EMS

Y
DO-RNC must be able to Ping:
- Node IP of all connected DOMs
- IP address of DOM PPP interfaces
- IP address of DO-EMS
- All PDSNs

Y
Is ABIS Peer status = Connected for all
DOMs? NORTEL-XX# show abis peer

Y
Is GPS present and locked for DOMs?
NORTEL-XX# show GPS health

Y
Is SNTP or system time correct / present?
NORTEL-XX# show SNTP Time

Begin RF Optimization Tests


(Stationary or Mobile)

Set up Data Collection tools


according to:
EVDO Data Collection Tools
Optis-M and XCAL-DO v 1.1

Y
Is GRE A10 Resequencing enabled for all
BIO modules?
NORTEL-XX (config) # fastpath a10 reseq 1

Nortel Networks Confidential and Proprietary

N
TAP Tests (Tap Users Guide)
(FTAP / RTAP) Passed

198

Stationary RF Troubleshooting
Stationary
Test Start #1
Go to site or location where
problems are and perform FTP
DL using data collection tool.
Verify that there are no
antenna feeder issues

Is Avg. C/I ~8 - 12dB?

Are DRC requests ~ 1.2288 2.457Mbps?

Is Avg. App tput ~ 1.5


2.0Mbps?

Y
Go to Page
5 (#3)

Go to next location or begin


mobility testing

Is DOS FTP tput equivalent


to what tool reports?

Are Results as
expected?

-Try a different AT
- Verify AT COM ports
are configured correctly?

Data Collection Tool Settings


-Verify BAUD rate in port settings is ~115200
-Verify phone model and chip set are correct
-Verify correct PPP adapter under port settings

Check User Size. Is it greater


than 1?

>> 1 user on site, test where or when


there is less traffic

N
Post Process AT logs
with RFO

Refer to Chapter 7 of EVDO Opto guide


for Drop and AF resolution guidelines.

Do AT logs indicate >5%


Drops or AF?

Go to Page
5 (#3)

N
Is Fwd. Physical Rate to Rvs. Data
rate ratio ~ 40:1 but not > 63:1?

Verify iuplink tput (FTP


PUT) in good RF ~140kbps?

- Is RL SHO factor >2.0?


show 1xevdo connection<uati> counters<all>

N
- Are there excessive RL
NAKS/Timeouts in AT logs?

Nortel Networks Confidential and Proprietary

Opt coverage to intro


dominant server

Is RL SHO failure due to resource allocation fail


show 1xevdo connection<uati>
counters<all>
N

Determine resource
issue and resolve

Verify data fill for RL Max


Rate and trans Probs
Go to Page
5 (#3)

199

Mobile RF Troubleshooting

Start
Mobility
Tests #2

Drive Cluster Routes performing


consecutive FTP DLs.

END
Collect AT & RNC logs
Refer to the EVDO Optimization
Guide, Chapter 10 for tips on
recognizing and addressing failures.

Process AT and RNC logs using


RFO and CROS tools

Is Avg. RX Active tput ~ (500


800kbps or greater) per cluster?

Y
- From NEWTUN identify DOMs
with high fail rates.
- From RFO and NEWTUN output,
categorize failures (missing neighbor,
border handoff, long tune away,
Session setup fail, poor coverage, etc).

N
From AT and RNC logs, are
there excessive drops and access
failures (i.e. >10%)?

N
From AT and RNC logs are there
areas with poor coverage (C/I < 2dB, PER >1%, > 3 ASet Pilots)?

Is Fwd Physical Rate to Rvs. Data


rate ratio ~ = 40:1 but not > 63:1?

N
Verify data fill for RL Max Rate and
RL trans Probabilities table
1xEVDO RF Dataill Guide

- Is RL SHO factor >2.0?


show 1xevdo connection<uati> counters<all>

- If there are Aset pilots > 2 tiers, try adjusting the


Cell Radius according to formula in section
2.3.1.5 of 1xEVDO RF Data fill Guide
- Add sites or make adjustments in antenna
configurations to introduce dominant server.
- Verify that there are no antenna feeder problems.

For site specific


issues, go to
Stationary Tests #1

Identify localized
areas with poor FL
tput

Go to Page 3 #1

N
RL SHO failure due to resource allocation fails?
show 1xevdo connection<uati> counters<all>

Optimize coverage to
introduce dominant server

Determine root cause of


resource issues and resolve

N
Go to areas where ratio is compromised
and verify max uplink rate via FTP PUT.

Are results as
expected?

Nortel Networks Confidential and Proprietary

Go to Page
5 (#3)

200

TCP / UDP / RLP Troubleshooting


Go to: Page 2
(#1) or
Page 3 (#2)

TCP Trace
analysis - #3

- From FL AT logs is the Ratio of


AT NAK Bytes Requested / RX
Total Bytes >= 5%?
- Are the number of NAK Timeouts
>> ((RX Total bits x 1.15) x .01^2)

Y
From CellDM (DOM) logs for RTC
(rtclogmask 0xFFFFFFFF) are
GRE / Abis packets arriving out of
order?

SDU Resequencing Wait Timer may be too


short.
NORTEL-XX# (config)#1xevdo sduresequencing-wait <period>

N
Use Ethereal or Win Dump to
collect TCP Trace logs

Do TCP Trace logs (tcptrace lr)


indicate > 1% retransmissions?

Y
Is RTT excessive between any
nodes (e.g. >>300ms)?

Y
Verify TCP/IP Settings See
EVDO Opto Guide Appendix E

Are there consistent T1/E1 Path or


line code violations?
1 Monitor Link Status alarms for the
interface for a period of time in DOEMS.
2 NORTEL-XX(config)#logging
console on

Y
Verify which node or interface is
problematic and troubleshoot.
See NTP411-2133-949

Nortel Networks Confidential and Proprietary

Incorrect or faulty T1 Timing Source. Set


the interface to the correct timing source.
Insure interfaces at both ends are the same.
NORTEL-XX>enable
NORTEL-XX#configure
NORTEL-XX(config)#controller t1 1/0/1
NORTEL-XX(config-controller)#clock
source line-loop
Confirm timing configuration change:
NORTEL-XX#clear counters
NORTEL-XX#show controller t1 1/0/1

201

Appendix B: CLI System Readiness Check


This appendix gives an example of how to do a basic readiness checkup using the command line interface.
In this example, the following addresses are used:
DOM IP Address = 47.135.203.191
ANC IP Address = 47.135.203.161.
PDSN IP Address = 47.135.203.234
FTP Server IP Address = 47.135.203.18
Note: A DOM with 2 T1s configuration is used in the example
1.

Telnet to <DOM node> (i.e., specific DOM IP address) via either Console Port (Dial up connection) or
Backhaul if available.

2.

Under DOM prompt (NORTEL-03), type:

NORTEL-03>en
NORTEL-03#show interface
Name
Connection IP Address
In Octets
Out Octets Operational
----------------------------------------------------------------------------------------------------ethernet1/3/1
10.10.32.156/20
0
0
Down
node1/3/1
47.135.203.191/32
0
0
Up
ppp1/3/1
0.0.0.0/0
0
0
Down
ppp1/3/2
10.11.2.2/30
452184005 340
Up
ppp1/3/3
10.11.3.2/30
440974939 51272523 Up
ppp1/3/4
0.0.0.0/0
0
0
Down
t11/3/1
0.0.0.0/0
0
0
Down
t11/3/2
0.0.0.0/0
452184005 330
Up
t11/3/3
0.0.0.0/0
440974939 58087164
Up
t11/3/4
0.0.0.0/0
0
0
Down
-----------------------------------------------------------------------------------------------------

This displays all interface status from DOM. For instance, the DOM Ethernet port is not used, the DOM
node is up, 2 physical links (ppp1/3/2 and ppp1/1/3/3) between DOM and router are up, 2 T1 ports are up,
and 2 T1 ports are not used. This information indicates that the interfaces are up and running.
This only indicates that the DOM side interfaces are up but not necessarily all interfaces running
properly. For example, the show interface command may indicate that the TWO T1s are up, but in fact one
of the T1s may not be carrying any traffic at all. We have to ensure that IP route table is properly filled,
and also check if the displayed values of In Octets Out Octets are incrementing for all the T1s. (In Octets
and Out Octets information is displayed using the show interface command in the enable mode).

Nortel Networks Confidential and Proprietary

202

NORTEL-03>show ip route
IP ROUTING TABLE
---------------------------------------------------------------------------Destination
Flags
Gateway
Owner Interface
---------------------------------------------------------------------------0.0.0.0/0
10.23.1.1
S
ppp1/0/1
10.23.2.1
S
ppp1/0/2
10.25.3.1
S
ppp1/0/3
10.25.4.1
S
ppp1/0/4
10.23.1.0/30
L
10.23.1.2
L
ppp1/0/1
10.23.2.0/30
L
10.23.2.2
L
ppp1/0/2
10.25.3.0/30
L
10.25.3.2
L
ppp1/0/3
10.25.4.0/30
L
10.25.4.2
L
ppp1/0/4
47.135.201.245/32 L
47.135.201.245 L
node1/0/1
3.

Under DOM prompt, type show abis peer in the enable mode as the following example:

NORTEL-03>en
NORTEL-03#show abis peer
--------------------------------------------------------------------PeerAddress Port State RxMsgs Rxhellos
47.135.203.161 5604 UP 18189 8163
---------------------------------------------------------------------This displays the link status between the DOM and the ANC.
4.

Under DOM prompt, type show rrc state in the enable mode as the following example:

NORTEL-03>en
NORTEL-03# show rrc state
----------------------------------------------------------------------------MODEM 0: ADMIN STATE: OPERATING OPERATION STATE: UP
=======================================
SECTOR 16:
ADMIN STATE: UP
OPERATION STATE: UP
=======================================
SECTOR 17:
ADMIN STATE: UP
OPERATION STATE: UP
=======================================
SECTOR 18:
ADMIN STATE: UP
OPERATION STATE: UP
------------------------------------------------------------------------------This displays sector status of the DOM. For instance, each sector under this DOM is up and running.
SECTOR 16 = Sector Alpha, SECTOR 17 = Sector Beta, and SECTOR 18 = Sector Gamma.
We can also use show sector-element command in the enable mode to display similar information:
NORTEL-03#show sector-element
Name
Carrier Sector CAI Channel PN Power Admin
------------------------------------------------------------------------------element1 carrier1 sector1 IS-856 1125 116 25 dBm UP
element2 carrier1 sector2 IS-856 1125 120 25 dBm UP
element3 carrier1 sector3 IS-856 1125 124 25 dBm UP

Nortel Networks Confidential and Proprietary

Oper
UP
UP
UP

203

5.

Under DOM prompt, type show traffic-channel as the following example:

NORTEL-03>en
NORTEL-03#show traffic-channel
BTS BSC
Flow Pri Dn Pkts Up Pkts
------------------------------------------------0x0010 0x000034d6 CCH 128 124
26
0x0011 0x000034d7 CCH 128 183
76
0x0012 0x000034d8 CCH 128 105
2
This information provides how many traffic channels are established between DOM and ANC. The
provided Example displays only CCH (Control Channel) since there is no active traffic channel at the time
the command show traffic-channel is used.
6. From the DOM prompt, ping ANC, PDSN (i.e. ISP), and FTP servers as the following example:
-----------------------------------------------------------------------------------------NORTEL-03>ping 47.135.203.161
Sending 5, 100-byte ICMP Echos to 47.135.203.161, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 0/6/10 ms
NORTEL-03>ping 47.135.203.234
Sending 5, 100-byte ICMP Echos to 47.135.203.234, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 0/6/10 ms
NORTEL-03>ping 47.135.203.18
Sending 5, 100-byte ICMP Echos to 47.135.203.18, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 0/8/10
------------------------------------------------------------------------------------------Repeat these steps for each DOM in DO network.
FTAP and RTAP Tests:
To insure the integrity of the data link between the AT and the RNC, FTAP and RTAP tests should be
performed. These tests are similar to Markov calls in CDMA voice as it sends simulated data packets from
the RNC to the AT to test the link between the two.
The following document should be used as a guideline for FTAP and RTAP tests:

D:\Profiles\mwoodley
My Documents\CDMA

Nortel Networks Confidential and Proprietary

204

Appendix C: Shakedown Form


(NEEDS TO BE UPDATED: IP ADDRESS, FTP INFO, ETC.)
Date/Time:
Site ID/AN#
AT Log file
Site Name
Other Logs
Zone ID
AT Model & Version
Address
Data Collection Tool/Version
Engineer
Network Software Version
Problems Resolved?
Problems

Status:

Handoff between Sectors


Alpha <-> Beta

Database Updated: _____________


Comments/Observation

Alpha <-> Gamma

ALPHA (X)
Datafill
Pilot PN
Height
Orientation
Mech. DT
Antenna
Neighbor List
Datafill

Gamma <-> Alpha

BETA (Y)
Observed

Observed

Datafill
Pilot PN
Height
Orientation
Mech. DT
Antenna
Neighbor List
Datafill

GAMMA (Z)
Observed

Observed

Datafill
Pilot PN
Height
Orientation
Mech. DT
Antenna
Neighbor List
Datafill

Call Initiation/Term.

Call Initiation/Term.

Call Initiation/Term.

Coverage/HO

Coverage/HO

Coverage/HO

Nortel Networks Confidential and Proprietary

Observed

Observed

205

Appendix D: Examples of Past Issues


Problem Description:
Dropped Connections due to long tune away:
Symptoms:
Drops will occur when tuning away to 1x (hybrid mode) for periods longer than 5 seconds.
When the AT tunes to 1x, it looks for the last PN that it was monitoring on 1x and then sees if
there is a stronger pilot. If the PN is the same as the last time, or it has tuned to that PN within
the last 10 minutes, the AT will look for a page, and if it sees no page for it, it goes back to
EV-DO. If the AT does an idle handoff to a new PN, then it must wait and read all the
overhead messaging for that new PN (system parameters, extended system parameters,
neighbor list, channel list, access parameters). This will typically add about 1.2s to the tuneaway time, for each new PN that it does idle handoff to. In an area with no dominant server, it
may hand off to several PNs, and since each new one adds 1.2s, it may drop the EV-DO call.
Another possible cause of a long tune away can occur when the Ec/Io of the reference 1x pilot
drops below -16dB. When this occurs the AT will perform a rescan of the entire neighbor list
which may take some time if there is a long neighbor list, or no dominant pilot to lock onto.
(i.e. areas with pilot pollution issues).
It should be noted that tune-away problems are caused by the 1x system, and not the EV-DO
system.
How to recognize a long tune away:
Figure 1 EVDO RFO graph of RF conditions during and after a long tune away

Nortel Networks Confidential and Proprietary

206

Note: RF conditions are good a start of tune away to 1x.


The figure above shows a graph view from EVDO RFO of the RF characteristics before and
after a long tune away. During a long tune away the ATs reported RX power and TX power
will appear constant until the AT retunes to EVDO.
Figure 2 CDMA RFO graph of RF conditions during a tune away

Nortel Networks Confidential and Proprietary

207

Figure 3 CDMA RFO Grid view of tune away

On a prior tune away, the mobile saw PN 324 with Ec/Io just 2dB below PN 320. As a result,
on the next tune away the AT tuned to PN 324 where, at that time, the Ec/Io was very poor.
Resolutions / Workarounds:

Insure neighbor reciprocity to reduce chances that mobile will idle handoff to a new PN
but cannot get back to the original if it needs to.
Avoid having neighbor lists with sites 2 tiers away from the reference site.
If a long tune away is the result of RF optimization needs on the 1x network, permission
from the customer will be required before going forward with any changes to the
underlying 1x network.

Nortel Networks Confidential and Proprietary

208

Required Tools:
EVDO RFO
CDMA RFO
XCAP-DO
Problem Description:
Connection Deny received from RNC if AT sends Route Update while RNC is in an Await
TCC state. This issue is more commonly referred to as an RNC State mismatch.
Symptom:
Typically occurs after the first access Attempt (Connection Request) following an AF in the
AT logs. This occurs when the mobile does not receive the TCA due to tune away or because
it was in poor coverage. The state mismatch will occur when a Route Update (Connection
Request) is received when the RNC is in a CSMAwaitTCCState rather than the
CSMDormantState. When the mismatch occurs the system will send a Connection Close with
Reason Code 2 which equates to a Connection Deny. The Connection Deny means that
system resources are currently unavailable.
Resolution:
In prior releases there was a 5 sec timer associated with the tear down of the MAC channel. In
a recent patch to release 2.1, this timer has been removed to insure that Connection Closes
happen more quickly.
In addition to removing the MAC channel timer to improve connection close performance, the
2.1 release also has been improved such that if a new Connection Request is received while a
previous Connection is not closed out, the system will release the prior connection and set up
the new one without pegging an Access Failure.
Problem Description:
Cannot exceed 1.5MBps on FL
Symptoms:
In good RF conditions where C/I >=10dB forward link throughput does not exceed 1.5MBps.
Resolution or Workaround:

Insure that backhaul T1s are bundled and not separate. If T1s are not bundled,
throughput will be limited to 1.544MBps.

Log into the backhaul aggregate router and insure that there are no Path line-code
errors on configured T1s.

Insure that the TCP window size and MTU settings for data collection tool are set to
the optimum configuration as discussed in the throughput troubleshooting section
below:

Inusre IP header compression is enabled.

Nortel Networks Confidential and Proprietary

209

Problem Definition:
Dropped connection during connection close This condition is more commonly referred
to as All or Nothing.(i.e. the connection is granted all or the connection is denied
nothing.
Symptoms:
In some cases after an FTP session has completed and the system is in the process of
closing the connection, the AT may send a handoff Route Update prior to receiving the
Connection Close (Race Condition). In these situations the system may send the TCA for
the Route Update prior to the Connection Close. If the AT receives the Connection Close
prior to receiving the TCA it may close out the connection without sending a TCC. As
such, the system closes the connection with an error instead of a normal connection. This
pegs as a drop, with reason code 2.
Resolution:
The system should ignore Route Updates received after a Connection Close has been sent
thus avoiding this problem. The patch (.21) to 2.1 that speeds up the Connection Close
will also help prevent occurrences of this symptom.

Nortel Networks Confidential and Proprietary

210

S-ar putea să vă placă și