Nice Perform Troubleshooting Guide - Rev. A2

NICE Perform®
Release 3.1
Troubleshooting Guide
February 2009
385A0755-01 Rev. A2
®
Insight from Interactions TM
Information in this document is subject to change without notice and does not represent a
commitment on the part of NICE Systems Ltd. The systems described in this document are
furnished under a license agreement or nondisclosure agreement.
All information included in this document, such as text, graphics, photos, logos and images, is the
exclusive property of NICE Systems Ltd. and protected by United States and international
copyright laws.
Permission is granted to view and photocopy (or print) materials from this document for personal,
non-commercial use only. Any other copying, distribution, retransmission or modification of the
information in this document, whether in electronic or hard copy form, without the express prior
written permission of NICE Systems Ltd., is strictly prohibited. In the event of any permitted
copying, redistribution or publication of copyrighted material, no changes in, or deletion of, author
attribution, trademark legend or copyright notice shall be made.
All contents of this document are: Copyright © 2009 NICE Systems Ltd. All rights reserved.
This product is covered by one or more of the following US patents:
4,893,197 5,185,780 5,216,744 5,274,738 5,289,368 5,325,292 5,339,203
5,396,371 5,446,603 5,457,782 5,819,005 5,911,134 5,937,029 6,044,355
6,115,746 6,122,665 6,192,346 6,246,752 6,249,570 6,252,946 6,252,947
6,330,025 6,542,602 6,564,368 6,694,374 6,728,345 6,775,372 6,785,369
6,785,370 6,856,343 6,865,604 6,870,920 6,871,229 6,880,004 6,937,706
6,959,079 6,965,886 6,970,829 7,010,106 7,010,109 7,058,589 7,085,728
7,203,655 7,240,328 7,305,082 7,333,445 7,346,186 7,383,199 7,386,105
7,436,887
360o View, ACTIMIZE, Actimize logo, Alpha, Customer Feedback, Dispatcher Assessment, Encorder,
eNiceLink, Executive Connect, Executive Insight, FAST, FAST alpha Blue, FAST alpha Silver, FAST
Video Security, Freedom, Freedom Connect, IEX, Interaction Capture Unit, Insight from Interactions,
Investigator, Last Message Replay, Mirra, My Universe, NICE, NICE logo, NICE Analyzer, NiceCall,
NiceCall Focus, NiceCLS, NICE Inform, NICE Learning, NiceLog, NICE Perform, NiceScreen, NICE
SmartCenter, NICE Storage Center, NiceTrack, NiceUniverse, NiceUniverse Compact, NiceVision,
NiceVision Alto, NiceVision Analytics, NiceVision ControlCenter, NiceVision Digital, NiceVision
Harmony, NiceVision Mobile, NiceVision Net, NiceVision NVSAT, NiceVision Pro, Performix, Playback
Organizer, Renaissance, Scenario Replay, ScreenSense, Tienna, TotalNet, TotalView, Universe, Wordnet
are trademarks and registered trademarks of NICE Systems Ltd. All other registered and unregistered
trademarks are the property of their respective owners.
Applications to register certain of these marks have been filed in certain countries, including Australia,
Brazil, the European Union, Israel, Japan, Mexico, Argentina and the United States. Some of such
registrations have matured to registrations.
385A0755-01 Rev. A2
For assistance please contact your local supplier or the nearest NICE Systems Customer Service
Center:
EMEA Region: (Europe, Middle East, Africa)
Tel: +972-9-775-3800
Fax: +972-9-775-3000
email: support@nice.com
APAC Region: (Asia/Pacific)

Tel: +852-8338-9818
Fax: +852-2802-1800
email: support.apac@nice.com
The Americas Region: (North, Central, South America)

Tel: 1-800-NICE-611
Fax: +720-264-4012
email: support.americas@nice.com
Israel:
Tel: 09-775-3333
Fax: 09-775-3000
email: support@nice.com
NICE invites you to join the NICE User Group (NUG).

Visit the NUG Website at www.niceusergroup.org, and follow the online instructions.
International Headquarters-Israel North America

Tel: +972-9-775-3100 Tel: 1-800-663-5601
Fax: +972-9-775-3070 Fax: +201-356-2197
email: info@nice.com email: na_sales@nice.com
United Kingdom Germany
Tel: +44-8707-22-4000 Tel: +49-(0)-69-97177-0
Fax: +44-8707-22-4500 Fax: +49-(0)-69-97177-200
France Hong-Kong
Tel: +33-(0)1-41-38-5000 Tel: +852-2598-3838
Fax: +33-(0)1-41-38-5001 Fax: +852-2802-1800
Feedback
All queries, comments, and suggestions are welcome!

• If you have access to niceweb.nice.com, click here to open a feedback form
-or-
• Email: nicebooks@nice.com
For more information about NICE, visit www.nice.com

Revision History
Troubleshooting Guide
Revision Modification Date Description
A1 December 2008 • Added the following cross-server troubleshooting

flows:
• Playback Troubleshooting Flow on page 77
• Archiving Troubleshooting Flow on page 117
• Multi Data Hub Troubleshooting Flow
on page 141
• Added a section for the playback error “There are no
recordings to retrieve”. See Understanding
Playback Retrieval on page 116.
• Added a section for the Log Analyzer tool. See Using
Log Analyzer to Check Logs on page 312.
• Added an explanation for the “Chain not found”
message. See Troubleshooting the “Chain Not
Found” Error Message on page 352.
• Added appendix for displaying NICE system
information in the System Administrator. See Enabling
Display of NICE System Information in System
Administrator on page 573.
A2 February • Added cross references and small changes to the

Playback Troubleshooting Flow on page 77.
• Added cross references and small changes to the
Archiving Troubleshooting Flow on page 117.
Blank page for double-sided printing.
Contents
1
Introducing NICE Perform Troubleshooting 23
About this Troubleshooting Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Sources of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Finding the Technical and User Guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Setting an Alert for any Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Synchronizing the Guides to Your Local Machine . . . . . . . . . . . . . . . . . . . . 33
Indexing and Searching the Guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2
Recording Troubleshooting Flow 37
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Checking Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
General Recording Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Troubleshooting Flow Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Troubleshooting Call Failures using Exceptions . . . . . . . . . . . . . . . . . . . . . 54
Troubleshooting No New Calls in Business Analyzer . . . . . . . . . . . . . . . . . . 59
Troubleshooting Calls with Poor Audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Troubleshooting Calls with Audio from Several Calls . . . . . . . . . . . . . . . . . 67
Troubleshooting TRS Recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Troubleshooting Integration Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3
Playback Troubleshooting Flow 77
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Understanding the Playback Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
General Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Contents 7
NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting Playback Problem for All Calls . . . . . . . . . . . . . . . . . . . . . 83
Troubleshooting Playback Problem for Specific Calls . . . . . . . . . . . . . . . . . 85
Setting the Log Level in System Administrator . . . . . . . . . . . . . . . . . . . . . . . 87
Setting the Log Level in the .Config File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Setting the Player Log Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Checking the Workstation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Checking the File Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Checking the Locate Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Checking the Storage Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Troubleshooting Player Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Playback Logic Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Logger Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Storage Center Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Checking the Playback Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Sample Player Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Sample Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Storage Center Disconnection Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Storage Center with Playback Logic Log . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Logger Disconnection Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Checking the NiceScreen Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Example of Troubleshooting using the Logs. . . . . . . . . . . . . . . . . . . . . . . . 108
PlaybackAdministration.txt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
LocateLog.txt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Collecting the Server Logs using the NICE Perform Logs Collector Tool . 113
Collecting the Client Logs using NICE Perform Logs Collector Tool. . . . . 113
Copying the Playlist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Dumping the Client Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Understanding Playback Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Other Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4
Archiving Troubleshooting Flow 117
General Archiving Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Tape/DVD Archiving Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Contents 8

Media Library Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Backup Server Device Detection Troubleshooting . . . . . . . . . . . . . . . . . . . 123
DVD-RAM Devices Not Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
USB Tapes Not Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
SCSI Tapes Not Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Other Backup Server Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Storage Center Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Setting the Log Level to DEBUG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Checking for Archived Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Gathering Storage Center Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Checking the Storage Retention and Deletion Values . . . . . . . . . . . . . . . . 133
Verifying that the Storage Center Database is Selected . . . . . . . . . . . . . . 133
Verifying the Used and Free Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Verifying the Storage Unit Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Checking that the Archiving Rule Inserts New Calls . . . . . . . . . . . . . . . . . 135
Troubleshooting the Archiving Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Checking the Storage Center Archiving Tables . . . . . . . . . . . . . . . . . . . . . 137
Checking the Storage Center Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Checking the Nice Storage Streaming Service Logs . . . . . . . . . . . . . . . . . 139
Checking the Player Logs for Storage Center Errors . . . . . . . . . . . . . . . . . 139
Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5
Multi Data Hub Troubleshooting Flow 141
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Multi Data Hub Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
System Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Rule Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Business Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Playback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Quality Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
My Universe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Contents 9

Interaction Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Storage Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Checking Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Checking Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Troubleshooting Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Troubleshooting Communication Problems between Primary and Secondary
Data Hubs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Troubleshooting Database Structure Differences . . . . . . . . . . . . . . . . . . . 147
Stopping Replication When Upgrading All Data Hubs . . . . . . . . . . . . . . . . 147
Troubleshooting Saving in a Multi Data Hub Environment . . . . . . . . . . . . . 148
Changing Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Enabling MSDTC to Function in Multi Site Environments. . . . . . . . . . . . . . 148
Other Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6
Media Encryption Troubleshooting Flow 151
Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Changing the Logging Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Checking Server Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Checking the Logger/ScreenAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Checking the SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Checking the KSM Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Verifying the KSM is Functioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Checking the Client-side Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Checking the Server-side Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Checking for Certificate Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Trap Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Trap Messages for CIM Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Trap Messages for KSM Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
SAMS Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
ScreenAgent Module Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Verifying the IIS User Permissions for the KSM Configuration . . . . . . . . . 164
Contents 10

Using a Fully Qualified Domain Name instead of a Host Name . . . . . . . . . 166
Data Not Entered in the nice_crypto Database . . . . . . . . . . . . . . . . . . . . . . 169
Renewing Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Screens Not Recorded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
ScreenAgent Not Working Properly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Ensuring Connection between SQL Server and Applications Server . . . . 175
Installing the CA certificate on Vista Workstations. . . . . . . . . . . . . . . . . . . 175
7
Channel Mapping Troubleshooting Flow 177
Channel Mapping Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . 179
Identifying a Channel Mapping Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Verifying the Channel Mapping Configuration . . . . . . . . . . . . . . . . . . . . . . 181
Troubleshooting an Out of Memory Error . . . . . . . . . . . . . . . . . . . . . . . . . 183
Troubleshooting Two Users Updating Channel Mapping Concurrently . . . 184
Troubleshooting Saving in a Multi Data Hub Environment . . . . . . . . . . . . . 184
Adding a New Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Open Case Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Open Case Procedures for System Administrator Parameters (CLS) . . . . 186
Open Case Procedures for the RCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Troubleshooting Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Error Message appears when Updating Configuration . . . . . . . . . . . . . . . 190
A New or Modified Mapping Definition Does Not Work . . . . . . . . . . . . . . . 191
New Channel Mapping Definitions are not Saved . . . . . . . . . . . . . . . . . . . 191
Channel Mapping does not Work! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Channel Mapping in System Administrator Differs from what is in Effect . . 192
Channels Definitions: Logger does not Appear . . . . . . . . . . . . . . . . . . . . . 192
Channels Definitions: Input Channels do not Appear . . . . . . . . . . . . . . . . 193
Source Definitions: Cannot define Virtual Extensions . . . . . . . . . . . . . . . . 193
Source Definitions: Physical Switch does not Appear . . . . . . . . . . . . . . . . 193
Dynamic Mapping: Screen Logger does not Appear . . . . . . . . . . . . . . . . . 194
Dynamic Mapping: Virtual Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Contents 11

Deleting or Changing Logger, Switch, or CLS Definitions . . . . . . . . . . . . . 194
Procedure for Reassigning a Logger to a Different CLS . . . . . . . . . . . . . . 195
8
Applications Troubleshooting Flow 197
Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Checking the Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
Troubleshooting a Communications Failure . . . . . . . . . . . . . . . . . . . . . . . . 202
Troubleshooting Communications Failure with Importer or Reporter . . . . 203
Checking the Applications Server Services . . . . . . . . . . . . . . . . . . . . . . . . 205
Using NICE Services Configuration Manager . . . . . . . . . . . . . . . . . . . . . . 206
Solving Application Services User Rights Problem . . . . . . . . . . . . . . . . . . 207
Checking the Installation Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Checking the Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Setting the Log Level in System Administrator . . . . . . . . . . . . . . . . . . . . . 209
Setting the Log Level in the Config File . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Checking the Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Solving Desktop Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Verifying the .NET Framework Versions . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Set Security Wizard Continues to Appear After Installation . . . . . . . . . . . . 218
Problem Occurs on One Workstation . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Problem Occurs on All Workstations . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Solving Set Security Wizard Hangs or Displays an Error . . . . . . . . . . . . . . 221
Set Security Manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Set Security Manually Using Caspol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Solving Post Installation HTTP Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Verifying the Applications are Not Loaded under the Internet Zone . . . . . . 228
Configuring NICE Perform as a Trusted Web Site . . . . . . . . . . . . . . . . . . . 228
Checking NICE Perform Initialization Errors . . . . . . . . . . . . . . . . . . . . . . . 229
Solving Applications Server Configuration Problem . . . . . . . . . . . . . . . 229
Troubleshooting Remote Name Could Not be Resolved Error . . . . . . . 230
Troubleshooting Unable to Connect to the Remote Server Error . . . . . 230
Troubleshooting Serialization Problem . . . . . . . . . . . . . . . . . . . . . . . . 230
Contents 12

Solving Service Unavailable Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
IIS AppPool Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Non-Paged Pool Memory Leak Problem . . . . . . . . . . . . . . . . . . . . . . . 233
Disabling Custom Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Solving ActiveX Control Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Troubleshooting Application Access (Power Management Issue) . . . . . . . 234
Troubleshooting Browser Problems Using Logs . . . . . . . . . . . . . . . . . . . . 237
Explorer Log (on the Client Machine) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Desktop Client Logs (on the Client Machine) . . . . . . . . . . . . . . . . . . . . . . 237
IIS - WWW Log File (on the Applications Server) . . . . . . . . . . . . . . . . . . . 238
IIS - HTTPERR Log File (on the Applications Server) . . . . . . . . . . . . . . . . 239
Application Event Viewer (on the Applications Server) . . . . . . . . . . . . . . . 239
Solving ActiveX Control Error on the Applications Server . . . . . . . . . . . . . 241
Optimizing Client Server Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
Optimizing Client Server Performance on Computers without Internet . . . 244
Redirecting the Client Request to the Local Server . . . . . . . . . . . . . . . . . . 244
Modifying the Security Settings in the Internet Explorer . . . . . . . . . . . . . . 245
Solving Email Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Verifying the SMTP Service Installation and Configuration . . . . . . . . . . . . 246
Troubleshooting SMTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Troubleshooting Crystal Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
Using the Process Explorer Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Using the Desktop Tester . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
9
Databases Troubleshooting Flow 253
General Databases Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 255
General Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Checking the Windows Event Viewer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Checking the SQL Server Error Logs and Events . . . . . . . . . . . . . . . . . . . . 256
Checking SQL Jobs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Checking the Connection to the NICE Perform Database Server . . . . . . . . 257
Checking the Database Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Checking for Missing Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Contents 13

Checking Free Space in the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Verifying the Log on as a Service User Rights Assignment . . . . . . . . . . . . 260
Verifying the Service Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
Verifying Domain Users Assigned to the User Rights Assignments . . . . . . 261
Checking the SQL Server Installation Folder . . . . . . . . . . . . . . . . . . . . . . . 263
Monitoring Performance Objects and Counters . . . . . . . . . . . . . . . . . . . . . 265
Checking for Performance Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
Scanning the Disk for Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Checking Replication in Multi Data-Hub Environments . . . . . . . . . . . . . . . 268
Changing the Database Passwords in SQL Authentication . . . . . . . . . . . . 269
Changing the nice sa Login Password . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Changing the nice Login Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Checking the Database Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Increasing the Size of a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
Checking Performance Using the NICE Performance Collector . . . . . . . . . 274
Defining Autogrowth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Using the NICE DBSpace Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Analyzing the Database Using NICE DBAgnose . . . . . . . . . . . . . . . . . . . . . 284
General Data Mart Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Data Mart Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Checking the Database Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Recreating Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
Checking the Data Mart Database Version . . . . . . . . . . . . . . . . . . . . . . . . 289
Checking Connectivity Between the Database and Data Mart Servers . . . 289
Checking a Data Mart Job Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
10
Customer Feedback Installation Troubleshooting Flow 291
General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
Troubleshooting Customer Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Troubleshooting Flow Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Troubleshooting - General Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Troubleshooting - DTMF Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
Troubleshooting - CTI Info or Business Data Routing . . . . . . . . . . . . . 298
Contents 14

Surveys and Interactions are not Properly Correlated . . . . . . . . . . . . . . . . 301
11
High Density Logger Troubleshooting Flow 303
General High Density Logger Troubleshooting Flow . . . . . . . . . . . . . . . . . 304
Verifying the Logger Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
Generating a Build Content Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
Checking Logger Initiation Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
Setting the Debug Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
Checking the Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
Using Bug Find to Check Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
Using Log Analyzer to Check Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
Checking for Logger Configuration Errors . . . . . . . . . . . . . . . . . . . . . . . . . 314
Checking Logger Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Checking Map Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Checking RCM Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
Synchronizing the Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
Testing the Logger COTS using the NICE Test Kit . . . . . . . . . . . . . . . . . . . 318
Tests and Log File Locations in a Windows Environment . . . . . . . . . . . . . 318
Tests and Log File Locations in a DOS Environment . . . . . . . . . . . . . . . . 320
Testing the NICE Boards using the NICE Test Kit . . . . . . . . . . . . . . . . . . . . 320
Testing the Logger using the NICE PST. . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Shutting Down and Starting Up the Logger . . . . . . . . . . . . . . . . . . . . . . . . . 322
Shutting Down and Starting CLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
Using the Nicelog Setup Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
Contents 15

12
VoIP Troubleshooting Flow 329
Isolating the VoIP Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
Troubleshooting VoIP Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Troubleshooting the VoIP Logger LAN Connection . . . . . . . . . . . . . . . . . . 332
Using Logs to Check that the VoIP Logger is Configured to Receive RTP
Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
Checking the RTP Packets using the Sniffer . . . . . . . . . . . . . . . . . . . . . . . 337
Troubleshooting Network and Environment Issues . . . . . . . . . . . . . . . . . . 338
Checking the CLS Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
Checking Channel Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
Using the Sniffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
Troubleshooting N+1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
mascm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Configuring after Replacing a NIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Replacing a CISCO Phone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Troubleshooting a Packet Size Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Troubleshooting RTP Timestamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
Troubleshooting RDP Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
Troubleshooting the “Chain Not Found” Error Message . . . . . . . . . . . . . . 352
Troubleshooting Active VoIP Integrations . . . . . . . . . . . . . . . . . . . . . . . . . 353
13
Interactions Center Troubleshooting Flow 355
Interactions Center Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . 356
Interactions-based Call Recording Data Flow . . . . . . . . . . . . . . . . . . . . . . . 357
Interactions-based Call Recording Data Flow Details . . . . . . . . . . . . . . . . 358
Understanding the Total Call Recording Data Flow . . . . . . . . . . . . . . . . . . 360
Understanding the Total Call Recording Data Flow Details . . . . . . . . . . . . 361
Understanding the Screen Call Recording Data Flow . . . . . . . . . . . . . . . . . 363
Screen Call Recording Flow Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
Contents 16

Open Case Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
RecordingDecisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
Call Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
DB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
TRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
RCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
System Administrator Parameters (CLS Plug-ins) . . . . . . . . . . . . . . . . . . . 371
Checking the SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
Setting the Reporting Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Running the RCM Information Dumper . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
Checking the CLS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
Starting/Stopping the Interactions Center Services . . . . . . . . . . . . . . . . . . 378
Checking the Status of Interactions Center Components. . . . . . . . . . . . . . 379
Checking Components with the NICE Services Configuration Manager . . 379
Checking Processes Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
Troubleshooting using the Interactions Center Logs . . . . . . . . . . . . . . . . . 381
Examples of Troubleshooting using the Logs. . . . . . . . . . . . . . . . . . . . . . . 383
Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
Checking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
Checking the CLS Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
Query the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
In the Business Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
Fine-tuning the CLS Server Configuration Parameters. . . . . . . . . . . . . . . . 411
Isolating a Screen Problem Further . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
Shutting Down and Starting CLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
14
NMS Troubleshooting Flow 417
Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
Contents 17

Troubleshooting NMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
Client-Side Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
Pinging the Monitored Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
Checking the SNMP Service Configuration . . . . . . . . . . . . . . . . . . . . . . . . 421
Verifying that SNMP Agent is Sending Traps . . . . . . . . . . . . . . . . . . . . . . 425
Repairing the SNMP Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
Reinstalling the NICE Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
Server-Side Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
Collecting the System Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
Checking the NMS Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
Checking the CA-Unicenter Service and Sub-Services . . . . . . . . . . . . 428
Checking the Unicenter TND Agent Technology Service . . . . . . . . . . . 432
Checking the SNMP Trap Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
Checking the SNMP Nice Trap Formatter Service . . . . . . . . . . . . . . . . 434
Checking the Trap Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
Checking the MIB File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
Checking Polling Timeout Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
Troubleshooting Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
Scenario 1: Monitored Server Does Not Appear in the 2D Map . . . . . . 442
Scenario 2: Drilldown Fails in the 2D Map . . . . . . . . . . . . . . . . . . . . . . 442
Scenario 3: Only Some Agents are Reporting . . . . . . . . . . . . . . . . . . . 442
Scenario 4: There are No Traps in the Console . . . . . . . . . . . . . . . . . . 442
Scenario 5: No Email Notifications Are Received . . . . . . . . . . . . . . . . 443
Changing the System Administrator Server Name . . . . . . . . . . . . . . . . . . . 445
Checking for a CAClient Serialization Issue . . . . . . . . . . . . . . . . . . . . . . . . 445
Using the NMSUtil Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
15
Telephony Services Server Troubleshooting Flow 449
Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
Gathering Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
Checking the Telephony Services Server Configuration . . . . . . . . . . . . . . 452
Checking the Dialogic Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Contents 18

Troubleshooting the Dialogic Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
Troubleshooting the Dialogic Analog Interface . . . . . . . . . . . . . . . . . . . . . 454
Troubleshooting the Dialogic Digital Interface . . . . . . . . . . . . . . . . . . . . . . 457
Troubleshooting the Dialogic IP Interface . . . . . . . . . . . . . . . . . . . . . . . . . 460
Using the ISDIAG Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
Testing the Extension Using the Dialogic Voice Demo . . . . . . . . . . . . . . . 464
Testing an Analog Extension Using the Dialogic Voice Demo . . . . . . . 464
Testing a Digital Extension Using the Dialogic Voice Demo . . . . . . . . . 467
Using the Universal Dialogic Diagnostics Utility . . . . . . . . . . . . . . . . . . . . 470
Troubleshooting using the NICE Playback Telephony Layer Tester . . . . . 474
Checking the Telephony Services Server Service . . . . . . . . . . . . . . . . . . . 478
Checking the Applications Server Service . . . . . . . . . . . . . . . . . . . . . . . . . 478
Checking the Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
Setting the Logs Reporting Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
Viewing the Telephony Services Server Logs . . . . . . . . . . . . . . . . . . . . . . 480
Viewing the NICE Playback Streaming Services Logs . . . . . . . . . . . . . . . . 481
Viewing the NICE Playback Administration Service Logs . . . . . . . . . . . . . 481
Viewing the NICE Player Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
Checking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
Troubleshooting Playback to Extension Problems . . . . . . . . . . . . . . . . . . . 482
Checking Ports Using Port Tester . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
Shutting Down and Starting the Telephony Services Server . . . . . . . . . . . 485
Troubleshooting Dial Plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486
16
NiceScreen Troubleshooting Flow 489
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
Isolating the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
Checking the NiceScreen Logger Service Status . . . . . . . . . . . . . . . . . . . 492
Checking the ScreenAgent Interactions Center Registration . . . . . . . . . . . 493
Checking the Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
Checking SAMS for Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
Contents 19

Isolating the Problem Further . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
Checking the NiceScreen Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
Checking the ScreenAgent Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
Checking the RCM Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
Checking the Call Server Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
Checking the Call Status in the Database . . . . . . . . . . . . . . . . . . . . . . . . . 497
Checking the Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
Checking the Naming Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498
Checking Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498
Verifying the ScreenAgent Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 500
Testing the ScreenAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501
NICE ScreenAgent Installation/Operational Problems . . . . . . . . . . . . . . . . 503
NICE ScreenAgent Optimization and Performance Issues . . . . . . . . . . . . . 505
NiceScreen Logger Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . 507
17
Storage Center Troubleshooting Flow 513
Common Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
Installation Freezes: SNMP Dependencies . . . . . . . . . . . . . . . . . . . . . . . . 514
Archiving Failure: Multiple Network Interface Cards . . . . . . . . . . . . . . . . . 515
Storage Center Is Not Communicating with the NMS . . . . . . . . . . . . . . . . 515
Verify SCLoader Service User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
Verify the SnmpAgent Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
Verify the SNMP Service Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
Allowing Duplicate Archiving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
Storage Center Does Not Archive Calls . . . . . . . . . . . . . . . . . . . . . . . . . . 521
Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
Verifying the Storage Center Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
Implementing the Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . . 527
Verifying that the Storage Center Database is Selected . . . . . . . . . . . . . . 527
Verifying the Used and Free Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
Gathering Storage Center Activity Statistics . . . . . . . . . . . . . . . . . . . . . . . 528
Contents 20

Checking that the Archiving Rule Inserts New Calls . . . . . . . . . . . . . . . . . 528
Troubleshooting the Archiving Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
Checking the Storage Center Archiving . . . . . . . . . . . . . . . . . . . . . . . . . . 531
Checking the Storage Center Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
Checking the Nice Storage Streaming Service Logs . . . . . . . . . . . . . . . . . 532
Verifying Playback from Storage Center Only . . . . . . . . . . . . . . . . . . . . . . . 533
A
Collecting Information Using the Log Collector 537
What is the Log Collector? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
What is a Component? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
What is a Report? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
FAQs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542
Collecting Information from NICE Perform Servers . . . . . . . . . . . . . . . . . . 543
Collecting Information from Your Local Machine . . . . . . . . . . . . . . . . . . . . 550
Saving and Exporting Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
Importing Saved Reports for Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
Scheduling Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
Viewing Collected Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560
Viewing and Saving Reports in other Applications . . . . . . . . . . . . . . . . . . 562
Reports Created with the Log Collector. . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
Server-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
General Server-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
Applications Suite Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564
Playback Telephony Services Server Reports . . . . . . . . . . . . . . . . . . . 566
Logger Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
SQL Server Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
Interactions Center Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
Storage Center Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
Reporter Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
NiceScreen Logger Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
ScreenSense Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
Contents 21

Stream Server Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
Business Services Framework (BSF) Server Reports . . . . . . . . . . . . . 569
CTI Integrations Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
Media Library Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
Audio Analysis Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
VRG Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570
MPCM (Media Provider Controller Manager) Reports . . . . . . . . . . . . . 570
RTS Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570
Client-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
General Client-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
Player Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
ScreenAgent Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
ScreenSense Agent Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
B
Enabling Display of NICE System Information in System
Administrator 573
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
Common Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
Verifying the System Administrator User . . . . . . . . . . . . . . . . . . . . . . . . . . 576
Adding the System Administrator User to Computers not in an Active
Directory Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
Editing DCOM Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582
Setting Security Permissions through the WMI Control . . . . . . . . . . . . . . . 588
Contents 22

1
Introducing NICE Perform
Troubleshooting
This troubleshooting guide includes troubleshooting flows and procedures that are intended for use
by support staff and NICE support engineers in Tier 1, 2 and 3. This guide does not include
procedures for Tier 4.
Contents
About this Troubleshooting Guide................................................................................ 24

Sources of Information .................................................................................................. 26
Finding the Technical and User Guides ....................................................................... 27
Setting an Alert for any Changes .................................................................................. 30
Synchronizing the Guides to Your Local Machine ...................................................... 33
Indexing and Searching the Guides.............................................................................. 34
Chapter 1: Introducing NICE Perform Troubleshooting 23

About this Troubleshooting Guide

This troubleshooting guide includes troubleshooting flows and procedures that are intended for use
by support staff and NICE support engineers in Tier 1, 2 and 3. This guide does not include
procedures for Tier 4.
This guide includes the following troubleshooting:
• Recording Troubleshooting Flow: Includes a general recording troubleshooting flow, as
well as several recording troubleshooting scenarios. This flow refers to the relevant
components, as required.
• Media Encryption Troubleshooting Flow: Includes a troubleshooting workflow, as well as
tables of trap messages and ScreenAgent Management System (SAMS) log messages.
• Channel Mapping Troubleshooting Flow: Includes a channel mapping troubleshooting
flow, as well as provides quick solutions for common problems you face when defining
channel mapping. The procedures are designed for the seasoned user who is familiar with the
NICE Perform system and the System Administrator application. For complete instructions on
using the System Administrator, see the System Administrator’s Guide.
• Applications Server Troubleshooting Flow: Contains a troubleshooting workflow for the
NICE Perform Applications Server, including solving Desktop problems and troubleshooting
Browser problems using Logs. A list of the services, config and log files is also provided.
• Databases Troubleshooting Flow: Includes procedures for troubleshooting NICE Database
Server and Data Mart. These procedures must be perform only by authorized personnel. For
full assistance, contact NICE Systems Customer Services.
• Customer Feedback Installation Troubleshooting Flow: Provides a review of some
information that will aid the installer in understanding the Customer Feedback workflow. Also
provided is a workflow recommending how to proceed when troubleshooting a Customer
Feedback system.
• High Density Logger Troubleshooting Flow: Contains a troubleshooting flow with
troubleshooting tips and procedures for High Density Loggers.
• Interactions Center Troubleshooting Flow: Includes a Interactions Center troubleshooting
workflow, Interactions-based call recording data flow, screen call recording data flow, total
call recording data flow, and other troubleshooting issues.
• NMS Troubleshooting Flow: Includes an NMS troubleshooting flow and other
troubleshooting issues.
• Playback Telephony Services Server Troubleshooting Flow: Includes a troubleshooting
flow and other troubleshooting issues.
• NiceScreen Troubleshooting Flow: Lists some of the most common problems you may
encounter and solutions.
• Storage Center Troubleshooting Flow: Describes common troubleshooting issues and
how to verify that the Storage Center was properly installed. It also includes a troubleshooting
workflow to solve various issues.

• VoIP Troubleshooting Flow: Contains troubleshooting tips and procedures for when you
experience a problem with VoIP.
• Collection Information Using the Log Collector: The Log Collector is an easy-to-use
utility that gathers the information from all your NICE Perform servers and local workstation,
in the form of reports, for easy transference for analysis.

Sources of Information
Sources of Information
There are several additional sources of information.
1. There are also over 150 Technical and User Guides on ExtraNICE (http://www.extranice.com)
• Hardware Guides
• History Release Notes
• Installation Guides
• Reference Guides
• Release Notes
• Switch/Driver Integration Guides
• Upgrade/Migration Information
• User Guides
2. The NICE Perform Release 3.1 Maintenance Guide is especially useful and should be read
together with this guide.
3. Internal (for NICE-use only) sources include:
• Technical Documentation Team Site for technical documents in progress:
http://niceweb.nice.com/sites/Technical_Writers_Documents/default.aspx
• Tier 4 Team Site for tools and utilities:
http://niceweb.nice.com/teams/Tier4_RD/default.aspx
• Tier 3 Team Site: http://niceweb/sites/Tier3_CS/Tier3 working documents and
manuals/Forms/AllItems.aspx
• Product Training Team Site:
http://niceweb.nice.com/sites/ProductTraining_RD/default.aspx

Finding the Technical and User Guides

There are over 150 guides on ExtraNICE.
1. Browse to http://www.extranice.com/. You must have access to the relevant content on
ExtraNICE.
2. If you are accessing ExtraNICE from outside of the NICE network, the following window
appears.
Figure 1-1 ExtraNICE Portal
3. Enter your User name and Password and click on Log On to ExtraNICE. The following
window appears.

Figure 1-2 ExtraNICE Portal Home Page
4. Select Products > NICE Perform > NICE Perform Release 3.1. The following window
appears.
Figure 1-3 NICE Perform Release 3.1
5. Expand the Content Type: Technical and User Guides.

6. Expand the required Files Types to view a list of manuals.

Figure 1-4 List of Manuals

Setting an Alert for any Changes

You can set a daily alert of any changes to the content in SharePoint.
1. In ExtraNICE, select Products > NICE Perform > NICE Perform Release 3.1. The
following window appears.
Figure 1-5 ExtraNICE Portal - NICE Perform Release 3.1
2. Click on NICE Perform Release 3.1. The following window appears.

Figure 1-6 NICE Perform Release 3.1 - Actions
3. Select the Actions drop-down menu.

Figure 1-7 Actions Drop-down Menu
4. Select Alert Me. The following window appears.

Figure 1-8 Alerts Page
5. Select Send a daily summary.

6. Click OK. A daily alert is emailed to you detailing any changes or updates made to the
selected page.
7. Copy the changed files to your local folder.

Synchronizing the Guides to Your Local Machine
Synchronizing the Guides to Your Local Machine

Use a tool of your choice to synchronize the Guides from SharePoint to your local machine.
For demonstration purposes only we describe how to use Colligo Reader.
1. Download and install http://colligo.com/products/sharepoint/reader.asp.
2. Select Start > Program > Colligo Reader > Colligo Reader.
3. From the Site menu, select Download New Sites. The following window appears.
Figure 1-9 Download New Site Window
4. In the URL field, add http://www.extranice.com/EIS/Products/NICEPerform.

5. Add your User name and Password.
6. Click Download. The files are synchronized to your local machine.
7. The files are located in a folder, for example, C:\Documents and
Settings\<username>\Local Settings\Application
Data\ColligoOfflineClient\Storage5\Files\Sites\1\NICE Perform Release 3.1 .

Indexing and Searching the Guides

Use a search tool of your choice to search for information.
For demonstration purposes only we describe how to use Windows Desktop Search.
1. Download and install Windows Desktop Search
http://www.microsoft.com/windows/products/winfamily/desktopsearch/getitnow.mspx.
2. Click on the Indexing icon in the Taskbar.
Figure 1-10 Index Now
3. Select Windows Desktop Search Options. The following window appears.

Figure 1-11 Indexing Options
4. Click Modify. The following window appears.

Figure 1-12 Indexed Locations
5. Select the location with the manuals and click OK.

6. Use your selected search tool to locate the relevant information.
Figure 1-13 Search Results

2
Recording Troubleshooting Flow
This section contains troubleshooting tips and procedures for when you experience a problem with
recording.
Contents
Gathering Information .................................................................................................... 38

Checking Exceptions ..................................................................................................... 39
General Recording Troubleshooting Flow ................................................................... 50
Troubleshooting Call Failures using Exceptions ........................................................ 54
Troubleshooting No New Calls in Business Analyzer................................................. 59
Troubleshooting Calls with Poor Audio ....................................................................... 66
Troubleshooting Calls with Audio from Several Calls ................................................ 67
Troubleshooting TRS Recordings................................................................................. 69
Troubleshooting Integration Issues.............................................................................. 75
Chapter 2: Recording Troubleshooting Flow 37

Gathering Information
In order to narrow down the problem answer the following questions:
1. Is the recording TDM or VoIP?
2. For VoIP, is the recording environment passive or active?
3. For TDM, is the recording environment trunk-side, extension-side or observation?
4. Is the recording total or QM (interaction based)?
5. In a total recording environment is TRS configured?
6. Is the site N+1?
7. Is the recording summed or with speaker separation?
8. Is the channel mapping static or dynamic? If necessary export the channel mapping
definitions. See Exporting Channel Mapping Definitions in the Channel Mapping Guide.
9. Did the problem occur for a specific agent, group of agents or for all agents?
10. Which CTI integration and what version?
11. Check for exceptions by checking the tblExceptionXX table. See Checking Exceptions
on page 39.

Checking Exceptions
Checking Exceptions
Query the database to see the exceptions. Most of the time exceptions are connected to recordings
and not directly to the interaction IDs. In order to see the connection between an exception and
interaction ID, query both the exception details and the related recording (which includes the
interaction ID).
To check the CLS Exceptions in the Database:

1. Query the tblInteractionCatalog table to check for which period the calls occur.
2. Get all the exceptions related to interactions:
SELECT * FROM tblExceptionXX, tblInteractionXX
WHERE
tblExceptionXX.iInteractionID = tblInteractionXX.iInteractionID
3. Get all exceptions related to recordings:
SELECT * FROM tblExceptionXX, tblRecordingXX
WHERE
tblExceptionXX.iRecordingID = tblRecordingXX.iRecordingID
4. Check the vcExceptionDetail field in tblException.
To check the CLS Exceptions in Business Analyzer:

• In Business Analyzer, select the interaction and click the Exceptions tab in the Details pane.
Figure 2-1 Checking the CLS Exceptions in Business Analyzer

Checking Exceptions
Exception Short
ID Meaning Possible Cause Recommended Action
Description
1 Duplicate call start. Duplicate start of call – Problem with the driver or Check the driver or PABX.
another call with the same PABX reports. Collect the driver and CLS log files.
call key has started while this
call was open.
2 Maximum duration Call too long – the call was Either the call was longer than Check if the value of the Call Server
exceeded. open for more seconds than the parameters value, or there parameter op_MaxOpenCallDuration
the value of the registry is a problem with the driver or matches the length of calls in the site.
parameter PABX reports.
MaxOpenCallDuration and
was therefore forcibly
closed.
3 Call flushed while Call flushed – a flush The driver may have gone Check that the driver is up.
open. command was executed down and come up. Review the driver log files, look for
while the call was open. errors. If there is no error no action is
needed.
4 N/A N/A N/A N/A
5 Call start not End call without start Problem with the driver or Check the driver or PABX.
reported. (Default Start Time). PABX reports. Review the driver log files, look for
Could be followed by reports errors. If there are errors, collect the
with exp02 because the start driver and CLS log files.
and end call reports have
different information.
6 N/A N/A N/A N/A

Checking Exceptions
Exception Short
Description
7 No available No available recording There are not enough voice Could be caused by unclosed
recording resource. resource. recording channels to meet recorded calls – check for exp02 calls.
the number of recording If there is a substantial amount of calls
requests in the site. with these exceptions, check RCM
logs for Logger disconnections
(Logger down) and if channels were
allocated after connection
reestablished with Logger.
8 Logger not Logger not responding. Problem with Logger or Check the Logger.
responding. connection to Logger. Check the network connectivity
between the Logger and the CLS.
Check the RCM logs.
9 Unspecified Unspecified recording Unknown failure cause. Collect the Integration and CLS log
recording failure. failure. Received from the RCM. files.
10 Agent logout during Agent logout during the call. Problem with the driver or Check if it is physically possible to
call. PABX call or logout reports. logout during a call.
Collect the Integration and CLS log
files.
11 Too many calls for MaxExtentionOpenCall - too The agent had more Check if the parameter value meets
agent/extension. many open calls for the same simultaneous open segments the requirements of the site.
agent/extension when
than the parameter value. Look for calls with exp02 to see if
compared to the
op_MaxCallPerExt registry Could be a problem with the there is a problem with the call reports.
parameter. driver or PABX call reports.

Checking Exceptions
Exception Short
Description
12 Voice recording Voice recording failed. Error code received from the • Collect the Integration and CLS
failed. telephony server. If this log files.
appears as e12 check the
• Sub-exception 12 310 indicates
sub-exception ID for more
the Logger is down.
information.
no VoIP audio due to a
configuration problem. This is
dependant on the VoIP
integration. Check the
configuration, the forwarding data
passed to the Capture (RCM
logs), the Capture logs, as well as
the forwarding device
configuration. For example, no
audio is received if no or the
wrong forwarding information is
passed to Capture, or if the audio
is not forwarded by the telephony
switch/forwarding device.
13 Screen Logger not Screen logger not Logger is down or network Check the NiceScreen Logger and
responding. responding. issue. ScreenAgent. Check the RCM logs for
errors when calling screen capture to
start record.

Checking Exceptions
Exception Short
Description
14 Screen recording Screen recording failed. If this appears as e14, check • Sub-exception 14 1002 indicates
failed. the sub-exception ID for more an error was received during
information. screen recording. Either
ScreenAgent was disconnected
from the Interactions Center or
ScreenAgent recording failed.
Partial recording (up until the time
of the error) may be available.
there was an unspecified failure.
15 Unmapped voice Unmapped voice recording. Problem in the voice channel Switch the logs to DEBUG.
recording. configuration. Check channels configuration. See if
there is a mapping configured for this
call.
Check the RCM logs.
Check what the RCM received in the
Start request.
16 Unmapped screen Unmapped screen recording. The recording request was If the allocation mode is by IP address
recording. received with empty Station or then check if the agent logged in.
IP address (depends on the Check ports. Verify in RCM logs that
screen allocation mode). the start request for recording the
agent's screen contained the screen
agent's IP.
17 Voice recording retry. Recording voice succeeded May be a temporary failure on Check Logger for possible reasons for
only after retry (partial retry). the Logger. temporary failures.

Checking Exceptions
Exception Short
Description
18 Call Server service Call Server was down during Call Server was down. Check reason for Call Server failure.
shutdown. the call. Collect CLS logs, event viewer and
CPU Performance Monitor.
19 N/A N/A N/A N/A
20 Logger not The Logger did not respond Stop record command arrived No action is needed if there was
responding. to the start record command. before a response for the start indeed a short call; otherwise collect
record request arrived. This CLS and driver log files.
may occur for one of the If the request was for 2 medias, try to
following reasons: understand the recording problem with
• The call was very short (1 the first media.
or 2 seconds). Check the RCM logs.
• The request was for Check the Call Server logs.
2 medias.
• Success in both was
required (usually QA).
• One media failed
immediately.
• Stop record is sent for
both medias.
21 N/A N/A N/A N/A
22 N/A N/A N/A N/A

Checking Exceptions
Exception Short
Description
23 Call start not Stop record without start Usually happens with very No action is needed if there was
reported. (screen or voice). short calls. indeed a short call; otherwise collect
The request was for 2 medias. CLS and driver log files.
Success in both was required If the request was for 2 medias, try to
(usually QA). One media understand the recording problem with
failed immediately. Stop the first media.
record was sent for both Check the RCM logs.
medias.
Check the Call Server logs.
Or may be an internal
problem.
24 Error in stop record Stop record with wrong ID. Usually happens with very No action is needed if there was
request. No start call request was short calls. indeed a short call; otherwise collect
found with this CLS Call ID. The request was for 2 medias. CLS and driver log files.
Success in both was required If the request was for 2 medias, try to
(usually QA). One media understand the recording problem with
failed immediately. Stop the first media.
record was sent for both Check the RCM logs.
medias.
Or may be an internal
problem.
25 Too many requests Too many recording requests May occur due to a problem Collect CLS and driver log files.
for channel. for the same Logger and with the driver or PABX call Check the Call Server logs.
channel (more than 30). reports – calls are not closed.
26 RCM service down RCM was down during the RCM was down. Collect CLS logs, event viewer and
during call. call. CPU Performance Monitor.
Check the RCM logs.
Check the reason that the RCM went
down.

Checking Exceptions
Exception Short
Description
27 Error on complete The contact started after its Problem in the driver or PABX Check the driver or PABX.
interaction start. segment. call or logout reports. Collect CLS and driver log files.
Check the RCM logs for a long period
in DEBUG mode.
28 No available No available recording Not enough Logger recording See if the configuration meets the site
recording resource. resource. channels. needs. If not change it.
The Resource Call Manager Bad resource management Collect CLS and driver log files.
cannot allocate more configuration. Restart RCM.
resources for this initiator. There maybe a problem with Check the RCM logs.
calls that are not closed and
Could be a problem with the driver or
therefore resources are not
PABX reports.
being freed up.
29 Error on complete Segment was open when the Could be a problem with the Check the driver or PABX.
interaction close. contact closed. drivers or PABX call reports. Collect CLS and driver log files.
30 Time Interval Block dummy call was Call Server restarted. Collect CLS log files.
recording aborted. closed due to a Call Server Check why Call Server was closed.
restart.
31 Client disconnect. Client was disconnected Client was disconnected. Collect CLS log files.
during the open block. Check why the client was
disconnected.
32 N/A N/A N/A N/A

Checking Exceptions
Exception Short
Description
33 Stop on demand not Stop on demand was Stop on demand was No action required.
by initiator. performed on the interaction performed on the interaction
recording by a client who recording by a client who was
was not the recording not the recording initiator.
initiator.
34 Invalid call time Time field was changed by Call Server error. Illegal time Collect CLS log files.
report. the DB Server. Interaction parameters were reported. Check the Call Server logs.
was inserted with time value
lower then 1970. Stop time
was lower than start time.
35 RCM service not The Call Server received a RCM is too busy. Collect CLS logs, event viewer and
responding. timeout error on the request Internal RCM error. CPU Performance Monitor.
to the RCM. Check the RCM logs.
Event caused the RCM not to
respond, for example,
detected many Loggers
initializing.
36 Business data value String field was truncated by Business data length is Collect CLS and driver log files.
too long. the DB Server. Call Server incorrectly specified. Check the maximum business data
sent a string longer than length, change it in the DB and restart
allowed by the DB schema. the CLS.

Checking Exceptions
Exception Short
Description
37 Failed to record VoIP. Failed to record. Audio not This usually results from a • Run a sniffer to check if packets
found in VoIP recording. failure to sniff VoIP packets by were coming in.
the Logger.
• Collect CLS and Logger log files.
After the Start command to
the Logger, the RCM waits 2.5 • Check for channel mapping
seconds and checks if the failures.
audio packets were received. • Check the RCM logs for the
allocated channel.
• Check the Capture logs.
38 Failed to update Update VoIP (which updates Occurs, for example, when Check that the device in the Update
VoIP data. the IP for an open request the update is for an VoIP sent from driver is mapped.
and its participant) sent by unmapped device or a closed
the Driver failed. request, or because the
update was not received by
the RCM.
39 Complete interaction Split Contact - Segments for Interaction Router could not No action required.
handled on multiple contact handled on more find mapped segment on the
servers. than one Interactions Center. Interactions Center that was
selected for the other open
segment under the same
contact.
40 Recording time Recording Time Mismatch – Problem with integration. Check the Call Server logs.
mismatch. recording time was out of call Calls may be reported late to
time range. the CLS when the system is
under stress.

Checking Exceptions
Exception Short
Description
41 Unknown Initiator. At the start of the call, the User is not defined under Ensure the user exists and is defined
initiator participant has no UserAdmin. in the DB.
User ID, that is, the User ID
was not found in the
UserAdmin Database.

General Recording Troubleshooting Flow

See Troubleshooting Flow Details on page 52 for a description of every step mentioned in the
following flow.
Figure 2-2 General Recording Troubleshooting Flow


Troubleshooting Flow Details

1. Is the problem a system-level problem, that is, no recording at all?
2. If the problem is a system-level problem:
• Check that all services are up and running (Applications Server, Interactions Center and
Loggers).
• Check that the Loggers are up.
• Check the communication between servers.
• Check that there is time synchronization between the various NICE servers.
3. Check if the problem is a data or audio problem, by running a query in Business Analyzer. If
there are no interactions, continue with Step 7.
4. Is the problem a Logger-level problem?
5. If the problem is a Logger-level problem:
a. Verify the Logger configuration in System Administrator, check that the Logger was
added to the CLS and that it is up (by selecting it and verifying that you see all the
Logger's details).
b. Check the Logger communication with the Applications Server and Interactions Center.
c. Check that the Loggers are time synchronized.
d. For TDM environments, see the High Density Logger Troubleshooting Flow.
e. For VoIP environments, see the VoIP Troubleshooting Flow.
6. Make a test call:
a. Monitor the test call.
b. If the test call does not appear in the Monitor, proceed with Step 7.
c. If the test call appears in Monitor, try to ROD the test call.
d. If the test call is not recorded, check the mapping.
• For a trunk-side recording environment, add the participant trunk group and
participant trunk number columns in Business Analyzer, and verify which trunk
groups and trunk numbers were assigned to the interactions. Check the mapping for
these trunks.
e. For TDM trunk-side recording, check the trunk connections.
f. If the test call is recorded, try to play back the test call in Business Analyzer.
g. If the test call can be played back in Business Analyzer, check if there are Scheduler rules.
Verify that you can see the rule in the Rules Manager table in the DB. Verify that you do
not see errors in the Scheduler and schedule sync.
h. If the test call cannot be played back in Business Analyzer, see the Playback
Troubleshooting Flow.
i. For VoIP recording problems, see the VoIP Troubleshooting Guide.

7. Is the problem a CTI Server to Driver problem?

8. If the problem is a CTI Server to Driver problem:
a. Open the driver window.
b. Make a call.
c. Check for event errors in the window (errors appear in red).
d. Trace the call in the Connection Manager and Driver logs (D:\Program Files\Nice
Systems\CTI\Log) and try to locate errors.
e. See the relevant Integration Guide to troubleshoot the problem.
9. Is the problem a Driver to CLS problem?
10. If the problem is a Driver to CLS problem:
a. Trace the call in the Call Server and RCM logs (D:\Program Files\Nice
Systems\Interactions Center\Log) and try to locate errors. Search by time or extension
number to identify the problem.
b. See the Interactions Center Troubleshooting Flow.
11. Is the problem an Interactions Center process problem?
12. If the problem is an Interactions Center process problem:
a. If the event triggers a recording, identify the call initiator: ScreenSense, Total,
QA/Rule-based, or ROD.
b. Check the call events and mappings in the Call Server logs and RCM logs (D:\Program
Files\Nice Systems\Interactions Center\Log). For ScreenSense tagging requests there
are no RCM logs.
c. Check the DBServer logs to check the end of the call.
d. Check the CLS exceptions. See Checking Exceptions on page 39.
e. See the NICE Interactions Center Troubleshooting Flow.
13. Is the problem a CLS to Database problem?
14. If the problem is a CLS to Database problem:
a. Query SQL Server database (tblInteractions). See the Interaction Data Reference Guide.
b. If the interaction is missing or not complete, check for a problem between the CLS and the
Database Server.
c. See the NICE Interactions Center Troubleshooting Flow.
d. See the Databases Troubleshooting Flow.

Troubleshooting Call Failures using Exceptions

This flow is appropriate for silent calls and other call failures.
1. In Business Analyzer, note the Interaction ID of the calls.
2. Check for interaction exceptions:
a. In the database expand nice_interactions > Tables.
b. Query the tblInteractionCatalog table to check for which period the calls occur.
Figure 2-3 tbllnteractionCatalog Table
c. Run a query, where XX is the period in which the exceptions occur, and YY is the
interaction ID:
SELECT * FROM tblExceptionXX
where iInteractionID = YY

Figure 2-4 Running a Query
d. Check the vcExceptionDetail field to identify the exception.

e. Troubleshoot the exception using Checking the CLS Exceptions in the Interactions Center
3. Check the recordings.
a. Run a query, where XX is the period in which the exceptions occur, and YY is the
interaction ID:
SELECT * FROM tblRecordingXX
where iInteractionID = YY.

Figure 2-5 Check the Recordings
b. Note the iLogger, iChannel and time of the recording. This will be used later to
troubleshoot the problem.
4. Depending on the type of recording make a note of the following information.
Table 2-1: Recording Type Information
Recording Type Information
Recording Type Information
Trunk/Digital Matrix Trunk number, time slot, Logger, as well as agent and
agent group (for interaction based recording)
Extension-side Extension number, channel mapped to the extension,

Logger, as well as agent and agent group (for
interaction based recording)
Passive VoIP Extension number, IP address, MAC address, Logger,

as well as agent and agent group (for interaction based
recording)
Active VoIP (Service Virtual extension, extension associated with the virtual
Observation) extension, Logger, as well as agent and agent group (for
interaction based recording)
5. For interaction based recording, check the rule:

a. On the Interactions Center open the D:\program files\Nice systems\Interactions
Center\BIN\Log\Call Server.log.xxx.txt log files.

b. The following line indicates that the rule engine is checking if the call should be recorded:
18/07/08 16:04:38.317 DEBUG [Command Thread, 5680]:
RecordingDecisions::Selector: Recieved a call for decision: Agent_ID=179389,
Extension=25938, Switch_ID=25, Direction=0, Dnis=17209, Dialed_Num=,
CLI=7172472759 ,BusinessData=
c. One of the following lines appears after the rule check to see if the call matches any of the
active recording rules:
• If the call does not match any of the active recording rules:
18/07/08 16:04:38.317 DEBUG [Command Thread, 5680]: CALLSRVR: No
Recording Command Exist for AgentID 179389, Ext 25938
a. If the call does match one of the active recording rules:
18/07/08 15:58:34.662 INFO [Command Thread, 5680]: Start record Request
to RCM.
b. If the call is not recorded because it does not match one of the active rules, check in Rules
Manager that the Scheduler rule that is recording this agent or group of agents is valid and
active.
6. Connect to the Logger using the Remote Desktop and check if there are any errors (use mstsc
/console and not mstsc). See the High Density Logger Troubleshooting Flow. For the VoIP
Logger, also check the IPCaptureLog.sav file. This contains the first error in the current
session, or if there are no errors in the current session, the first error in the last session when an
error occurred. See the VoIP Troubleshooting Flow.
7. For trunk-side recording:
a. Use the Functional Tester to check the trunk status.
b. If there is a problem with frame loss, trunk not synchronized, etc, use the NiceLog Setup
Tool to solve the problem. See the Using the Nicelog Setup Tool in the High Density
Logger Troubleshooting Flow.
8. For extension-side recording:
a. Make a call from the extension, speak and leave the call open.
b. Use the Functional Tester to check which channel has activity.
c. If the activity is on a channel different to that noted in Step 4, check the channel mapping.
d. If there is no activity, check the Logger. See to the High Density Logger Troubleshooting
Flow.
9. For VoIP recording:
a. Use the Functional Tester to check which channel has activity. If there is no channel
activity, the problem is most likely a VoIP Logger problem.
b. Use the sniffer to check the RTP packets. See Checking the RTP Packets Using the Sniffer
in the VoIP Troubleshooting Flow. If there are packets, check the sniffing NIC
configuration in the Logger IP tool.

c. If there are no packets, troubleshoot the network and environment. See the VoIP Logger
Troubleshooting Flow. In particular check:
• Passive VoIP: Check the mirroring and network topology.
• Active VoIP: Check that the NIC IP is routable from the source IP address, and verify
that the port intervals configured in the Logger IP Tool are open on the firewall for
UDP packets.
10. For Avaya DMCC active VoIP recording:
a. Check in the Observer logs that the virtual extension is registered to the correct extension.
b. If the correct extension was used, run the sniffer to check the packets.
c. If the correct extension was not used, check the configuration.

Troubleshooting No New Calls in Business Analyzer

1. Checking free space in the NICE_Interactions database. If required, increase the database
size, see the Databases Troubleshooting Flow.
a. In the Start menu, select Programs > Microsoft SQL Server 2005 > SQL Server
Management Studio, and click Connect.
b. In the Object Explorer, expand Databases, right-click on the relevant database, select
Properties, and check the Space available. This provides the space of the entire DB with
no distinction between the data size and log size.
c. Right-click the database, select Reports > Standard reports > Disk usage. The usage
is displayed graphically with the separation for data and log.
d. In System Administrator, expand the Master Site > Database Servers > Databases
and select each database to see the used and free space percentages.
Figure 2-6 Database Size
2. Check the NMS for trap 1023. To find the reason for the failure to insert the record, the
DBSrvr log files (on the Interactions Center in D:\program files\Nice
systems\Interactions Center\BIN\Log\DBServer.log.xxx.txt files) should be backed up
and sent to NICE Customer Support. The metadata for the Call might be inserted into the
database by the TRS (if it exists), but in order to ensure that this is the case, NICE Customer
Support should be consulted.
3. Check in the Driver window that calls are being sent to the CLS by looking for Establish
events.
13699296: [07/21/06 18:37:49]
13699296: DELIVERED TO APPLICATION:
13699296: MonitorCrossRefID 63
13699296: EstablishedEvent ::=
13699296: {
13699296: establishedConnection
13699296: {
13699296: callID 15008,
13699296: deviceID "5204",
13699296: devIDType 0
13699296: },
13699296: answeringDevice

13699296: {
13699296: deviceID "5204",
13699296: deviceIDType 55,
13699296: deviceIDStatus 0
13699296: },
13699296: callingDevice
13699296: {
13699296: deviceID "97228776",
13699296: },
13699296: calledDevice
13699296: {
13699296: deviceID "1691",
4. Check the Call Server log files (on the Interactions Center in D:\program files\Nice
systems\Interactions Center\BIN\Log\Call Server.log.xxx.txt files) for the call reported
event. For more information on troubleshooting using log files, see the NICE Interactions
Center Troubleshooting Flow.
• Insert Interaction to DB
InteractionData
***********
properties:
========
LocalStartTime = 17/02/2008 13:16:34 . LocalStopTime =
17/02/2008 13:16:39 . GMTStartTime = 17/02/2008 11:16:34 .
GMTStopTime = 17/02/2008 11:16:39 . OriginatorInteractionID =
11269314 .
members:
========
InteractionId : 11269314
interactionOriginalID : 11269314
Duration : 00:00:05
OpenReasonId : 128
CloseReasonId : 128
SwitchId : 1
InitiatorUserId : 10
QATypeId : False
InteractionTypeId : 2
RecordedTypeId : 0
TypeData :
CallInteractionData
***********
properties:
========
members:
========
PbxCallIndex : 11269314

CallDirectionTypeId : 1
CompoundId : 11269311
IsPlaybackCall : False
*********************
IsEvaluated : False
iMediaTypesId : 2
additionalDataCollection[0] : 1
participantsArray[0] :
ParticipantData
***********
properties:
========
members:
========
ParticipantId : 11269312
Station : 1
SwitchId : 1
IsFirstUser : True
IsInteractionInitiator : False
DeviceTypeId : 0
DeviceId : 0
ParticipantTypeId : 1
UserId : 10
*********************
participantsArray[1] :
ParticipantData
***********
properties:
========
members:
========
ParticipantId : 11269313
Station : 144
SwitchId : 0
IsFirstUser : False
IsInteractionInitiator : True
DeviceTypeId : 0
DeviceId : 0
ClientId : 0
ParticipantTypeId : 2

*********************
recordingsArray[0] :
RecordingData
***********
properties:
========
RecordingGMTStartTime = 17/02/2008 13:51:26 . RecordingGMTStopTime
= 17/02/2008 13:51:29 .
members:
========
Logger : 10225
Channel : 5
iMediaTypeId : 2
InitiatorTypeId : 64
InitiatorUserId : 0
RecordingSideTypeId : 3
RecordingRecordedTypeId : 3
RecordedParticipantId : 11269312
TimeDiff : -02:34:50.2600000
WrapupTime : 00:00:00
m_iTransientIndex : 0
*********************
recordingsArray[1] :
RecordingData
***********
properties:
========
RecordingGMTStartTime = 17/02/2008 13:51:26 . RecordingGMTStopTime
= 17/02/2008 13:51:29 .
members:
========
Logger : 10225
Channel : 5
iMediaTypeId : 2
InitiatorTypeId : 64
InitiatorUserId : 0
RecordingSideTypeId : 3
RecordingRecordedTypeId : 3
RecordedParticipantId : 11269313
TimeDiff : -02:34:50.2600000
WrapupTime : 00:00:00
m_iTransientIndex : 0
*********************

*********************
• Insert Contact (If Compound Ended)
ContactData
***********
properties:
========
GMTStartTime = 17/02/2008 11:16:34 . GMTStopTime = 17/02/2008
11:16:39 .
members:
========
ContactId : 11269311
Duration : 00:00:05
OpenReasonId : 256
CloseReasonId : 256
TransferSiteId : 0
TransferContactId : 0
RecordedTypeId : 0
QATypeId : False
DirectionTypeId : 1
IsEvaluated : False
iMediaTypesId : 2
InitiatorSwitchId : 1
InitiatorStation : 1
InitiatorUserId : 10
m_attachedInteractionInfo :
InteractionUpdatableData
***********
properties:
========
members:
========
m_interactionsInfo[0] :
InteractionUpdatableInfo
***********
properties:
========
InteractionId = 11269314 . QATypeId = False . IsOverwrite =
False .
members:
========
*********************
*********************
*********************

5. Check the Call Server log files (on the Interactions Center in D:\program files\Nice
systems\Interactions Center\BIN\Log\DBServer.log.xxx.txt files) for the call inserted
event. For more information on troubleshooting using log files, see the NICE Interactions
Center Troubleshooting Flow.

6. Check if the call was inserted into the database table:

a. Query the tblInteractionCatalog table to check for which period the calls occur.
b. Check the tblInteractionXX and tblRecordingXX tables.
7. If the calls are in the database the problem may be in the Business Analyzer query:
a. Try to use the NICE user when querying.
b. Ensure that you do not have any expression or filter applied to the query.
c. Collect the logs from the Applications Server.

Troubleshooting Calls with Poor Audio
Troubleshooting Calls with Poor Audio

1. If VoIP calls are recorded with poor audio, see the VoIP Troubleshooting Flow.
2. For extension-side recording, check the connections between the handset, PABX, ALI board
and Punch Down Block (Krone).
3. For trunk-side recording, check for an error on the channel using the Functional Tester.

Troubleshooting Calls with Audio from Several Calls

The following symptoms may occur:
• Call duration equals the "op_MaxOpenCallDuration" parameter (by default 5 hours).
• tblException shows INF002, EXP002, Exp005, INF029.
• Calls have the wrong voice, calls are cut off or start from the middle and this is not a mapping
problem.
To troubleshoot the problem:

1. Check the integration troubleshooting in the relevant Integration Guide.
2. Ensure that all devices are monitored. Some integrations require that all ACDs, VDNs, hunt
group IVRs be monitored.
3. Ensure that the latest Update is installed.
4. Check what information is required to open a case as described in the Integration Guide.
5. Check which call to analyze:
a. Business Analyzer, note the Interaction ID (XXXX) of the call.
b. In the database expand nice_interactions > Tables.
c. Query the tblInteractionCatalog table to check for which period the calls occur.
Figure 2-7 tblInteractionCatalog Table
d. Check the Compound ID for this call, by running the following query:
select iCompoundID from tblCallInteractionXX where iInteractionID = 'XXXX'
e. Check how many calls belong to the Compound ID, by running the following query:
select * from tblCallInteractionXX where iCompoundID = 'XXXX'
6. If there are multiple results it means that the compound was wrongly associated with other
calls. Focus on the first call and note the:

• PBX Call ID:

select IPBXCallID from tblCallInteraction where iInteractionID = 'XXXX'
• Compound ID
select IcompoundID from tblCallInteraction where iInteractionID = 'XXXX'
• Station:
select nvcStation from tblParticipantXX where iInteractionID = 'XXXX'
• Switch ID:
select iSwitchID from tblParticipantXX where iInteractionID = 'XXXX'
• Time of the call:
select dtInteractionLocalStarttime from tblInteractionXX where iInteractionID =
'XXXX'
7. Investigate the call events:
init: The caller picks up the phone.
Delivered: The call between the two parties is ringing.
Establish: The recipient picks up the phone.
Connection clear: The call is over.
Transfer: The call is transferred from one agent to another.
Held / Retrieve: One of the parties pressed hold or retrieved the call.
a. Look for the Call ID and time in the driver and CTI logs (D:\NICECTI\Integration\Log).
b. Look for the device number and try to follow the flow of events up to the connection clear
or transfer. If there is no connection clear or transfer, then it is a third party problem or not
all devices are being monitored (see Step 2).
c. If there is a transfer event then there will be a new call ID, so continue the check using the
device number.

Troubleshooting TRS Recordings

1. Switch the log reporting level to debug:
a. In System Administrator, verify that you are working in Technician mode.
b. In the Organization tree, select CLS Definitions > CLS Server. Click the Report Level
tab.
c. Expand TRS.
d. Set the File to Debug. Do not set the Screen to Debug as this will affect performance.
e. Repeat for Call Server and RCM.
2. Switch the driver logs to Debug:
a. In System Administrator, verify that you are working in Technician mode.
b. In the Organization tree, select CTI Integrations > Drivers, and select the relevant
driver.
c. Expand the Driver Reporting Level.
d. Select the desired row and click Edit.
e. Select the checkboxes for the reporting levels you want to include, and click OK.
3. Check the volume of TRS calls. In Reporter, run the TRS Interaction Volume Report and
TRS Interaction Volume by Logger Report . In order to identify the TRS status, this report
presents the interaction volume up until to the latest TRS run.
Figure 2-8 TRS Interaction Volume Report
4. Are there too many TRS calls? Continue with step 15.

5. If there are fewer calls than expected or no calls, check if the CLS/Driver was down for an
extended period while the Logger was recording.
6. If there are fewer calls than expected or no calls, check if the problem is related to specific
Loggers by checking the TRS Interaction Volume by Logger report. If it is a general problem,
continue with Step 9.
7. Check the connectivity between the Loggers and the Interactions Center:
a. Ping the Loggers and Interactions Center.
b. Check if Trap ID 1095 clsTRSFailed QueryLogger or 1096
clsTRSSucceededQueryLogger appears. For further details, see the NMS User's
Guide.
c. Check the TRS logs (on the Interactions Center in D:\program files\Nice
systems\Interactions Center\BIN\Log\TRS.log.xxx.txt files).
d. Check the Logger logs (in D:\NTLogger\logger\log\LogFilex.DAT). See the High
Density Logger Troubleshooting Flow.
8. Check if the problematic Loggers are associated with the same Interactions Center. If the
problem is related to a specific Interactions Center:
a. Check if the NICE CLS TRS service is started.
b. Check the TRS traps. For further details, see the NMS User's Guide.
Table 2-2: TRS Traps
Trap ID Description Recommended Action
1089 The TRS service is not running. Check the SNMP messages and
clsTRSisDown Interactions Center Module: the TRS log files and try to
TRS determine the reason that the
service is down.
Check if the service was closed
down manually.
1090 TRS Service has started and

clsTRSIsUp initialized properly.
Interactions Center Module:
TRS
1093 The TRS has failed to execute Check if the SQL Server is up
clsTRSFailed an SQL command. and running. Verify that the
toQueryCLS Interactions Center Module: server machine where the
DB TRS Interactions Center runs has
connection to the server machine
where the nice_interactions
database is installed. Analyze the
TRS log files.

Table 2-2: TRS Traps (Continued)
Trap ID Description Recommended Action
1094 TRS successfully connected

clsTRS and queried the
SucceededTo nice_interactions database.
QueryCLSDB Interactions Center Module:
TRS
1095 Indicates a communication Verify that the Logger is up and

clsTRSFailed problem with the Logger. running.
QueryLogger Interactions Center Module: Verify that the Interactions Center
TRS is properly connected to the
Logger. Check if RCM also has a
communication problem with the
same Logger.
1096 TRS successfully connected

clsTRS and queried the Logger.
Succeeded Interactions Center Module:
QueryLogger TRS
c. In System Administrator, check the TRS configuration.

d. Check the TRS logs (on the Interactions Center in D:\program files\Nice
systems\Interactions Center\BIN\Log\TRS.log.xxx.txt files).
9. If the problem is not related to a specific Interactions Center:
a. Change the Logger log levels to debug:
• Open d:\NTlogger\logger\config\SysRC.dat in Notepad.
• In [system -default] set the LogeLevel2File and LogLevel2Screen to 5 instead
of 3.
• Reboot the Logger. See Shutting Down and Starting Up the Logger in the High
Density Logger Troubleshooting Flow.
b. Check that the Logger is up.
c. Check the Logger logs. See the High Density Logger Troubleshooting Flow.
10. Check that the Data Mart Population Schedule job ran:
a. In System Administrator, expand the Master Site > Data Marts and select the required
Data Mart server.
b. Click the Monitor tab.
c. In the SQL Jobs area, select the Data Mart Population Schedule job. The job steps are
detailed in the History area.

Figure 2-9 Data Mart Population Schedule Job
11. Collect the relevant log files using Perform Mate including the Driver, Call Server, RCM,
TRS, DBServer and any problem Logger logs. See the High Density Logger Troubleshooting
Flow and NICE Interactions Center Troubleshooting Flow.
12. Collect the following data tables:
• tblInserterLoggers
• tblInserterCalls (relevant part only):
13. In System Administrator, check the TRS parameters:

a. In the Organization tree, select CLS Definitions > CLS Server.
b. Select the required CLS Server, and click the Configuration tab.
c. Click next to TRS. Note the parameter values.
IMPORTANT
Any change of parameters should be done ONLY in consultation with NICE Support.
d. For a list of the parameters see the NICE Interactions Center Troubleshooting Flow.
14. Check if the TRS parameters were modified:
a. In Audit Trail create and run the following advanced query.

Figure 2-10 Advanced Query
b. Check if the parameter value has changed, and what the previous and new values are.
Figure 2-11 Audit Trail
15. If there are too many TRS calls:

a. In Business Analyzer, check if the same calls appear as inserted by both the Interactions
Center (an agent name appears) AND the TRS (User- Unmapped appears). If this is the
case, provide a list with these calls with their recording times, Logger ID and channel ID.
b. In Reporter, run the TRS Interaction Volume Report and TRS Interaction Volume by
Logger Report in order to identify whether the problem is related to a specific TRS or
specific Loggers.

c. If the problem is related to a specific Logger:

• Run a query in Business Analyzer to see whether the increase is for a specific channel.
• Try to playback several of these calls and see whether the calls are invalid, noisy, ring
tones, DTMF or IVR calls.
d. If the increase is not related to a specific Logger, check the following:
• Check for driver problems, for example, internal errors, disconnections between the
drivers and the PABX or internal PABX, and so on. See the troubleshooting in the
relevant Integrations Guide.
• Check for Interactions Center or Interactions Center subcomponents problems. For
further information, see the NICE Interactions Center Troubleshooting Flow.
e. Collect the relevant log files using Perform Mate including the Driver, Call Server, RCM,
TRS, DBServer and any problem Logger logs. See the High Density Logger
Troubleshooting Flow and NICE Interactions Center Troubleshooting Flow.
f. Collect the following data tables:
• tblInserterLoggers
• tblInserterCalls (relevant part only):
16. Switch all log levels (CLS Server, Drivers and Loggers) back to what they were before you
started this troubleshooting procedure.

Troubleshooting Integration Issues
Troubleshooting Integration Issues

Various tools and utilities for various additional troubleshooting and performance issues are
detailed in the relevant Integration Guide:
• NICE Events Spy enables you to trace events after they were transferred from the PABX to
the Connection Manager, enabling you to detect bugs or malfunctions.
• NICE Debug Service enables you to gather data critical for solving problems stemming from
the transfer of events between the switch and the Connection Manager.
• NICE Connection Manager Monitor tool enables you to view the contents of the
Connection Manager's tables. It also enables you to verify if:
• Devices are monitored
• Monitored devices are filtered
• Displays the loaded CTI links
• Displays connected clients.
• Log Manager system logs all significant system activity and maintains a log of all data,
enabling you to view the history of all relevant system activity. The Log Manager system has
three main components:
• Log Manager
• Log Manager Services
• Log Viewer
• CAPI Spy enables you to monitor all messages sent by the CTI driver to the CLS CAPI (Call
Server). Examination of these messages enables you to pinpoint whether the problem is in the
CTI driver or in the CLS CAPI server. CAPI Spy has two main components:
• CAPI Spy Plug-in
• CAPI Spy Utility

3
Playback Troubleshooting Flow
This chapter describes the playback flow and common troubleshooting issues when playback is
not functioning properly.
Contents
Gathering Information .................................................................................................... 78

Understanding the Playback Flow ................................................................................ 79
General Troubleshooting Flow ...................................................................................... 81
Troubleshooting Playback Problem for All Calls......................................................... 83
Troubleshooting Playback Problem for Specific Calls ............................................... 85
Setting the Log Level in System Administrator........................................................... 87
Setting the Log Level in the .Config File ...................................................................... 88
Setting the Player Log Levels ....................................................................................... 90
Checking the Workstation Ports ................................................................................... 92
Checking the File Versions ............................................................................................ 92
Checking the Locate Order............................................................................................ 94
Checking the Storage Center ........................................................................................ 95
Troubleshooting Player Errors ...................................................................................... 96
Example of Troubleshooting using the Logs.............................................................108
Example of Troubleshooting using the Logs.............................................................108
Collecting the Server Logs using the NICE Perform Logs Collector Tool .............. 113
Collecting the Client Logs using NICE Perform Logs Collector Tool...................... 113
Copying the Playlist ..................................................................................................... 114
Dumping the Client Memory ........................................................................................ 115
Understanding Playback Retrieval.............................................................................. 116
Other Troubleshooting ................................................................................................. 116
Chapter 3: Playback Troubleshooting Flow 77

Important issues to consider when troubleshooting for playback misfunction are:
• Is this a new installation? If yes, when was the installation executed?
• Were there any changes to the environment that might have affected the system? If yes, check
these changes first.
• How often does the problem occur?
• Can the problem be retraced? If yes, specify the steps.
• Were the Servers restarted? If yes, did this solve the problem?

Understanding the Playback Flow

The Playback Flow is described in the chart and procedure below.
Figure 3-1 Playback Flow
1. The user selects an interaction in Business Analyzer (Client) and clicks Play.
2. The Business Analyzer (Client) sends the playback request to the Player.
3. The Player sends the playback request, including the call parameters (site ID, call ID, segment
or complete, user ID of users requesting playback, user ID of the agent to play back) to the
Playback Administration on the Applications Server.
4. The Playback Administration checks the privileges with the User Admin to ensure that the
user is allowed to playback the interaction.
5. The Playback Administration then gathers the interaction data from the Database Server:
• tblRecordingXX
• tblInteractionXX
• tblParticipantXX
• tblContacXX for complete calls

6. The Playback Administration uses tblRecordingXX to locate where the interaction is stored
based on the locate order.
NOTE: Recordings with Recording Type = Not Recorded are ignored.

7. The Playback Administration gathers the locate results.

8. The Playback Administration builds the playlist (xml) for each located media, including the
start and end times and where the media is stored.
9. The playlist is sent to the Player. If the playlist contains no medias, the user receives a No
recordings were found message. See Understanding Playback Retrieval on page 116 for
an explanation of this message.
10. The NICE Player eliminates any redundant medias according to the participants.
11. The NICE Player/Stream Server fetches the media from the Loggers and/or Storage Center,
and plays the call.

General Troubleshooting Flow

General troubleshooting is described in the flowchart and procedures below.
Figure 3-2 Troubleshooting Flow
1. What method of playback are you using?

• Playing back to an extension.
• Playing back directly from the Logger or Storage Center, that is, with a direction
connection between the Player and the Logger or Storage Center.
• Playing back via a Stream Server. See Choosing a Stream Server Configuration in the
System Administrator's Guide.
2. If your problem is playing back to an extension, see Playback Telephony Server Services
Troubleshooting.
3. Is the problem playing back a specific call or all calls?

4. If the problem is for all calls, see Troubleshooting Playback Problem for All Calls
on page 83.
5. If the problem is for specific calls only, see Troubleshooting Playback Problem for Specific
Calls on page 85
6. If the site is a Multi Data Hub environment:
• Check if the agent and the Logger are located in the same site and physical location.
• Try to playback the call locally. If this works, the problem is most likely a network issue,
for example, timeout, latency, and so on.
7. If you have N+1 or 1+1 the call may be recorded on a couple of Loggers:
• Check where the call is physically located in locatelog.txt. See Example of
Troubleshooting using the Logs on page 108.

Troubleshooting Playback Problem for All Calls

Troubleshooting playback problems for all calls is described in the flowchart and procedures
below.
Figure 3-3 Troubleshooting Flow for Playing Back All Calls

If the problem is for all calls:

1. Set the appropriate log level:
• Set the Log Level in the System Administrator. Setting the Log Level in System
• Set the Log Level in the Config File. See Setting the Log Level in the .Config File
on page 88.
• Set the Player Log Level. See Setting the Player Log Levels on page 90.
2. Check that the Applications Server and Database Servers are up. See Applications
Troubleshooting Flow on page 197 and Databases Troubleshooting Flow on page 253.
3. Check the time synchronization between the Applications Server and Database Server.
4. If playing back over a WAN, check if there is a latency problem or communication errors.
5. If other users can play back the calls on the same workstation when logging in, check the user
permissions in the User’s Admininstrator application.
6. If you can play back the calls when logged-in to a different workstation, check the workstation
connectivity and also check workstation problems. See Solving Desktop Problems
on page 217.
7. In System Administrator, check the range of the workstation ports, and ensure that these ports
are open. See Checking the Workstation Ports on page 92.
8. On the Applications Server, check for a conflict between the NICE updates in the ServerBin
and ClientBin folders. See Checking the File Versions on page 92.
9. If a No Recording Found message appears for the calls in the Player, check the Locate
Order. See Checking the Locate Order on page 94.
10. Check that the Storage Center is archiving. See Checking the Locate Order on page 94 and
Checking the Storage Center on page 95.
11. If an error message other than No Recording Found message appears in the Player, see
Troubleshooting Player Errors on page 96.
12. Troubleshoot using the Playback Administration Service Logs. See Example of
13. If this is an encrypted environment, see Media Encryption Troubleshooting Flow
on page 151.
14. Submit information to NICE Customer Support:
• Dump the Client Memory. See Dumping the Client Memory on page 115.
• Copy the Playlist. See Copying the Playlist on page 114.
• Collect the Server Logs using the NICE Perform Logs Collector Tool. See Collecting
Information Using the Log Collector on page 537.
• Collect the Client Logs using the NICE Perform Logs Collector Tool. See Collecting

Troubleshooting Playback Problem for Specific Calls

Figure 3-4 Troubleshooting Flow for Playing Back Specific Calls
If the problem is for specific calls:

1. Set the appropriate log level:
• Set the Log Level in the System Administrator. See Setting the Log Level in System
• Set the Log Level in the Config File. See Setting the Log Level in the .Config File
on page 88.

• Set the Player Log Level. See Setting the Player Log Levels on page 90.
2. If other users can play back the calls on the same workstation when logging in, check the user
permissions in User Admininstration.
3. Is there anything unique about the calls? (for example, were they all recorded on the same
Logger, or at the same time, or are they recordings of the same agent or phone number).
4. If the problems relate to a specific remote Logger, check for Firewall or network issues, for
example DNS configuration. See High Density Logger Troubleshooting Flow on page 303.
5. If a No Recording Found Message appears for the calls in Player, check the Locate Order.
See Checking the Locate Order on page 94.
6. Check that the Storage Center is archiving. See Checking the Storage Center on page 95.
7. If an error message other than a No Recording Found message appears in the Player, see
Troubleshooting Player Errors on page 96.
8. Troubleshoot using the Playback Administration Service Logs. See Example of
9. If this is an encrypted environment, see Media Encryption Troubleshooting Flow
on page 151.
10. Submit information to NICE Customer Support:
• Dump the Client Memory. See Dumping the Client Memory on page 115.
• Copy the Playlist. See Copying the Playlist on page 114.
• Collect the Server Logs using the NICE Perform Logs Collector Tool. See Collecting
• Collect the Client Logs using the NICE Perform Logs Collector Tool. See Collecting

Setting the Log Level in System Administrator
To set the log level in System Administrator:

1. In System Administrator, verify that you are working in Technician mode: In the
Organization tree, select Organization. Then mark the Technician Mode checkbox and click
Save.
2. In the Organization tree, select Applications > Configurations > General.
3. Select the Logs Level tab.
Figure 3-5 Logs Level Tab
4. In the Master Level and File Level columns, note the current log level for Playback
Administration and Playback Locate.
5. In the Master Level and File Level columns, change the log level for Playback
Administration and Playback Locate to Debug.
6. After troubleshooting ensure that you reset these values according to the values you noted in
Step 4 on page 87.
WARNING
Failure to switch the log levels back to the default values might cause system problems due to a
performance overload.

Setting the Log Level in the .Config File

Each of the components on the Applications Server has .config files stored in the following
folder:
<Application Server Installation Path>\ServerBin\.
Playback NiceApplications.Playback.Administration.Startup.exe.
Administration config
Loggers LocateService
PBResourceManager
IMPORTANT
Backup your .config files before editing them.
If you edit the configuration files, ensure that you use a plain text editor such as Notepad without
any options.
DO NOT use text wrap as this will corrupt your configuration file.
In any event, any change of configuration files should be done ONLY in consultation with NICE
Support!!!!.
To set the log level in the .config file:

1. Navigate to <Application Server Installation Path>\ServerBin\.
2. In Notepad, open the relevant .config file.
3. Expand the <log4net> section.
4. Expand the required logger name sections (<logger name="NiceApplications.Playback">,
<logger name="LocateService"> and <logger name="PBResourceManager">).
5. Note the values in both Level and Threshold.
6. Set the debug level, by changing the values on both Level and Threshold to be defined as
ALL.

Figure 3-6 Log Levels in the .config File
7. Save and close the .config file. There is no need to restart the Applications Server.
8. After troubleshooting ensure that you reset these values according to the values you noted in
Step 5 on page 88.
WARNING

Setting the Player Log Levels

You need to set the Player log level in System Administrator and in the
NiceApplications.Playback.GUI.exe.config file to enables playback logic logging. For further
information, see Troubleshooting Player Errors on page 96.
To set the Player log levels:

1. To set the Player log levels in System Administrator:
a. Verify that you are working in Technician mode.
b. In the Organization tree, select Playback > Player Settings.
c. Click the Log Level tab.
Figure 3-7 Log Level Tab
d. Note the current Log Level setting.

e. Set the Log Level to Debug.
f. Logout and then login for the change to take effect.
g. After troubleshooting ensure that you reset these values according to the values you noted
in Step d on page 90.
IMPORTANT
To create a log file of the playback logic errors:

1. To create a log file of the playback logic errors in the
NiceApplications.Playback.GUI.exe.config config file:
a. Set the EnablePBLogicLog value to true:
<add key="EnablePBLogicLog" value="true" />
b. In the Registry Editor, navigate to: HKEY_LOCAL_MACHINE\SOFTWARE\Nice
Systems\PBLogic\Log, and note the values of the MMLClient_LogLevel and the
PBLogic_LogLevel.
c. Set the MMLClient_LogLevel and PBLogic_LogLevel to 0 for maximum details.

Figure 3-8 Registry Key
d. After troubleshooting ensure that you reset these values.

• Set the EnablePBLogicLog value to false.
• In the Registry Editor, reset the MMLClient_LogLevel and PBLogic_LogLevel
values according to the values you noted in Step b on page 90.
IMPORTANT

Checking the Workstation Ports
Checking the Workstation Ports

The settings apply to all Workstations!
To check the Workstation ports:

1. In System Administrator, navigate to Applications > Configurations > General.
2. Expand the Workstation Settings area. Then select as follows:
• Random Ports - all ports on each workstation must be open.
• Specific Ports Range - a range between 1024 and 5000 (the minimum range is 11 ports.
For example, 2001 - 2011 inclusive).
Checking the File Versions

On the Applications Server, check for a mismatch between the NICE updates in the ServerBin
folder (contains binaries and DLL files for server-based applications) and the ClientBin folder
(contains the binaries and DLL files needed by clients).
To check the file versions:

1. Browse to D:\Program Files\Nice Systems\Applications\ServerBin.
2. Check the NiceApplications.Playback.xxx.dll version numbers:
a. In Explorer, right-click on Name, and select More.
Figure 3-9 Choose Details Window
b. Select File Version and check OK.

c. Check the file version numbers against the D:\Program Files\Nice
Systems\Applications\ClientBin.

Checking the File Versions
3. If the NICE Playback Telephony Services service or the NICE Playback Streaming
service is installed, ensure that all the required NICE updates are installed. Check the
following version numbers:
• NiceApplications.Playback.Streaming.xxx.dll
• NiceApplications.Playback.Telephony Services.xxx.dll

Checking the Locate Order
Checking the Locate Order

The Locate Order is the order in which all storage media are searched when a query is run. If your
site is installed with a Storage Center, you can select to search the Storage Center before the
Loggers when running a query. If most of the interactions in your query results are stored on a
Storage Center, this will produce faster results.
Locate Order can also be defined per Location. If a user is defined on a location which has a
Locate Order defined, the locate order of the Location takes precedence over the locate order
defined for the site and is used.
For more information on the logs, Example of Troubleshooting using the Logs on page 108.
Locate
Description Troubleshooting
Order
No Storage Do not search the Storage Center If there is a Storage Center and this is
Center (there is no Storage Center at the incorrectly configured the calls will not be
site). played back.
Before First, search the Storage Center You will not be able to playback
Source and then search the Loggers. If interactions that are not yet archived to the
Logger most of the interactions in your Storage Center if the Logger that they are
query results are stored on a stored on is down.
Storage Center, this will produce
faster results.
After Source First, search the Logger where You will not be able to playback
Logger the interaction was recorded and interactions that are not yet archived to the
then search the Storage Center. Storage Center if the Logger that they are
stored on is down.
After All Search all the Voice Loggers and

Loggers then search the Storage Center.
Storage Search only the Storage Center You will not be able to playback
Center Only and not the Loggers. interactions that are not yet archived to the
Storage Center.
Customized You define which Loggers to If a Logger is missing from the customized
Order search, and in what order to order you will be unable to playback those
search them. interactions.
The source and spare Logger will
always be checked first, prior to
this customized order.

Checking the Storage Center
Checking the Storage Center

Check if the Storage Center is archiving the calls.
To check the Storage Center:

1. Check the directories on the Storage Center to check if the calls are archiving there.
2. Check the archiving to the Storage Center.
a. In Reporter, import the Health Checks Report template.
b. Run the Archiving Coverage Report.
c. Check the Archiving Coverage per Storage Center View.
Figure 3-10 Archiving Coverage Report
3. To further troubleshoot Storage Center problems, see Storage Center Troubleshooting Flow
on page 513.

Troubleshooting Player Errors
Troubleshooting Player Errors

The player errors are recorded in the Player log files, *.txt, in
%APP_DATA%\NicePlayer\Release3. There are various types of errors that may occur in the
Player:
• Playback errors
• Logger errors
• Storage Center errors
Playback Logic Errors

The following playback logic errors may appear in the Player Logs. These errors have the
MMLAPIBase.PbLogicLog prefix.
Error No. Description Action Required
OK 1 OK. None
Fail -1 General failure. Collect the Player logs and submit them
to NICE Customer Support. Refer to
Collecting the Client Logs using NICE
Perform Logs Collector Tool
on page 113.
Illegal -3 One or more of the input Collect the Player logs and submit them
parameter parameters has an invalid to NICE Customer Support. Refer to
value. Collecting the Client Logs using NICE
on page 113.
Improper -4 The playback logic is not in a Collect the Player logs and submit them
mode proper mode to perform the to NICE Customer Support. Refer to
requested operation. Collecting the Client Logs using NICE
on page 113.
Connection -5 Failed to connect to (or Check the connectivity to the Storage

failed disconnect from) the Center or the NiceScreen Logger.
NiceScreen Logger or the Troubleshoot the Storage Center or
Storage Center. NiceScreen NiceScreen Logger. See the Storage
Loggers are sometimes Center Troubleshooting Flow
referred to as Multi Media on page 513 or the NiceScreen
Loggers (MML). Troubleshooting Flow on page 489.

Logger Errors
Error No. Description Action Required
End of -7 No more buffers are available Collect the Player logs and submit them
requested for the requested segments. to NICE Customer Support. See the
segment Collecting the Client Logs using NICE
on page 113.
No packets -8 The segment exists but its Collect the Player logs and submit them
yet (for next requested buffer was not to NICE Customer Support. See
monitoring) found. Collecting the Client Logs using NICE
on page 113.
Segment -9 The requested segment was Collect the Player logs and submit them
not found not found in the NiceScreen to NICE Customer Support. See
Logger or the Storage Center. Collecting the Client Logs using NICE
on page 113.
Logger Errors
The following Logger errors may appear in the Player Logs.
Error
Description Action Required
No.
-9 Handler is invalid.
-1 Communication failure. Failed to connect to the Logger. To check

this, ping the Logger from the client, and
check any Firewall settings between the
Logger and client.
0 Operation successful.
1 Command failed.
11 Invalid parameter, for example, invalid

tag.
12 Token does not exist.
57 Not enough resources for playback.
200 Audio Over LAN & Import: Invalid token
201 Audio Over LAN & Import: DB error or

record aborted.

Logger Errors
Error
No.
203 Audio Over LAN & Import: No free virtual The Logger has no free connections.
output channel. Each media uses a connection to the
Logger. To check this, wait a while and
then retry playing back the interaction.

Storage Center Errors
Storage Center Errors

The following Storage Center errors may appear in the Player Logs.
Error
No.
300007 The Storage Center failed to connect Check the Logger where the backup
to a backup device on a Logger. This device is located.
should not be considered an error if
there are no backup devices in the
site.
300008 The Storage Center did not archive or See:

it had an error in the locate flow. • Checking the Locate Order
on page 94
• Checking the Storage Center
on page 95
• Setting the Player Log Levels
on page 90
• Storage Center Troubleshooting
Flow on page 513.
Checking the Playback Logs

The server log files are found in the <Application Server Installation Path>\ServerBin\ folder
unless specified otherwise.
PlaybackAdministration.txt.x
Playback NiceApplications.PlaybackAdministration.SystemFramework.log
Administration PlaybackRM.txt.x
LocateLog.txt.x
Business NBA.log.
Analyzer NBADefault.log.

Sample Player Logs
Sample Player Logs

Sample Log
2008-07-31 13:39:15,781 DEBUG; [MMLAPIBASE.PBLOGICLOG] - DebugInfo - CNMLApi :
CNMLApi : NML_ReleaseBuffer() - Released buffer context 20000004. [Issued by component]
[ThreadId 12144, ErrorCode 0]
2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - CNMLApi :
CNMLApi : NML_ReleaseBuffer() - Released buffer context 20000004. [Issued by component]
2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp :
LogicImp : [I] GetNextPacket() - Got Packet (pContext=536870916, Type=0, Size=4096,
TimeTag=29941032.3332150528, Flags=0, HeaderSize=24, DataSize=4096 [ThreadId 12144,
ErrorCode 0]
LogicImp : [I] GetNextPacket() - Got Packet (pContext=536870916, Type=0, Size=4096,
TimeTag=29941032.3332150528, Flags=0, HeaderSize=24, DataSize=4096 [ThreadId 12144,
ErrorCode 0]
LogicImp : [I] GetNextPacket() - Got Packet (PacketGUID=0-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0,
MACSize=0, PacketMAC=) [ThreadId 12144, ErrorCode 0]
LogicImp : [I] GetNextPacket() - Got Packet (PacketGUID=0-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0,
MACSize=0, PacketMAC=) [ThreadId 12144, ErrorCode 0]
LogicImp : <<GetNextPacket() [Duration = 16 msec] [ThreadId 12144, ErrorCode 0]
LogicImp : <<GetNextPacket() [Duration = 16 msec] [ThreadId 12144, ErrorCode 0].
Storage Center Disconnection Log

2008-09-25 12:33:05,250 INFO ; [ExpandView.OnTmrScaleTick] - Position: 32515
2008-09-25 12:33:05,906 ERROR; [MMLAPIBase.GetPacket] m_pbLogic.GetNextPacket failed
res = CONNECTIONFAILED
2008-09-25 12:33:05,921 INFO ; [VoiceMMLServer.GetPacket] - screen GetNextPacket failed
at NiceApplications.Playback.MediaSourceAPI.MMLAPIBase.GetPacket(Boolean& lastPacket,
Byte[]& data, tSecuredPacketParams& packetParams)
at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetActivePacket(Boolean&
lastPacket, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime,
CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32&
actualDataSize)

Sample Player Logs
at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetPacket(Boolean& lastPacket,
Boolean& active, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime,
actualDataSize)
at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket()
2008-09-25 12:33:05,921 ERROR; [AudioBufferBase.FetchingThreadFunction] screen
GetNextPacket failed at
NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket()
at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.GetPacket(Boolean&
fetchDone)
at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.GetPacket()
at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Pop()
at NiceApplications.Playback.MediaServices.Logic.AudioBuffer.GetPacket()
at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.FetchingThreadFunction()
2008-09-25 12:33:05,921 INFO ; [AudioBufferBase.FetchingThreadFunction] - m_state changed
to Seek
res = CONNECTIONFAILED
at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetActivePacket(Boolean& lastPacket,
Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType&
compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize)
actualDataSize)
fetchDone)

Sample Player Logs

to Seek
2008-09-25 12:33:05,921 ERROR; [CommonViewManager.ErrorOccurred] ErrorOccurred,
description: SCREEN_SERVER_SCREEN_FAILED_TO_GET_NEXT_PACKET
2008-09-25 12:33:05,921 DEBUG; [CommonViewManager.DisableCloseForms] -
>>DisableCloseForms
Storage Center with Playback Logic Log

2008-09-25 12:58:41,218 WARN ; [MMLAPIBASE.PBLOGICLOG] - Warning -
MMLPoint::SendBlockingMsgToChannel : timeout expired. ChannelId(1), Opcode(203)
2008-09-25 12:58:41,218 WARN ; [MMLAPIBase.PbLogicLog] - Warning -
MMLPoint::SendBlockingMsgToChannel : error detected while waiting. ChannelId(1),
Opcode(203), lastError(553648143) [ThreadId 5464, ErrorCode 0]
2008-09-25 12:58:41,218 FATAL; [MMLAPIBase.PbLogicLog] FatalError -
PlaybackPoint::GetBuffer : SendSyncCommandToChannel failed. Channel id(1) [ThreadId 5464,
ErrorCode 0]
ErrorCode 0]
2008-09-25 12:58:41,218 FATAL; [MMLAPIBase.PbLogicLog] FatalError - CNMLApi :
CNMLApi : NML_GetBuffer() - NMLAPI_PB_GetBuffer() returned 0,0.GetLastError returned 0
2008-09-25 12:58:41,218 ERROR; [MMLAPIBase.PbLogicLog] Error - CNMLApi : CNMLApi
: ConnectionFailureCallback() - Disconnection occured for ClientHandle a86860. [ThreadId 5964,
ErrorCode 0]
res = ConnectionFailed

Sample Player Logs
2008-09-25 12:58:41,218 ERROR; [MMLAPIBase.PbLogicLog] Error - CNMLApi : CNMLApi

: ConnectionFailureCallback() - Disconnection occured for ClientHandle a86860. [ThreadId 5964,
ErrorCode 0]
actualDataSize)
fetchDone)
to Seek

Sample Player Logs

ErrorCode 0]
ErrorCode 0]
res = ConnectionFailed
actualDataSize)
fetchDone)
to Seek

Sample Player Logs
Logger Disconnection Log

2008-09-25 12:43:38,828 INFO ; [RenderVoiceMediaBase.WriteAudioToDSBuffer] - MediaId 3
writeSize 16000 clentBufferLength 16000
2008-09-25 12:43:38,843 ERROR; [LoggerPBAPI880.GetNextPacketFromLogger]
NC_StreamFetchData failed res = -16
2008-09-25 12:43:38,843 ERROR; [VoiceServer.GetPacket] GetPacket error
VOICE_SERVER_CONNECTION_FAILURE
2008-09-25 12:43:38,843 ERROR; [AudioBufferBase.FetchingThreadFunction] FETCH DATA
FAILED at NiceApplications.Playback.Streaming.Nice.VoiceServer.GetPacket()
fetchDone)
to Seek
2008-09-25 12:43:38,843 INFO ; [RenderVoiceMediaBase.OnAudioBufferError] - Got buffering
event while playing propogating event
2008-09-25 12:43:38,859 INFO ; [PlaybackController.OnBufferingEvent] - Controller got
Buffering event while playing or seeking 44
2008-09-25 12:43:38,859 INFO ; [PlaybackController.PlayFrom] - PlayFrom 9250
2008-09-25 12:43:38,859 INFO ; [PlaybackController.Seek] - Seek 9250
2008-09-25 12:43:38,890 INFO ; [RenderVoiceMediaBase.GetFreeSpaceSize] - playCursor
151948, LastPlayCursorPosition 135118
2008-09-25 12:43:38,921 INFO ; [PBViewManager.OnBuffring] - [WMS] Buffering Started
2008-09-25 12:43:38,921 INFO ; [ExpandView.OnBuffring] - *** Not Gui thread using
this.Invoke ***
2008-09-25 12:43:38,921 INFO ; [AudioBufferBase.Seek] - Seek m_seekPosition=9250,
m_state=Working
2008-09-25 12:43:38,921 INFO ; [CompressedInBuffer.Seek] - Seek : 9250
2008-09-25 12:43:38,921 DEBUG; [VoiceServer.InternalJumpToPos] - JumpToPos Token 191
startTime 25/09/2008 09:45:13 404 StopTime 25/09/2008 09:47:04 403

Sample Player Logs
2008-09-25 12:43:38,937 INFO ; [AudioBuffer.GetPacket] - Poped:Packet: Media 1, Time

25/09/2008 09:45:22.120-25/09/2008 09:45:26.210, Position 8716-12806 (=4090ms), Data
4090by (=4090ms), Active
2008-09-25 12:43:38,937 ERROR; [LoggerPBAPI880.StartPlayback] AUD_AudioOverLAN_Ex
failed res = -1
2008-09-25 12:43:38,937 INFO ; [VoiceServer.InternalJumpToPos] - AUD_AudioOverLAN_Ex
failed at NiceApplications.Playback.MediaSourceAPI.LoggerPBAPI880.StartPlayback(Int32
token, DateTime startTime, DateTime endTime)
at NiceApplications.Playback.Streaming.Nice.VoiceServer.InternalJumpToPos(DateTime
startTime)
2008-09-25 12:43:38,937 ERROR; [VoiceServer.InternalJumpToPos] JumpToPos error
VOICE_SERVER_START_OVER_LAN_FAILED
2008-09-25 12:43:38,937 DEBUG; [AudioProcessing.DecodePacket] - DecodePacket,
startPos=9250, endPos=110999, packetStartPos=8716, packetEndPos=12806
2008-09-25 12:43:38,937 INFO ; [RenderVoiceMediaBase.Seek] - AUD_AudioOverLAN_Ex
failed at NiceApplications.Playback.Streaming.Nice.VoiceServer.InternalJumpToPos(DateTime
startTime)
at NiceApplications.Playback.Streaming.Nice.MediaDataServerBase.JumpToPos(Int64 ticks)
at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.MediaServerSeek(Int64
position)
at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.StartFetch(Int64 position)
at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.SetPosition(Int64
position, Boolean& isWorking)
at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.Seek(Int64 position)
at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Seek(Int64 position)
at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.Seek(Int64 position)
at NiceApplications.Playback.MediaServices.Logic.RenderVoiceMediaBase.Seek(Int64 position)
2008-09-25 12:43:38,937 INFO ; [OutputDS.RenderVoiceSeekCallback] -
RenderVoiceSeekCallback stat = VOICE_SERVER_START_OVER_LAN_FAILED
2008-09-25 12:43:38,953 INFO ; [AudioBuffer.GetPacket] - Poped:Packet: Media 1, Time
25/09/2008 09:45:26.210-25/09/2008 09:45:30.300, Position 12806-16896 (=4090ms), Data
4090by (=4090ms), Active

Sample Player Logs
Checking the NiceScreen Logs

Settings for your NiceScreen Logger are stored in the MMLogger.cfg configuration file located in
the NiceScreen Logger installation directory. LogFile_Path indicates the location of the
NiceScreen log files.
The logs are located in the installation folder, by default in:
C:\Programs Files\Nice Systems\Multimedia Logger.
To check the NiceScreen Logs:
Review the NiceScreen Logger logs (logfileXX.log).

Example of Troubleshooting using the Logs

This section includes sample logs useful for troubleshooting playing back an interaction. For
further information on the playback process, see Understanding the Playback Flow on page 79.
To Troubleshoot using the Logs:

1. In PlaybackAdministration.txt (see PlaybackAdministration.txt on page 108), search for:
a. Start PreparePlayListForInteractions: This indicates the start of the playback process.
b. Locate Results: Check if the locate process succeeded. In this example two parts of the
call are located.
c. RecordingId:XXX: This indicates the recording ID for each part of the call.
d. Build media per participant: This indicates the start of the media building process (per
media).
e. New Voice Media start or New Screen Media start: This indicates that the interaction
has been located and that the media has been built. If the locate succeeded, but the media
is missing, it could indicate a timing problem on the client.
2. If the locate failed, open LocateLog.txt (see LocateLog.txt on page 110) and search for:
a. AddRequest … recording Id: XXX: This indicates that a locate request for a recording
was initiated.
b. StorageCenterLocator.LocateRecordingCallback … Operation Result = 0: A zero
(0) indicates that the recording was located on the Storage Center. A value other than zero
(0) indicates that the part of the call is missing from the Storage Center.
c. with result Found: This indicates that the locate process was successful.
d. with result NotFound: This indicates that the locate process was not successful. This
may indicate a time difference between the Interactions Center and Logger, or the
Interactions Center and Applications Server. Refer to TN0752: Windows Network Time
Protocol Configuration. This note provides procedures for configuring the Windows
Network Time Protocol on NICE servers.
e. If there are Storage Center errors, see the Storage Center Logs to establish the reason for
the error. Review the Storage Center Logs. By default, the log files are stored in the
same directory where the Storage Center is running. The default log file names are:
• StorageCenterLog.log
• StorageCenterLog1.log through StorageCenterLog100.log
PlaybackAdministration.txt
2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Start
PreparePlayListForInteractions
2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Create
play list items

2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Bring

segments from db for requested calls
2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - query url:
zandberg:62074
2008-08-24 09:59:04,620 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Build
stages for interactions using Analyzer class
2008-08-24 09:59:04,620 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Create
play media
2008-08-24 09:59:04,620 DEBUG; [PlayListManager.CreatePlayMedia] - : prepare locate
requests
2008-08-24 09:59:04,620 DEBUG; [PlayListManager.CreatePlayMedia] - Add locate requests for
call 0
2008-08-24 09:59:04,620 INFO ; [Participant.BuildRecordings] - Added another recordings: 417
for the participant ID 11
2008-08-24 09:59:04,635 DEBUG; [PBAdminServer.GetServerIdOfLogger] -
GetServerIdOfLogger took:00:00:00
2008-08-24 09:59:04,635 INFO ; [Participant.BuildRecordings] - Added another recordings: 416
for the participant ID 13
2008-08-24 09:59:04,651 DEBUG; [PBAdminServer.GetServerIdOfLogger] -
GetServerIdOfLogger took:00:00:00.0156250
2008-08-24 09:59:04,651 DEBUG; [PlayListManager.CreatePlayMedia] - : run Locate
2008-08-24 09:59:04,651 DEBUG; [PBAdminServer.GetLoggersOnSite] - GetLoggersOnSite
took: 00:00:00
took: 00:00:00
took: 00:00:00
took: 00:00:00
2008-08-24 09:59:04,698 DEBUG; [LocateManager.LocateFinished] - Locate callback recieved 0
2008-08-24 09:59:04,698 INFO ; [LocateManager.LocateFinished] - Locate Callback Request 0
Times: 633518804008930000, 633518804238930000
2008-08-24 09:59:04,729 DEBUG; [LocateManager.LocateFinished] - Locate callback recieved 1
2008-08-24 09:59:04,729 INFO ; [LocateManager.LocateFinished] - Locate Callback Request 0
Times: 633518804008930000, 633518804238930000
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.CreatePlayMedia] - Locate Request took:
00:00:00.0781250

2008-08-24 09:59:04,729 DEBUG; [PLAYLISTMANAGER.WRITELOCATERESULTSTOTRACE] -

:LOCATE RESULTS 2008-08-24 09:59:04,729 DEBUG;
[PLAYLISTMANAGER.WRITELOCATERESULTSTOTRACE] - LOGGERID :1722315 CHANNELID:49
TOKEN:0 OFFLINEMEDIANUMBER:0 DEVICENUMBER:0 ISPARTIAL:FALSE RECORDINGS:
INTERACTIONID:326924 RECORDINGID:417 SERVERID:30 SITEID:1/N
2008-08-24 09:59:04,729 DEBUG; [PLAYLISTMANAGER.WRITELOCATERESULTSTOTRACE] -
LOGGERID :1722315 CHANNELID:19 TOKEN:0 OFFLINEMEDIANUMBER:0 DEVICENUMBER:0
ISPARTIAL:FALSE RECORDINGS: INTERACTIONID:326924 RECORDINGID:416 SERVERID:30
SITEID:1/N
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.CreatePlayMedia] - : BUILD MEDIA PER
PARTICIPANT
2008-08-24 09:59:04,729 INFO ; [PlayListManager.BuildMediaForParticipant] - Building Media:

Stage Times : 7/17/2008 8:33:20 AM, 7/17/2008 8:33:43 AM Locate Times:
633518804008930000,633518804238930000
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.BuildMediaForParticipant] - : NEW VOICE
MEDIA START
2008-08-24 09:59:04,729 INFO ; [PlayListManager.BuildMediaForParticipant] - Building Media:
Stage Times : 7/17/2008 8:33:20 AM, 7/17/2008 8:33:43 AM Locate Times:
633518804008930000,633518804238930000
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.BuildMediaForParticipant] - : NEW VOICE
MEDIA START
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.CreatePlayMedia] - : Finish
CreatePlayMedia
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.PreparePlayListForInteractions] - : Build
xml document
2008-08-24 09:59:04,729 DEBUG; [PlayListManager.PreparePlayListForInteractions] - : Return
PlayList
LocateLog.txt
2008-08-24 09:59:04,651 [20] DEBUG; [LocateServiceServer.AddRequest] - ADDREQUEST:
Request id:23 LocateOrder = Custom start time: 7/17/2008 8:33:20 AM Milli: 893 stop time:
7/17/2008 8:33:43 AM Milli: 893 for RECORDING ID: 417 interactionId: 326924 serverId:30
siteId: 1
2008-08-24 09:59:04,651 [20] DEBUG; [LocateServiceServer.EnterCheckLoggerState] - The
request 23 for recording Id: 417 interactionId: 326924 serverId:30 siteId: 1 moved to
CheckLogger state
2008-08-24 09:59:04,667 [20] DEBUG; [LocateServiceServer.AddRequest] - ADDREQUEST:
Request id:24 LocateOrder = Custom start time: 7/17/2008 8:33:20 AM Milli: 893 stop time:
7/17/2008 8:33:43 AM Milli: 893 for RECORDING ID: 416 interactionId: 326924 serverId:30
siteId: 1
2008-08-24 09:59:04,667 [20] DEBUG; [LocateServiceServer.EnterCheckLoggerState] - The
request 24 for recording Id: 416 interactionId: 326924 serverId:30 siteId: 1 moved to
CheckLogger state

2008-08-24 09:59:04,667 [STP SmartThreadPool Thread #31] DEBUG;

[StorageCenterLocator.OriginalLocate] - Locate calls Storage Center: zandberg:30003 recordings:
Id: 417 interactionId: 326924 serverId:30 siteId: 1
2008-08-24 09:59:04,667 [STP SmartThreadPool Thread #31] INFO ;
[StorageCenterLocator.OriginalLocate] - Call LocateRecording for Storage Center:
zandberg:30003 recordings: Id: 417 interactionId: 326924 serverId:30 siteId: 1
[StorageCenterLocator.OriginalLocate] - Locate calls Storage Center: zandberg:30003 recordings:
Id: 416 interactionId: 326924 serverId:30 siteId: 1
2008-08-24 09:59:04,698 [14] INFO ;
[STORAGECENTERLOCATOR.LOCATERECORDINGCALLBACK] - GOT LOCATE CALLBACK FOR
RECORDING 1;30;326924;417 OPERATION R ESULT = 0

[StorageCenterLocator.OriginalLocate] - Locate for recording 1;30;326924;417 returned with 1
results
[LocateServiceServer.HandleState] - The request 23 for recording Id: 417 interactionId: 326924
serverId:30 siteId: 1 finished state CheckLogger with result Found
[LocateServiceServer.SendLocateFinishedCallback] - FinishLocate for recording Id: 417
interactionId: 326924 serverId:30 siteId: 1 Results: Request id:23 results number: 1 results:
source: SC start time: 7/17/2008 8:33:20 AM Milli: 893 stop time: 7/17/2008 8:33:43 AM Milli:
893
[LocateServiceServer.SendLocateFinishedCallback] - Locate callback called with times:
633518804008930000, 633518804238930000
[LocateServiceServer.SendLocateFinishedCallback] - Finish request Avail: 499 Active999
2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #32] INFO ;
[StorageCenterLocator.OriginalLocate] - Call LocateRecording for Storage Center:
zandberg:30003 recordings: Id: 416 interactionId: 326924 serverId:30 siteId: 1
2008-08-24 09:59:04,714 [14] INFO ;
[STORAGECENTERLOCATOR.LOCATERECORDINGCALLBACK] - GOT LOCATE CALLBACK FOR
RECORDING 1;30;326924;416 OPERATION R ESULT = 0

[StorageCenterLocator.OriginalLocate] - Locate for recording 1;30;326924;416 returned with 1
results
[LocateServiceServer.HandleState] - The request 24 for recording Id: 416 interactionId: 326924
serverId:30 siteId: 1 finished state CheckLogger WITH RESULT FOUND


[LocateServiceServer.SendLocateFinishedCallback] - FinishLocate for recording Id: 416
interactionId: 326924 serverId:30 siteId: 1 Results: Request id:24 results number: 1 results:
source: SC start time: 7/17/2008 8:33:20 AM Milli: 893 stop time: 7/17/2008 8:33:43 AM Milli:
893
[LocateServiceServer.SendLocateFinishedCallback] - Locate callback called with times:
633518804008930000, 633518804238930000
[LocateServiceServer.SendLocateFinishedCallback] - Finish request Avail: 499 Active999
2008-08-20 19:02:56,620 [8860] INFO ; [StorageCenterLocator.LocateRecordingCallback] - Got
locate callback for recording 1;26;239481090;4323375 Operation Result = 300008
2008-08-20 19:02:56,620 [8860] INFO ;
[STORAGECENTERLOCATOR.LOCATERECORDINGCALLBACK] - ERROR WHEN LOCATE RECORDING
1;26;239481090;4323375 OPERATION RESULT = 300008
[StorageCenterLocator.OriginalLocate] - Locate for recording 1;26;239481090;4323375 returned
with 0 results
[LocateServiceServer.HandleState] - The request 7 for recording Id: 4323375 interactionId:
239481090 serverId:26 siteId: 1 finished state StorageCenterLocate with result NotFound
2008-08-20 19:05:52,077 [8860] INFO ; [StorageCenterLocator.LocateRecordingCallback] - Got
locate callback for recording 1;26;253423018;4569139 Operation Result = 300007
2008-08-20 19:05:52,077 [8860] INFO ; [StorageCenterLocator.LocateRecordingCallback] -
Error when locate recording 1;26;253423018;4569139 Operation Result = 300007
[StorageCenterLocator.OriginalLocate] - Locate for recording 1;26;253423018;4569139 returned
with 0 results
[LocateServiceServer.HandleState] - The request 70 for recording Id: 4569139 interactionId:
253423018 serverId:26 siteId: 1 finished state StorageCenterLocate with result NotFound

Collecting the Server Logs using the NICE Perform Logs Collector Tool
Collecting the Server Logs using the NICE Perform Logs

Collector Tool
The Log Collector Tool collects all relevant playback information at one click of a button. This
information includes all configuration parameters and all playback related log files. This
information can be saved and used when opening or escalating cases. For more information, see
Collecting Information Using the Log Collector on page 537.
Collecting the Client Logs using NICE Perform Logs

Collector Tool
Use the NICE Perform Logs Collector to collect the Player and ScreenAgent Log files. For
more information, see Collecting Information Using the Log Collector on page 537.
Player:
• All *.txt; *.comments; *config; *.xml files from %APP_DATA%\NicePlayer\Release3
directory.
ScreenAgent:
• All *.* files from directory specified by the Common AppData registry key.
NOTE: Collecting all the necessary information can speed up resolution of a support
case.

Copying the Playlist
Copying the Playlist

The playlist is built each time an interaction is played, and the previous playlist is overwritten.
The playlist.xml comprises several elements.
Element Description
Item Part of an interaction. There maybe multiple items in an interaction.
Stage Typically an item comprises one stage. An additional stage may be added by,
for example, a conference call.
Time The Start and End time of the stage (Year, Month, Day Hour, Minute, Second,
Millisecond).
Participant A participant in the interaction stage. There will usually be multiple participants.
This includes information:
• Media information including the media type (voice or screen), the media
original location (Logger ID and Channel ID) ,and the Logger token.
• Participant information, including Agent ID, User ID, and Device ID. No
participant information will appear for customers.
ContactKey Included only if this is a complete call.
To copy the playlist:

1. If you have a problem playing back an interaction, copy the following file immediately and
submit it to NICE Customer Support:
<Installation Folder>\NICE Player\temp\Playlist.xml
2. For further information on troubleshooting using the playlist, see Troubleshooting Player
Errors on page 96.

Dumping the Client Memory
Dumping the Client Memory

If requested to by NICE Customer Support, you may need to dump the client memory. Do this
using ADPlus, a Windows debugging tool that starts the CDB debugger and executes a set of
debugger commands to gather information. For more information regarding ADPlus, see
http://support.microsoft.com/kb/286350.
To Dump the Client Memory:

1. Copy the debug tools directory onto the client machine (C:\Program Files\Debugging
Tools for Windows\). Alternatively, connect a USB Flash drive to the problematic machine
and run the dump command from the USB Flash drive.
2. Identify the process ID:
a. Run Internet Explorer. You should only have one Internet Explorer window open.
b. In Windows Task Manager, click the Processes tab.
c. From View > Select Columns.
d. Select PID (Process Identifier) and click OK.
e. Note the Internet Explorer Process ID.
3. From the Debugging Tools for Windows directory or USB Flash drive, run the following
command:
adplus -hang -p XXXX -quiet
• The dump is created in the Debugging Tools for Windows directory. To specify an output
path use the -o option.
• If the process throws a fatal exception and then terminates, replace the -hang option with
the -crash option. This takes the dump before the process terminates.
• If any question dialogs appear during the dump process, click OK.
4. The cdb.exe command window appears, and the dump process starts.
5. The command window closes when the dump is finished and the results are saved in a file
with the format <Mode>_Date_<Date> _Time_<Time>, for example:
Hang_Mode__Date_12-27-2005__Time_17-37-1919.
6. Verify that this is the date and time that you took the dump, ZIP the entire directory (take into
account that the file can be large), and mail or FTP it to NICE Customer Support.

Understanding Playback Retrieval
Understanding Playback Retrieval

When you cannot play back a recording from the Business Analyzer application and receive the
error There are no recordings to retrieve, you need to verify that the Media Library and
Storage Center are installed. If they are not installed, you can retrieve the entire storage media that
includes the specific recording from the Backup Manager.
Other Troubleshooting
A No recording was found message could be related to activation of CTI Analysis in the
integration. Check the Playback Administration service log file as well as the Business Analyzer
service log file.

4
Archiving Troubleshooting Flow
This chapter describes common troubleshooting issues for the various archiving methods used
with NICE Perform Release 3.1.
Contents
General Archiving Troubleshooting Flow .................................................................. 118

Tape/DVD Archiving Troubleshooting ........................................................................ 119
Media Library Troubleshooting ...................................................................................121
Backup Server Device Detection Troubleshooting ...................................................123
Other Backup Server Troubleshooting.......................................................................125
Storage Center Troubleshooting.................................................................................126
Setting the Log Level to DEBUG..............................................................................128
Checking for Archived Calls .....................................................................................129
Gathering Storage Center Information .....................................................................132
Checking the Storage Retention and Deletion Values .............................................133
Verifying that the Storage Center Database is Selected ..........................................133
Verifying the Used and Free Space..........................................................................133
Verifying the Storage Unit Threshold........................................................................134
Checking that the Archiving Rule Inserts New Calls ................................................135
Troubleshooting the Archiving Rule .........................................................................135
Checking the Storage Center Archiving Tables........................................................137
Checking the Storage Center Logs ..........................................................................138
Checking the Nice Storage Streaming Service Logs................................................139
Checking the Player Logs for Storage Center Errors ...............................................139
Other Troubleshooting Issues ..................................................................................140
Chapter 4: Archiving Troubleshooting Flow 117

General Archiving Troubleshooting Flow
General Archiving Troubleshooting Flow

1. Which archiving activity are you troubleshooting?
• Online archiving on a Logger.
• Archiving on tape/DVD.
• Organizing the archiving media at a site with Media Library. The Media Library assigns
sequential numbers to the cassettes, and the Media Library printer prints labels
automatically.
• Archiving on Storage Center.
2. If your problem is archiving online to a Logger, see the High Density Logger
Troubleshooting Flow on page 303 and VoIP Troubleshooting Flow on page 329.
3. If your problem is archiving to tape, see Tape/DVD Archiving Troubleshooting on page 119.
4. If your problem is with the Media Library, see Media Library Troubleshooting on page 121.
5. If your problem is with the Backup Server, see Backup Server Device Detection
Troubleshooting on page 123.
6. If your problem is archiving to Storage Center, see Storage Center Troubleshooting
on page 126.

Tape/DVD Archiving Troubleshooting

See the Backup Configuration Guide and Archiving Devices and Media Reference Guide for
further information.
To troubleshoot tape/DVD archiving:

1. View the status of the drive:
a. Log in to the NICE Perform Applications Suite.
b. From the Accessories menu, select Backup Manager.
c. Double-click the NiceLog icon .

d. Select a Logger. Logger details (Server Name, Logger ID, Host/IP Address and Backup
Mode) appear in the Logger Details area; device information (if media is loaded)
appears in the Devices Information area including:
• Device Assignment shows the current operation taking place: Automatic
Archiving, Retrieval, Retrieve while Archiving, Manual Archiving, Erase.
• Device Status shows the current status: Loading, Empty, Ejecting, Retrieving,
Suspended.
e. In the Device Information area, select a row representing a device.
f. Click the Device Info button on the Backup Manager toolbar.

The Device Info window appears.
Figure 4-1 Device Info Window
2. Ensure that there is a media in the drive.

3. Check that the retention period of the media has not expired yet..
4. Check that the media was closed properly and is valid for usage in append mode.
5. Ensure that the media is not full or write protected.
6. Ensure that the media version is supported. For a list of the supported media, see the Archiving
Devices and Media Reference Guide.
7. Verify that the archiving device can be viewed as a drive.
8. Verify that the correct driver is installed for the archiving device. See Archiving Devices
Driver Installation in the Archiving Devices and Media Reference Guide.

9. If necessary, use a cleaning media to clean the archiving drive. For a list of supported cleaning
media and how to clean the archiving device, see the Archiving Devices and Media Reference
Guide.
WARNING
If an archiving drive is accidentally cleaned with a spent cleaning cassette, the used-up cassette
will slowly contaminate the drive. The contamination can spread to the archiving medium. The
archiving medium can then contaminate a new archiving drive and cause it to fail.
10. Test the archiving drive using the NICE Test Kit.
a. If you do not have the NICE Test Kit you can download it from ExtraNICE > Services >
Diagnostic Utilities.
b. See the General High Density Logger Troubleshooting Flow on page 304 and the
NICE Test & Recovery Kit User Manual.
c. Use the following tests that run under Windows:
Test/Operation Test Description Log File Location
Backup Sony AIT & Checking SONY AIT Root:\Programs\SONY Tape Tool
DDS Tape Tool IDE Backup Devices. for Windows\S.N.dat
Firmware Upgrade for S.N indicates the tape serial
SONY IDE & SCSI number
Backup Test DVD Creating a Test DVD No log file.

Creator Application
Backup DVD Read & Checking DVD Drives Root:\Programs\BurnInTest\BIT_lo

Verify g*.log
d. Use the following tests that run under DOS:
Backup SONY AIT & Checking SONY AIT Root:\Programs\SONY\NTAU\*.ana

DDS Test (SCSI) & DDS SCSI Backup
Devices
Root:\Programs\SONY\Ship\*.abc

Media Library Troubleshooting

See the Backup Configuration Guide.
To troubleshoot Media Library:

1. Verify that all Media Library Loggers appear in the Media Library window as Active =1.
Figure 4-2 Media Library Window
2. If automatic archiving is not working, try to archive manually.

3. If cannot print lables because of archiving issues using the Backup Server, see Tape/DVD
Archiving Troubleshooting on page 119.
4. Check that all the required updates are installed.
5. Check if the Media Library is not printing labels for one or more of the Loggers.
6. Check the Media Library printer status and which Loggers are associated to a specific Media
Library printer:
a. In System Administrator, in the Organization tree, select Media Library Servers and
select the required Media Library Server.
b. Selected the required Media Library printer.
c. Check the Printer Status in the General tab.
d. Click the Loggers tab.
e. Check which Loggers are associated to the Media Library Printer.
7. Check the network connection between the Media Server and the affected Loggers.
8. Check that the printer is plugged in and online.

9. Check the location of the printer sensors and check the label cartridge in the printer.
10. Use the test button on the printer to print a test label.
11. The following are the relevant Media Library tables:
• Nice_admin.tblMLibLoggers
• nice._admin.tblSystemResources includes all defined Media Library Loggers with

iLoggerId (iLoggerid must be = iResouseId from tblSystemResources)
• Nice_interaction.tblMLCassestes describes cassettes that Media Library created labels
for.
• Nice_interaction.tblMLChannels describes for a given tape the channels and times.
• Nice_interaction.tblMLLog is the Media Library log.

12. In System Administrator, navigate to Applications > Configurations > Locate Settings
and check that Storage Center is configured for locating and check the Locate Order. For more
information, see Checking the Locate Order on page 94.
13. Increase the reporting level for the NICE Playback Administration service, which includes the
Locate service and Storage Center service.
14. Check the Locate and Storage Center log files at the following paths:
• Locate service log file: NiceSystem\Applications\ServerBin\Locate.log
• Storage Center log file: NICE Systems\Storage Center\StorageCenterLog.log
a. Check that the StorageCenterLog.log does not contain error messages related to the
Storage Center startup process.
b. Check that the Locate.log reflects the correct locate process for off-line call playback:
• A locate request for a specific InteractionID is received.
• The NICE Playback Administration service (locate) checks the source Logger and
returns Result NotFound.
• The NICE Playback Administration service (locate) checks the rest of the Loggers and
returns Result NotFound.
• The NICE Playback Administration service (locate) checks the Storage Center and
returns Result OfflineFound.
• The NICE Playback Administration service (locate) activates RetrieveServiceSystem
and gets a list of tape drives.
• If one or several interactions are not online, the Retrieval Wizard appears. See
“Retrieving Recordings of Interactions” in the Business Analyzer Guide.
15. Ensure that the NICE Storage Center Service and Nice Storage Streaming Service
(PlaybackServiceSC) are running (Note: A site using the Media Library must have the NICE
Storage Center to enable retrieval of recordings.)

Backup Server Device Detection Troubleshooting

The Backup Server uses DVD-RAM devices, USB tapes, or SCSI tapes to back up data. This
section troubleshoots how to enable detection of each of these backup methods:
• DVD-RAM Devices Not Detected on page 123
• USB Tapes Not Detected on page 124
• SCSI Tapes Not Detected on page 124
DVD-RAM Devices Not Detected

Follow the procedure below when the Backup Server does not detect DVD-RAM devices.
To troubleshoot DVD-RAM devices not detected by the Backup Server:

1. Check that the hardware connections of the DVD-RAM devices are correctly attached.
2. Navigate to Control Panel > System , and click the Hardware tab. Click the Device
Manager button to open the Device Manager. Check that the DVD-RAM devices are listed.
3. Navigate to Control Panel > Add or Remove Programs and check that the WriteDVD!
application appears in the list of installed programs.
4. Restart the computer.
5. From My Computer, check that a removable device appears for each DVD-RAM device
installed on the Backup Server.
6. From the Device Manager (Control Panel > System > Hardware > Device Manager),
check that the name of each DVD drive includes a specific identifier.
Figure 4-3 DVD Drive Name with Identifier
Correct
Wrong
7. If the Logger is installed as a service:

a. From the Start menu of the Backup Server machine, select Run and enter gpedit.msc.
The Group Policy window appears.

b. Navigate to Windows Settings > Security Settings > Local Policies > Security
Options, and check that the following Policy is disabled: Devices: Restrict CD-ROM
access to locally logged on user only.
USB Tapes Not Detected

Follow the procedure below when the Backup Server does not detect USB tapes.
To troubleshoot USB tapes not detected by the Backup Server:

1. Check that certified drivers are installed for the USB tapes.
2. Additional installed backup devices prevent the detection of USB tapes. Check that no
DVD-RAM devices or SCSI/IDE tapes are connected to the Backup Server.
SCSI Tapes Not Detected

Follow the procedure below when the Backup Server does not detect SCSI tapes.
To troubleshoot SCSI tapes not detected by the Backup Server:

• Installed DVD-RAM devices prevent the detection of SCSI tapes. Check that no DVD-RAM
devices are connected to the Backup Server.

Other Backup Server Troubleshooting
Other Backup Server Troubleshooting

• If an error occurs in the DVD, do not restart the Logger before ejecting the archiving media
from all DVD drives in the system.
• Before performing a DVD device assignment change, such as from auto-archiving to retrieval
or vice versa, ensure that the Logger's front panel doors are fully open because the DVD tray
will automatically eject after the assignment change. Failing to do so can cause errors in the
Logger.
• When browsing in the My Computer folder, do not browse in the removable disk drives or
CD drives as this might cause an interruption in the Logger archiving operation.

Storage Center Troubleshooting

The diagram below describes the workflow for troubleshooting NICE Storage Center.

NOTE:
• In the Storage Center troubleshooting flow on page 126, SC refers to Storage
Center, as a matter of convenience.
• The main steps for troubleshooting the NICE Storage Center are detailed below. The
numbers ( ) in the flow on page 126 refer to these steps.
1. Set the log level to debug. See Setting the Log Level to DEBUG on page 128.
2. Check if the system is recording.
3. Check if the Storage Center is archiving any calls (See Checking for Archived Calls
on page 129):
• In Business Analyzer run a query and check if the calls are archived.
• Physically check on the Storage Center where the files are archived.
• Use Storage Center Investigator to check the archiving.
• Use Reporter and run the Archiving Coverage Report.
4. If the system is not archiving any calls:
a. Check the connectivity between the Storage Center and Loggers, the Storage Center and
Database Server and the Storage Center and Rule Engine (located on the Applications
Server).
b. Check that the Storage Center ports are open. See the Server Hardening Guide.
c. Check the relevant services on the (for further information on service, see the
Maintenance Guide):
• Check the Storage Center services: LogService, Nice Storage Streaming Service, and
Nice Storage Center Service.
• Check the rule services on the Applications Server: Nice Rule Engine service and
Nice RulesManager Service.
• Check the Nice Playback Administration service.
• Check the SQL Services on the Database Server.
d. Check that active storage rules are defined. In Rules Manager, click Storage and check the
storage rules.
e. Check the storage deletion and retention values. See Checking the Storage Retention
and Deletion Values on page 133.
f. Verify that the Storage Center Database is selected. See Verifying that the Storage
Center Database is Selected on page 133.
g. Check if there is a space problem. See Verifying the Used and Free Space on page 133.
h. Check the Storage Unit maximum threshold configured in System Administrator. See
Verifying the Storage Unit Threshold on page 134.

i. If you need to define more storage space, add another quota unit (under the quota group)
and restart the Storage Center. The Storage Center swaps to the next quota. There is no
need to create a new rule, as this can lead to calls not being archived (from the previous
rule).
j. Check that all the required updates are installed.
k. Check the Locate Order. See Checking the Locate Order on page 94.
l. Check the Playback logs. See Example of Troubleshooting using the Logs on page 108.
m. For playback problems, see the General Troubleshooting Flow on page 81.
5. If the system is not archiving specific calls, check for a pattern and establish which calls are
not being archived:
a. Calls from a specific Logger. See the General High Density Logger Troubleshooting
Flow on page 304 or the VoIP Troubleshooting Flow on page 329.
b. Calls which meet certain criteria, for example, long calls (these may be regarded as
personal calls and excluded via a rule). In this case, update the rule.
c. Calls from new agents. In this case, update the rule.
d. Calls from new extensions. In this case, update the rule.
e. Calls from extensions that have been changed. In this case, update the rule.
6. Check that the storage rule inserts calls. See Checking that the Archiving Rule Inserts New
Calls on page 135.
7. Troubleshoot the storage rules. See Troubleshooting the Archiving Rule on page 135.
8. Check the archiving table in the database. See Checking the Storage Center Archiving
Tables on page 137.
9. Check the relevant logs. See Checking the Storage Center Logs on page 138, Checking the
Nice Storage Streaming Service Logs on page 139, and Checking the Player Logs for
Storage Center Errors on page 139.
Setting the Log Level to DEBUG

You change the level of the Storage Center log files and of the Nice Storage Streaming Service log
files to DEBUG.
To set the log level to DEBUG:

1. Set the Storage Center log level from LOW to DEBUG by changing the entry in the registry:
a. Click Start > Run.
b. Enter Regedit.
c. From the Edit menu, select Find, and enter DEBUGCATEGORYMASK.
d. Set the value to 255.
e. Restart the Storage Center service.

2. Change the log level in Nice Storage Streaming Service log files to DEBUG.
a. Navigate to the Nice Storage Streaming Service storage directory and open
Nice.Storage.StorageStreaming.StorageStreamingHost.exe.
b. In the Nice Storage Streaming Service configuration file, change the log Filter attribute to
DEBUG.
c. Save the Nice Storage Streaming Service configuration file.
Checking for Archived Calls

You can check if calls are archived by running a query in Business Analyzer, checking physically
in the folders on the Storage Center or using the Storage Center Investigator.
To check for archived calls:

1. Run a query in Business Analyzer to check which segments were archived:
a. Run a query, for example, Segment - Last 7 days.
b. Click Preferences .
c. In the Table View area, move the Voice Archive Status and Screen Archive Status
columns from the Available Columns to the Selected Columns list box.

d. Click Save. The icon indicates a segment that is not archived.
2. Check the folders on the Storage Center to see if files have been written:
a. The Storage Center contains Storage Groups which are a logical grouping of one or more
Storage Units. Each time you create an archiving rule (via the Rules Manager), you
associate the rule with a Storage Group. The a Storage Unit contains either:
• Directory Per Day: Creates subdirectories under the defined directory path (for each
day of the year).
• Multiple Directories: Creates subdirectories under the defined directory path (based
on the destination type). Files are written to the subdirectories in a cyclic manner:
When one subdirectory reaches full capacity, files are written to the next subdirectory.
b. In the example below, recordings are archived in Directory Per Day storage units,
organized by server name\task name\archiving date.

3. Use Storage Center Investigator to check the archiving. See Using the Storage Center
Investigator on page 109.
a. Double-click the icon on the desktop to activate the application.

b. Click Start Storage Center Investigator in automatic mode.
c. In the Storage Center Installation directory field, verify that the Storage Center path is
correct.
d. In the Available Databases area, select the checkbox of the database you want to query.
e. In the Automatic Reporting Settings fields:
• Enter the desired time interval to generate automatic reports.
• Enter the desired number of days backwards to include in the reports.
f. Click Next to view the results.
g. In the NICE Storage Center Investigator Results tab, click the Database overview
sub-tab to view query results.

4. Check the archiving to the Storage Center. In Reporter, run the Archiving Coverage Report,
and check the Archiving Coverage per Storage Center view.
Gathering Storage Center Information

Use the Storage Center Investigator to verify that all the required files are archived on the file
system. The Storage Center Investigator is installed together with the Storage Center in the same
installation directory.
The default path is: C:\Program files\NICE Systems\Storage Center.
See Using the Storage Center Investigator on page 107 for detailed instructions on how to use
the Storage Center Investigator.

Checking the Storage Retention and Deletion Values

The Storage retention value is the number of retention days for the files in the Storage Group,
meaning the number of days that the recordings must remain in the Storage Group. By default,
there is no mandatory retention. The retention value is usually set to the amount of days that the
site is obligated to save recordings. The maximum retention period is seven years (based on the
NICE database design).
When you define a storage rule, you can define a value for Force Deletion, meaning that files
archived by a specific rule will be deleted after a certain number of days.
What happens if a conflict occurs between the group's Retention value and a rule's Force Deletion
value (such as the group's Retention value is set to 100 days)?
The Force Deletion value of the rule takes preference!
The group Retention value is a more general definition; the archiving rule Force Deletion value is
specific to a group of archived files.
Verifying that the Storage Center Database is Selected

In the System Administrator, check that the Storage Center database is selected.
• In System Administrator, in the Organization tree, under Master Site > Database Servers,
General tab, check that the nice_storage_center database is selected in the Databases on
this SQL Server area.
Verifying the Used and Free Space

Check the used and free space on the Storage Center servers and storage areas.
1. In System Administrator, in the Organization tree, under System Monitoring > Storage
Center Servers branch, expand the hierarchy to view a list of all Storage Centers.
2. Expand a Storage Center to view its storage areas.

3. Select a Storage Center or one of its storage areas. The Status window appears.
Verifying the Storage Unit Threshold

Check the storage unit threshold on the Storage Center server.
To verify the storage unit threshold:

1. Log in to the NICE Perform Application Suite. In the navigation bar, click Accessories, and
select System Administrator.
2. From the Settings menu, select Technician Mode.
3. In the Organization Tree, select the Storage Center Servers branch.

4. Expand a Storage Center to view its storage groups and units (SCGroup > SCUnit), and
check the Maximum Threshold.

Figure 4-4 Storage Details
Maximum Threshold
Checking that the Archiving Rule Inserts New Calls

Archiving to Storage Center starts with the archiving rule. The rule determines which calls needs
to be archived and then adds the required information to the tblStorageCenter and tblSCTasks
tables in the database.
1. Check the Storage Center tables, tblStorageCenterXX in the nice_interactions of the
relevant calls to verify that the Rule Engine inserts new calls. From Storage Center
Investigator, query iArchiveID to see if the ID number increases, and check the Start and Stop
times of the last interactions. When you know the time of the last interaction, you can verify
whether new interactions were inserted.
2. Check whether new tasks are added to the tblSCTasks table in the nice_admin database. If
tasks are added, check that they match the rule filters.
If the Rule Engine inserts new calls into tblStorageCenterXX, the archiving rule is probably
correct and the problem is in the Storage Center. In this case, continue with Checking the Storage
Center Archiving Tables on page 137. If there are no new calls, continue with Troubleshooting
the Archiving Rule on page 135.
Troubleshooting the Archiving Rule

Troubleshooting the archiving rule comprises the following steps:
1. Check the Rule Engine Log file for errors.
2. Check that there is an Active Rule in the Rule Manager, and check its schedule.
3. Check the Storage Center rule schedule. See the procedure below.
4. Check that the Rule Engine generates actions. See page 136.
To check the Storage Center rule schedule:

1. In the nice_rule database, check the next scheduled time of the Storage Center rule by
running the following query:
SELECT tblREStorageCenterRule.nvcRuleName,
tblRELocalRuleSchedule.iScheduleID,
tblRELocalRuleSchedule.dtNextScheduleTime,

tblRELocalRuleSchedule.dtLastRunTime
FROM tblRELocalRuleSchedule INNER JOIN
tblREStorageCenterRule ON tblRELocalRuleSchedule.iScheduleID =
tblREStorageCenterRule.iScheduleID
WHERE (tblREStorageCenterRule.iStatus = 1)
A section of the tblREStorageCenterRule table appears below, showing the format of the date
fields.
dtNextScheduleTime is the GMT time.

dtLastRunTime is the local time of the Applications Server.
2. Check that the Rule Engine database is not full.
To check that the Rule Engine generates actions:

1. Check that the Rule Engine generates actions for the Storage Center Rules by running the
following query:
SELECT *
FROM tblREStorageCenterAction
2. In the query results, check the start time, stop time and interaction IDs in the results of the
query.
3. If the table does not fill up, check that the Event Providers collect the interactions by running
the query:
SELECT *
FROM tblRECallServerEvent
4. Check that the results contain calls that are not archived.
• If the table does not contain the calls, the error can be in the Event Collection step. Call
NICE Customer Services.
• If the interaction is in the Event table, the problem can be in the Generation step. The
reason could be one of the following:
• The Generation Query takes a lot of time and does not complete because of timeout.
Maybe there are missing indexes. See the Database Administrator.
• The rule has problems with the file name mask and the post process of the Storage
Center rule. The database function spREStorageCenterRulePostProcessing
failed. Check that the File name mask is not empty by running the following query:

SELECT nvcRuleName, iRuleId, nvcDescription, iStatus,

nvcFileNameMask
FROM tblREStorageCenterRule
Check that nvcFileNameMask is not empty for the required rule.
• The problem could be in the Execution step. In a Multi Data Hub environment, the
secondary Rule Engine Log file can contain the following message:
[StorageCenterExecuter.OnProcessWorkItem] Failed to execute Storage Center action
System.Data.SqlClient.SqlException: The operation could not be performed because the
OLE DB provider 'SQLOLEDB' was unable to begin a distributed transaction.
[OLE/DB provider returned message: New transaction cannot enlist in the specified
transaction coordinator.]
OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin:JoinTransaction
returned 0x8004d00a].
• It may be that the Microsoft Distribute Transaction Coordinator (MSDTC) is not
configured properly. Storage Center rules require that the MSDTC works correctly. This
can be checked by performing the following command:
BEGIN TRANSACTION
SELECT * FROM nice_rule_link.nice_admin.dbo.tblSCTasks
COMMIT
If the command fails, MSDTC must be fixed.
• In a Multi Data Hub environment, the problem could be in the Rules Manager behavior.
Rules defined by the Rules Manager are relevant for all sites. The Rules definitions are
written to the Master site and are replicated to all Secondary Data Hubs.
Checking the Storage Center Archiving Tables

1. Check the iFSArchiveClass column in the tblStorageCenterXX table. The values represent
the following:
ID Status Action
0 Not archived yet to Storage Center Check if the Storage Center service is up
file system. and check the Storage Center log files.
1 Archiving in progress to Storage Check the Storage Center log files.

Center file system.
2 Archived to Storage Center file Everything is OK.

system.
3 Call is being deleted from Storage

Center file system.

ID Status Action
4 Call deleted from Storage Center file

system.
5 Archiving failed to Storage Center Check the Storage Center log files.
file system.
6 Call deletion from Storage Center Check the Storage Center log files.
file system failed.
7 Call was not found in the Logger. Check why the call is not found in the
Logger
2. Check the iEsmArchiveClass column in the tblStorageCenterXX table. The values

represent the following:
ID Status Action
0 Not archived yet to ESM. Check if the Storage Center service is up

and check the Storage Center log files.
1 Archiving in progress to ESM. Check the Storage Center log files.
2 Archived to ESM. Everything is OK.
3 Call is being deleted from ESM.
4 Call deleted from ESM. Check the Storage Center log files.
5 Archiving failed to ESM Check the Storage Center log files.
6 Call was not found in the Logger. Check the Storage Center log filesr
Checking the Storage Center Logs

1. Review the Storage Center logs. By default, the log files are stored in the same directory
where the Storage Center is running. The default log file names are:
Time Event Category Description
Owner Thread ID

2. Track the task snapshots in the log files. Each task snapshot details the current tasks and the
number of elements distributed among Loggers.
Task Name Task ID Rule ID
Logger ID Number of Elements
3. Ensure that the logs do not contain errors relating to the Storage Center startup process.
4. To collect the logs, use the Log Collector. See the Collecting Information Using the Log
Collector on page 537 for further information.
Checking the Nice Storage Streaming Service Logs

You check the Nice Storage Streaming Service logs to verify that playback performed from
Storage Center is successful.
By default, the log files are stored in the following directory:...\Program Files\Nice
Systems\Storage Center\Nice Playback Service SC\StorageStreaming<Process
ID>Logs. The Process ID is a value that changes each time the system is restarted.
The following procedure lists the text strings you need to find in the Nice Storage Streaming
Service logs in order to verify that playback from the Storage Center was successful.
To check the Nice Storage Streaming Service logs:

1. Search for the text ERR:, and verify that it does not appear in the log file.
2. Search for the text StartOutgoingSessionEvent, and verify that it appears.
3. Search for the text OpenSession, and verify that it appears.
4. In the same row as the OpenSession text, verify that Result=OK appears.
5. Search for the text StartStreamingPackets, and verify that it appears.
6. Search for the text StopOutgoingSessionEvent, and verify that it appears.
7. Search for the text StopStreamingPackets, and verify that it appears.
Checking the Player Logs for Storage Center Errors

The player errors are recorded in the Player log files, in *.txt format, in
%APP_DATA%\NicePlayer\Release3. The following Storage Center errors may appear in the
Player logs.

Error
No.
300007 The Storage Center failed to connect to a Check the Logger where the backup
backup device on a Logger. This should device is located.
not be considered an error if there are no
backup devices in the site.
300008 The Storage Center did not archive or it See the Playback Troubleshooting
had an error in the locate flow. Flow on page 77.
Other Troubleshooting Issues

• If the Storage Center is not communicating with the NMS, see the Storage Center Installation
Guide. Perform the following verification tests to ensure that all the objects are configured
properly:
• Verify Nice Storage Center Service User
• Verify the SnmpAgent Object
• Verify the SNMP Service Settings
• If interactions are not being archived on a Storage Center server with multiple active network
interface cards, check that the Storage Center IP address defined in the System Administrator
is the primary network connection. For more details, see the Storage Center Installation
Guide.
• If there are non-archived calls in the Storage Center tables, use the Storage Center Investigator
to check if the iRuleID of the calls is still active. Storage Center will not archive calls that
belong to an inactive rule. In this case, change the iRuleID to an active one. See Using the
Storage Center Investigator on page 107.

5
Multi Data Hub Troubleshooting Flow
This chapter describes common troubleshooting issues when employing a Multi Data Hub with
NICE Perform Release 3.1.
Contents
Gathering Information ..................................................................................................142

Multi Data Hub Implications.........................................................................................143
Checking Connectivity .................................................................................................144
Checking Replication ...................................................................................................145
Troubleshooting Replication .......................................................................................147
Troubleshooting Saving in a Multi Data Hub Environment ......................................148
Changing Passwords ...................................................................................................148
Enabling MSDTC to Function in Multi Site Environments ........................................148
Other Troubleshooting .................................................................................................149
Chapter 5: Multi Data Hub Troubleshooting Flow 141

Important issues to consider are:
• Is this a new installation? If yes, when was the installation performed?
• Were there any changes to the environment that might have influenced the system? If yes,
check these changes first.
• Is the problem reproducible? If yes, what were the steps?
• Were the Servers restarted? If yes, did this resolve the problem?

Multi Data Hub Implications
Multi Data Hub Implications

A Multi Data Hub system has site implications that affects the different system components.
Changes in the components are all done automatically with no user intervention or configuration.
Database
• The Admin, Interaction, Audit and Rule databases are mandatory on all Data Hubs.
• The nice_interactions databases are not replicated across Data Hubs.
System Administrator
• Administration updates are blocked when the Primary Data Hub is not available.
• The login mechanism will check login token on all Data Hubs. Users will belong to a specific
Data Hub.
Rule Engine
• All rules are replicated from the Master Data Hub to the Secondary Data Hubs.
• Each local rule engine will handle the local rules.
• Storage Center rules are an exception as they are all stored only on the Master Data Hub. This
means that all Data Hubs operate using the same Storage Center rules.
Business Analyzer
• Public, private and saved items are replicated between all Data Hubs.
• User preferences are local.
• Running a query on Multi Data Hubs:
• The Unified Query engine queries the local Data Hub.
• Each remote Data Hub with agents from the query filter will be also queried.
• All results will be combined into one result.
• Locate
• The Unified Query service will connect all relevant Locate services in all Data Hubs.
• The Locate process looks only in the local Data Hub according to the locate order.
Playback
• Each Data Hub has its own Media Server.
• One or more Playback Servers can be configured in each Data Hub.

Checking Connectivity
Monitor
• User preferences are saved on the local Data Hub.
• Monitor will subscribe to events to the local CLS or remote Monitor service.
Quality Management
• All QM objects (Evaluations, Coaching Packages, Calibrations and Clips) will be saved in the
user's local Data Hub.
• Evaluations on remote calls will be replicated to the remote Data Hubs.
• Form definitions are replicated to all remote servers.
My Universe
• Preferences are stored locally.
Interaction Server
• All recording rules are replicated.
• Only handles local Integrations.
Storage Center
• Tasks are read from the Master Data Hub only.
• Archiving can be done from all Data Hubs.
Check connectivity between sites (PING using IP address and hostname).

Checking Replication
Use the following methods to check replication in Multi Data Hub environments.
To check replication in multi data hub environments:

1. In the NICE System Administrator, verify that the replication is defined using only the
machine names.
2. Check the connectivity between the Primary and Secondary Data Hubs.
3. Connect to the SQL Server Management Studio. In the Object Explorer, navigate to
Replication and check if there are components with an error.
4. In the NICE System Administrator:
a. Navigate to Master Site > Database Servers > Databases and click the Replication
tab.
b. Verify that no errors appear in the Status column.
5. If the Primary and Secondary Data Hubs have different database structures, the replication will
fail at those specific points. To ensure that the replicated data is identical, run tests on all Data
Hubs to ensure that additional information (business data, any external additions such as
Professional Services solutions) can be accessed from different sites.
6. Verify that the same version of the NICE Perform Database and the same updates are installed
on all Data Hubs. Check the following tables in the nice_admin database:
a. The database version appears in the tblDatabaseDetails table.
Figure 5-1 tblDatabaseDetailsTable
b. The update versions appear in the tblDatabaseHotFix table.

Figure 5-2 tblDatabaseHotFixTable
NOTE: Only those updates that were installed using an Update Pack appear in the
tblDatabaseHotFix table; updates that were installed manually do not appear.

7. When a Multi Data Hub environment is upgraded, replication should be removed during the
procedures. After the upgrade has been completed, create the replication again. Then test that
the replication is functioning properly.

Troubleshooting Replication
Troubleshooting Replication
In most problems that deal with replication, the Primary Data Hub is usually updated with
information. If there is a problem updating the Primary Data Hub, the replication itself will not
begin and the problem lies elsewhere. It is therefore important to first verify that the Primary Data
Hub has been updated and only then start troubleshooting replication problems.
Troubleshooting Communication Problems between Primary and

Secondary Data Hubs
Secondary Data Hubs are read only. All updates are written to the Master Data Hub and replicated
to the Secondary Data Hub.
To troubleshoot communication problems:

1. If there is a communication problem between the Primary and Secondary Data Hubs, data
updates on the Secondary Data Hub will not take effect. These data updates will not be stored
locally and will have to be completed once the communication problems have been resolved.
2. If a Secondary Data Hub is currently disconnected from the Master Data Hub, updates from
other Data Hubs will not be seen. Since all information is stored in the distribution database on
the Master Data Hub, these data updates will be completed once the Data Hub is connected
again.
Troubleshooting Database Structure Differences

The database structure can be changed in different ways, for example, names, number of fields,
field type, field length, additional or fewer tables. If a Master Data Hub and Secondary Data Hub
have different database structures the replication will fail at those specific points.
NOTE: It is very important not to make "manual" changes to the Secondary Data Hubs
as these will be overwritten when the data is replicated from the Primary Data Hub.
To check the database structure:

1. To ensure that the data is identical, run tests on all Data Hubs to ensure that additional
information (business data or any external additions such as Professional Services solutions)
can be accessed from the different Data Hubs.
2. For example, an error from the Nice DW Population Schedule job may indicate a problem
in the tblReservedXX tables (containing the business data fields).
Stopping Replication When Upgrading All Data Hubs

Upgrades must be applied to all Data Hubs simultaneously. An upgrade can be an Update or
Service Pack. Since upgrades may cause a change within the SQL structure, it is important to
remove the replication before implementing the upgrade.

Troubleshooting Saving in a Multi Data Hub Environment
NOTE: You must bring down communication between all the Data Hubs when
upgrading.
To stop replication when upgrading all data hubs:

1. Remove the replication prior to the upgrade. The replication should be stopped to all Data
Hubs, not just the Data Hub that is being upgraded
2. Once the upgrade is complete, create the replication again.
3. Test the site operation.

• Replication works with the Machine Name only.
• Any change to the database scheme of replicated or un-replicated tables requires that the
replication be shut down for that site.

In a Multi Data Hub environment, some of the information in nice_admin is replicated. If
replication is down, you will be unable to save in System Administrator.
Changing Passwords
If the nicesa user password is changed, the replication agents will indicate that the password is not
valid and the replication will not work and must be recreated.
To change nicesa password in Multi Data Hub environment:

1. Remove the replications.
2. Change the nicesa password.
3. Recreate the replications.
Enabling MSDTC to Function in Multi Site Environments

The SQL Server uses Microsoft Distribute Transaction Coordinator (MSDTC) to execute
distributed transactions. MSDTC will not function properly in an environment where the two
servers are using the same Windows OS image, for example where the 'Ghost' program is used to
copy the Windows image to the disk. The reason for the problem is that both servers that are
installed from the same image, have an identical key in the MSTDC section in the SQL registry. If
the distributed transaction fails and the registry keys are different, there may be other reasons for
the failure. See TN0676 Enabling Microsoft Distribute TransactionCoordinator (MSDTC) to
Function in Multi Site Environments. This note describes how to fix the problem without
re-installing the entire system.

• A replication error Cannot insert explicit value for identity column in table
'tblColumnInfo' appears. This occurs if the subscription table tblColumnInfo.iColumnId
identity is set to true. To solve the problem set the identity value to False.
• If there is a problem with replication, submit the following log file to NICE Customer Support:
Program Files\NICE Systems\
Applications\ServerBin\NiceApplications.SystemAdministrator.Host.exe.log

6
Media Encryption Troubleshooting Flow
This chapter includes tables of trap messages and ScreenAgent Management System (SAMS) log
messages as well as common troubleshooting issues.
Contents
Troubleshooting Workflow ..........................................................................................152

Trap Messages ..............................................................................................................159
SAMS Log Messages ...................................................................................................163
Verifying the IIS User Permissions for the KSM Configuration................................164
Using a Fully Qualified Domain Name instead of a Host Name ...............................166
Data Not Entered in the nice_crypto Database ..........................................................169
Renewing Certificates ..................................................................................................171
ScreenAgent Not Working Properly............................................................................172
Ensuring Connection between SQL Server and Applications Server .....................175
Installing the CA certificate on Vista Workstations ...................................................175
Chapter 6: Media Encryption Troubleshooting Flow 151

Troubleshooting Workflow
If you experience a problem with Media Encryption follow the steps below to isolate the problem.
1. Change the logging level to ALL. See Changing the Logging Level on page 153.
2. On the Key Storage Manager (KSM), check the connectivity between the Applications Server
and the Database. See Checking Server Connections on page 153.
3. Verify the connectivity between the Logger/ScreenAgent and the KSM on the Applications
Server by pinging the Applications Server.
4. Verify the connectivity between the KSM on the Applications Server and the Cryptographic
Database by pinging the SQL Server which has the NICE Cryptographic Database
(nice_crypto).
5. Use Telnet to check that the Logger/ScreenAgent can reach the Applications Server on port
443.
6. Check the Logger/ScreenAgent. See Checking the Logger/ScreenAgent on page 153.
7. Check the KSM status. See Checking the KSM Status on page 154.
8. If the KSM status is FAILED, verify that the KSM is functioning properly. See Verifying the
KSM is Functioning on page 154.
9. Check the CIM trap messages. See Checking the SNMP Traps on page 154.
10. Check the KSM trap messages. See Checking the SNMP Traps on page 154.
11. Check the client-side logs. See Checking the Client-side Logs on page 156.
12. Check the server-side logs. See Checking the Server-side Logs on page 156.
13. Check for certificate problems. See Checking for Certificate Problems on page 157.
1. Is this a new installation? If yes, when was the installation performed?
2. Were there any changes to the environment that might have influenced the system? If yes,
3. How often does the problem occur?
4. Is the problem reproducible? If yes, what were the steps?
5. Were the Servers restarted? If yes, did this resolve the problem?

Changing the Logging Level

To generate more detailed logs, you change the logging level in the Logger and the ScreenAgent.
To change the logging level:

• On the Logger/ScreenAgent, in ConfigPlugin, set the debug level, by changing the values in
both Level and Threshold to ALL:
<log4net>
<logger name="NiceApplications.SystemAdministrator">
<param name="Level" value="ALL" />

<appender name="RollingLogFileAppender"
type="log4net.Appender.RollingFileAppender">
<param name="Threshold" value="ALL" />
Checking Server Connections

You can check on the KSM whether the files were sent to the SQL Server database. If the folder
contains files, this indicates a connection problem between the Applications Server and the SQL
Server database.
To check the Applications Server to Database connection:

• On the KSM, check the EncryptionBackupFiles folder located at the following path:
...\<location of applications>\KeyStoreManager\WebService\EncryptionBackupFiles.
If this folder contains files, this indicates a problem with the connection between the
Applications Server and the SQL Server database.
Checking the Logger/ScreenAgent

You need to verify that the encryption services are running properly on the Logger/ScreenAgent.
To check the Logger/ScreenAgent:

1. On the Logger/ScreenAgent, verify that the LogService located in C:\Program Files\Common
Files\Nice\LogService\Bin\LogSysSrvr.exe is running.
2. Verify that the CIMService.exe service is running.
3. Verify that CIMService.exe service has Local System Account privileges.
4. Verify that the service has privileges to write to the disk.
5. Verify that there is enough space on the disk.

6. Check if the CimClinet, CimHistory and CimKSM files exist in the C:\document and
settings\all users\application data\Nice\CryptoInfo folder on the Logger/ScreenAgent.
The size of the CimKSM file is 0 KB. If the CimKSM file is not 0 KB, then there is a problem
with the connection between the Logger and the KSM.
Checking the SNMP Traps

In the SNMP Console, view the events generated by the CIM and KSM.
See the NMS User's Guide or Trap Messages on page 159 for a list of the trap messages and
recommended actions.
Checking the KSM Status
To check the KSM status:

1. In the navigation bar, click Accessories, and select System Administrator.
2. Verify that you are working in Technician mode: From the Settings menu, select Technician
Mode.
3. In the Organization tree, select Security > Media Encryption.
4. Click Test to check the KSM status. If the status is Failed, verify that the KSM is functioning
properly. See Verifying the KSM is Functioning on page 154.
Verifying the KSM is Functioning

When you cannot connect to the KSM, you do not know if the problem lies with the KSM or with
the client machine. In this procedure, you verify that the KSM is functioning properly.

To verify if KSM functions properly:

1. On the KSM server, navigate to IIS > Web Sites > Default Web Site > KSMWebService.
2. In the right pane, right-click KSMService.asmx and select Browse.
The Internet Explorer opens.
3. In the Address field, change the URL address to: https://<KSM machine
name>/KSMWebService/KSMService.asmx.
If the following screen appears, then the problem is probably with the client workstation.

NOTE: If you did not replace the name of the local host with the KSM machine name,
the following window appears:
Make sure that you click No. (If you click Yes, the verification process is skipped.)
Checking the Client-side Logs

Check the following client-side logs:
• Nice Player
<Drive>:\Documents and Settings\Administrator\Local Settings\Application
Data\NicePlayer\Release3\NicePlayer.txt
• Logger/ScreenAgent
<Drive>:\Documents and Settings\All Users\Application
Data\Nice\Log\CryptoInfoManagerLogExceptions.log
• Encryption
<Drive>:\Documents and Settings\Administrator\Local Settings\Application
Data\NICE\EncryptionPluginClientLog.txt
Checking the Server-side Logs

Check the following server-side logs:
• KSM
C:\Program Files\NICE
Systems\Applications\KeyStorageManager\WebService\KeyStorageManager.txt
• Encryption
C:\ Program Files\NICE
Systems\Applications\ServerBin\NiceApplications.EncryptionPlugin.log

Checking for Certificate Problems

The certificates might have expired.
To check for certificate problems:

1. Check with your certification authority if your certificates have expired.
2. On the Applications Server, check in the IIS log files (KeyStorageManager.txt.x) for the
following type of error:
ERROR: [SNMP.RegistrateToTheSNMPConnector] SnmpConnector: New
SNMPConnectorObject was added to the connector.
This indicates a problem with the site certificate. Check for other errors as well.
3. On the Logger/ScreenAgent, check in the Event Viewer if there are any CIMService error
messages.
In the Source EFS and Description areas, the following error indicates a problem with the site
certificate.
An error with Source: EFS and Description: EFS recovery policy contains invalid
recovery certificate.
4. On the Logger/ScreenAgent, check the properties of the CryptoInfo folder:

a. Go to C:\Documents and Settings\All Users\Application Data\CryptoInfo,
right-click the CryptoInfo folder and select Properties.

b. Select Advanced. The Advanced Attributes window appears.
c. Select Encrypt contents to secure data. If you receive the following error, this
indicates a problem with the site certificate.
5. Issue new certificates for the SQL Server and IIS.

Trap Messages
Trap Messages
Trap Messages for CIM Module
Trap No. Trap Cause Action
6141 Cryptographic There is no certificate. (This Install CA root on the

Information Manager trap will be sent with the no machine.
cannot connect to certificate trap). Check the KSM.
KSM. There is a problem in the KSM Check the DB of the
(not running). SysAdmin for the KSM URL
The KSM URL obtained from – see that its valid.
the System Administrator is
invalid.
6142 Cryptographic There is no connection to the Check that the SysAdmin is

Information Manager System Administrator. up.
cannot connect to System Administrator URL is Install the client machine
System Administrator. invalid. once again and change the
SysAdmin URL.
6143 Cryptographic The file attribute is read only. Change the attribute of the
Information Manager files to read/write.
cannot save encryption The path is C:\Documents
data to a persistent file. and Settings\All
Users\Application
Data\Nice\CryptoInfo.
6144 Cryptographic There is probably a problem in

Information Manager is the initialization (such as
not initialized. encryption not configured).
This trap will be sent together
with another trap.
6145 Encryption is not The encryption is not yet Check the entire encryption
configured in the configured in the system. The components and check that
system. KSM is not yet configured. the KSM is configured
properly.
6146 The Customer has no There is no certificate installed Install the CA root in client
valid certificate for on the client machine. machine.
encryption. (Make sure that the CA is
installed both on current
user and on the local
machine.)
6147 SystAdmin URL There is no SysAdmin in the Run the client installation
registry key is empty. registry – it was not provided in and set the SysAdmin URL.
the client installation.

Trap Messages
6148 KSM URL does not The SysAdmin URL is not Check the SysAdmin
exist in the System aware of the KSM URL, database for the KSM URL.
Administrator. therefore when asked for the
KSM URL it returned null.
6149 There is no license for The encryption feature does Check that there is license
encryption in the not exist in the client site. for encryption for the client.
system.
6150 The Logger login token The login token against Check the version of the
is not valid. SysAdmin is invalid probably SysAdmin DLLs in the CIM
because of mismatch between Server - check that this is
DLLs in the CIM server and in the correct version.
the SysAdmin.
Trap Messages for KSM Module
5200 Cannot establish a a. Database is down. a. Start the database.

secure connection b. Connection problem to the b. Verify that the
between Key Storage database. connection to the
Manager and the database is OK (using
c. Secure connection problem
database. the Query Analyzer).
to the database: no CA root
installed on the KSM c. Check that there is a CA
machine, problem with the root installed on the
SQL certificate. KSM machine. Verify
that there is a certificate
assigned to the SQL
service and that it is
valid.
5201 Secure connection Fixes the above trap.

between Key Storage
Manager and database
was established
successfully.
5202 The database The database certificate is Call NICE Support Services.
certificate is missing. Therefore, no
missing/corrupted: data encryption or decryption of the
cannot be encrypted data is possible.
encrypted/decrypted.
5203 The database Fixes the above trap.

certificate is valid: data
can be
encrypted/decrypted.

Trap Messages
5204 Cannot establish a a. Remote KSM machine is a. Start the machine.

secure connection to down.
remote Key Storage b. No connection to the b. Verify that the
Manager. remote KSM machine. connection to the
c. Remote KSM web service remote machine is OK.
is unavailable. c. Use the Media
Encryption branch in
order to test the
d. There is no CA root
connection to the
installed on the local KSM
remote KSM.
machine.
d. Verify that there is a CA
root installed on the
local KSM machine.
Also, use the Media
Encryption branch from
the local KSM machine
in order to test the
connection to the
remote KSM.
5205 Secure connection to Fixes the above trap.

remote Key Storage
Manager was
established
successfully.
5206 Database master key Restore procedure of the Check in the nice_crypto
cannot be opened: data database was incomplete: the database, security,
cannot be database master key was not certificate folder, that the
encrypted/decrypted. restored correctly. certificate exists.
Notify NICE Support
Services.
5207 Database master key Fixes the above trap.

opened successfully.
5208 Duplicated GUID was A GUID that already exists. Notify NICE Support
stored to the database. Services.
KSM module is up. The KSM web service is up.
5209 Failed while writing The KSM failed to write to the Use the Media Encryption
Cryptographic backup file. This could happen branch in order to test the
Information to file. if the KSM application pool user KSM status. KSM also tries
does not have privileges to the to create a file in the
EncryptionBackUpFiles folder. EncryptionBackUpFiles
folder.

Trap Messages
5210 Succeeding in writing Fixes the above trap.

Cryptographic
Information to file.
5211 Failed while reading a. The KSM failed reading the a. Verify that the KSM
Cryptographic backup file. This could application pool user
Information from file. happen if the KSM has read privileges to
application pool user does the
not have privileges to the EncryptionBackUpFiles
EncryptionBackUpFiles folder.
folder. b. Check which user
b. Can also happen if the file encrypted the file, by
was written and encrypted right-clicking the file >
by a different user (The Properties. Then in the
KSM application pool user General tab, click
was replaced). Advanced > Details.
If the user is not the
current KSM application
pool user, change it to
the user who encrypted
the file, and wait for the
file’s content to be
written to the database.
5212 Succeeded in reading Fixes the above trap.

Cryptographic
Information from file.

SAMS Log Messages
SAMS Log Messages

ScreenAgent Module Messages
Message. Cause Action ScreenAgent Behavior
Problem to connect CIM Error ScreenAgent will retry until

to server the operation succeeds.
Server not initialized CIM Error ScreenAgent will retry until

the operation succeeds
Problem connecting Network, Server is Verify server is up, IP is ScreenAgent will record
to KSM down, wrong IP, etc. correct, etc. with encryption.
Problem connecting Network, Server is Verify server is up, IP is ScreenAgent will retry until
to SysAdmin down, wrong IP, etc. correct, etc. the operation succeeds.
Encryption not Encryption is not Configure Encryption. ScreenAgent will retry until
configured configured in the operation succeeds
SysAdmin
Client is not CIM Error - ScreenAgent will retry until

connected the operation succeeds
Problem to create CIM Error - ScreenAgent will retry until

client connection the operation succeeds.
Problem in client CIM Error - ScreenAgent will retry until

connection the operation succeeds.
SysAdmin URL does Enter the SysAdmin URL ScreenAgent will retry until
not exist in the registry. the operation succeeds.
KSM URL does not Enter the KSM URL in the ScreenAgent will retry until
exist registry. the operation succeeds.
Problem saving CIM Error - ScreenAgent will retry until

Crypto to file the operation succeeds.
No license for No encryption license If the site is to be an ScreenAgent will record

encryption in site in the System. encryption site, install a without encryption.
license. If the site is not an
encryption site, reinstall
the ScreenAgent with no
encryption.
No certificate on the Certificate was not Install certificate. ScreenAgent will record
machine installed on the with encryption.
workstation.

Verifying the IIS User Permissions for the KSM Configuration
Verifying the IIS User Permissions for the KSM

Configuration
If there is an error when configuring the KSM, you need to verify that the appropriate permissions
are granted to the IIS user for the following folders:
• Temp folder
• EncryptionBackupFiles folder
• WebService folder
IMPORTANT : Make sure that all permissions are also inherited by sub-folders.
To verify IIS user identity permissions:

1. Open the C:\Windows\Temp folder.
2. Click the Security Tab.
The following window appears.
3. Select the group or user names of the current IIS user.

4. In the Permissions for Administrators area, verify that the following permissions are selected
in the Allow column:

Verifying the IIS User Permissions for the KSM Configuration
• Read & Execute

• Read
• Write
5. Click OK.
6. Navigate to the
...\<location of applications>\KeyStoreManager\WebService\EncryptionBackupFiles
folder and repeat Step 2 and Step 3.
7. In the Permissions for Administrators area, verify that the Full Control permission is selected
in the Allow column.
8. Navigate to the ...\<location of applications>\KeyStoreManager\WebService\WebService
folder and repeat Step 2 and Step 3.
9. In the Permissions for Administrators area, verify that the Full Control permission is selected
in the Allow column.

Using a Fully Qualified Domain Name instead of a Host Name
Using a Fully Qualified Domain Name instead of a Host

Name
If the IIS and SQL certificates require a Fully Qualified Domain Name (FQDN), you need to
ensure that all calls to the KSM Web service and connections to the Database Server are made
through the FQDN, as opposed to through the host name.
IMPORTANT : Ping the Database Server using the FQDN to verify that it can connect in this
manner before changing the host name.
The FQDN is displayed in the Full computer name field of the Computer Name tab in the
System Properties window. (Right-click My Computer and select Properties to open the
System Properties window.)
To use an FQDN for the KSM Web service and SQL Server:
1. Ping the Database Server using the FQDN to verify that it can connect in this way.
2. From the System Administrator application, expand the Master Site (Primary Data Hub)
branch and select Applications > Application Servers > <name of server where the Key
Storage Manager is installed>.

The Application Server Services screen appears.
3. In the Host Name/IP Address field of the Key Storage Manager (KSM), enter the FQDN
instead of the host name.
4. Restart the CIM service on all Loggers and ScreenAgents.
5. Select Database Servers.
The Database Server screen appears.

Figure 6-1 Database Server Screen
6. In the Host Name/IP Address field of the SQL database, enter the FQDN instead of the host
name.
7. Restart the SQL Server.

Data Not Entered in the nice_crypto Database

Sometimes the data in the CryptoInfoData.1.txt file is not entered as expected into the
nice_crypto database. This happens if the current KSM user is not the user who originally
encrypted the file. In this procedure, we do the following:
• Verify the identify of the original user who encrypted the file
• Change the KSM user back to the original user in order to access the file
• Change the original KSM user back to the current KSM user
To enter data from CryptoInfoData.1.txt file into the nice_crypto database:

1. Right-click the CryptoInfoData.1.txt file, and select Properties.
NOTE:
By default, the CryptoInfoData.1.txt file is located at the following path:
...\Program Files\NICE Systems\Applications\KeyStorageManager\WebService\
EncryptionBackupFiles
2. On the General tab, click Advanced.

The Advanced Attributes window appears.
3. Click Details.

The Encryption Details window appears.
4. The user who originally encrypted the file appears in the Data Recovery Agents For This
File As Defined By Recovery Policy area.
5. If the original user is different than the current user, configure the Key Storage Manager server
with the original user.
6. Restart IIS.
7. Wait for the nice_crypto database to access the CryptoInfoData.1.txt file.
8. Change the original user back to the current user.

Renewing Certificates
Renewing Certificates
Instead of renewing certificates, you need to issue new certificates for the SQL Server and IIS.
Screens Not Recorded

At times, screens are not recorded when encryption is enabled, and the following error message
appears:
CCimHelper: GenerateCryptoInfo failed. Retrying...: Problem saving Crypto to file. state
= -4&gt.
In this case, the Antivirus software is interfering with screen recording.

ScreenAgent Not Working Properly

ScreenAgent is not working properly, and the following appears in the ScreenAgent log file:
EVENT: 28/03/07 14:07:08:276 CCimHelper:GenerateCryptoInfo failed. Retrying...:
Problem saving Crypto to file. state = -4; Context = (3464).
This problem may occur if the CimClinet, CimHistory and CimKSM files are missing from the
C:\document and settings\all users\application data\Nice\CryptoInfo folder on the
workstation.
This problem occurs as a result of a Windows EFS (Encrypting File System) configuration issue.
The certificate must be renewed.
To check ScreenAgent:
1. Check if the CIMService.exe service is running.
2. Verify that CIMService.exe service has Local System Account privileges.
3. Verify that the service has privileges to write to the disk.
4. Verify that there is enough space on the disk.
5. Check if the CimClinet, CimHistory and CimKSM files exist in the C:\document and
settings\all users\application data\Nice\CryptoInfo folder on the workstation. The size of
the CimKSM file is 0 KB.

6. Check in the Event Viewer if there are any CIMService error messages. An error with
Source: EFS and Description: EFS recovery policy contains invalid recovery
certification indicates a problem with the site certificate.
7. Check if there are any error messages in the SAMS log.

8. Check the CryptoInfo folder properties:
a. Right-click on the CryptoInfo folder and select Properties.
b. Click Advanced.

c. Select Encrypt contents to secure data. If you get the following error this indicates a
problem with the site certificate.
9. Renew the certificate as described in Renewing Certificates on page 171.

Ensuring Connection between SQL Server and Applications Server
Ensuring Connection between SQL Server and

Applications Server
Sometimes the FQDN of the SQL Server in the SQL certificate differs from the FQDN configured
in the Applications Server. When this is the case, the following error message appears in the log
file:
The certificate's CN name does not match the passed value.
To correct this problem, you need to make sure that the FQDN of the SQL Server and its assigned
FQDN in the System Administrator are the same.
To ensure a connection between the SQL Server and the Applications Server:
1. Ping the SQL Server from the Applications Server to get its FQDN in the Applications Server.
2. If the FQDN in the Applications Server differs from the FQDN in the SQL certificate, add the
FQDN of the SQL Server to the DNS.
The connection between the SQL Server and the Applications Server is ensured.
Installing the CA certificate on Vista Workstations

See the Microsoft Vista Workstation Configuration Guide for information regarding installing the
CA certificate on Vista workstations.

7
Channel Mapping Troubleshooting Flow
This section provides quick solutions for common problems you face when defining Channel
Mapping. The procedures in this section are designed for the seasoned user who is familiar with
the NICE Perform system and the System Administrator application. The procedures in this
section are designed to be directly solve the problem at hand. For complete instructions on using
the System Administrator, see the System Administrator’s Guide.
Contents
Channel Mapping Troubleshooting Workflow ..........................................................179

Gathering Information ..............................................................................................179
Identifying a Channel Mapping Problem ..................................................................179
Verifying the Channel Mapping Configuration..........................................................181
Troubleshooting an Out of Memory Error.................................................................183
Troubleshooting Two Users Updating Channel Mapping Concurrently....................184
Troubleshooting Saving in a Multi Data Hub Environment.......................................184
Adding a New Agent ................................................................................................184
Open Case Procedures ................................................................................................186
Open Case Procedures for System Administrator Parameters (CLS) .....................186
Open Case Procedures for the RCM .......................................................................187
Troubleshooting Scenarios .........................................................................................190
Error Message appears when Updating Configuration ............................................190
A New or Modified Mapping Definition Does Not Work............................................191
New Channel Mapping Definitions are not Saved....................................................191
Channel Mapping does not Work! ............................................................................191
Channel Mapping in System Administrator Differs from what is in Effect ................192
Channels Definitions: Logger does not Appear........................................................192
Channels Definitions: Input Channels do not Appear ..............................................193
Source Definitions: Cannot define Virtual Extensions ..............................................193
Source Definitions: Physical Switch does not Appear..............................................193
Chapter 7: Channel Mapping Troubleshooting Flow 177

Dynamic Mapping: Screen Logger does not Appear ...............................................194
Dynamic Mapping: Virtual Extensions......................................................................194
Deleting or Changing Logger, Switch, or CLS Definitions........................................194

Channel Mapping Troubleshooting Workflow

If you experience a problem with Channel Mapping follow the steps below to isolate the problem.
• Gathering Information
• Identifying a Channel Mapping Problem
• Verifying the Channel Mapping Configuration
• Troubleshooting an Out of Memory Error
• Troubleshooting Two Users Updating Channel Mapping Concurrently
• Troubleshooting Saving in a Multi Data Hub Environment
• Adding a New Agent
• Other Troubleshooting Issues
• Open Case Procedures for System Administrator Parameters (CLS)
• Open Case Procedures for the RCM
Identifying a Channel Mapping Problem

Check whether the problem is a Channel Mapping problem using one of the following methods.
To identify a Channel Mapping problem:

1. Check for a Channel Mapping problem using ROD:
a. Generate a simple test call.
b. Verify that you can see the test call in the Monitor application. If the call does not appear
in the Monitor application, see the Recording Troubleshooting Flow and the Interactions
Center Installation Guide.
c. If the test call appears in Monitor, try to ROD the test call. If the test call is not recorded,
check the mapping.

For a trunk-side recording environment, add the participant trunk group and participant
trunk number columns in Business Analyzer, and verify which trunk groups and trunk
numbers were assigned to the interactions. Check the mapping for these trunks.
2. Check for a Channel Mapping problem in the logs:
a. Generate a simple test call.
b. Verify that you can see the call in the Interactions Center RCM log files. See the
Interactions Center Installation Guide
3. Check for a Channel Mapping problem by querying the database for Channel Mapping
exceptions:
a. Query the tblInteractionCatalog table to check over what period the calls occur.
b. Get all the exceptions relating to interactions:
WHERE
c. Get all exceptions relating to recordings:
WHERE
d. Check the vcExceptionDetail field in tblException.
4. Check for a Channel Mapping problem by checking for exceptions in Business Analyzer.
Select the interaction and click on the Exceptions tab in the Details pane.
No Exception Description Possible Cause Recommended Action
15 Unmapped Unmapped Problem in the voice Switch the logs to DEBUG.

voice voice recording. channel Check channels
recording. configuration. configuration. See if there is
a mapping configured for this
call. Check the RCM logs.
Check what the RCM
received in the Start request.
16 Unmapped Unmapped The recording If the allocation mode is by IP

screen screen request was address, then check if the
recording. recording. received with empty agent logged in. Check ports.
Station or IP address Verify in RCM logs that the
(depends on the start request for recording
screen allocation the agent's screen contained
mode). the screen agent's IP.

Verifying the Channel Mapping Configuration

The Channel Mapping configuration is dependent on:
• Recording Mode: Total or Interaction based.
• Recording Type: Plain, Time Selective, Dynamic Matrix, Passive VoIP or Active VoIP
• Mapping Type: Static or Dynamic
• Recording Method: Extension or Trunk
To verify the Channel Mapping configuration:

1. Check if changes were made to the site without making the necessary changes to the Channel
Mapping configuration, for example:
• Logger was reinstalled.
• Channels were changed.
• Capture type was changed.
• Changes were made to the monitored devices is System Administrator without reflecting
the changes in Channel Mapping.
• For Avaya DMCC, changes in virtual extensions were made without remapping all
sources.
• For N+1, any change in the chain, for example, an addition of a Logger to a chain,
converting all Loggers to 1+1, and so on, without reflecting the changes in Channel
Mapping.
2. For extension-side recording, verify the following:
a. Channel definition (Recording type = Plain or Time Selective, Summation Support and
Inserter Support).
b. Source definition (extensions).
c. Static Channel Mapping definition.
3. For trunk recording, if the call was recorded but with the wrong voice or silence, the trunk
may be mapped to the wrong Logger channel. Verify that the mapping matches the physical
connection to the Loggers.
4. For trunk-side recording, verify the following:
a. Channel definition (Recording type = Plain, Time Selective or Digital Matrix, Summation
Support and Inserter Support).
b. Source definition (trunks).
c. For Plain or Time Selective channels, static Channel Mapping definition.
d. For Digital Matrix channels, static Channel Mapping or dynamic Channel Mapping
definition.

5. If you are using dynamic Channel Mapping a recording may fail if there is an attempt to
record more calls simultaneously that there are assigned channels. Depending on your system
you can:
• Buy more channels.
• Ensure in your dynamic mappings that all sources are mapped to more than one Logger.
• For QA recording, try to change your rule percentage/timing.
6. Verify that all ACDs (Hunt Groups), IVRs, and Pickup Groups participating in the call are
mapped to Logger channels.
7. Check that the mapping details match the Logger configuration in the Logger summary file
(\NTLOGGER\CONFIG\summary.txt), for example, total versus interaction-based
recording, the Logger summation support definition (Summed or Co-PABX/Stereo), and so
on.
8. Use the Functional Tester to verify that the channel definitions on the Logger match the
devices that were mapped to the channels, for example, summation and in the mapping and in
the Logger Channel Mapping import file. See the High Density Logger Troubleshooting Flow
and the Channel Mapping Guide for Importing Mapping Definitions.
9. Depending on the integration, check that the mapping is according to the extension number,
MAC #, Device Unique Identifier (DUI) or IP address.
10. Verify that the mapping configuration was updated to the CLS. See the Channel Mapping
Guide.
NOTE: It may take up to 10 minutes for the RCM to apply the configuration when there
are many mappings and Logger channels.

11. Check the RCM logs to verify that the mapping configuration was successfully updated to the
Loggers:
• Check that the mappings have been applied, and that all the relevant change commands
have been sent to the Loggers.
• Search for ERROR/WARN/Exception messages during the execution of the
configuration changes. Otherwise, the only way you will find the errors is when the
mapping does not work, or specific changes are not applied, and you may lose
recordings as a result.
Troubleshooting an Out of Memory Error

An Out of Memory error appears in System Administrator while defining Channel Mapping (that
is, Internet Explorer is connected to the Applications Server). The System Administrator’s
memory consumption increases after many configuration changes.
NOTE: The out of memory error does not necessarily indicate a problem with Channel
Mapping and can also be caused by other parts of System Administrator.
To troubleshoot an Out of Memory error:

1. On the Applications Server, in Task Manager, click the Processes tab, and check the
memory usage for NICE System Administrator.
2. On the client workstation, in Task Manager, click the processes tab, and check the memory
usage for iexplore.exe.
3. Check the performance counters:
a. In the Run window enter perfmon.
b. Click Add.
c. For each counter (Processor, Memory, Private Bytes, Threads, Handles,% CPU, Page
Faults / Sec), do the following:
• From the Performance Object list, select the Object.
• If enabled, select All instances.
• Select Select counters from list.
• Select a counter and then click Add.
d. Click Close.
e. Click View Report and monitor the results and check for irregularities.
4. On the Applications Server, create a memory dump file:
a. Navigate to <NICE Perform installation folder>\NICE Applications\Tools.
b. Double-click NICE Services Configuration Manager.exe.

c. Click Dump .
d. Click Yes. The Browse For Folder window appears.

e. Select the folder that will contain the memory dump files and click OK. Memory dump
files are created.
5. On the client workstation, create a dump file using the Windows ADPlus utility. For more
information regarding ADPlus, see http://support.microsoft.com/kb/286350.
6. Restart the Applications Server.
7. Re-launch Internet Explorer.
Troubleshooting Two Users Updating Channel Mapping Concurrently

Prior to NICE Perform Release 3.1, in the unlikely event of two users simultaneously updating the
Channel Mapping configuration, the changes saved by the second user did not take into account
the first user’s changes, resulting in a potential conflict.
In NICE Perform Release 3.1, a message appears informing the second user that their changes will
be discarded. The Channel Mapping configuration will reload, and the second user may then make
the required Channel Mapping configuration changes. This is called an atomic save.

In a Multi Data Hub environment, some of the information in nice_admin is replicated. If
replication is down, you will be unable to save in System Administrator.
NOTE: This affects all the branches in System Administrator, not only Channel
Mapping.
The atomic save (described in Troubleshooting Two Users Updating Channel Mapping
Concurrently on page 184) in a Multi Data Hub environment relies on the Primary Data Hub
being up.
Adding a New Agent

You may need to add a new agent.
To add a new agent:

1. If required, for TDM Loggers, connect the audio input. For further information, see the
NiceLog High Density Logger Installation and Configuration Guide.

2. In System Administrator > Channel Mapping (for further information, see the relevant
Integration Guide):
a. Define Logger channels
b. Define sources
c. Map channels to sources
d. Update the Interactions Center
3. In the Users Administrator:
a. Add the new agent. See Adding a New User in the Users Administrator Guide.
b. Add the agent to the relevant group. See Adding Group Members in the Users
Administrator Guide.
c. If required, give the user privileges. See Profile Privileges in the Users Administrator
Guide.
4. In the Rules Manager, ensure that the new agent is covered by any relevant rules, for
example, if the agent will be recorded according to a Scheduler rule, or whether the
interactions of the agent’s group will be archived according to Storage rule.
5. Make a test call:
a. Monitor the test call.
b. If the test call does not appear in the Monitor, there may be a CTI Server to Driver
problem, Driver to CLS problem or Interactions Center process problem. See the
Recording Troubleshooting Flow.
c. If the test call appears in Monitor, try to ROD the test call.
d. If the test call is not recorded, check the mapping.

If you edit the configuration files:
• Ensure that you use a plain text editor such as Notepad without any options.
• DO NOT use text wrap as this will corrupt your configuration file.
• DO NOT save the files in Unicode as this will cause the System Administrator to crash on
startup.
• In any event, any change of configuration files should be done ONLY in
consultation with NICE Customer Support!!!!

Open Case Procedures

This section tells you what information to provide if you need to send a problem to NICE
Customer Support for the following:
• Open Case Procedures for System Administrator Parameters (CLS)
• Open Case Procedures for the RCM
Open Case Procedures for System Administrator Parameters (CLS)

Data to be provided in Interactions Center Parameters (CLS Plug-ins) case reports, when
contacting Customer Support:
• Details regarding System configuration:
• Exact Interactions Center release, service pack, and installed Updates
• Any special site configuration, for example, Multi Data Hubs
• Problem description:
• Expected behavior
• Actual behavior
• Actions taken
• Is the problem related to a specific configuration (such as a specific integration or
mapping type) or does it effect all configurations?
• Does the problem reoccur after restarting the System Administrator or browser?
• Is there a work around?
• Are there also problems with additional parameters, such as Integrations parameters,
Loggers parameters, DB Server parameters.
• Provide specific examples and details
• Complete log files, in debug level from the server (located on the computer where the
Interactions Center sits, under NICE Systems > Applications> ServerBin)
• NiceApplications.CLSPlugin.log.*.txt
• A backup copy of the nice_admin database (located on the computer where the database is
installed)
• Please point to the relevant times in the logs when the problem can be seen in the logs. Quote
the lines if you can.
• Provide a screenshot of an error if one is displayed.
• Provide Performance Counters of SysAdmin process:
• Private Bytes
• Threads

• Handles
• % CPU
• Page Faults/Sec
• All performance counters under the .Net Memory Category.
• In case of Out Of Memory issues, please provide dump of SysAdmin process.
• In case of Online Configuration Update issues, please also provide full Interactions Center
Logs.
• Provide any other relevant data, for example, memory usage or screen shots.
Open Case Procedures for the RCM

Data to be provided in RCM case reports, when contacting customer support:
• Exact Interactions Center release, service pack and NICE Updates (Update Packs)
installed
• Details of all Loggers configured to each Interactions Center: version, capture type,
number of channels, recording method, GMT and local time of the Logger.
• Mapping details for each Logger attached to the Interactions Center:
• Total/Interaction Based
• Recording method of Logger channels (for example, Passive VoIP, DMCC, Nortel
DMS, Digital Matrix)
• Type of Sources and their number
• Static/Dynamic Mapping
• Connection between Logger and Interactions Center (LAN/WAN). If WAN, please
indicate Bandwidth and Latency.
• N+1 chain details (when relevant)
• Details from CTI plug-in on the drivers and the type of interfaces (for example, Avaya TSAP,
Genesys, CISCO ICM etc.).
• Site's SBC/Gateway topology (when relevant).
• Number of ScreenAgents reporting to the Interactions Center (when relevant)
• Call Rate on the Interactions Center Machine
• Work hours of agents mapped to the Interactions Center, including peak hours (for example,
hours of shifts, 24/7)

• Actual behavior
• Actions taken
• Provide specific examples and call details (for example, Interaction ID, Segment ID,
extension number, Agent number, time of the call, relevant DB Records for the call).
• If the issue occurs in specific call scenarios, describe the scenario.
• Does the issue occur in a specific type of call? (for example, only internal calls)
• Complete log files, in debug level (located on the Interactions Center computer, under NICE
Systems > Interactions Center > Log):
• Call Server.log.*.txt
• RCM.log.*.txt
• Complete log files, in debug level, of drivers attached to the Interactions Center.
• Complete log files of Loggers (including capture logs) attached to the Interactions Center.
• All RCM configuration backup files found under NICE Systems > Interactions Center >
Loggers
• Files found under NICE Systems > Interactions Center > bin
• RCM.log.Persistency.soap
• RCM.exe.config
• Provide the following database tables in CSV format (query from before the time the problem
occurred, preferably overlapping with the time in the provided logs):
• nice_interactions
• tblInteraction
• tblRecording
• tblParticipant
• tblException
• nice_admin
• Channel Mapping tables
• The following Performance Counters of the RCM.Exe process:
• Private Bytes
• Threads
• Handles
• % CPU

• Page Faults / Sec

• All Performance Counters under the .Net Memory Category.
• Provide any other relevant data, for example, the users included, memory usage, screen shots.

Troubleshooting Scenarios
Troubleshooting techniques are provided for the following issues which can occur when your
system is up and running.
Error Message appears when Updating Configuration
A New or Modified Mapping Definition Does Not Work
New Channel Mapping Definitions are not Saved
Channel Mapping does not Work!
Channel Mapping in System Administrator Differs from what is in Effect
Channels Definitions: Logger does not Appear
Channels Definitions: Input Channels do not Appear
Source Definitions: Cannot define Virtual Extensions
Source Definitions: Physical Switch does not Appear
Dynamic Mapping: Screen Logger does not Appear
Dynamic Mapping: Virtual Extensions
Deleting or Changing Logger, Switch, or CLS Definitions
Procedure for Reassigning a Logger to a Different CLS
Error Message appears when Updating Configuration
Scenario:
You click Update Configuration and receive an error message, similar to the one below, that
update was unsuccessful.
Solution 1:
1. Check the Network connection between the Applications Server and the Interactions Center.
2. Check the status of the CLS Monitor service on the Interactions Center.
3. Click Update Configuration again.
Solution 2:
Only if Solution 1 fails, do the following:


1. Restart the following:

• RCM
• TRS Service
• In Media Provider Controller Manager (MPCM - FLM) environments, restart the Call
Server
2. Click Update Configuration again.
A New or Modified Mapping Definition Does Not Work
Scenario:
You add and/or modify Channel Mapping definitions. Then you click Update Configurations.
One or more of the new or modified mapping definitions does not work.
IMPORTANT
Some configurations require that you restart the RCM to complete the update process. For a
complete list, see the Channel Mapping Guide.
Solution:
1. Stop the CLS service.

2. Delete all RCM persistency files located under the Interactions Center/Loggers folder.
3. Restart the CLS service.
New Channel Mapping Definitions are not Saved

If you click Save and your changes do not appear to be saved, the following occurred:
Another user was updating Channel Mapping definitions at the same time that you were and
clicked Save first.
What to do:
1. Refresh the screen to see the new changes.
2. Recreate your changes.
See also Channel Mapping does not Work! on page 191.
Channel Mapping does not Work!

When you define Channel Mapping and save your definitions, they are saved only in the interface.
To apply them to the system, you must update the CLS. For instructions, see the Channel Mapping
Guide.
See also New Channel Mapping Definitions are not Saved on page 191.

Channel Mapping in System Administrator Differs from what is in Effect
Scenario:
You open the System Administrator application and view the Channel Mapping definitions. They
appear to be different from what you know to be currently running.
Cause:
Changes were made to the Channel Mapping definitions in the System Administrator and saved.
However, Update Configuration was not applied, so therefore the Interactions Center was not
updated and the new Channel Mapping definitions were not applied.
Channels Definitions: Logger does not Appear
Scenario 1
Loggers in the Channels Definition branch are identified by their Logger ID, not by the Logger
name that appears in the Organization tree. Verify that you are searching for the Logger ID.
Logger IDs are defined during Logger installation.
To determine the Logger ID:

1. In the System Administrator, expand the Loggers branch and select the Logger.
2. The Logger ID appears in the General tab of the Logger definition.
Scenario 2
If the Logger is attached to more than one CLS, then it will not appear in the Channel Definition
window. If the Logger is not attached to a CLS, then attach it now.
To attach a Logger to a CLS:

1. In the System Administrator, click the Loggers branch and define the Logger.
2. Click the Channels tab to check that input channels are defined.
3. Click the CLS branch and select a CLS. In the Logger tab, select the new Logger.
4. Go back to the Channel Mapping branch and click the Channels definition branch.
5. Select the new Logger. The Update Channels Details window opens. Loggers are identified
here by the Logger ID, not by the Logger name you supplied.
6. For Screen Loggers, you cannot edit channel details. However, you must open their definition
and click Save.

Channels Definitions: Input Channels do not Appear
To display input channels:

1. In the System Administrator, expand the Loggers branch and select the Logger.
2. Click the Channels tab and move the input channels to the Configured Input Channels list.
3. Save the Logger definition.
4. Open the CLS servers branch, detach the Logger. (All mapping in the Channel Mappings
branch will be removed.)
5. Click the Channel Mapping branch and expand one of the branches. This step is required to
update the database.
6. Open the CLS servers branch and reattach the Logger.
7. Return to the Channel Mapping branch and continue.
Source Definitions: Cannot define Virtual Extensions

When working with Virtual Extensions, the Location of both the Logger and the Connection
Manager must be identified using the same method: either by IP Address or by Host Name.
Source Definitions: Physical Switch does not Appear
Scenario:
A Physical Switch is present in the CTI Integrations, but does not appear in the Sources definition
branch.
Solution:
To attach a Physical Switch:

1. Define the Physical Switch on a CTI Interface:
a. Expand the CTI Integrations branch and click the CTI Interfaces branch.
b. From the Actions menu, select New CTI Interface.
c. Define the new Interface with the Physical Switch.
2. Attach the Interface to a Connection Manager:
a. Expand the CTI Integrations branch and then expand the Connection Manager branch.
Select a Connection Manager.
b. Click the Interfaces tab and attach the Interface (physical switch).
3. Attach the CLS to the Driver.
a. Expand the CTI Integrations branch and then expand the Drivers branch. Select a Driver.
b. In the General tab, expand the Attach CLS section.

c. Attach the CLS to the Driver.

4. Attach the Driver to the Connection Manager.
a. Expand the CTI Integrations branch and then expand the Drivers branch. Select a Driver.
b. Click the Interfaces tab and select the Physical Switch/Connection Manager.
Dynamic Mapping: Screen Logger does not Appear
Scenario:
A Screen Logger in your Data Hub does not appear in the Dynamic Mapping branch.
Solution:
Screen Loggers do not need to be defined in the Sources branch. However, you must open their
definition and click Save.
To add a Screen Logger:

1. Expand the Channel Mapping branch and click the Channels definition branch.
2. Click the Screen Logger to expand its definition.
3. Click Save.
4. Return to the Dynamic Mapping branch.
Dynamic Mapping: Virtual Extensions

If you receive an error message when attaching channels pools, verify that the number of Virtual
Extensions is greater than or equal to the number of Channels in the Channels pool.
Confirm that the Location of both the Logger and the Connection Manager are identified using
the same method: either by IP Address or by Host Name.
Deleting or Changing Logger, Switch, or CLS Definitions

Channel Mappings are dependant upon the Logger, Switch, and CLS definitions made in the
System Administrator. Changing or deleting attachments from a switch to a CLS or from a CLS to
a Logger, affect the recordings once the RCM is restarted, in the following ways.
If you... Then...
Delete a Logger definition All channel definitions and mapping configurations for that
-or- Logger are deleted.
Detach a Logger from a CLS Source definitions remain.
Delete a switch definition All Source definitions and mapping configurations for that
-or- switch are deleted.
Detach a CLS from a driver Channel definitions remain.

If you... Then...
Delete a CLS definition • All mapping configurations for that CLS are deleted.
• All Channel definitions for Loggers that were attached to
the CLS are deleted.
• All Source definitions for switches to which the CLS was
attached remain.
Attach a Logger to a second The Logger’s Channel Mapping channel definitions and
CLS after the Logger’s mapping configurations are deleted.
channels have been defined
and mapped.
To update Channel Mapping definitions, see the Channel Mapping Guide.
Procedure for Reassigning a Logger to a Different CLS

If you move a Logger from one Interactions Center to another, you must change its assignment in
the System Administrator. After removing the Logger from its current CLS, you must update the
Channel Mapping configuration and wait 10 minutes for the update to be complete. Then you can
attach the Logger to the new CLS and continue. These steps are outlined in the following
procedure.
To attach a Logger to a different CLS:

1. In the System Administrator Organization tree, expand a Data Hub and select
CLS Definitions > CLS Server.
2. Select the CLS Server to which the Logger is currently attached and click the Loggers tab.
3. Move the Logger from the Attached Loggers list to the Available Loggers list. The
following message appears.
4. Click Yes. Then click Save .

5. Select CLS Definitions > Channel Mapping. Then click Update Configurations.
6. Wait 10 minutes for the CLS to be completely updated!
IMPORTANT
Wait 10 minutes before continuing with this procedure!

7. Select CLS Definitions > CLS Server. Then select the new CLS server and click the
Loggers tab.
8. Move the Logger from the Available Loggers list to the Attached Loggers list. Then click
Save .
9. Select CLS Definitions > Channel Mapping.
10. Define Channel Mapping for the Logger following the regular Channel Mapping procedures.

8
Applications Troubleshooting Flow
This section contains troubleshooting workflow for the NICE Perform Application Server,
including solving Desktop problems and troubleshooting Browser problems using Logs. A list of
the services, config and log files is also provided.
Contents
Workflow........................................................................................................................198
Checking the Basics ....................................................................................................200
Troubleshooting a Communications Failure..............................................................202
Troubleshooting Communications Failure with Importer or Reporter ....................203
Checking the Applications Server Services...............................................................205
Checking the Installation Files ....................................................................................208
Checking the Log Files ................................................................................................209
Solving Desktop Problems ..........................................................................................217
Troubleshooting Browser Problems Using Logs ......................................................237
Solving ActiveX Control Error on the Applications Server ......................................241
Optimizing Client Server Performance .......................................................................242
Optimizing Client Server Performance on Computers without Internet..................244
Solving Email Failure ...................................................................................................246
Troubleshooting Crystal Reports ................................................................................250
Using the Process Explorer Utility..............................................................................251
Using the Desktop Tester.............................................................................................252
Chapter 8: Applications Troubleshooting Flow 197

Workflow
Workflow
If you experience a problem with the Applications Server follow the steps below to isolate the
problem.


Checking the Basics
Checking the Basics

For Applications Server problems, check the following:
1. Troubleshoot a Communication Failure (see Troubleshooting a Communications Failure
on page 202)
2. Troubleshoot a Communication Failure between the Applications Server and the Importer
and/or Reporter Packages when using Language Pack (see Troubleshooting
Communications Failure with Importer or Reporter on page 203)
3. Check the Applications Server Services (see Checking the Applications Server Services
on page 205)
• Use NICE Services Configuration Manager (see Using NICE Services Configuration
Manager on page 206)
• Solve Application Services User Rights Problem (see Solving Application Services User
Rights Problem on page 207)
4. Check the Installation Files (see Checking the Installation Files on page 208)
5. Check the Logs (see Checking the Log Files on page 209)
6. Solve Desktop Problems
• Verify the .NET Framework Versions (see Verifying the .NET Framework Versions
on page 217)
• Solve Set Security Wizard Continues to Appear After Installation (see Set Security
Wizard Continues to Appear After Installation on page 218)
• Solve Set Security Wizard Hangs or Displays an Error (see Solving Set Security Wizard
Hangs or Displays an Error on page 221)
• Set Security Manually (see Set Security Manually on page 222)
• Set Security Manually Using Caspol (see Set Security Manually Using Caspol
on page 224)
• Solve Post Installation HTTP Error (see Solving Post Installation HTTP Error
on page 224)
• Verify the Applications are Not Loaded under the Internet Zone (see Verifying the
Applications are Not Loaded under the Internet Zone on page 228)
• Configure the NICE Perform as a Trusted Web Site (see Configuring NICE Perform as a
Trusted Web Site on page 228)
• Check NICE Perform Initialization Errors (see Checking NICE Perform Initialization
Errors on page 229)
• Solve Service Unavailable Error (see Solving Service Unavailable Error on page 230)
• Disable Custom Errors (see Disabling Custom Errors on page 233)
• Solve ActiveX Control Error (see Solving ActiveX Control Error on page 233)

Checking the Basics
• Troubleshoot Application Access (Power Management Issue) (see Troubleshooting

Application Access (Power Management Issue) on page 234)
7. Troubleshoot Browser Problems Using Logs
• Explorer Log (on the Client Machine) (see Explorer Log (on the Client Machine)
on page 237)
• Desktop Client Logs (on the Client Machine) (see Desktop Client Logs (on the Client
Machine) on page 237)
• IIS - WWW Log File (on the Applications Server) (see IIS - WWW Log File (on the
Applications Server) on page 238)
• IIS - HTTPERR Log File (on the Applications Server) (see IIS - HTTPERR Log File (on
the Applications Server) on page 239)
• Application Event Log File (on the Applications Server) (see Application Event Viewer
(on the Applications Server) on page 239)
8. Solve ActiveX Control Error on the Applications Server (see Solving ActiveX Control Error
on the Applications Server on page 241)
9. Solving Email Failure (see Solving Email Failure on page 246)
10. Troubleshoot Crystal Reports (see Troubleshooting Crystal Reports on page 250)
11. Optimizing Client Server Performance (see Optimizing Client Server Performance
on page 242)
12. Optimizing Client Server Performance on Computers without Internet (see Optimizing Client
Server Performance on Computers without Internet on page 244)
WARNING
Manually modifying registry settings can cause your system to malfunction. This procedure
should only be done by qualified personnel.
IMPORTANT
Back up your config files before editing them.
any options.
Support!

Troubleshooting a Communications Failure
Troubleshooting a Communications Failure

If you ping the server and it responds, but the communications failure persists, you may have a
problem with the fully qualified domain name.
To identify the problem:

• Ping the server, using its fully qualified name: <ServerName>.<Domain Name>. In order to
resolve this, the FQDN should either be resolved by the DNS (requires assistance of IT) or
added to the hosts file (C:\WINDOWS\system32\drivers\etc\hosts).

Troubleshooting Communications Failure with Importer or Reporter
Troubleshooting Communications Failure with Importer or

Reporter
NOTE: This procedure applies only to NICE Perform systems that use the Language
Pack.
If your NICE Perform system uses a Language Pack, there may be a conflict between the
Applications Server and the Importer and/or Reporter Packages features. To prevent such
conflicts, you must configure the Applications Server to apply the language settings to the default
user profile.
To configure the language settings:

1. In the Start menu, navigate to Settings > Control Panel > Regional and Language
Options.
The Regional and Language Options window appears.
2. Click the Advanced tab.
Figure 8-1 Regional and Language Options Window - Advanced Tab
3. In the Default user account settings area, select Apply all settings to the current user
account and to the default user profile.
When you select the checkbox, the Change Default User Settings window appears.

Troubleshooting Communications Failure with Importer or Reporter
Figure 8-2 Change Default User Settings Window
4. Click OK to accept the changes.

The Regional and Language Options window reappears.
5. Click OK.
NOTE: If, after changing the language settings, you open the Regional and Language
Options window again, the checkbox appears not selected. However, the language
settings have been changed and are in effect despite the fact that the checkbox is
cleared.
6. Restart your computer.

Checking the Applications Server Services

The following services run on the Applications Server:
• NICE System Administrator: The main Applications Server service that controls the login
process to NICE applications. If this service is not running, users will be unable to login.
• NICE Business Analyzer: Runs the Business Analyzer application (the main query
application for searching and playing back interactions) and the queries in the DB. There are
some sub-services under this service.
• NICE Locate Service: Relevant to SP3 and SP4 only. Runs under the NICE Business
Analyzer service, and finds the actual location of the audio in the preparation for playback
(Loggers, Storage Center, or Media Library).
• NICE Rule Engine: Reads all defined rules from the database, ensures that the rules run at the
defined times, and notifies the relevant unit that needs to run the rule (Storage Center, Audio
Analysis, and soon) with details of what to do.
• NICE Rules Manager Host: Supports the Rule Manager application where users create rules.
• NICE Monitor Server: Handles all ROD and monitor requests, and communicates with the
Interactions Center (directly to the Call Server).
• NICE Audit Trail Service: Runs the Audit Trail, logging all user and server operations and
functions to the relevant database (nice_audit).
• NICE Coaching Server: Manages all Coaching activity that is created by supervisors and
evaluators that is delivered to Agents (via "My Universe").
• NICE MyUniverse: Manages the My Universe application, and is responsible for collecting
the required information and displaying the relevant components, based on settings defined by
them, and/or their managers.
• NICE Media Server: Relevant to SP3 only. Fetches media from the Loggers and Storage
Center. See the Storage Center Troubleshooting Flow.
• NICE Playback Resource Manager: Relevant to SP3 and SP4 only. Manages the state of
each of the ICB (Instant Callback) resources, for example, ports configured for ICB, and so
on, and displays an error if no resources available. See the Playback Telephony Server
• NICE Playback Administration: Relevant to SP4 and 3.1. Builds the play list. In SP4,
replaces the NICE Media Server. In 3.1 this service incorporates the NICE Playback Resource
Manager service and NICE Locate service functionality.
• NICE Playback Streaming: Relevant to SP4 and 3.1 only. Ensures that the media is
delivered to the NICE Player (according to the configuration in the System Administrator).
• NICE Evaluations Form Server: Creates evaluation forms and the evaluations themselves,
as well as runs the Form Designer and Evaluator.
If the NICE Reporter is running on the Applications Server, the following services will also be
running:

• NICE Reporter Engine: Executes reports and allows the Reporter application to run.
• NICE Reporter Scheduler: Searches the DB for any active report schedules and, at the
scheduled times, sends a request to the Reporter Engine to run the report(s).
• Crystal Report Application Server: Must be running on the Reporter server. This is a
third-party program that NICE Perform uses as its underlying reporting technology.
NOTE:
The NICE Reporter usually runs on the Data Mart.
To solve services problems:

• Use NICE Services Configuration Manager (see Using NICE Services Configuration
Manager on page 206)
• Solve Application Services User Rights Problem (see Solving Application Services User
Rights Problem on page 207)
Using NICE Services Configuration Manager

The NICE Services Configuration Manager enables you to manage the services on the
Applications Server.
To configure the NICE Services Configuration Manager to run on startup:

1. Copy a shortcut from Applications\Tools\Nice Services Configuration Manage\NICE
Services Configuration Manager.exe to C:\Documents and
Settings\Administrator\Start Menu\Programs\Startup.
2. Right-click the Shortcut to Nice Services Configuration Manager shortcut link and select
Properties.
3. From the Run drop-down list, select Minimize.
4. Click OK.

Solving Application Services User Rights Problem

A service may not start if the user does not have Log on as a service rights.
To solve the problem:

1. To check which user is running the service, use the NICE Services Configuration Manager
or right-click My Computer > Manage > Services and Applications > Services, select
Properties and click the Log On tab.
2. Open Control Panel > Administrative Tools > Local Security Policy.
3. Expand Local Policies > User Rights Assignment > Log on as a service.
4. Verify that user used to run the service has this permission.

Checking the Installation Files
Checking the Installation Files

The files are typically installed on the D:\ partition of the Applications Server, located under
D:\Program Files\Nice Systems\Applications.
Sub-folders include:
• ServerBin: Contains binaries and DLL files for server-based applications.
• ClientBin: Contains the binaries and DLL files needed by clients.
• Assemblies: Contains COM wrappers for many applications.
• Desktop: Contains, among other things, ASPX files for the login page.
• NICE Database Suite: Contains SQL scripts used by the applications.
• NICE_Help: Contains help files for the web applications suite.
• Tools: Contains tools used by installers and NICE customer support.
• Client Side Applications: Contains all installations files for client side applications (Player,
ROD, setsecurity.exe and so on).
• KeyStorageManager: Contains the Key Storage Manager Web service assemblies and log
files.

Checking the Log Files

The configuration and log files for each application are found in either (or both) the ServerBin,
and ClientBin folder, together with the binary files.
• Set the Log Level in the Config File (see Setting the Log Level in the Config File
on page 210)
• Set the Log Level in System Administrator (Setting the Log Level in System Administrator
on page 209)
• Check the Logs (see Checking the Logs on page 214)
To set the log level in System Administrator:

Organization tree, click Organization. Then select the Technician Mode checkbox and
click Save.
2. In the Organization tree, select Applications > Configurations > General. Click the Logs
Level tab.
3. Set the Master Level and File Level to Debug.

4. After troubleshooting ensure that you reset these values.
WARNING
Failure to do so may cause system problems due to a performance overhead.

Setting the Log Level in the Config File

Each of the components on the Applications Server has config files in the following folder:
<Applications Server Installation Path>\ServerBin\.
Each config file includes a <log4net> section with several loggers:
Each Logger includes several appendices, that is, the types of log file output:
Component SA* Configuration File
Application Y NiceApplications.SystemAdministrator.Host.exe.config
Messages <logger name="NiceApplications.ApplicationMessages">
Audio Analysis Y NiceApplications.SystemAdministrator.Host.exe.config

branch (System <logger name="NiceApplications.AudioAnalysis">
Administrator)
Audio Analysis Y NiceApplications.AASearchController.Host.exe.config

Search Controller
Audit Trail Y NiceApplications.AuditTrail.Host.exe.config
Branches Y NiceApplications.SystemAdministrator.Host.exe.config
<logger name="NiceApplications.Branches">
Business NiceApplication.UnifiedQuery.host.exe.config
Analyzer
Coaching Y NiceApplications.CoachingTool.Host.exe.config
NiceApplications.CoachingTool.Host.vshost.exe.config

Encryption Plugin Y NiceApplications.SystemAdministrator.Host.exe.config

<logger name="NiceApplications.EncryptionPlugin">
Evaluation Forms Y NiceApplications.EvaluationForms.Host.exe.config
FTF Query Y NiceApplications.FTFQuery.Host.exe.config
Importer Y NiceApplications.SystemAdministrator.Host.exe.config
<logger name="NiceApplications.Importer">
Insight Manager Y NiceApplications.Investigations.Host.exe.config

(Investigations)
Interactions NiceApplications.SystemAdministrator.Host.exe.config
Server Plugin <logger name="NiceApplications.CLSPlugin">
IP Phone Tester Y IPPhoneTester.exe.config
License Manager Y NiceApplications.SystemAdministrator.Host.exe.config

<logger name="NiceApplications.LicenseManager">
Lists Editor Y NiceApplications.SystemAdministrator.Host.exe.config

<logger name="NiceApplications.ListsEditor">
Locate Service NiceApplications.UnifiedQuery.LocateService.Startup.exe.config

Relevant to SP3 and
SP4 only
Location Service NiceApplications.SystemAdministrator.Host.exe.config

<logger name="NiceApplications.LocationService">
Locator NiceApplications.UnifiedQuery.LocateService.LocateServiceHost.
exe.config
Loggers Plugin NiceApplications.SystemAdministrator.Host.exe.config

<logger name="NiceApplications.LoggersPlugin">
Media Server NiceApplications.Playback.MediaServer.Startup.exe.config

Relevant to SP3 only
Monitor NiceApplications.Monitor.Server.ServerStartup.exe.config
NiceApplications.Monitor.Server.ServerStartup.exe.config
MyUniverse Y NiceApplications.MyUniverse.Host.exe.config
NiceApplications.MyUniverse.Server.ServerStartup.exe.config
Playback NiceApplications.Playback.Administration.Startup.exe.config
Administration
Relevant to SP4 and
3.1 only

Playback NiceApplications.Playback.ResourceManager.RMServerStartup.e
Resource xe.config
Manager
Relevant to SP3 and
SP4 only
Playback NiceApplications.Playback.Streaming.Encoder.exe.config
Streaming NiceApplications.Playback.Streaming.Startup.exe.config
Relevant to SP4
and 3.1 only.
Playback branch NiceApplications.SystemAdministrator.Host.exe.config

(System <logger name="NiceApplications.PlaybackPlugin">
Administrator)
Retention Y NiceApplications.RetentionService.Host.exe.config
Service
Rule Engine Y NiceApplications.RuleEngine.Service.exe.config
Rules Manager Y NiceApplications.RulesManager.Host.exe.config
Storage Center NiceApplications.SystemAdministrator.StorageCenterPlugins.

branch (System config
Administrator)
Summary Page Y NiceApplications.SystemAdministrator.Host.exe.config

<logger name="NiceApplications.SummaryPage">
System Y NiceApplications.SystemAdministrator.Host.exe.config
Administrator <logger name="NiceApplications.SystemAdministrator">
System Y NiceApplications.SystemAdministrator.Host.exe.config
Administrator <logger name="NiceApplications.SystemAdminSNMP">
SNMP
Users Y NiceApplications.SystemAdministrator.Host.exe.config
Administrator <logger name="NiceApplications.UsersAdministrator">
IMPORTANT
Backup your config files before editing them.
any options.
Support!!!!.

To set the log level in the config file:

1. Navigate to <Applications Server Installation Path>\ServerBin\.
2. In Notepad, open the relevant .config file. See Log File Config Files (see Setting the Log
Level in System Administrator on page 209).
3. Expand the <log4net> section.
4. Expand the required logger name section.
5. Set the debug level, by changing the values on both Level and Threshold to be defined as
ALL.
6. Save and close the config file. There is no need to restart the Applications Server.
7. After troubleshooting ensure that you reset these values.
WARNING
Failure to do so may cause system problems due to performance overhead.

Checking the Logs

The server log files are found in the <Applications Server Installation Path>\ServerBin\ folder
unless specified otherwise.
Component Log File
Analytics NiceApplications.ClearSightApp.txt
Application NiceApplications.ApplicationMessages.log
Messages
Audio Analysis NiceApplications.AAPlugin.log

branch (System
Administrator)
Audio Analysis AASearchController.log

Search
Controller
Audit Trail AuditTrailLog.txt

AuditTrailDefaultLog.txt
Branches NiceApplications.SystemAdministrator.Branches.log
Business NBA.log
Analyzer NBADefault.log
Coaching CoachingHostLog.txt
Encryption NiceApplications.EncryptionPlugin.log
branch (System
Administrator)
Evaluation EvaluationFormsLog.txt
Forms
FTF Query NiceApplications.FTFOpenCalls.Log

NiceAppliactions.FTFQuery.Log
Importer NiceApplications.Importer.log
Insight NiceApplications.InvestigationsApp.txt
Manager
(Investigations)
Interactions NiceApplications.CLSPlugin.log
Center - CLS
branch (System
Administrator)
IP Phone Tester XmlAppsHost.log

XmlAppDefaultLog.txt

Component Log File
License NiceApplications.LicenseManager.log
Manager
List Editor NiceApplications.ListsEditor.log
Locate Service LocateDefaultLog.txt

Relevant to LocateLog.txt
SP3 and SP4
only.
Location NiceApplications.LocationService.log
Service
Locator Locator.log
Loggers branch LoggersPlugin.log

(System
Administrator)
Media Server MediaServer.txt

Relevant to SP3
only
Monitor MonitorLog.txt
monitorDefaultLog.txt
MyUniverse MyUniverseApp.txt
MyUniverse MyUniverseBasicPlugin.txt
Playback PlaybackAdministration.txt
Administration NiceApplications.PlaybackAdministration.SystemFramework.log
Relevant to SP4
PlaybackRM.txt (3.1 only)
and 3.1 only
LocateLog.txt (3.1 only)
Playback ResourceManagerDefaultLog.txt
Resource ResourceManagerLog.txt
Manager
Relevant to SP3
and SP4 only
Playback NiceApplications.Workflow.Log
Organizer
Playback NiceApplications.PlaybackStreaming.SystemFramework.log
Streaming PlaybackStreaming.txt
Relevant to SP4
and 3.1 only
Playback NiceApplications.PlaybackPlugin.log
branch (System
Administrator)

Component Log File
Reporter ReporterBin\ReporterManager.txt
ReporterBin\ReporterEngine.txt
ReporterBin\ReporterSchedulerLog.txt
The Reporter application also includes logs within the application itself.
The log provides information regarding the execution of reports. You can
also export it to csv format.
To view the Reporter logs:
1. In Reporter, click Administration.
2. Click Log.
3. Click Export to save the log files as a csv file.
Retention RetentionService.log
Service
Rule Engine RuleEngine.ActionQueue.txt

RuleEngine.SystemFramework.txt
RuleEngine.txt
Rules Manager NiceApplications.RulesManager.Host.exe.log
Summary Page NiceApplications.SummaryPage.log
System NiceApplications.SystemAdministrator.Host.exe.log
Administrator NiceApplications.SystemAdministrator.SystemFramework.log
System NiceApplications.SystemAdministrator.SNMP.log
Administrator
SNMP
Users Admin NiceApplications.UsersAdmin.log

Solving Desktop Problems

There are several types of Desktop problems:
• Verify the .NET Framework Versions (see Verifying the .NET Framework Versions
on page 217)
• Solve Set Security Wizard Continues to Appear After Installation (see Set Security Wizard
Continues to Appear After Installation on page 218)
• Solve Set Security Wizard Hangs or Displays an Error (Solving Set Security Wizard Hangs
or Displays an Error on page 221)
• Set Security Manually (see Set Security Manually on page 222)
• Set Security Manually Using Caspol (see Set Security Manually Using Caspol on page 224)
• Solve Post Installation HTTP Error (see Solving Post Installation HTTP Error on page 224)
• Verify the Applications are Not Loaded under the Internet Zone (see Verifying the
Applications are Not Loaded under the Internet Zone on page 228)
• Configure the NICE Perform as a Trusted Web Site (see Configuring NICE Perform as a
Trusted Web Site on page 228)
• Check NICE Perform Initialization Errors (see Checking NICE Perform Initialization
Errors on page 229)
• Solve Service Unavailable Error (see Solving Service Unavailable Error on page 230)
• Enable Custom Errors (see Disabling Custom Errors on page 233)
• Solve ActiveX Control Error (see Solving ActiveX Control Error on page 233)
• Troubleshoot Application Access (Power Management Issue) (see Troubleshooting
Application Access (Power Management Issue) on page 234)
Verifying the .NET Framework Versions

Verify that the correct version of .NET Framework is installed:
• SP 3: .NET 1.1 or .NET 2.0 or .NET 3.0 or .NET 3.5
• SP 4 and Release 3.1: .NET 2.0 or .NET 3.0 or .NET 3.5
Ensure that the following Updates are installed:
• .NET 1.1:
• .NET Framework 1.1 dotnetfx.exe, version: 1.1.4322.573
• .NET Framework 1.1 hotfix NDP1.1sp1-KB867460-X86.exe, version 1.0.871.2738
• .NET 2.0 and SP1:
• .NET Framework 2.0 dotnetfx.exe, version: 2.0.50727.42
• .NET Framework 2.0 hotfix NDP20-KB914460-X86.exe, version 1.0.36.79
• .NET 3.0 and 3.5

Set Security Wizard Continues to Appear After Installation

After installation, when opening applications in the browser the Set Security page appears even
though the SetSecurity.exe application was run.
Problem Occurs on One Workstation

If this happens on one workstation this may be because:
• .NET Framework is not installed.
• Applications are running under the internet zone
If .NET framework is installed, check if the Applications work when running Desktop Tester (this
runs the Applications without using the browser). For more information, see Using the Desktop
Tester on page 252. If the Applications work when running Desktop Tester but do not work in the
browser, possible causes include:
• SetSecurity.exe was run with the wrong URL
• There is a problem with the browser download cache not been refreshed.
• The browser is running with .NET 1.1. (from Release 3.1)
To solve the problem for one workstation:

1. If you see the following error when running SetSeurity.exe, install the .NET framework by
typing http://ServerName/Nice in Internet Explorer. You will be prompted to download
.NET.

2. Check if the Applications are running under the internet zone. See Verifying the Applications
are Not Loaded under the Internet Zone on page 228.
3. On the Workstation, run SetSecurity.exe with the correct URL. Verify that the URL you
entered is the same one used in the browser.
4. Clear the browser download cache:
a. Close all open Internet Explorer Browser windows. If these are left open, an error may
occur when clearing the cache.
b. Right-click on the Internet Explorer icon, click Properties (in Internet Explorer 7.0
click Internet Properties and then click Delete), click Delete Files and then click OK.
-or-
Open the Run window and run gacutil.exe -cdl.
c. Verify that Download cache deleted successfully appears. If an error occurs, ensure
that all browser windows are closed, and repeat Step b.
TIP:
gacutil.exe can be found in either of the following locations:
C:\WINDOWS\Microsoft.NET\Framework\<.NET version>
C:\Program Files\NICE Systems\Applications\Tools
You can drag-and-drop the file into the Run window.
d. In Internet Explorer enter the URL again. Verify that you are entering the server name
correctly. (URL format is: http://ServerName/Nice).
5. Verify/set the security manually. See Set Security Manually on page 222 or Set Security
Manually Using Caspol on page 224.
6. From Release 3.1:
a. Verify that .NET 2.0 is installed on the machine.

b. Open C:\Program Files\Internet Explorer\iexplore.exe.config and check that the

<supportedRuntime version="v2.0.50727" /> is not commented out.
<configuration>
<startup>
<!--<supportedRuntime version="v2.0.50727" />
<supportedRuntime version="v1.1.4322" />
</startup>
</configuration>
NOTE:
The iexplore.exe.config file typically does not exist. This is not a problem.
c. Either update the file and remove the commenting, or delete the file.
Problem Occurs on All Workstations

If this happens on all workstations, the .customconfig Mime Type is not defined, that is, the
installation failed to define it.
To solve the problem for all workstations:

1. On the Applications Server, from the command line run inetmgr.
2. Right-click on the local computer and select Properties.
3. Click MIME Types.
4. Click New.
5. Create a new MIME type with:
a. Extension: .customconfig
b. Mime type: .customconfig (in 2008 = customconfig/customconfig)

6. Click OK.
7. Click OK.
8. No restart is required.
Solving Set Security Wizard Hangs or Displays an Error

When running the SetSecurityApp.exe application, it hangs or displays a Policy level Enterprise
cannot be saved error message. The user logged in to Windows does not have the necessary
permissions required to run the SetSecurityApp.exe application, that is, the user who is logged in
to the machine belongs to the Users group in Windows.

1. Log in to Windows as an Administrator user, that is, a user who belongs to the
Administrators group in Windows.
2. Run the SetSecurityApp.exe application.

Set Security Manually

The Set Security utility sets all NICE Perform downloaded assemblies to receive full trust. When
you are unable to set security automatically, you can set it manually by following the procedures
below.
To set security manually:

1. From the Start menu, select Settings > Control Panel > Administrative Tools >
Microsoft .NET Framework Configuration.
IMPORTANT
If there is more than one version of .NET Framework installed on your computer, be careful to
select the relevant .NET Framework version.
2. In the tree, navigate to the following branch: Runtime Security Policy > Machine > Code
Groups > All_Code > NICE Perform Code Group.
3. Right-click NICE Perform Code Group and click New.
4. In the Name field, enter a relevant name.

5. Click Next.
6. From the Choose the condition type for this code group drop-down list, select Site.
7. In the Site name field, enter a relevant name.
8. Click Next.
9. Leave the Use existing permission set drop-down list as FullTrust.

10. Click Next.

11. Click Finish.
Set Security Manually Using Caspol

An additional option for configuring the security setting is to run a script that uses Caspol.exe
(Code Access Security Policy). Ensure that you use the version of Caspol that matches your
version of .NET.
To set security manually using Caspol:

• Create the following script and run it in the command window:
C:\WINDOWS\Microsoft.NET\Framework\<.NET Version>\Caspol.exe -machine -pp
off -addgroup "All Code" -site <ComputerName>FullTrust -name "NICE CEM Code
Group" -d "Allow full trust privileges to NICE CEM applications",
replacing <.NET Version> with the version of .NET that is installed on the client desktop, and
the <ComputerName> with the NICE Perform server name in the format that is used to access
it (Host Name/ IP /FQDN).
NOTE:
If there is more than one version of .NET Framework is installed on your computer, run
this script for each version.
Solving Post Installation HTTP Error

When trying to open application after the installation, a Server Error in
'NiceApplication/Desktop/WebPage' Application HTTP error appears. Possible causes:
• ASP.NET is not installed/allowed on the machine

• The machine is not configured to work with ASP.NET 2.0 (from NICE Perform Release 3.1)

1. On the Workstation, check that ASP.NET 2.0 is installed by:
a. From the Start Menu, select Settings > Control Panel > Add Remove Programs.
b. Click Add/Remove Windows Components.
c. Double-click Application Server.

d. Check that the ASP.Net subcomponent is selected.
e. Click OK.
f. Click Next.
2. From NICE Perform Release 3.1, verify that ASP.NET 2.0 is configured on the IIS:
a. Select Start > Settings > Control Panel>Administrative Tools>Internet
Information Services (IIS) Manager.
b. In the tree, right-click Web Sites > Default Web Site, and select Properties.
c. Click the ASP.NET tab.

d. Ensure that the ASP.NET version is set to 2.0.50727. If not, select 2.0.50727.
e. Click OK.
f. If the ASP.NET tab is missing from the properties window, or the ASP.NET version
drop-down list does not include version 2.0.50727, the ASP.NET for .NET
Framework 2.0 is not configured. In this case, register ASP.NET 2.0 on the IIS:
• Open the command window (Select Start > Run… then enter cmd in the Open field
and click OK).
• In the command window, navigate to the .Net framework 2.0 directory, located
under windows installation dir in the following path:
C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727
• Type aspnet_regiis.exe -i and press <Enter>. The system will install the
ASP.NET 2.0 and register it on the IIS.

• Wait until the command finishes, go back to the IIS manager and make sure that the
ASP.NET tab is added and version 2.0 has been selected on the combo box (as
mentioned above).
Verifying the Applications are Not Loaded under the Internet Zone
Verify that the NICE Perform application is not loaded under the Internet zone on your Internet
Explorer window.
To verify the applications are not loaded under the Internet zone:
1. Log in to NICE Perform.
2. Verify that Local Intranet or Trusted appears in the right side of the status bar (and not
Internet).
Configuring NICE Perform as a Trusted Web Site

It is recommended to define your NICE Perform Applications Server as a trusted site.

1. In Internet Explorer, from the Tools menu, select Internet Options.
2. Click the Security tab.
3. Select Trusted Sites and click Sites.
4. In the Add this Web site to the zone field, enter your server name in the following format:
http://ServerName
5. Click Add.
6. Clear the Require Server verification checkbox.
7. Click OK.

8. Restart your computer, open Internet Explorer window, and enter the URL again. Verify that
you are entering the server name correctly. (URL format is: http://Server Name/Nice).
Checking NICE Perform Initialization Errors

A NICE Perform initialization error appears when accessing NICE Perform. Possible problems:
• Solving Applications Server Configuration Problem (see Solving Applications Server
Configuration Problem on page 229)
• Troubleshooting Remote Name Could Not be Resolved Error (see Troubleshooting Remote
Name Could Not be Resolved Error on page 230)
• Troubleshooting Unable to Connect to the Remote Server Error (see Troubleshooting Unable
to Connect to the Remote Server Error on page 230)
• Troubleshooting Serialization Problem (see Troubleshooting Serialization Problem
on page 230)
Solving Applications Server Configuration Problem

If there is a problem accessing NICE Perform from the workstations, but the problem does not
occur when accessing NICE Perform from the Applications Server, the Applications Server host
name may not have been configured correctly in the System Administrator.

Organization tree, click Organization. Then select the Technician Mode checkbox and
click Save.
2. In the Organization tree, select Applications > Applications Server, and select the relevant
Applications Server.
3. Verify that the ServerName appears for each Host Name/IP Address.

Troubleshooting Remote Name Could Not be Resolved Error

If a <Remote Name> Could Not be Resolved Error occurs when you are accessing the URL
with a hostname/IP address, this may be a DNS problem. Remoting uses DNS names for
communication and there is a problem resolving the DNS name mentioned in the error.

• Check for a communication failure. See Troubleshooting a Communications Failure
on page 202.
Troubleshooting Unable to Connect to the Remote Server Error

The problem occurs when the NICE System Administrator service is either stopped or has failed to
initialize.

• Verify that the NICE System Administrator service is running. If the service fails to start,
check the system admin log file (NiceApplications.SystemAdministrator.Host.exe.log) for
errors.
Troubleshooting Serialization Problem

The problem occurs when there is a problem with a Microsoft Windows update installed on the
machine.

1. Check for a .Net Framework HF problem. See Verifying the .NET Framework Versions
on page 217.
2. Check for Windows Update problems. See the latest NICE Technical Note on Microsoft
Security Patches (issued monthly).
Solving Service Unavailable Error

When trying to open application a Service Unavailable error appears. Possible problems:
• IIS AppPool Problem (see IIS AppPool Problem on page 231)

• Non-Paged Pool Memory Leak Problem (see Non-Paged Pool Memory Leak Problem
on page 233)
IIS AppPool Problem

The problem occurs when there is a problem with the IIS AppPool.

1. Identify the problem:
a. In the command line, enter inetmgr to open IIS Manager.
b. Verify that DefaultAppPool is running.
c. In the command line, enter eventvwr to open Event Viewer.
d. Identify logon issues by looking for W3SVC Warnings under System. This may occur
when the AppPool identity is set to a specific user name, and the password for this user
was updated in Windows, without been updated in the AppPool. Continue with Step 3 to
resolve the problem.
e. Look for W3SVC Errors. Unhandled exceptions from an application running under the
pool can cause it to lock. Continue with Step 4.
2. Start the DefaultAppPool:
a. Start Internet Information Services (IIS) Manager.
b. Expand Internet Information Services, expand <Server Name>, expand Application
Pools, and then expand DefaultAppPool.
c. Right-click DefaultAppPool, and then click Start.
3. Update the DefaultAppPool logon credentials:
a. Right-click DefaultAppPool, and select Properties.

b. Click the Identity tab.
c. In the Configurable area, update the logon credentials.

d. Click OK.
e. Right-click DefaultAppPool, and then click Stop.
f. Right-click DefaultAppPool, and then click Start.
4. Modify the DefaultAppPool health:
a. Right-click DefaultAppPool, and select Properties.
b. Click the Health tab.
c. Clear Enable rapid-fail protection (this should only be used for an interim period until
the source of the problem is identified).

Non-Paged Pool Memory Leak Problem

A non-paged pool memory leak can cause a connection to the service to be denied.
To identify the problem:

1. Check the errors in the IIS HTTPERR log file to identify if the denial of service was caused by
a memory leak (See IIS - HTTPERR Log File (on the Applications Server) on page 239).
2. Open Task Manager.
3. Click the Processes tab.
4. In the View menu, select Select Columns.
5. Select Non-paged Pool.
6. Click OK.
7. Click on NP Pool to sort by non-paged pool.
8. Look for unusually high values. This indicates a leak.
9. Identify the application causing the memory leak and take appropriate action.
Disabling Custom Errors

Custom errors should be not be displayed by default. If custom errors are switched on a Server
encountered an internal error appears.

1. In Notepad, open the relevant service .config file (according to the exceptions source). See
Setting the Log Level in System Administrator on page 209.
2. Search for the following:
<system.runtime.remoting>
<customErrors mode="On" />
</system.runtime.remoting>
3. Change the value to Off.
4. Recreate the problem and check the error returned.
Solving ActiveX Control Error

If you get an ActiveX Control error on the Desktop, set the appropriate options ActiveX options.
For details on how to set the ActiveX Control options for the Applications Server, see Solving
ActiveX Control Error on the Applications Server on page 241.



3. Select Local Intranet and click Custom Level.
4. Scroll down to ActiveX controls and plug-ins and select options as follows:
• Download signed ActiveX Controls - Prompt
• Script ActiveX controls marked safe for scripting - Enable
5. Click OK.
Troubleshooting Application Access (Power Management Issue)

If the Network Interface Card (NIC) of the Applications Server is set to power save mode, it can
turn off the power on your Applications Server network card, prohibiting workstations from
accessing the server. To avoid this situation, the power save mode on the NIC of the Applications
Server must be turned off. Depending on your NIC, power save mode can be located in two
different places. You must check both locations. If power save mode appears in both locations, you
must turn it off twice!
To turn off Power save mode:

1. On the Applications Server, select Start > Settings > Network and Dial-up Connections.
2. Double-click the Local Area Connection icon.

3. Click Properties.
4. In the General tab, click Configure, and then click the Power Management tab.
5. Click the Power Management tab and clear all checkboxes. (This is the first location to
check. Some NICs might not have this tab.)

6. Click the Advanced tab. Browse through the Property list and determine if there is a property
for power management. Disable it. (This is the second location to check. Some NICs do not
have a property for power management listed here.)
7. Click OK.

Troubleshooting Browser Problems Using Logs

You can troubleshoot browser problems using various log files:
• Explorer Log (on the Client Machine).
• Desktop Client Logs (on the Client Machine)
• IIS - WWW Log File (on the Applications Server)
• Application Event Log File. See Application Event Viewer (on the Applications Server)
Explorer Log (on the Client Machine)

The Explorer log is on the client machine and requires a registry key to be updated. To
troubleshoot client downloading and execution issues, enable the Internet Explorer logging
mechanism.
To enable smart client downloading and security logging:

1. Open the Registry Editor. (Select Start > Run. In the Run window, enter regedit and click
OK.)
2. Locate and select the following registry key:
HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework
3. Select Edit > New > DWORD Value.
4. Name the new DWORD DebugIEHost. Assign it any non-zero value.
5. Select Edit > New > String Value.
6. Name the new String IEHostLogFile. Assign it the full path (including the file name) of the
file in which you want to record the debug trace (for example, C:\Temp\IEDebug.log)
7. Open the browser and enter the requested NICE Perform URL.
8. After the Set Security page appears (or any other error page which is not the login page), check
the browser log file.
Desktop Client Logs (on the Client Machine)
To view the desktop client logs:

• View the Desktop Client logs at C:\ Documents and Settings\[User]\ Local
Settings\Application Data\NICE.
NOTE:
This path may contain hidden folders.

Component Log File
Coaching C:\Documents and Settings\[user]\Application

Data\NICE\CoachingPackageClient.txt and CoachingNugget.txt
This file contains errors and info. The debug level cannot be changed for the
client log file.
Evaluation C:\Documents and Settings\[user]\Application

Forms Data\NICE\EvalFormsDesignerLog.txt
The debug level cannot be changed for the client log file.
MyUniverse C:\Documents and Settings\[user]\Application

Data\NICE\MyUniverseClient.log.
MyUniverse client logs are only written if the Desktop Tester is activated.
The debug level cannot be changed for the client log file.
System C:\Documents and Settings\[user]\Application

Administrator Data\NICE\SystemAdminClientLog.txt, LoginUtilsLog.txt and
Autolock.log.
This file contains errors only. The debug level cannot be changed for the
client log file.
User Admin UsersAdminClient.log
Desktop DesktopLog.txt
General ClientComponents.txt, ClientMessages.log
IIS - WWW Log File (on the Applications Server)

The Internet Information Services (IIS) World Wide Web Publishing Service (W3SVC),
sometimes referred to as the WWW Service, manages the HTTP protocol and HTTP performance
counters.
This log file records the W3SVC errors, for example, crashes, recycle, ping failures, and so on.
To view the IIS www logs:

• View the IIS www logs at C:\WINDOWS\system32\LogFiles\W3SVC1.
To verify the log location:

1. On the Applications Server, open Internet Information Services Manager (from the command
line run inetmgr or from the Start menu, select Settings > Control Panel > Administrative
Tools > Internet Information Services Manager).

2. Right-click on the Default Web Site, select Properties and click the Web Site tab.
3. Verify that logging is enabled, by checking that Enable logging is selected.

4. To check the log file location, click Properties.
5. Under Log file directory, check the directory where log files are saved.
IIS - HTTPERR Log File (on the Applications Server)

This file enables you to diagnose why requests are rejected by HTTP.sys. Each entry includes URL
and a brief description of the error. A list of the most common errors follows:
• Connection_Dropped
• AppOffLine
• BadRequest
• QueueFull
• Connection_Abandoned_By_AppPool
To view the IIS HTTPERR logs:

• View the IIS HTTPERR logs at C:\WINDOWS\system32\LogFiles\HTTPERR.
Application Event Viewer (on the Applications Server)

Check the Applications Server event logs using the Event Viewer.
To check the event log:

1. In the command line, enter eventvwr to open Event Viewer
-or-
Right-click My Computer, click Manage, expand System Tools > Event Viewer.

-or-
Select Start > Settings > Control Panel > Administrative Tools > Event Viewer.
2. Check for the following types of errors:
• W3SVC worker process errors, for example, ASP.NET reports an unhealthy status
Deadlock detected.
• .NET runtime errors, for example, FatalExecutionEngineException.
• ASP.NET errors, for example, side-by-side not supported within single AppPool and
ASP.NET state server communication errors.
• To check for a problem with the IIS AppPool, identify logon issues by looking for
W3SVC Warnings under System .

Solving ActiveX Control Error on the Applications Server
Solving ActiveX Control Error on the Applications Server

If you get an ActiveX Control error on the Applications Server, set the appropriate options
ActiveX options. For details on how to set the ActiveX Control options for the Desktop, see
Solving ActiveX Control Error on the Applications Server.

3. Select Local Intranet and click Custom Level.
4. Scroll down to ActiveX controls and plug-ins and select options as follows:
• Download signed ActiveX Controls - Prompt
• Run ActiveX Controls and Plug-ins - Enable
5. Click OK.
To solve the problem using the Registry:
WARNING
Manually modifying registry settings can cause your system to malfunction. This procedure
should only be done by qualified personnel.
1. Open a registry editor and navigate to the following key:

HKEY_CURRENT_USER\Software\Microsot\Windows\CurrentVersion\Internet
Settings\Zones\1
(Zone 1 is the Local Intranet Zone.)
2. Verify the settings as follows:
Setting Value
Download signed ActiveX Controls Key 1001 = 1
Run ActiveX Controls and Plug-ins Key 1200 = 0
3. Click OK.

Optimizing Client Server Performance

You can increase the speed at which the pages of the NICE Perform Applications Suite appear in
your Internet Explorer window.
To optimize client server:

1. In your Internet Explorer, navigate to Tools > Internet Options.
2. On the General tab, in the Temporary Internet files area, click Settings.
3. In the Settings window:

• In the Check for newer versions of stored pages area, select Automatically.

• In the Temporary Internet files folder area, in the Amount of disk space to use field,
enter 100.
4. Click OK.
6. Browse to the Security section and clear Empty Temporary Internet Files folder when
browser is closed.
7. Click OK.

Optimizing Client Server Performance on Computers without Internet
Optimizing Client Server Performance on Computers

without Internet
When opening the NICE Perform Login page without internet connection available, the client
server will not be able to reach the VeriSign server to validate the NICE Perform digital signature.
In this event, there will be a failure timeout. This failure may take several minutes.
In order to work around this failure and to reduce failure timeout to a minimum, you can modify
the security settings. You can do this by performing one of the following:
• Redirecting the Client Request to the Local Server
• Modifying the Security Settings in the Internet Explorer

Although both options have a security impact, redirecting the client request to the local server has
less of a security impact as only validation of VeriSign certificates is skipped. Other certificates are
validated.
Redirecting the Client Request to the Local Server

You can optimize the client server performance in computers without internet connection by
redirecting the client request to the local server in one of two ways:
• By modifying the DNS Server
• By modifying the hosts file
To redirect the client request to the local server:

• Modify the DNS Server or hosts file to include the following IP addresses and server names.
IP Address Server Name
127.0.0.1 crl.verisign.com
127.0.0.1 CSC3-2004-
crl.verisign.com
NOTE: The server names may change if VeriSign updates their server names.

Optimizing Client Server Performance on Computers without Internet
Modifying the Security Settings in the Internet Explorer

You can optimize the client server performance in computers without internet connection by
modifying the security section in the Internet Explorer.
To modify the security section in the Internet Explorer:

1. In your Internet Explorer, navigate to Tools > Internet Options.
3. Browse to the Security section and clear Check for publisher's certificate revocation.
NOTE:
Clearing Check for publisher's certificate revocation has a security impact as the
revocation phase is skipped and is not recommended.
4. Click OK to close the Internet Options window.

Solving Email Failure
To solve an email problem:

• Verify the SMTP Service Installation and Configuration
• Troubleshoot SMTP
Verifying the SMTP Service Installation and Configuration

If emails are not sent, you need to verify that the SMTP service is installed, configured correctly,
and running on the NICE Perform Applications Server.

1. Verify that the SMTP service is installed on the NICE Perform Applications Server.
2. From the Start menu, select Run. In the Run window, enter inetmgr.
3. Verify that the SMTP service is running.
4. Verify that the SMTP service is configured properly:

a. Right-click Default SMTP Virtual Server and select Properties.
b. In the General tab, in the IP address drop-down list, confirm that (All Unassigned) is
selected.
c. Click the Access tab. Then click Relay and select All except the list below.
d. Click OK.

e. Click the Delivery tab. Then click Advanced.
f. Verify that:
• The Fully-qualified domain name is defined correctly.
• The Smart host text field displays the name of the SMTP server of your organization,
in the format MyExchange.Domain.com.
g. Click OK.
Troubleshooting SMTP
Check that the SMTP logging is enabled and review the logs.
To troubleshoot SMTP:
1. In the Event Viewer, look for SMTP service messages under the Application Log and for
relevant messages under the Security Log.
2. Navigate to the following folder and verify that it does not contain any email messages:
C:\Inetpub\mailroot\Queue.
3. If there are email messages in this folder, this is due to either the SMTP configuration, or an
antivirus issue (known issue). To remedy the antivirus issue, for example, for Norton
AntiVirus:
a. From the Start menu, select Programs > Symantec Client Security > Symantec
Antivirus > Configure > Internet Email Auto Protect.

b. Clear the Enable Internet E-mail Auto-Protect checkbox.
4. From the Start menu, select Run. In the Run window, enter inetmgr.
5. Right-click Default SMTP Virtual Server and select Properties.
6. On the General tab, ensure that Enable logging is selected.

7. Click Properties.
8. Specify the settings that you want to use. You can specify settings to include log time period
and the location and the name of the log file.
9. Click OK, and then click OK.
10. Check the logs (the default location is C:\WINDOWS\System32\LogFiles\SMTPSVC\).

Troubleshooting Crystal Reports
Troubleshooting Crystal Reports

If Crystal Reports crashes you may need to update the Data Execution Prevention settings.
To troubleshoot Crystal Reports:

1. In the command line, enter eventvwr to open Event Viewer.
2. Check for errors.
3. Select My Computer > Properties.
5. In the Performance area, click Settings.
6. Click the Data Execution Prevention tab.
7. Select Turn on DEP for all programs and services except those I select.
8. Select crystalras.exe.
9. Click OK to close the Performance Options window.
10. Click OK to close the System Properties window.

Using the Process Explorer Utility
Using the Process Explorer Utility

To enable diagnosis of performance issues, the NICE Perform installation includes the Process
Explore utility (located in the Tools directory).
This utility displays the full image path, command line, environment variables, parent process,
security access token, opened handles, loaded DLLs, and mapped files.
To run the Process Explorer:

• Double-click procexp.exe.

Using the Desktop Tester
Using the Desktop Tester

The Desktop Tester is used to run the Applications without the browser, enabling you to narrow
down a troubleshooting problem to a browser/proxy issue.
To use the Desktop Tester:

1. (Optional) To use the Desktop Tester on a computer other than the Applications Server:
a. Copy the <Applications Server Installation Path>\ClientBin\ folder from the
Applications Server to the computer.
b. Open DeskTopTester.exe.config using Notepad.
c. Modify the TesterPort to <HostName>:<SysAdminPort(default 62070)>, for example,
Teddy:62070.
d. Save the config file.
2. Double-click DeskTopTester.exe. Verify that the Applications run.

9
Databases Troubleshooting Flow
This chapter describes some of the most common problems that may arise during and after
installing the NICE Perform databases, and offers probable solutions.
IMPORTANT
These procedures must be perform only by authorized personnel. For full assistance, contact
NICE Systems Customer Services.
Contents
General Databases Troubleshooting Flow .................................................................255

General Considerations ...............................................................................................256
Checking the Windows Event Viewer .........................................................................256
Checking the SQL Server Error Logs and Events .....................................................256
Checking SQL Jobs......................................................................................................257
Checking the Connection to the NICE Perform Database Server ............................257
Checking the Database Connections .........................................................................257
Checking for Missing Databases ................................................................................258
Checking Free Space in the Database ........................................................................259
Verifying the Log on as a Service User Rights Assignment.....................................260
Checking the SQL Server Installation Folder.............................................................263
Monitoring Performance Objects and Counters........................................................265
Checking for Performance Problems .........................................................................266
Scanning the Disk for Errors .......................................................................................267
Checking Replication in Multi Data-Hub Environments............................................268
Changing the Database Passwords in SQL Authentication .....................................269
Chapter 9: Databases Troubleshooting Flow 253

Checking the Database Backup ..................................................................................271
Increasing the Size of a Database...............................................................................273
Checking Performance Using the NICE Performance Collector ..............................274
Defining Autogrowth ....................................................................................................276
Using the NICE DBSpace Utility ..................................................................................278
Analyzing the Database Using NICE DBAgnose .......................................................284
General Data Mart Troubleshooting Flow...................................................................287
Data Mart Troubleshooting Issues ..............................................................................288

General Databases Troubleshooting Flow
General Databases Troubleshooting Flow

If you experience a problem with databases, the following workflow lists the steps to isolate the
problem. Each step in the workflow is a section in this chapter.

General Considerations
General Considerations
Before attempting to troubleshoot any database problems, it is advisable to consider the issues
listed below. These will better enable you to analyze your problems and determine if the issues are
indeed related to the database.
• Is this a new installation or an upgrade of an existing system?
• For new installations, when was the installation performed? How long after the
installation did the problems appear?
• For upgrades, when was the upgrade performed? Did the problems arise before or after the
upgrade?
• How often does the problem occur? Are the circumstances similar?
Checking the Windows Event Viewer

In is often recommended to look for errors in the Windows Event Viewer. I/O errors are the most
common when dealing with read/write errors to and from the database.
NOTE: These error messages are also displayed in the SQL Server Log files, as
described in the next section.
Checking the SQL Server Error Logs and Events

You can use the Log File Viewer to check the SQL Server Agent logs, SQL Server error logs and
Windows event logs.
To check the SQL Server Error Logs and Events:

1. Connect to the SQL Server Management Studio.
2. Navigate to your SQL Server 2005 instance.
3. Navigate to Management > SQL Server Logs.
4. Double click on any of the log files.
5. Use the Export, Search, and Filter options as required.

Checking SQL Jobs
Checking SQL Jobs

Use the SQL Server to check that jobs are configured correctly and that they run properly.
To check SQL jobs:

1. In the SQL Server Management Studio, navigate to SQL Server Agent > Jobs.
2. Verify that the job durations are reasonable (may vary from job to job).
3. Verify that numerous jobs have not been configured to run at the same time. This is especially
relevant for sites in which numerous jobs run during off-hours.
4. Verify the job outcome, as a job may be successful even if internal steps have failed.
5. Check the job history to verify if all previous jobs have run successfully. Failed jobs are
identified by an error icon .
Checking the Connection to the NICE Perform Database

Server
Check the connection to the Database Server making sure you use the name configured on the
NICE Applications Server.
To check the connection to the NICE Database Server:

1. On the Applications Server, ping the Database Server using the host name and IP address.
-or-
2. On the Applications Server:
a. Navigate to D:\Program Files\NiceSystems\Applications\ServerBin.
b. In the NiceApplications.SystemAdministrator.Host.exe.config file, search for
SystemFramework.DataAccess.ConnectionString value=<DB Server Name> .
This is the Database Server name defined in the application.
c. Ping the Database Server using this name.
Checking the Database Connections

Use the SQL Server to check the database connections listed below.
To check the database connections:

1. In the SQL Server Management Studio navigate to Management > Activity Monitor.
2. In the right-click menu select View Process.

Checking for Missing Databases
3. Check the Activity Monitor as follows:

a. Scroll to the Blocked By and Blocking columns. Any non-zero value in these two
columns indicates an error (a deadlock).
b. Each row in this list represents a database connection. Verify that no database has
excessive connections.
4. Filter the Activity Monitor for blocked processes as follows:
a. Click the Filter button.
b. Specify a filter with Blocking Type = Blocked.
c. Click Apply filter, and then click OK.
Checking for Missing Databases

You can check for missing databases using either the System Administrator or the SQL Server.
To check for missing databases using the System Administrator:
NOTE: It is recommended not to work in Technical Mode in order to prevent making

unintentional changes to the database configuration.
1. In the System Administrator, navigate to Master Site > Database Servers > Databases and
verify that all five mandatory databases are selected:
• nice_admin
• nice_audit
• nice_ca
• nice_rule
2. In addition, verify that all the optional databases that are supposed to be installed at your site
are also selected.
To check for missing databases using the SQL Server:

1. Connect to the SQL Management Studio.
2. In the Object Explorer, navigate to Databases and verify that the mandatory and optional
databases are listed and that the tables for each one contains data.

Checking Free Space in the Database
Checking Free Space in the Database

There are several ways to check the free space in the NICE Perform databases.
To check free database space:

1. Check the NMS alarms.
-or-
a. Navigate to Databases and select the relevant database.
b. From the right-click menu select Properties.
c. In the General page check Space available.
NOTE: The space available relates to the free space of the entire database, and does
not distinguish between the sizes of the Data files and the Log files.
-or-
3. Right-click the database and select Reports > Standard reports > Disk usage. The usage is
displayed graphically and indicates both the Data files and the Log files separately.
-or-
4. Use the Nice DBSpace utility to check the database space (see Using the NICE DBSpace
Utility on page 278).
-or-
5. In the NICE System Administrator, navigate to Master Site > Database Servers >
Databases and select each database to see the used and free space percentages.
NOTE: It is recommended not to work in Technical Mode in order to prevent making

unintentional changes to the database configuration.

Verifying the Log on as a Service User Rights Assignment
Figure 9-1 NICE System Administrator
-or-
6. Use Nice DBSpace to check the database space.

If a service does not start, make sure your user has the Log on as a service User Right
Assignment.
There are two steps:
• Verifying the Service Account (see page 260)
• Verifying Domain Users Assigned to the User Rights Assignments (see page 261)
Verifying the Service Account
To verify the service account:

1. In the Computer Management window, open the Services window, and select the service that
you cannot access.
2. From the right-click menu select Properties.
The Properties window appears.
3. Click the Log On tab.

Figure 9-2 MSSQLSERVER Properties Window
4. The domain account and the domain user appear in the This Account field.
Verifying Domain Users Assigned to the User Rights Assignments
To verify domain users assigned to the User Rights Assignments:

1. In the Control Panel navigate to Administrative Tools > Local Security Policy > User
Right Assignments, and select Log on as a service.

Figure 9-3 Local Security Settings Window
2. From the right-click menu select Properties.

The Log on as a service Properties window appears.

Checking the SQL Server Installation Folder
Figure 9-4 Log on as a service Properties Window
Domain User for NICE Services
Domain User for SQL Services
3. All domain users defined with Log on as a service policy are listed. Verify that the domain
user that appears in Figure 9-2 appears in the list.

If there are SQL I/O problems, verify if the SQL software may be installed on a compressed folder.
To check the SQL Server installation folder:

1. Run the NICE DBAgnose utility. Verify that OK appears in the status column of the
Compressed Files row, as in Figure 9-5.

Figure 9-5 NICE DBAgnose
-or-
2. In Windows Explorer:
a. Navigate to the SQL Server installation folder.
b. In the right-click menu, select Properties.
c. Click the General tab.
d. Click Advanced.
Figure 9-6 Advanced Attributes Window
e. Ensure that Compress contents to save disk space is not selected.

Monitoring Performance Objects and Counters
Monitoring Performance Objects and Counters

You can use the Performance Monitor to check the activity on the SQL Server.
To monitor performance objects and counters:

1. In the Run window enter perfmon and click OK.
The Performance window appears.
2. Click the Add button .

3. For each object listed in the table below, do the following:
Object Counters
Processor %Processor Time
Memory Pages/sec
PhysicalDisk %Disk Time
Avg Disk Queue Length
Avg Disk Write Queue Length
Avg Disk Read Queue Length
Disk Read/sec
Disk Write/sec
SQL Server:BufferManager Buffer catch hit ratio
Page life expectancy
SQL Server:MemoryManager SQL Cache Memory
Total Server Memory
a. From the Performance Object drop-down list, select the object listed in the table.
b. Select Select counters from list and select the counter corresponding to that object in
the table above.
c. Select All instances if it is enabled.
d. Click Add.
The object you selected is added to the Performance chart.
4. Click Close.
5. Click the View Report button .

6. Check the results for irregularities.

Checking for Performance Problems
Checking for Performance Problems

You can check for performance problems in several ways.
To check for performance problems:

1. Use SQL Profiler to run a trace:
a. In the Start menu, select Programs > Microsoft SQL Server 2005 > Performance
Tools > SQL Server Profiler.
b. In the File menu, click New Trace.
c. Connect to the computer on which the SQL Server is installed.
The Trace Properties dialog box appears.
d. In the Use The Template name drop-down list, select TSQL.
e. Click the Events Selection tab.
f. Select Show all events and Show all columns.
g. Scroll down to the TSQL.
h. Ensure that the following events are selected:
• SQL:BatchStarting
• SQL:StmtCompleted.
• SQL:StmtStarting
i. For each of these event classes, click Column Filters.
The Edit Filter window appears.
Figure 9-7 Edit Filter Window
j. Select the Duration filter.

Scanning the Disk for Errors
k. Select Greater than or equal to and enter a value. A value of 1000 milliseconds, for
example, would cause the filter to show only those queries that take longer than one
second to run.
l. Remain in the Events Selection tab of the Trace Properties window, and scroll up to
Security Audit.
m. Clear Audit Login and Audit Logout.
n. Click Run to run the trace.
The SQL Server Profiler enables you to view the trace data. Only information satisfying
the filter is captured.
-or-
2. To check consistency and integrity for different tables and databases, in the SQL Query
Analyzer, run the following commands:
a. To check database consistency, run the following query using the following syntax:
dbcc checkdb (database name).
b. To check the tables, run the following query using the following syntax:
dbcc checktable (table name).
c. To check table index fragmentation, run the following query using the following syntax:
dbcc showcontig (<table name>) with all_indexes.
Verify that the value for Scan Density [Best Count:Actual Count] is greater than 60%.
-or-
3. Use Nice Performance Collector to check the performance. See Checking Performance
Using the NICE Performance Collector on page 274.
Scanning the Disk for Errors
To scan the disk for errors:

1. On the SQL Server, run a disk scan on the drives where the data and log files are located.
2. Select the drive, and from the right-click menu select Properties.
3. Click the Tools tab, and then click Check Now.

Checking Replication in Multi Data-Hub Environments
Checking Replication in Multi Data-Hub Environments

Use the following methods to check replication in Multi Data Hub environments.
NOTE:
• If you want to make any changes to the database schema of replicated or
un-replicated tables, you must first shut down the replication for that site.
• When running the replication, the created scripts and log files are saved in
Documents and Settings\[System Admin User]\Application Data.
To check replication in multi data hub environments:

1. In the NICE System Administrator, verify that the replication is defined using only the
machine names.
-or-
2. Check the connectivity between the Primary and Secondary Data Hubs.
-or-
3. Connect to the SQL Server Management Studio. In the Object Explorer, navigate to
Replication and check if there are components with an error.
-or-
4. In the NICE System Administrator:
a. Navigate to Master Site > Database Servers > Databases and click the Replication
tab.
b. Verify that no errors appear in the Status column.
-or-
5. If the Primary and Secondary Data Hubs have different database structures, the replication will
fail at those specific points. To ensure that the replicated data is identical, run tests on all
data hubs to ensure that additional information (business data, any external additions such as
PS solutions) can be accessed from different sites.
-or-
6. Verify that the same version of the NICE Perform Database and the same updates are installed
on all sites. Check the following tables in the nice_admin database:
a. The database version appears in the tblDatabaseDetails table.
Figure 9-8 tblDatabaseDetails

Changing the Database Passwords in SQL Authentication
b. The version of the updates appears in the tblDatabaseHotFix table.
NOTE: Only those updates that were installed using an Update Pack appear in the
tblDatabaseHotFix table; updates that were installed manually do not appear.
-or-
7. When a Multi Data Hub environment is upgraded, the replication is automatically removed
during the procedures. After the upgrade has been completed, create the replication again.
Then test that the replication is functioning properly.

It is generally recommended to change the user passwords after installation. You can change the
passwords after the DBSetup installation, after completing the entire site installation, or in a live
production system. Since passwords are kept in several places, you must update the password
using the System Administrator to ensure that the passwords are updated in all these locations.
There are two passwords:
• nice sa - this password the same privileges as the Microsoft SQL Server sa user, and is used
for the following tasks:
• Changing passwords
• Replication
• Linking servers (such as Data Mart)
The nice sa password must be identical across all SQL Servers.
• nice - this password is used as a connection string for all NICE applications to the database.
You can use a different nice password for each SQL Server.
The nice and nice sa passwords are stored in encryption in the tblDbPassword table in the
nice_admin database.
The nice password is also stored in encryption in the System Administrator configuration file.
Figure 9-9 System Administrator Configuration File

Changing the nice sa Login Password

IMPORTANT
• You must work in Technician Mode to change the password.
• You must remove all replication before changing the password.
To change the nice sa login password:

1. In the System Administrator, under Master Site select Database Servers.
2. From the Actions menu, select SQL Server Security.
The SQL Server Security window for nice sa login appears.
Figure 9-10 SQL Server Security Window for nice sa Login
Make Sure You are in the

Security Window for
nice sa Login
3. Enter and confirm the new password.

4. To use a blank password, select Blank Password.
5. Click OK.
The password is changed on all SQL Servers in both the Primary and the Secondary Data
Hubs.
Changing the nice Login Password

IMPORTANT
• You must work in Technician Mode to change the password.
• You must remove all replication before changing the password.
To change the nice login password:

1. In the System Administrator, under Master Site > Database Servers, select the database
server on which you want to change the password.

Checking the Database Backup
2. From the Actions menu, select SQL Server Security.

The SQL Server Security window for nice login appears.
Figure 9-11 SQL Server Security Window for nice Login
Make Sure You are in the

Security Window for
Nice Login
3. Enter and confirm the new password.

4. To use a blank password, select Blank Password.
5. Click OK.
The password is changed on this SQL Server only.
NOTE:
• It may take up to one minute for the configuration file to be updated; do not use the
new password during this time.
• If your site includes the Data Mart server, you must recreate links. For details see
the NICE Perform System Administrator’s Guide.

You can check the database backup in System Administrator.
To check the database backup:

1. In System Administrator, navigate to Master Site > Database Servers:
a. Select the desired database and click the Backup tab.
b. Verify that all backup jobs are predefined and that there no errors in the Status column.
-or-
2. Connect to the SQL Server Management Studio:
a. Navigate to SQL Server Agent > Jobs.
b. If an error icon appears in the Differential Backup job or the Full Backup job, it indicates
that the last database backup job has failed.

-or-
3. Check if the source of the problem is the cassette as follows:
a. Check that there is a DAT/AIT cassette in the tape drive.
b. Check that the cassette is not in write-protect mode.
c. Check that the cassette is not corrupted.
-or-
a. Navigate to SQL Server Agent > Jobs.
b. Select the desired job and select View History in the right-click menu.
-or-
5. Run the job again:
a. Right-click the job and select Start Job.
b. Wait until the job finishes and check if the backup succeeded.
-or-
6. Check if the source of the problem is the tape driver:
• Open the operating system backup program and try to backup a file to the tape drive. If
you succeed, the tape driver is working properly and the reason for the failure is not the
tape.
-or-
a. Select the nice_admin database.
b. In the right-click menu select Tasks > Back Up.
c. Configure the destination to be a local file on the hard disk and try to backup the database.
If the backup succeeds, then the problem is not in the database.
-or-

Increasing the Size of a Database
8. Verify that the total database space used does not exceed the cassette capacity, as indicated in
the following table:
Cassette Type Capacity
DAT DDS2 4 GB
DAT DDS3 12 GB
DAT DDS4 20 GB
AIT1 35 GB
AIT2 50 GB
DAT72 72 GB
DVD Panasonic 9576 9.4 GB (double-sided)
Increasing the Size of a Database
To increase the size of a database:

2. Under Databases, select the database for which you want to increase the size.
3. From the right-click menu, select Properties.
The Database Properties window appears.
4. Select the Files page.

Checking Performance Using the NICE Performance Collector
Figure 9-12 Database Properties Window - Files Page
5. In the Initial Size column, click inside the cell for either the data file or the log file.
6. Enter the desired size for the database.
7. Click OK.
Checking Performance Using the NICE Performance

Collector
This tool can also be found on the Tier 4 Team Site. Ensure that you are using the newest version
which also supports SQL Server 2005. This tool collects SQL Profiler and Performance Monitor
traces.
To check performance using the NICE Performance Collector:

1. On the NICE Database Server, navigate to
Program Files\Nice Systems\Applications\Tools\Performance Collector.
2. Double-click Nice Performance Collector.exe.
3. The Nice Performance Collector window appears.

Checking Performance Using the NICE Performance Collector
Figure 9-13 NICE Performance Collector
4. In the SQL Server field, enter the SQL Server name or the IP Address.
5. Select the authentication method you are using.
6. Click Get Data/Log Drive.
The program automatically detects the location of the data and log files and enters them in the
respective fields.
7. In the Destination Folder field, define a location for the files which the Nice Performance
Collector creates.
8. In the schedule area, define a start and stop time.
9. Click Start.
Two files are created, one for counters and one for SQL Profiler.

Defining Autogrowth
Defining Autogrowth
To define autogrowth:
1. Open the Microsoft SQL Server Management Studio.
2. Under Databases, select the database for which you want to define the autogrowth.
Figure 9-14 Microsoft SQL Server Management Studio
3. In the right-click menu select Properties.

The Database Properties window appears.
4. In the Select a Page area, select Files.

Defining Autogrowth
Figure 9-15 Database Properties Window
Autogrowth
Parameters
Buttons
5. In the Autogrowth column, select either the Data file or the Log file and click the
corresponding Autogrowth Parameters button.
6. The Change Autogrowth window appears.
Figure 9-16 Change Autogrowth Window
7. To enable autogrowth, select Enable Autogrowth.

-or-
To disable autogrowth, clear Enable Autogrowth.

Using the NICE DBSpace Utility
8. If you enabled autogrowth:

a. Under File Growth select either In Percent or In Megabytes and select the desired
parameter from the corresponding drop-down box.
b. Under Maximum File Size select either Restricted File Growth and choose the desired
parameter from the corresponding drop-down box, or select Unrestricted File Growth.
9. Click OK.
The parameters you defined appear in the Autogrowth column of the Properties window.

The DBSpace utility enables you to verify database space usage and to monitor its status.
DBSpace offers the following features:
• Indicates databases that have less than 20% space available.
• Shows the used and free disk space for each database.
• Shows the used and free disk space for data and log files.
• Shows all tables in each database and their current status.
• Predicts the approximate number of days that remain until the tables reach full capacity.
IMPORTANT
You must run the DBSpace utility locally. Therefore, before running the DBSpace utility, you must
copy it to the NICE Perform server(s) on which the NICE Perform databases are installed.
To run the DBSpace utility:

1. Copy the DBSpace utility from the NICE Perform Applications Server to any convenient
location on the NICE Perform server.
2. Extract the files from the Nice DBspace.zip file.
3. In the folder containing the extracted files, double-click Nice DBspace.exe.
The Nice DBSpace window appears.

Figure 9-17 Nice DBSpace Window - Data/Log Tab

SQL Server Low Space
Connection Databases
Area Area
Database Status
Area
4. In the SQL Server field, enter the SQL Server name or the IP Address, or select the SQL
Server from the drop-down list.
Select either Use Windows Authentication or Use SQL Authentication. If you selected
SQL Authentication, enter your SQL Server Login name and the SQL Server password in the
Login and Password fields respectively.
NOTE: For SQL Authentication only:

By default, the sa Login password is blank. It is highly recommended to define a unique
sa Login password for your site. However, if you did not define a unique sa Login
password, leave the Password field blank.
5. Click the Connect button.

The Database Status area is populated.

6. In the Data/Log tab, select either Data to see the status of the SQL Server data files or Log to
see the status of the SQL Server log files.
The DBSpace utility displays the following:
Area Status
Low Space Databases Lists all databases in which the amount of free space
is less than 20%, and the percentage of the remaining
free space.
Drive Status Area Displays the amount of used space (displayed in blue)
and free space (displayed in green) on the drive on
which either the Data files or the Log files are located.
Databases Status Area Displays the amount of used space (displayed in blue)
and free space (displayed in green) in each database
in the NICE Perform Database Suite. The status of
each database is displayed both as a table and a
graph.
TIP: Use the scroll bar under the table to display the
portion of the table not visible.
NOTE: The DBSpace utility does not display the status of the databases in real time.
To refresh the display, click the Connect button.
7. Click the Tables tab.

Figure 9-18 Nice DBSpace Window - Tables Tab
8. From the Databases drop-down list, select the desired database.

The Tables list displays all the tables in that database and the status of each table.
To display fragmentation details, double-click the desired table. The fragmentation details
appear in the Fragmentation area.
9. Click the Predict tab.

Figure 9-19 Nice DBSpace Window - Predict Tab
10. The Predict table gives an estimate of how much time remains until each of the listed tables
reaches full capacity. The calculations are based on the parameters listed at the top of each
column.
11. Click the Activity tab.

Figure 9-20 Nice DBSpace Window - Activity Tab
12. In the Refresh Rate drop-down box select the desired number of seconds and click the Start
button.
To see the most relevant connections of any of the databases, click the desired database. To see
all the connections of that database click the Show All button.

Analyzing the Database Using NICE DBAgnose

The DBAgnose utility checks your NICE Perform Database Suite and SQL Server configurations,
and offers an analysis of many of the potentially serious database problems. By running the
DBAgnose utility you ensure that your database and the SQL Server are configured properly.
IMPORTANT
You must run the DBAgnose utility locally. Therefore, before running the DBAgnose utility, you
must copy it to the NICE Perform server(s) on which the SQL Server is installed.
To run the DBAgnose utility:

1. Copy the DBAgnose utility from the NICE Perform Applications Server to any convenient
location on the NICE Perform server.
2. Open the Nice DBAgnose folder you just copied and double-click Nice DBAgnose.exe.
The Nice DBAgnose window appears.
Figure 9-21 Nice DBAgnose Window

3. In the SQL Server field, enter the SQL Server name or the IP Address, or select the SQL
Server from the drop-down list.
Select either NT Authentication or SQL Authentication. If you selected SQL
Authentication, enter your SQL Server Login name and the SQL Server password in the
Login name and Password fields respectively.
NOTE: For SQL Authentication only:

By default, the sa Login password is blank. It is highly recommended to define a unique
sa Login password for your site. However, if you did not define a unique sa Login
password, leave the Password field blank.
4. Click the Diagnose button.

The DBAgnose utility checks the SQL Server configurations.
Figure 9-22 Nice DBAgnose Window
5. The DBAgnose utility displays the status of each SQL Server configuration.
IMPORTANT
Some of the displayed configurations may not be applicable to the database migration, and you
may be able to successfully conclude the migration procedures despite some warning or even
error messages. For details or assistance contact NICE Systems Customer Services.

NOTE: You can save the results by clicking the Save Results button. You can then
print them out or email them to NICE Systems Customer Services.
TIP: For a detailed description of each configuration, place your cursor at the beginning
of the row; a pop-up window appears displaying the details of that configuration.

General Data Mart Troubleshooting Flow
General Data Mart Troubleshooting Flow

If you experience a problem with the Data Mart, the following workflow lists the steps to isolate
the problem. Each step in the workflow is a section in this chapter.

Data Mart Troubleshooting Issues

Data Mart Troubleshooting covers the following topics:
• Checking the Database Size (see page 288)
• Recreating Links (see page 288)
• Checking the Data Mart Database Version (see page 289)
• Checking Connectivity Between the Database and Data Mart Servers (see page 289)
• Checking a Data Mart Job Failure (see page 290)
Checking the Database Size

Check to see if the size of the database is sufficient. If necessary, increase the database size. For
details, see Checking Free Space in the Database on page 259.
Recreating Links
To recreate links to the Data Mart:

1. In the System Administrator, expand the Master Site > Data Marts and select the required
Data Mart server.
2. Click the Sites tab.
Figure 9-23 Data Marts - Sites Tab
3. Click Recreate Links.

Checking the Data Mart Database Version

Ensure that the database version is the same on all sites.
To check the Data Mart database version:

• Verify that the database version in the tblDatabaseDetails table of each database is identical.
Figure 9-24 tblDatabaseDetails
Checking Connectivity Between the Database and Data Mart Servers

The Data Mart adds the Database Server as a linked server. It is crucial that these two servers can
communicate with one another.
To check connectivity between the Database and Data Mart servers:

1. In the SQL Server Management Studio expand the Security > Linked Servers branch.
Figure 9-25 Linked Servers Branch
2. From the Data Mart server, ping the Database Server.

Checking a Data Mart Job Failure
To check a Data Mart job failure:

1. Check for error messages in Windows Event Viewer.
-or-
2. Check for error messages in the SQL log files.
-or-
3. Check the job history to see which step failed and why. This isolates the problematic domain
or problematic communication with the database.
a. In System Administrator, expand the Master Site > Data Marts and select the required
Data Mart server.
b. Click the Monitor tab.
c. In the SQL Jobs area, select the required job. The job steps are detailed in the History
area.
Figure 9-26 Monitor Tab

10
Customer Feedback Installation
Troubleshooting Flow
This section provides a review of some information that will aid the installer in understanding the
Customer Feedback workflow.
Also provided is a workflow recommending how to proceed when troubleshooting a Customer
Feedback system.
Contents
General Information .....................................................................................................292

Troubleshooting Customer Feedback ........................................................................294
Troubleshooting Flow Expanded..............................................................................295
Troubleshooting - General Issues......................................................................295
Troubleshooting - DTMF Routing ......................................................................296
Troubleshooting - CTI Info or Business Data Routing .......................................298
Surveys and Interactions are not Properly Correlated .............................................301
Chapter 10: Customer Feedback Installation Troubleshooting Flow 291

General Information
General Information
The following information is made available to help the installer troubleshoot the NICE Customer
Feedback system. For additional tips, see Troubleshooting Customer Feedback on page 294.
For NICE Customer Feedback to function properly, certain information must be made available
from the PBX.
Binding between call and survey
To bind a call to a survey, the Telephony Services must receive a call event from the CLS (CAPI).
In order to receive the events, the Feedback extensions must be monitored by the drivers.
In most cases, the Telephony Services uses the Compound ID of the call to identify its previous
segments (segments that have the same Compound ID). It then binds the survey to one of
segments, choosing either the first or last segment, according to the Feedback configuration in the
System Administrator.
For Avaya Automatic Routing, the Telephony Services cannot use the Compound ID since the
Feedback call and the previous calls have different Compound ID values. In this case, the
Telephony Services uses the Avaya UCID value to bind the calls. It is important that the PBX
reports the UCID to the driver and that the CLS reports the UCID in its events table.
Dynamic loading of surveys
There are two main ways to dynamically load a survey:

• By DTMF - the DTMF is sent by the switch to the Feedback extension immediately after the
call to the Feedback extension is established. For a sample of the System Administrator
definition, see Figure 10-1 on page 293.
-or-
• By Call Data (DNIS, Phone Number, VDN or Business Data) - the call data is received in
events reported by the CLS. If the Feedback extensions are not monitored by the drivers,
events will not be received and surveys cannot be loaded. For a sample of the System
Administrator definition, see Figure 10-2 on page 293.
Environment Setup
The following should exist in your NICE Perform environment:

• Playback Server extensions which are allocated for Feedback must be monitored by the driver.
Use the Driver Tester for verification. The important thing to check is that events such as Start
and End Call from agent extensions as well as from playback extensions are monitored.
• For Avaya Automatic Routing, the PBX should be configured such that the UCID information
is enabled. This can also be verified using the Driver Tester. If the PBX is sending the
information correctly, the CLS will have to save it in the database. For this to happen, a value
must be defined for the for the vcPBXUniversalCallInteractionId field in the Business Data
tab of the CLS branch of the System Administrator application.
• The VDN architecture should be known and Telephony Services extension should be
configured to auto-login as described in the Release Notes.

General Information
Configuration in the System Administrator
The following are sample configurations in the System Administrator application:

• By DTMF - in the Ports tab of the Telephony Services > Server > Feedback branch there
must be at least one port defined with an extension number that is configured as Feedback.
Figure 10-1 System Administrator - Feedback Branch, Ports Tab
• By Call Data - in the Surveys tab of the Telephony Services > Server > Feedback branch,
surveys must be attached to a DTMF for Avaya Automatic Routing or the DNIS number for
Manual Transfer.
Figure 10-2 System Administrator - Feedback Branch, Surveys Tab

Troubleshooting Customer Feedback

In the event that you need to troubleshoot your Customer Feedback system, use the following flowchart and accompanying tables to locate
the source of your problem and promptly correct it. Details appear in Troubleshooting Flow Expanded on page 295.
Figure 10-3 Customer Feedback Troubleshooting Flowchart

Troubleshooting Flow Expanded

In the event that you need to troubleshoot your Customer Feedback system, the troubleshooting flowchart is explained in the following
tables. See the accompanying flowchart to locate the source of your problem and promptly correct it. The flowchart appears in
Troubleshooting Customer Feedback on page 294.
Troubleshooting - General Issues on page 295
Troubleshooting - DTMF Routing on page 296
Troubleshooting - CTI Info or Business Data Routing on page 298
Troubleshooting - General Issues

Table 10-1: Troubleshooting - General Issues
# Problem Symptom Possible Cause & Solution
1 Surveys are not The survey was done correctly. In the Database table Make sure there is an agent defined
assigned to an agent in tblPbsNifSurveyResult there are values in the in the Users Administrator, and that
the Business Analyzer iCompundID and iInteractionID, however the iUserID field the agent identity (agent ID or
Feedback. is set to -99. extension number) matches the
parameters of the call that preceded
the survey.
In the following line, taken from the
log, the agent should be either
associated with extension 40005 or
agent ID 46149:
INFO ; [CapiServer.HandleEvent] – internal
participant: agentId=’46149’
IsInteractionInitiator=False PhoneNum=40005
UniqueDeviceID=

Table 10-1: Troubleshooting - General Issues (Continued)
2 An error message The call will be disconnected by the Telephony Services and A prompt file is missing. A new
“Prompt file is missing” there will be a new record for the survey in the database. prompt file must be recorded using
is played at the The value of fltSurveyScore field depends on the total weight the Prompt Recorder.
beginning of, or during of the questions answered. If the total weight is above the
a survey. minimum weight, the actual score will be saved. Otherwise, the
value will be -1.
3 Surveys can be viewed A record exists in the database table The UCID is not reported to the driver
in the Business tblPbsNIfSurveyResult. The iCompoundID field contains a by the PBX. The PBX should be
Analyzer Feedback, value but the nvcUCID, iInteractionID and iUserID are configured to pass this value.
but they are not empty.
attached to any Furthermore, a search for interaction in the
interactions. vwCallInteraction with the iCompoundID value returns an
interaction which has a blank
vcPBXUniversalCallInteractionID.
Troubleshooting - DTMF Routing

Table 10-2: Troubleshooting - DTMF Routing
4 Surveys can be A record of the call appears in the database table The Feedback extensions are not
viewed in the tblPbsNifSurveyResult. However the fields nvcDNIS, nvcUCID, monitored by the driver. In this case
Business iCompoundID, iInteractionId, iUserID and iSwitchID have no CAPI event is received by the
Analyzer empty values. Telephony Services.
Feedback, but The following messages can be found in the Telephony Services log: The Feedback extensions should be
they are not INFO ; [CFBasicActivity.OnHasNextQuestion] – Line[1] Quit questions loop added to the monitored devices list in
attached to any … the Integrations branch of the
interactions. WARN ; [TelephonyManagerService.GetCapiInformation] – Line[1] No CAPI System Administrator application.
interaction was found

Table 10-2: Troubleshooting - DTMF Routing (Continued)
5 Surveys can be A record of the call appears in the database table Feedback ports are configured in the
viewed in the tblPbsNifSurveyResult. However, the fields nvcDNIS, nvcUCID, Telephony Services branch of the
Business iCompoundID, iInteractionId, iUserID and iSwitchID have System Administrator with an
Analyzer empty values. incorrect extension number.
Feedback, but The following messages can be found in the Telephony Services log
they are not at the start of the survey:
attached to any INFO ; [CapiServer.HandleEvent] – Received event:
interactions. CLS_START_CALL_EVENT
INFO ; [CapiServer.HandleEvent] – internal participant: agentId=’46150’
IsInteractionInitiator=False PhoneNum=40008 UniqueDeviceID=
The following messages can be found in the Telephony Services log
at the end of the survey:
INFO ; [CFBasicActivity.OnHasNextQuestion] – Line[1] Quit questions loop
…
WARN ; [TelephonyManagerService.GetCapiInformation] – Line[1] No CAPI
The phone number in the log file (in this case 40008) is not the
extension number configured in the System Administrator for the line
(in this case 1).
6 Survey is not The call is routed to the Telephony Services but the server The Telephony Services cannot find
loaded. Feedback immediately hangs up the call. There is no record of the call in the the received DTMF in the System
call is immediately database. In the log file the following lines will appear: Administrator’s configuration.
disconnected. INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] – Line[2] Get survey ID In Avaya Automatic Routing, the
by load information [Load by DTMF] received DTMF is the VDN that
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] – Line[2] DTMF initiates the Feedback call.
sequence 48016
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] – Line[2] No survey
A DTMF routing should be added to
ID was found the Survey Assignments tab of the
Feedback branch in the System
Administrator with the received
DTMF sequence.

Troubleshooting - CTI Info or Business Data Routing

Table 10-3: Troubleshooting - CTI Info or Business Data Routing
7 Survey is not After the call was transferred to the Feedback extension, there is The Feedback extensions are not
loaded. silence for 15-20 seconds and then the line is disconnected. No data monitored by the driver. In this case a
Feedback call was saved in the database. CAPI event is not received by the
is In the log, CAPI information is not received: Telephony Services.
disconnected The Feedback extensions should be
INFO ; [CFMainWF.OnIsCtiInfoLoadType] – Line[1] Survey load type is
after more than CTI_INFORMATION added to the monitored devices list in
15 seconds. … the Integrations branch of the System
INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start Administrator.
call, 15 seconds left
INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start call,
14 seconds left
…
1 seconds left
8 Survey is not After the call was transferred to the Feedback extension, there is If CAPI events are constantly received
loaded. silence for 15-20 seconds and then the line is disconnected. No data but for a different extension, the
Feedback call was saved in the database. Feedback ports are configured in the
is In the log, CAPI information is received, but not used. Telephony Services plug-in with a
disconnected wrong extension number.
In the log below, the CAPI reports calls for extension 40005, while the
after more than
Telephony Services configuration specifies that extension 40008 is
15 seconds.
used in this line (line 1):
INFO ; [CapiServer.HandleEvent] – Received event: CLS_START_CALL_EVENT
…
[continued on next page...]

Table 10-3: Troubleshooting - CTI Info or Business Data Routing (Continued)
8 [...continued from previous page]

INFO; [CapiControl.AddInteraction] – Added interaction to hashtable [Extension 40005,
SwitchId 1, CompoundId 300866, InteractionId 300869, DNIS 40005, UCID
00001062061213527184, Phone number, VDN, IsTransferInteraction False]
…
INFO ; [TelephonyManagerService.AttachWorkflowToExtension] – Line[1] Workflow
instance added to extension 40008
(InstanceId=ccd70c25-7e60-4db9-a32a-1c02e2eb5fdd)
…
INFO ; [CFMainWF.OnIsCtiInfoLoadType] – Line[1] Survey load type is
CTI_INFORMATION
…
INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start
call, 15 seconds left
INFO ; [CapiServer.HandleEvent] – Received event: CLS_START_CALL_EVENT
…
INFO; [CapiControl.AddInteraction] – Added interaction to hashtable [Extension 40005,
SwitchId 1, CompoundId 300866, InteractionId 300872, DNIS 40005, UCID
00001062061213527184, Phone number T6205#1, VDN, IsTransferInteraction
True]
…
INFO ; [CapiServer.HandleEvent] – Received event: CLS_END_CALL_EVENT
…
14 seconds left
…
1 seconds left

Table 10-3: Troubleshooting - CTI Info or Business Data Routing (Continued)
9 Survey is not After the call is transferred to the Feedback extension it is immediately No survey is configured for the CTI
loaded. disconnected. No data was saved in the database. information or the Business Data of
Feedback call In the log below, the survey should be loaded by DNIS. A call, the call.
is immediately previously received from the CAPI, is found with DNIS 40005, but there A CTI information or Business Data
disconnected. is no routing configured in the System Administrator for this DNIS routing should be added to the Survey
value: Assignments tab in the Feedback
INFO ; [TelephonyManagerService.GetCapiInformation] - Line[1] Wait for CAPI start call, branch of the System Administrator
15 seconds left with the received information.
INFO ; [TelephonyManagerService.GetCapiInformation] - Line[1] Found CAPI
interaction ID 300895
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] - Line[1] Get survey ID by
load information [Load by DNIS]
INFO; [TelephonyManagerService.GetSurveyIdByLoadInfo] - Line[1] interaction
information Extension 40005, SwitchId 1, CompoundId 300889, InteractionId 300895,
DNIS 40005, UCID 00001062561213527869, Phone number T6255#1, VDN,
IsTransferInteraction True
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] - Line[1] No survey ID
was found
10 After a survey In Manual Transfer, after the agent transferred the call and the survey Both Manual Transfer and Automatic
ends another was completed, the customer hears once again the survey prompts Routing are used. The agent first
survey starts in from the beginning. transfers the customer to the
the customer’s Also, in the database there are two surveys with the same Feedback extension. When the
call. CompoundID and nvcUCID. Feedback session ends, the VDN
transfers the customer to another
extension.
Only one of the methods should be
used. Either the Automatic Routing
will be disabled, or the agent will stop
using the Manual Transfer.

Surveys and Interactions are not Properly Correlated
Surveys and Interactions are not Properly Correlated

If you have correlation issues between your surveys and the agents’ interactions, this may be due
to a faulty UCID.
Check your switch configuration. See the Playback Server Installation Guide.
Review the Customer Feedback Setup Guide.

11
High Density Logger Troubleshooting
Flow
This chapter contains troubleshooting tips and procedures for High Density Loggers.
Contents
General High Density Logger Troubleshooting Flow................................................304

Verifying the Logger Configuration ............................................................................306
Generating a Build Content Report.............................................................................307
Checking Logger Initiation Failure..............................................................................308
Setting the Debug Level...............................................................................................309
Checking for Logger Configuration Errors ................................................................314
Checking Logger Activity ............................................................................................316
Checking Map Errors ...................................................................................................316
Checking RCM Issues ..................................................................................................317
Synchronizing the Time ...............................................................................................317
Testing the Logger COTS using the NICE Test Kit ....................................................318
Testing the NICE Boards using the NICE Test Kit .....................................................320
Testing the Logger using the NICE PST .....................................................................321
Shutting Down and Starting Up the Logger ...............................................................322
Shutting Down and Starting CLS ................................................................................323
Using the Nicelog Setup Tool ......................................................................................323
Other Troubleshooting Issues.....................................................................................325
Other Useful Tools........................................................................................................326
Chapter 11: High Density Logger Troubleshooting Flow 303

General High Density Logger Troubleshooting Flow
General High Density Logger Troubleshooting Flow

If you experience a problem with a High Density Logger, the following workflow lists the steps to
isolate the problem. Each step in the workflow is a section in this chapter.
Figure 11-1 General High Density Logger Troubleshooting Flow

Checking the Basics

Perform the following basic checks:
1. Check that the Logger services are up.
2. Reboot the Logger. See Shutting Down and Starting Up the Logger on page 322.
3. Check the connection to the power supply.
4. Check the network connection.
5. Verify that the TCP/IP properties correspond with the system administrator's site definition.
6. If the Logger is not responding in the SNMP traps, stop the CLS and then reboot the CLS and
Logger. See Shutting Down and Starting Up the Logger on page 322 and Shutting Down
and Starting CLS on page 323.
7. Check any changes were made to the switch, for example, frame format, signaling type, switch
version, and so on.
8. Check the connection between the Logger and the NICE Interactions Center.
9. Check the cables between the Logger and the switch.
10. Use the NICE Test Kit to check the COTS components. See Testing the Logger COTS using
the NICE Test Kit on page 318.
11. Use the NICE Test Kit to check the NICE boards. See Testing the NICE Boards using the
NICE Test Kit on page 320.
12. Use the PST to check the Logger. See Testing the Logger using the NICE PST on page 321.

Verifying the Logger Configuration
Verifying the Logger Configuration

Check that the Logger has been correctly configured in System Administrator.
To verify the Logger configuration:

1. Ensure that the Logger is up and running.
2. Verify that you are working in Technician mode. In the Organization tree, click Organization.
Then select the Technician Mode checkbox, and click Save.
3. In the Organization tree, select the Logger Servers branch.
4. Select the relevant Logger and verify the:
a. IP address or Host Name
b. Input Trunks
c. Input Channels
d. Predefined Selective Recording
e. Global parameters
5. In the Organization tree, expand CLS Definitions > CLS Servers, and select a CLS server.
6. Click the Loggers tab.
7. Ensure that the Logger is attached to the CLS.
8. In the Organization tree, expand Logger Servers > NiceLog > <Logger name> >
Recording Channels.
9. Verify the Audio Recording Channels setup.

Generating a Build Content Report
Generating a Build Content Report

Use the NICE Test Kit Application to generate a build content report of the software and hardware.
You can collect the following logs in the zip file together with the build content report:
• Logger Cfg Files
• Logger Log Files
• OS Event Log File
• MSInfo32 File
For further details, see Testing the Logger COTS using the NICE Test Kit on page 318, and the
NICE Test & Recovery Kit User Manual.
To build a content report:

1. If you do not have the NICE Test Kit Application, you can download it from ExtraNICE >
Services > Diagnostic Utilities.
2. In the NICE Test Kit, click Build Content Report.
Figure 11-2 NICE Test Kit Window
3. In the Options area, select the required log files, and in the Destination Folder field, enter a
Destination Folder for the Content Report.
Figure 11-3 Log File and Destination Folder Options
Select log files

Checking Logger Initiation Failure
4. Click Take Build Content. The build content is recorded.

5. To see the report, click View Build Content.
6. Close the Build Content Viewer window.
7. Click Save Build.
8. Click OK.
The build content is saved in a zip file.
Checking Logger Initiation Failure

In the event of a Logger initiation failure, ensure that all the new hardware has been installed.
To check Logger initiation failure:

1. In Computer Management, select Device Manager.
2. Look for errors. There is no specific error messages for Logger initiation failure. Different
error messages may appear according to which devices were installed.
Figure 11-4 Computer Management Window Displaying Errors
Errors
3. From the Action menu, run Scan for hardware changes.

The device identification process starts.
4. Ensure that all the hardware is installed.

Setting the Debug Level

Switch the debug level to FATAL.
To set the debug level:

1. Open d:\NTlogger\logger\config\SysRC.dat in Notepad.
Figure 11-5 SysRC.dat File in Notepad
2. In [system -default], set the LogLevel2File and LogLevel2Screen to 1 instead of 3.

3. Reboot the Logger. See Shutting Down and Starting Up the Logger on page 322.
Checking the Logs

Bug Find and Log Analyzer are two tools used to check the logs of High Density Loggers:
• Using Bug Find to Check Logs on page 309
• Using Log Analyzer to Check Logs on page 312
Using Bug Find to Check Logs

Bug Find automatically searches in the Logger log files for FATAL and BUG errors. Other files
can be dragged and dropped into it and it will search in them as well.
The logs are located in D:\NTLogger\logger\log. The log files are plain text. The directory
includes:
• Log files for monitoring the recording activity:
• Logs all Logger software messages and errors.

• Last ten files, up to 10 MB each.

• Named LogFile.DAT (current), then LogFile1.DAT, …, LogFile9.DAT (oldest).
• Event log files:
• Logs important events and user actions on Logger.
• One file for each day of the month.
• Named EVELogFileN.DAT, where N=1..31
Figure 11-6 Log File
Figure 11-7 Event File
To check the logs:

1. Double-click D:\NTLogger\Logger\Testers\LoggerUtils\bugfind.exe to run Bug Find.
This tool can also be found on the Tier 4 Team Site.

Figure 11-8 Bug Find
2. In the Search String field, enter - logger. This is part of the string --logger is up and
running- indicates that the Logger started up.
Figure 11-9 Bug Find - “Logger is up and running” String
3. For a FATAL error, check which component name is listed after FATAL:. The example below
shows a DBM fatal error start, indicating a problem with the database.

Figure 11-10 Bug Find - Fatal Error
Using Log Analyzer to Check Logs

Log Analyzer searches multiple log files for any string, such as an Extension, Agent ID, or IP
address.
To use Log Analyzer to check logs:

1. Double-click D:\Program Files\Common Files\Nice\LogService\Bin\LogAnalyzer.exe
to run Log Analzyer.
Figure 11-11 Log Analyzer

2. In the Search String field, enter the string you want to find in the logs.
3. Add log files to Log Analyzer in the following ways:
• Drag and drop log files into the Log Analyzer window.
• Click Add , and browse to each log file you want to add.
4. Click Search to start searching the logs.
Log Analyzer searches the logs, and the Search button changes to Stop.
Figure 11-12 Log Analyzer Searching Logs
5. Click Stop to stop the search, or Exit to close Log Analyzer.

Checking for Logger Configuration Errors

Line errors can be seen in the Functional Tester. A warning error (on one or more lines) on the
Logger may indicate an issue with the Logger configuration. This tool should only be used in
consultation with NICE Support!!!!
Error thresholds are set which indicate how and when an error is reported.
To view the error thresholds:

1. To view the error threshold, open d:\NTLogger\logger\config\AudHW.cfg and search for:
[NATI2_ERRORS_PARAMS] / [ETAI2_ERRORS_PARAMS] /
[ETAI3_ERRORS_PARAMS]/ [ALI4_ERRORS_PARAMS]
2. There are 2 parameters per error:
• The number of times that the board checks the line and finds the error.
• The total number of times that the board checks the line, that is, the number of checks in
window of checks.
EXAMPLE:
11 ; Min Errors: Max Frame Length - Error 0 - LOS (Loss of sync) - in order to report this
error, the error must occur every time the line is checked.
2 4 ; Min Errors: Max Frame Length - Error 2 - UBAL (PBX terminal balance error) - in order to
report this error, the error must occur at least twice in 4 checks.
To view the errors:

1. The error message format is:
Board Error
ALI4 ALI4_CheckAllBoards: Board has real error, <Board, Stream, Error Index,
Error Count, Frame Length>
ETAI2 ETAI2_CheckAllBoards: Board has real error, <Board, Line, Error Index,
ETAI3 real error, <Board, Stream/Line, Error Index, Error Count, Frame Length>
BTAI2 BTAI2_CheckAllBoards: Board has real error, <Board, Line, Error Index,
NATI2 NATI2_CheckAllBoards: Board has real error, <Board, Stream, Error Index,

Where:
Parameter Description
Board Board ID.
Stream/Line Stream/line one the board.
Error Index The type specific error ID:

ALI4:
0 – NOS
ETAI2:
0 – FER, 1 – MLOS, 2 – PRA, 3 – FLOS, 4 – AIS, 5 – NOS
ETAI3:
0 – FER, 1 – MLOS, 2 – RS, 3 – XS, 4 – PRA, 5 – FLOS, 6 – AIS, 7 – NOS
BTAI2:
0 – FLOS, 1 – NOS
NATI2:
0 – LOS, 1 – NOS, 2 – UBAL, 3 – LSIG, 4 – MFAS, 5 – INDICATOR TIME
OUT, 6 – SYNCWINKFLASH
Error Count The number of times the board checked the line and an error occurred in
the current window of checks.
Frame The length of the frame in which the errors were counted.
Length
2. For example, error message 4 20 1 1 1 indicates a critical problem that is reported even if
there is only one error out of one check:
• Board = 4
• Line on Board = 20
• Error Index = the first error in the config file
• Error Count = 1
• Frame Length = 1
This error may indicate a signal that is too low and require an amplifier or active coupler.
A real error is reported when the error rate is greater than or equal to that specified by the
parameters:
WARN: NATI2_CheckAllBoards: Board has real error
This is followed by another error when the error rate drops below that specified by the
parameters:
WARN: NATI2_CheckAllBoards: Board error was reset

Checking Logger Activity
When the Capture driver does not find all the hardware boards the following log message
appears:
System_Init: Failed to find all boards. <0,0>.
From Logger 9.06 SP5, a warning message appears while waiting for all boards to be found:
System_Init: Wating for all boards to be found by Pnp. This may take several
minutes... <SafeCounter,0>
SafeCounter indicates the number of tries to find the boards. The maximum number of retries
is 60 with an interval of 10 seconds, that is, the plug and play manager has 10 minutes to locate
the NICE boards.
Checking Logger Activity

If there is no activity on the Logger, use the Functional Tester to check the mapping and input type.
This is not usually a Resource Coordination Module (RCM) issue.
IMPORTANT
This tool should only be used in consultation with NICE Support.
Checking Map Errors

The following may cause an error when trying to create a map on the logger (Digital Matrix
configurations):
• The board may be configured as Co-PaBX and there is no summation mate in the map.
• The summation mate trunk is not configured on the board.
• There are not enough channels and the Logger is mapped with static inputs.

Checking RCM Issues
Checking RCM Issues

RCM communicates with different components, such as Loggers, ScreenAgents and other CLS
components.
To check the RCM Logs:

• On the NICE Interactions Center, open the RCM process (Resource Manager console), or
review the RCM.log.xxx.txt files in D:\Program files\NICE Systems\NICE CLS\Bin\Log.
To check the Call Server Logs:

• On the NICE Interactions Center, the CallServer.log.xxx.txt logs are located in a \Log folder
in the installation path.
The following RCM problems may occur:
• If the RCM reports that there are no resources to allocate (indicated as an Unmapped
exception 15 in the Call Server logs), check the following:
• For Total and Interaction-based recording, check the mappings in System
Administrator.
• For Interaction-based recording, also check that you have enough channels in your
channel pool when recording simultaneous calls.
• If the RCM cannot connect to a Logger, check if the Logger is assigned to the CLS Server
in System Administrator and if the settings were saved.
Synchronizing the Time

Each computer in the Logging System must be accurately time synchronized to ensure correct
operation. Time synchronization of Logging System components is achieved using the Windows
Time service known as W32Time that uses the Network Time Protocol (NTP).
Open a command prompt and type: w32tm /resync.
NOTE: Loggers clock should not be modified while the Logger is running. Modify the
time with NTP only.

Testing the Logger COTS using the NICE Test Kit

The NICE Test Kit is provided on a USB flash drive and contains several utilities used to perform
diagnostic testing of COTS (Commercial Off The Shelf) hardware. This enables you to determine
if a component is faulty, and thus avoid replacing properly functioning components.
The NICE Test Kit includes tests for PC peripherals, hard disks and backup devices. It also
contains System Recovery Tools that enable you to restore the NICE system on your
Logger/Recorder. You can also detect the devices on the Logger/Recorder and create a Build
Content Report.
The NICE Test Kit on the USB can be ordered as a spare part.
You can also download the NICE Test Kit Application from ExtraNICE. This can be used without
the USB and offers limited functionality. The NICE Test Kit Application enables you to detect
hardware devices on your Logger (SBC/MB, NICE Cards, Hard Disk Devices and Backup
Devices). It also enables you to create a hardware build content report and collect the following
files.
• Logger Cfg Files
• Logger Log Files
• OS Event Log File
• MSInfo32 File
To test the Logger:

1. If you do not have the NICE Test Kit, you can download it from ExtraNICE > Services >
Diagnostic Utilities.
2. For further details, see the NICE Test and Recovery Kit User Manual.
Tests and Log File Locations in a Windows Environment

The table below lists the location of tests and log files in a Windows Environment.
Table 11-1: Test and Log File Locations in a Windows Environment
Detected Device List List of detected H.W No log file.

devices.
Build Content Reports Report of the Build Root:\Reports\*.zip & *.html

content (Software and
Hardware)
Live Update (Web) Update the NICE Test Kit No log file.
PC Peripherals - CPU Checking the CPU Root:\Programs\BurnInTest\BIT_log*.log

Table 11-1: Test and Log File Locations in a Windows Environment (Continued)
PC Peripherals - Checking the Network Root:\Programs\BurnInTest\BIT_log*.log

Network
PC Peripherals - Checking the Memory Root:\Programs\BurnInTest\BIT_log*.log

Memory
PC Peripherals – VGA Checking the VGA 2D Root:\Programs\BurnInTest\BIT_log*.log

2D
HDD Fujitsu (SCSI) Checking Fujitsu SCSI Root:\Programs\HDD Fujitsu Diag -

Hard Drives SDIAG for Windows
HDD Maxtor (IDE & Checking Maxtor IDE & Root:\Programs\HDD Maxtor - NICE Disk
SATA) SATA Hard Drives Tool for Windows
HDD Seagate Checking Seagate SCSI Root:\Programs\HDD Seagate -

Enterprise (SCSI) Hard Drives SeaTools Enterprise for Windows
HDD Seagate Checking Seagate IDE & Root:\Programs\SeaTools for Windows

Seatools (IDE & SATA) SATA Hard Drives –
Requires DotNet2
software
HDD Western Digital Checking Western Digital Root:\Programs\HDD Western Digital -

(IDE, SATA & SCSI) IDE, SATA & SCSI Hard Diagnostic for Windows\WinDlg.htm
Drives
HDD Other (SCSI) Checking undefined Root:\Programs\HDD Seagate -

manufacturers SCSI SeaTools Enterprise for Windows
Hard Drives
NICE EIS Products Checking PC peripherals Root:\Programs\BurnInTest\BIT_log*.log

for various NICE EIS
products
NICE VIM Products Checking PC peripherals Root:\Programs\BurnInTest\BIT_log*.log

for various NICE Security
products
Backup Sony AIT & Checking SONY AIT IDE Root:\Programs\SONY Tape Tool for
DDS Tape Tool Backup Devices. Windows\S.N.dat
Firmware Upgrade for S.N indicates the tape serial number
SONY IDE & SCSI
Backup Test DVD Creating a Test DVD No log file.

Creator Application
Backup DVD Read & Checking DVD Drives Root:\Programs\BurnInTest\BIT_log*.log

Verify

Testing the NICE Boards using the NICE Test Kit
Tests and Log File Locations in a DOS Environment

The table below lists the location of tests and log files in a DOS Environment.
Table 11-2: Test and Log File Locations in a DOS Environment
Test / Operation Test Description Log File Location
Backup SONY AIT & Checking SONY AIT & Root:\Programs\SONY\NTAU\*.ana

DDS Test (SCSI) DDS SCSI Backup Root:\Programs\SONY\Ship\*.abc
Devices
HDD Seagate and Checking Seagate & Root:\Programs\SEAGATE

Maxtor Test (IDE and Maxtor IDE & SATA Hard
SATA) Drives.
HDD Western Digital Checking Western No log file.

Test (SATA) Digital SATA Hard Drives
HDD Hitachi-IBM Test Checking IBM or Hitachi Root:\Programs\DFT\*.BLZ

(IDE, SATA and SCSI) IDE, SATA or SCSI Hard Location depends on the user. The
Drives (DOS) log files can only be read by the
manufacturer.
Memory Diagnostic Memory test No log file.

Tools
System Recovery Performing System No log file.

Tools Recovery over IP
Testing the NICE Boards using the NICE Test Kit

You can use the NICE Boards Diagnostics Utility (NBDU) to analyze the operation of the NICE
boards within the recording platform, and diagnose whether a board is defected. The NBDU
allows Field Engineers to easily test the hardware boards in the field and get an immediate
indication of its functioning. The NBDU requires the use of the NICE Boards Diagnostics Device
(NBDD) self test accessory which must be purchased separately.
The following NICE boards can be tested:
• ALI IV
• NATI II
• ADIF IV
• ETAI III
For further details, see the NICE Test & Recovery Kit User Manual.

Testing the Logger using the NICE PST
Testing the Logger using the NICE PST

The NICE Portable Service Tester (PST) is a system level tester that improves hardware failure
diagnostics on site. The PST enables the field support engineer to test audio Loggers. See the
NICE PST User Manual.
The following tests are available:
• Activity: Checks for the channel input signal.
• Threshold: Tests the channel signal threshold.
• Monitor Telephone Side: Checks the signal on the telephone side (for live monitoring).
• Monitor PBX Side: Checks the signal on the PBX side (for live monitoring).
• Record and Play: Records the channel for a few seconds of signal, and then plays it back.
• Compression: Compresses and records a call, and then plays it back.
• Manual Archiving: (If archiving device is connected). Archives a call and retrieves it from
the backup device.
Specifications
• Suitable for use with the following Loggers: NiceLog Logger, High Density Logger, NiceCall
Focus, ICU and MIRRA IV.
• Compatible with the following PABXs: Analog, E1, T1, BT, Nortel, Norstar, Lucent, Siemens,
Nec, Philips, Ericsson, Aspect, Alcatel. (The PST is ready to add any new PABXs).
• Testing time per channel of between 5 and 60 seconds.
• Supports up to two NATI boards, two ALI boards, one ETAI board and one BTAI board
(future use).

Shutting Down and Starting Up the Logger
Shutting Down and Starting Up the Logger

Follow the procedures below to shut down and start up the Voice Logger processes correctly,
BEFORE you restart or shutdown the Windows operating system.
To shut down and start up the Logger:

1. There are multiple Voice Logger modules running in the Windows Taskbar.
Figure 11-13 Multiple Voice Loggers Running in the Windows Taskbar
Multiple Loggers running
2. From the Start menu, select Shutdown Logger. All of the modules, except for the VL Snmp
Agent, will shutdown. You can either close the VL Snmp Agent manually, or ignore it. If you
are not rebooting the Operating System, do not close the SNMP Agent.
-or-
Stop the NiceLog service.
3. To start the Voice Logger either:
• Reboot the Windows operating system and let the system startup automatically.
• Select Start > Programs > NiceLog Server > Monitor LOG or Start > Programs >
Startup > Monitor LOG.
• Start the NiceLog service.
The Voice Logger starts up and all of the Voice Logger modules appear in the Windows
Taskbar. When the Voice Logger is completely started up, it plays the 7-note couplet "Shave
and a Haircut, Two Bits."
NOTE: If you close the VL Snmp Agent, but do not reboot the Windows operating
system, then you must restart the Windows SNMP.

Shutting Down and Starting CLS
To shut down the Interactions Center:
1. On the Interactions Center, double-click the icon in the task bar, and then in the
Interaction Server Service Manager, click Stop NICE CLS.
-or-
Stop the NICE CLS service.
2. Shut down Windows normally.
To start the Interactions Center:

1. Power the computer on and log in via the console or PC Anywhere using the admin account.
This should launch all services, startup apps, the CLS Dispatch process, and all CLS daughter
processes. If auto-login is configured or the CLS is running as a service, even a Windows login
is unnecessary.
2. To start the NICE CLS service, either use the Interactions Server Service Manager or start the
NICE CLS service.
Using the Nicelog Setup Tool

The Nicelog Setup Tool is installed on the Loggers. This enables you to make setting changes.
This tool should only be used in consultation with NICE Customer Support!!!!
To run the Nicelog Setup Tool:

1. In D:\NTLogger\Logger\Utilities, double-click SetupTl.exe. The options vary according to
the boards installed.

Using the Nicelog Setup Tool
Figure 11-14 Nicelog Setup Tool
2. Click GENERAL.
Figure 11-15 GeneralForm Window
3. From the Pcm Mode list, select A_law or U_law, and then click OK.
4. To modify the ETAI settings, click ETAI.

Figure 11-16 ETAI3 Setup Window
5. Make the necessary changes (typically Frame Format) and click OK.
6. Click OTHERS, and select LMOP if you have a High Density Logger, or Alarm Board if you
have NiceCall Focus III or ICU.

• If you edit the configuration files, ensure that you use a plain text editor such as Notepad
without any options. DO NOT use text wrap as this will corrupt your configuration file. In any
event, any change of configuration files should be done ONLY in consultation with NICE
Support!!!!
• In the event of a change in configuration, restart the relevant service.
• Backup browser related errors that are viewed in the System Event Viewer logs may be due to
one of the following reasons:
• Network outages
• Name resolution problems
• An overloaded master browser
• A master browser that is unavailable because, for example, it is shut down or has lost network
connectivity
• Solutions for various additional troubleshooting and performance issues are detailed in the
Channel Mapping Guide:
• Channels Definitions: Logger does not Appear

Other Useful Tools
• Channels Definitions: Input Channels do not Appear

• Source Definitions: Cannot define Virtual Extensions
• Source Definitions: Physical Switch does not Appear
• Dynamic Mapping: Screen Logger does not Appear
• Dynamic Mapping: Virtual Extensions
• Deleting or Changing Logger, Switch, or CLS Definitions
Other Useful Tools

Various hardware diagnostic utilities are available to test Logger hard drives and storage devices
on ExtraNICE > Services > Diagnostic Utilities.
• Perfmon is a Windows built-in tool for monitoring the utilization of operating system
resources, as well as system availability. The Perfmon Counter Logs Creator helps in
automatically creating counter logs for using with the Perfmon tool. See Perfmon Counter
Logs Creator on the Tier 4 Team Site.
Figure 11-17 Perfmon Counter Logs Creator Window

Other Useful Tools
• If you do not have the Perfmon Counter Logs Creator, follow the procedure below to
collect the Performance Monitor logs:
a. From the Start menu, select Run.
b. In the Open field, enter perfmon and click OK.
c. In the new window (on the left hand side), under Console Root > Performance Logs &
Alerts, right-click Counter Logs and select New Log Settings.
d. In the new window, enter in a suitable filename and click OK.
e. In the new window, in the General tab, click Add Objects, and select the Use local
computer counter settings option. In the Performance Objects list, add all the
processes that are relevant to the Logger and click Close.
f. The file is saved in the path specified in the General tab.

12
VoIP Troubleshooting Flow
VoIP.
Contents

Isolating the VoIP Problem ..........................................................................................330
Troubleshooting VoIP Problems .................................................................................331
Troubleshooting the VoIP Logger LAN Connection ..................................................332
Troubleshooting N+1....................................................................................................347
Chapter 12: VoIP Troubleshooting Flow 329

Isolating the VoIP Problem

There are two VoIP recording modes:
• Passive
• Active:
• VRG
• VRA
• CISCO Active
• Avaya DMCC
• Nortel DMS
• Alcatel IPDR

Troubleshooting VoIP Problems
Troubleshooting VoIP Problems

1. If the problem is a system-level problem, that is, there is no recording at all:
• Check that all services are up and running (Applications Server, Interactions Center).
• Check that the VoIP Loggers are up.
• Check the communication between servers and VoIP Loggers.
• Check that there is time synchronization between the various NICE servers.
2. If the problem is a VoIP Logger-level problem:
a. Verify the VoIP Logger configuration in System Administrator, check that the VoIP
Logger was added to the CLS and that it is up (by selecting it and verifying that you see all
the VoIP Logger's details).
b. Troubleshoot the VoIP Logger LAN Connection (see Troubleshooting the VoIP Logger
LAN Connection on page 332):
• Checking the Logger Network Cables on page 332
• Troubleshooting VoIP Logger Connections on page 332
• Troubleshooting Speed Synchronization Failure on page 332
• Verifying the TCP/IP Configuration on page 333
3. Using Logs to Check that VoIP Logger is Configured to Receive RTP Packets
4. Checking the RTP Packets using the Sniffer
5. Troubleshooting Network and Environment Issues
6. Checking the CLS Exceptions
7. Checking Channel Mapping

Troubleshooting the VoIP Logger LAN Connection

This section provides procedures for:
• Checking the Logger Network Cables on page 332
• Troubleshooting VoIP Logger Connections on page 332
• Troubleshooting Speed Synchronization Failure on page 332
• Verifying the TCP/IP Configuration on page 333
Checking the Logger Network Cables
Check that the following cables are connected:

• The network cable for communication with NICE Perform.
• The Sniffing network cable.
Troubleshooting VoIP Logger Connections
If you see that the physical connection between the Logger and the LAN is down (the LED on the
NIC is off), you must check the hardware at the site.
To troubleshoot VoIP Logger connections:

• Check the physical connection between the Logger or NICE Interactions Center and the
switch or hub by checking the following hardware along the path:
• Network adapter
• Cable
• Switch/hub
• LAN outlet (if one exists)
Troubleshooting Speed Synchronization Failure
This procedure enables you to check and configure your speed synchronization. Optimally, the
Logger's speed and duplex should be configured to 100 Full Duplex.
To troubleshoot your speed synchronization:

1. On the desktop, right-click My Network Places. From the drop-down menu, select
Properties.
2. Right-click Local Area Connection and select Properties.
3. Click Configure.
5. Select the Property called Link Speed & Duplex, or Connection Type, or Media Type.

NOTE:
This property name may vary, depending on the LAN chipset.
6. Change the value as follows:
• If your site is using a hub, set the Duplex Mode to 100 Half Duplex or 10 Half Duplex
(depending on the hub at the site).
• If your site is using a network switch, set the Duplex Mode to 100 Full Duplex.
TIP: If the connection still fails, gradually reduce Duplex Mode to 100 Half Duplex, then
10 Full Duplex, then 10 Half Duplex.
7. Verify the connection speed by checking the speed on the switch/hub, or on the Network
Adapter LEDs.
8. If connectivity still fails:
a. Reset the switch/hub port.
b. Check that the switch/hub configuration (speed and Duplex) matches the Network
Interface Card configuration.
9. If you still have a problem, contact NICE Customer Service for further assistance.
Verifying the TCP/IP Configuration
verify the TCP/IP configuration:

• Check that the following TCP/IP properties correspond with your system administrator's site
definitions:
• IP address
• Subnet mask
• Default gateway
• DNS server address
• WINS server address

Using Logs to Check that the VoIP Logger is Configured to Receive RTP
Packets
The VoIP Logger Capture log files are located in D:\NTLogger\VOIPCapture\Logs:
• IPCapture.<#>.log
EXAMPLE:
You have 4 log files. Their names are shown as follows:
IPCapture.1.log, IPCapture.2.log, IPCapture.3.log,IPCapture.4.log
• IPCapture.sav: Contains the first error in the current session, or if there are no errors in the
current session, the first error in the last session when an error occurred.
• IPCapture.crash: Exists when the Logger or Capture process crashes, that is when the
session was not closed.
The VoIP Logger log files are located in:
D:\NTLogger\Logger\Logs
NOTE:
If you open a support case, zip all the log files in the D:\NTLogger\VOIPCapture\Logs
and D:\NTLogger\Logger\Logs folders.
To use logs to check that VoIP Logger is configured to receive RTP packets:
1. Search for the commands that configure the channel for recording.
Figure 12-1 Commands
NOTE:
The Channel number is different from the Logger channel and the channel mapping
number.
Dir=Both indicates recording in both directions - Incoming (Rx) and Outgoing (Tx).
In the log in Figure 12-2 on page 335, Dir=0 indicates Incoming (Rx), and Dir=1
indicates Outgoing (Tx).

2. Verify that the IP address and port is correct.
NOTE:
The port number is important for Gateway-based recording. In the log file, the port
number is the second parameter in the ADDR attribute, after the colon (:).
This is the port

number
3. Check that the channel was configured successfully (per direction requested). The Logger now
starts to sniff for RTP packets from this IP address and port, in the direction specified.
Figure 12-2 Channel Configured Successfully

4. If the VoIP Logger is configured to start sniffing for RTP packets use the sniffer to check the
RTP packets. See Checking the RTP Packets using the Sniffer.
NOTE:
The Interactions Center pauses and resumes channels and for selective recording this
may cause a problem.

Checking the RTP Packets using the Sniffer

For detailed information on how to use the sniffer see Using the Sniffer on page 340.
To check the RTP Packets using the Sniffer:

1. Once the sniffer is connected and running, generate the following information:
• If there is a RTP capture issue on Logger, or Skinny capture issue on Interactions Center,
make some calls from the relevant IP phone/s.
• If there is a RAS capture issue on Interactions Center, wait enough time for "Keep Alive"
messages to be sent from IP phones.
2. Analyze the relevant packets and confirm that the settings are correct:
• Protocol (RTP, RAS and so on)
• IP addresses of source and destination
• Compression (G.711, G.729, G723_6.3, G723_5.3 or G.722)
3. Check for packet loss to see if all the relevant packets have been sniffed and check for packet
delay. If there has been any packet loss or delay, possible reasons may include:
• A problem with any one of the switches between the VoIP Logger and the PBX.
• A hardware problem with the network adapter.
• A problem with the configuration of teaming on the network adapter.
• A problem with the network, environment or switch configuration. See Troubleshooting
Network and Environment Issues on page 338.
4. Check if there is jitter. If there has been jitter, contact the site administrator to check the media
provider configuration.
5. Check if there are multiple instances of the same traffic. Possible reasons may include:
• If the same stream arrives twice, the network configuration may be causing the Logger to
receive the stream from two NICs. Contact the site administrator.
• If there are multiple streams, and the recording is Gateway-based, check if the port for the
Gateway is defined.
• In Active recording, check if two different streams are been sent to the same socket.
• Check if the rx and tx streams are been sent to the same port. This may be a vendor system
problem.
• In passive recording, if the call is between two devices on the same switch that is used for
spanning, there will be duplicate streams. This is not a problem.
6. If the RTP appears to be sniffed correctly but there are still no recordings:
a. Check if the VoIP Logger is overloaded. Check as follows:
• In Task Manager, check for high CPU consumption.

• In Task Manager, check the network utilization.

• Run the Performance Counter and monitor the crucial elements.
Figure 12-3 Performance Counter
b. For selective-recording, there may be a problem with the IP addresses received from the
Interactions Center. See Using Logs to Check that the VoIP Logger is Configured to
Receive RTP Packets on page 334.
c. Check the CLS exceptions. See Checking the CLS Exceptions on page 339.
Troubleshooting Network and Environment Issues

Contact the site administrator in order to troubleshoot network and environment issues.
To troubleshoot network and environment issues:

1. Check the following with the system administrator:
• If there were any destination port range changes.
• If any of the ports were moved.
• If there were any source port changes for the phones or gateways.
• If a switch is down.
• If a switch is in "error disable" mode, indicating a malfunction of the physical port.
• If there were any VLAN configuration changes.

• For active VoIP:

• Check that the NIC IP is routable from the source IP address.
• Verify that the port intervals configured in the Logger IP Tool are open on the firewall
for UDP packets.
2. If audio is not being recorded, the network board may not be sniffing the right ports on the
switch because:
• Check if VoIP Logger can ping all of the relevant components (Gateway, Call Manager
and IP phones).
• Check for switch limitations that prevent sniffing of the relevant components.
• If Conference Calls are not recorded, check if a Conference Resource is in use (Cisco
CCM environments), and verify that it is mirrored.
• Check the switch port mirroring configuration. The switch configuration may not have
been saved before disconnecting from the programming interface, in which case, all the
relevant spanning settings will be lost.
Figure 12-4 Switch Port Mirroring Configuration
Checking the CLS Exceptions

interaction ID). For more details, see the Interaction Data Reference Guide.
To check the CLS Exceptions:

1. Query:
WHERE

2. Check over what period the exceptions occur.

3. Select the vcExceptionDetail field in tblException.
4. Sub-exception 12 20 indicates no VoIP audio. This may be due to:
• A configuration problem. This is dependant on the VoIP integration. Check the
configuration, the forwarding data passed to the Capture (RCM logs), the Capture logs, as
well as the forwarding device configuration. For example, no audio is received if no or the
wrong forwarding information is passed to Capture, or if the audio is not forwarded by the
telephony switch/forwarding device.
• The RCM CheckForAudioDelay parameter specified in the System Administrator
specifies the period, in milliseconds, for RPT sniffing that the RCM waits after starting a
call before checking for audio availability on the Logger. If the RTP takes too long, the
CLS will mark the call as not recorded for total recording, and for interactions-based
recording it will send a stop command to the VoIP Logger. If this occurs try to increase the
duration of this parameter.
5. Exception 38 occurs, for example, when the update is for an unmapped device or a closed
request, or because the update was not received by the RCM.
Checking Channel Mapping

The VoIP Logger may have been configured with the wrong configuration in channel mapping (for
example, IP addresses instead of MAC address). For more information, see Channel Mapping
Troubleshooting.
Using the Sniffer

There are many Sniffer applications available, some of which are freeware. Wireshark (previously
called Ethereal) (www.wireshark.org) is easy to install and a convenient tool to use.
To use the sniffer:

1. Run the sniffing application on a PC connected to the port on the switch where the sniffing
should be performed, that is, either:
• On the VoIP Logger if the Logger software is not running.
• A different PC connected to the switch port.
2. From the Capture menu, select Interfaces.

Figure 12-5 Capture Interfaces
3. To capture all the traffic on the network adapter, click Start next to the required network
adapter to start capturing.
4. To filter while capturing:
a. In the Wireshark Capture Interfaces window, click Options next to the required network
adapter.
b. Check that the IP address is correct.
Figure 12-6 Wireshark Capture Options
c. Ensure that Capture packets in promiscuous mode is selected.

d. (Optional) In Capture Filter, type host x.x.x.x, where x.x.x.x is the required IP address.
e. In the Capture File(s) area:
• Click Browse and specify a folder and file name. Verify that the drive has sufficient
space available, for example 2 GB for 20 files of 100 MB.
• Select Use multiple files.

• Select Next file every 100 megabytes.

• Select Ring buffer with 20 files.
f. Click Start.
g. Make a test call.
5. From the Capture option, select Stop to stop capturing.
6. If the files are too small or too few to cover the required period, extend the number and size of
the capture files.
7. To filter an offline capture file for RTP packets:
a. From the File menu, select Open to load a capture file (*.pcap).
b. Right-click on the list of packets and select Decode As.
Figure 12-7 Wireshark Decode As
c. Select Decode.
d. In the Transport tab, select UDP both port(s) as RTP.
e. Click OK.

Figure 12-8 Wireshark Offline Capture Filter
8. To analyze a stream:
a. From the Statistics menu, select RTP > Show All Streams.
Figure 12-9 Wireshark RTP Streams
b. Check for packet loss in the Lost field.

c. For more detailed information, select the relevant stream and click Analyze.

Figure 12-10 Stream Analysis
9. To save the voice (G.711) for playback in a third-party player, click Save payload. This
enables you to check whether voice quality issues are related to the network or to the NICE
system.
10. Check for jitter.
11. It may sometimes be necessary to manually check a softphone stream (Wireshark may not
report the correct number of expected packets). Typically in this case the playback will not be
clear. This requires you to calculate the expected number of packets for the duration of the call
and compare it to the actual number of packets received.
Figure 12-11 Softphone Stream

a. Note the start time of the call, for example, 12.03.45.

b. Note the end time of the call, for example, 12.04.02.
c. From the start and end times, calculate the duration of the call.
d. Check the size of the UDP packet (User Datagram Protocol), for example, 180 bytes.
e. Calculate the size of the payload: UDP Packet Size in Bytes - 8 Byte Header - 12 Byte
RTP Header)/2, that is, 180-8-12=160 bytes.
f. Check the codec size/msec.
Codec Codec Size/msec
G.711/G.722 80 bytes per 10 msec
G.729 10 bytes per 10 msec
G.723_6.3 24 bytes per 30 msec
G.723_5.3 20 bytes per 30 msec
g. Calculate the duration of the RTP packet, that is, Payload / (Codec Size/msec).
See the following examples:
Codec Payload Duration
G.711/G.722 160 bytes/(80 bytes/10 msec)=20 msec
G.729 20 bytes/(10 bytes/10 msec)=20 msec
G.723_6.3 24 bytes/(24 bytes/30 msec)= 30 msec
G.723_5.3 20 bytes/(20 bytes/30 msec)=30 msec
h. Calculate the number of packets per second: 1000/Duration of the RTP packet, for
example, 1000/20=50 packets per second.
See the following examples:
Codec Packets per Second
G.711/G.722 1000/20 msec=50 packets per second
G.729 1000/20 msec =50 packets per second
G.723_6.3 1000/30 msec=33.3 packets per second
G.723_5.3 1000/30 msec=33.3 packets per second

i. Calculate the expected number of packets: Duration of the Call in seconds x Number
of Packets per second, for example, 13x50=650 packets.
j. If there exists a deviation of 50 packets per minute or greater between the number of
expected and actual packets, this could be due to a problem in the media provider, for
example, the phone, PBX, etc.
EXAMPLE:
A USB headset can cause such a deviation, since soft phones tend to generate a time drift.
If, after checking the media provider, there still exists a deviation of 50 packets per minute
or greater between the number of expected and actual packets, open a case with NICE
support.

Troubleshooting N+1
Troubleshooting N+1
When a normal Logger goes down and the spare Logger takes over the sequence of events can be
viewed in the log files.
Spare Logger
The spare Logger determines that something happened to the normal Logger and notes that the
normal Logger failed to report heartbeats.
27/07/2008 12:31:42.507 WARN: AUD_NPlus1SW_CheckNormalLoggers: Logger
failed to report heartbeats. <LoggerHostID,0> 1 0 [ThreadID=3952]
The spare Logger starts the switching process.
27/07/2008 12:31:42.507 INFO: SpareLoggerThreadProc: spare logger is in the
process of switching state. <OverridingLoggerID,0> 1 0 [ThreadID=3952]
The spare Logger is going to take over the function of Logger number <X>, where <X>is the
position of the normal Logger in the N+1 chain, and not the Logger number in System
Administrator.
27/07/2008 12:31:42.507 INFO: EVTA1020 Nplus1: Spare logger identified Normal
switching from logger no. 1 [ThreadID=3952]
27/07/2008 12:31:42.507 INFO: AUD_SetQuietBuffsToSwitchFromFax got num
of buffers 16 0 [ThreadID=3952]
The current configuration on the spare Logger is queried.
27/07/2008 12:31:42.507 INFO: VoIPDrvr.VoIP_GetChannelConfiguration :
Log Channel : 1 0 [ThreadID=3952]
The channels on the spare Logger are configured according to the masc view of the normal
Logger.
27/07/2008 12:31:43.007 INFO: SetVoipDMascConnections: Finished
configuring all channels. 0 0 [ThreadID=3952]
The spare Logger is now overriding Logger number <X>, and the switching process is over.
27/07/2008 12:31:43.007 INFO: EVTA1021 Nplus1: Spare logger is
overriding logger no. 1 - 5629 [ThreadID=3952]
The spare Logger is emulating the normal Logger, but with all channels started.
27/07/2008 12:31:43.007 INFO: Spare starts recording audio of logger : 1
5629 [ThreadID=3952]
The channels that should not be recording are stopped. Note that this works as designed, and that
no buffers were been recorded for the channels that are stopped.
27/07/2008 12:31:43.007 WARN: AudMgr Nplus1StopRecord stopping the
channels. <Status,0> 0 0 [ThreadID=3952]

Troubleshooting N+1
NOTE:
In the event of a brutal failure in Active VoIP environments, the spare Logger may need
resources that are still in use by the normal Logger, for example, in Avaya DMCC the
Virtual Extensions may still view themselves as in use by the normal Logger. These
resources are released after a timeout period. After this short period the channels that
could not be configured initially will be recovered.
Normal Logger
An error is reported, for example, after the sniffing network card is disconnected. This error
appears both in the Logger log file, as well as the Capture log file. The time to report this error is
now only 10 seconds (in previous versions it might take up to 2 minutes).
27/07/2008 12:32:12.291 ERR: NormalLoggerThreadProc: Received a capture
critical error event 0 0 [ThreadID=4452]
The normal Logger tries to stop the channels and release the resources, and the spare Logger starts
to take over immediately.
27/07/2008 12:32:12.291 INFO: EVTA1010 Nplus1: Normal logger switching
audio to spare due to Manual Switch [ThreadID=4452]
27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc Audio has
been switched 0 0 [ThreadID=4452]
27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc perform
switching to spare 0 0 [ThreadID=4452]
27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc stopping the
channels 0 0 [ThreadID=4452]
27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc
ExternalCapture!! 0 0 [ThreadID=4452]
27/07/2008 12:32:12.291 INFO: VoIPDrvr.VoIP_StopRecord : Log Channel : 1
0 [ThreadID=4452]
27/07/2008 12:32:12.291 INFO: EVTA0101 Record Stop On Channel [1,1]
[ThreadID=4452]
27/07/2008 12:32:12.291 WARN: AudMgr Nplus1StopRecord stopping the
channels. <Status,0> 0 0 [ThreadID=4452]
27/07/2008 12:32:12.978 INFO: EVTA0751 AUM_AudioImport stop on input
channel 1 with token number 1 [ThreadID=4520]
27/07/2008 12:32:13.025 INFO: DMA: DM_RecordProcessOnChannel- Stop
record 1 0 [ThreadID=4424]
IMPORTANT
After the normal Logger resumes its normal functioning, it MUST be restarted. The spare Logger
does not need to be restarted.


mascm
The VoIP Logger saves the configuration received from the CLS in the mascmap file. The VoIP
Logger reconfigures itself on restart from this file. If there is a discrepancy between the CLS and
the Logger, the CLS sends a request to the Logger to update the mascmap. There is therefore no
need to delete this file.
Configuring after Replacing a NIC

If you replace a NIC, run the IP Tool (D:\NTLogger\VoIPCapture\Bin) and reconfigure the NIC
properties.
Replacing a CISCO Phone

The Device Unique Identifier (DUI) is built into the CISCO physical phone. If you replace such a
phone, the channel mapping must be updated as this affects the mascmap. See the VoIP Logger
Installation Guide.
Troubleshooting a Packet Size Problem

There may be a problem with the maximum packet size if the following error appears in the
Capture logs:
PacketHandler::HandleReceivingCompleted. Completion operation failed.
ErrorCode=14040
This indicates that the message is too long, that is, a message sent on a datagram socket was larger
than the internal message buffer or some other network limit, or the buffer used to receive a
datagram was smaller than the datagram itself.
This may occur with Avaya DMCC integrations. The MaxPacketSize parameter is set to 320 by
default. Avaya DMCC may use a packet duration of 60 msec. In G.711 this means a packet size of
480 bytes without the header, that is, over 500 bytes per packet.
To troubleshoot a packet size problem:

• In the Registry, change the MaxPacketSize to 512.

Figure 12-12 change the MaxPacketSize
Troubleshooting RTP Timestamp

Depending on the environment, the RTP packets should be transmitted every x milliseconds. You
can use Wireshark to check that the RTP packets are in fact been transmitted and sniffed every x
milliseconds. For more information, see Using the Sniffer on page 340.
An increase of 8 in the timestamp is equal to one millisecond.
Example 1:
Figure 12-13 Check the RTP Packets
In this example if the phone uses 30 millisecond packets the timestamp should be incremented by
8 x 30 =240. Instead the timestamp is incremented by only 8, indicating a problem. This causes the
Capture to mishandle the packets.
Example 2:
After about a minute a call starts to drift and the IPCapture generates errors and starts beeping. In
this case the problem was caused by a specific soft phone with USB headset.

1. Open the trace file.

2. Decode the IP address packets to RTP.
3. In the Filter field, filter the calls for the source IP address by typing in:
ip.src = = <X>.<X>.<X>.<X> and rtp
4. The first few packets are usually the ring tone and not related to the voice RTP stream.
5. The packets that follow are the voice RTP stream. In this example, the first was captured at
10:22:06.625 and the last one was captures at 10:24:16:107. This gives a stream duration of
00:02:09.50 hours, that is, 129.5 seconds.
6. The timestamp of the first packet is 0 and the timestamp of the last packet is 1048960. In
8KHz packets this translates to 131.12 seconds ((1048960-0)/8000).
7. There is more than a second between these two calculation, that is, the phone sends 131.12
seconds of voice in 129.5 seconds.
Troubleshooting RDP Delay

When connecting to a machine using RDP (Remote Desktop Protocol), a serious delay could occur
both on the client side (console “freeze”) and on the server side (processes hang). As a result,
crucial processes (for example, IP Capture process on the VoIP Logger) face unexpected situations
due to not receiving the necessary resources.
Cause
There are several different parameters and system configurations that could cause such delays.
Some are related to the network configuration and others to the RDP version installed on either the
client or the server.
1. The order of Network Providers in the registry could be incorrect. The key to be checked is
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\NetworkProvider\Ord
er. The default order is "RDPNP,LanmanWorkstation,WebClient". Note that in Vista clients
(prior to SP1) there is a bug and the order is not as stated above (when installing Vista SP1
from a clean installation this bug is fixed). RDPNP should be the first provider in the list
(before the LanmanWorkstation).
2. In RDP version 6.0, a network query is launched during the connection attempt (which does
not happen in RDP version 5.x). If a WINS server is configured on the client, and this server
does not respond, the query will time out. As a result, the server waits for a response from the
client and this causes the symptoms as described above.
Solution
In general, the Remote Desktop Connection could affect both the client and the server, causing
delays and process starvation.
Verify the following to solve the delays:
1. The registry key stated above
(HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\NetworkProvider\Ord
er) is set to the default order ("RDPNP,LanmanWorkstation,WebClient").

2. If using RDP version 6.0 make sure all network configurations (for example, WINS servers)
are available.
Troubleshooting the “Chain Not Found” Error Message
NOTE: For more in depth information on normal and spare Loggers in a chain, see
“Resiliency and N+1” in the System Administrator’s Guide.
The “Chain not found” error message appears when the normal Loggers in the chain do not have
the same multicast IP address as the spare Logger.
Each chain has a unique multicast IP address, and all the Loggers in the chain (spare and normal)
share this same multicast IP address.
To troubleshoot the “Chain not found” error message:

1. In the System Administrator, navigate to Master Site > Logger Servers > NiceLog, and
check that the normal Loggers and the spare Logger are all added to the system.
2. Create a new chain with these Loggers. See “Resiliency and N+1” in the System
Administrator’s Guide.
3. If the error message still appears, complete the following:
a. Stop the NICE VoIP Logger Service.
b. Navigate to D:\NTLogger\Logger\Testers\LoggerUtils, and double-click
SWNPlus1Configurator.exe.
The Software N+1 Configuration window appears.

Figure 12-14 Software N+1 Configuration Window
c. In the Multicast Group IP field, enter the multicast IP address for the chain. This IP
address is the same for each Logger in the chain.
d. In the Chain ID field, enter a Chain ID for the Logger.
TIP: Note the Chain ID for each Logger so that you can reconstruct these IDs when you
recreate the chain in Step 4 on page 353.
e. Click Save and Close.
f. Restart the NICE VoIP Logger Service.
g. Repeat Step a through Step f for each Logger in the chain. When repeating Step d, make
sure that you enter the Chain ID correctly. The Chain ID of each normal Logger increases
by 1 for each additional Logger in the chain. The spare Logger is the last Logger in the
chain and has the highest Chain ID.
4. Create a new chain with the Loggers you configured in Step 3 on page 352. See “Resiliency
and N+1” in the System Administrator’s Guide.
Troubleshooting Active VoIP Integrations

See the following guides:
• VoIP Recording Agent (VRA) Installation Guide.
• VoIP Recording Gateway Installation Guide
• Avaya DMCC Integration Guide.

• Cisco IP Phone-based Active Recording Integration Guide.

• Integration with Nortel CS1000 and Duplicate Media Stream over IP Guide.
• Integration with Alcatel-Lucent OXE Including IP DR-Link Guide.

13
Interactions Center Troubleshooting Flow
the NICE Interactions Center.
Contents
Interactions Center Troubleshooting Workflow.........................................................356

Open Case Procedure ..................................................................................................366
Checking the SNMP Traps ...........................................................................................374
Setting the Reporting Level .........................................................................................375
Checking the CLS Configuration ................................................................................377
Starting/Stopping the Interactions Center Services..................................................378
Checking the Status of Interactions Center Components ........................................379
Troubleshooting using the Interactions Center Logs ...............................................381
Examples of Troubleshooting using the Logs...........................................................383
Checking Ports .............................................................................................................408
Checking the CLS Exceptions.....................................................................................410
Fine-tuning the CLS Server Configuration Parameters ............................................ 411
Isolating a Screen Problem Further ............................................................................412
Shutting Down and Starting CLS ................................................................................413
Other Useful Tools........................................................................................................415
Appendix 13: Interactions Center Troubleshooting Flow 355

Interactions Center Troubleshooting Workflow
Interactions Center Troubleshooting Workflow

If you experience a problem with the Interactions Center follow the steps below to isolate the
problem.
Figure 13-1 Troubleshooting Workflow

Interactions-based Call Recording Data Flow

Figure 13-2 shows an interactions-based call flow, starting from the point where a call starts, and
details how the information appears in the log files. This enables you to monitor the call flow, and
troubleshoot the problems that may arise during the recording process, up to the point when the
call is inserted into the database. See Interactions-based Call Recording Data Flow Details
on page 358 for a description of every step mentioned in Figure 13-2.
Figure 13-2 Interactions Center Call Flow

Interactions-based Call Recording Data Flow Details

1. The CTI Server sends a CTI event to the Driver.
2. The Driver processes the event.
3. The Driver sends a Start Call event to the Call Server.
4. The Call Server checks and processes the Start Call event:
a. The Call Server verifies that the agent ID or extension is recognized and exists. This check
is done by accessing an internal login table (Calls_LT) which holds all agent login
information.
b. The RecordingDecisions checks if the new calls match a recording request or a
QA/Negative/Selective rule.
c. The Call Server updates an internal table (Calls_CT) with the call details, that is, with the
start time, agent details, extension number, and so on.
5. The Call Server sends a Start Record event request (including all participants) to the RCM
and per participant.
6. The RCM returns channel allocation information to the Call Server (Synchronous Answer)
including the Logger #, Logger channel and time information.
7. The RCM sends the Logger a Start Recording command.
8. The Call Server updates Calls_CT with the recording details, including the Logger #, Logger
channel, Logger start time, and so on.
9. The Call Server sends a Start Call event to the Client, for example, Monitor (the client must
be registered for these type of events according to the Agent or Station and switch).
10. The Logger records the call.
11. The Logger returns a Recording Succeeded/Failed event to the RCM.
12. The RCM returns an Asynchronous Response to the Call Server. In interactions-based
recording, the real call start time is returned here. The call start time is the time that RCM
received a response from the Logger that it has started recording, normalized to the Logger
local time.
13. The RCM sends an Audio Check command to the Logger 2.5 seconds (configurable) after the
Asynchronous Response is sent to the Call Server.
14. The Call Server sends a Start Record event to the Client, for example, Monitor (the client
must be registered for these type of events according to the Agent or Station and switch).
15. For VoIP configurations only, the RCM waits a configured amount of time and performs an
audio check with the VoIP Logger. If the Logger has not received any audio packets from the
beginning of the call, an asynchronous Record Failure event is sent to the Call Server.
16. If one of the media recordings fail:
a. The RCM reports a Record Failure event to the Call Server.
b. The Call Server asks RecordingDecisions if the second media should be stopped.

c. The RecordingDecisions answers the Call Server as to whether the second media should
be stopped.
d. The Call Server sends a Stop Record request to the RCM for the second media.
18. The Driver sends an End Call event to the Call Server.
19. The Call Server checks if the calls are recorded in its internal database.
20. The Call Server sends a Stop Record event request to the RCM.
21. The RCM returns the stop request information with the recording status of the call to the Call
Server.
22. The RCM sends a Stop Record event to the Logger.
23. The Logger stops recording the call.
24. The Call Server updates the call details including the stop time and removes this call from
Calls_CT.
25. The Call Server sends an Insert Interaction event request to the DBServer (this is based on
having a Start and Stop message for the same call Interaction ID).
26. The Call Server sends an Insert Contact request to the DBServer including data about nested
interactions.
27. The Call Server sends a Stop Call (with recording information) event to the Client.
28. The DBServer inserts the interaction into the SQL Database on the Database Server.

Understanding the Total Call Recording Data Flow

Figure 13-3 shows a total call recording Data flow, starting from the point where a call starts, and
details how the information appears in the log files. This enables you to monitor the call flow, and
troubleshoot the problems that may arise during the recording process, up to the point when the
call is inserted into the database. See Understanding the Total Call Recording Data Flow
Details on page 361 for a description of every step mentioned in Figure 13-3.
Figure 13-3 Total Call Recording Data Flow

Understanding the Total Call Recording Data Flow Details

information.
per media (voice and screen) and per participant.
including the Logger #, Logger channel and time information. In total-recording
environments, the call recording start time returned is the start time reported by the Driver,
normalized to the Logger local time. This is because the Logger is already recording at the
time the call starts.
11. The RCM returns an Asynchronous Response to the Call Server.
13. The Call Server sends a Start Record event to the Client, for example, Monitor (the client must
15. If one of the media recording fails:
a. The RCM reports a Record Failure request to the Call Server.
b. The Call Server asks RecordingDecisions if the second media should be stopped.
c. The RecordingDecisions answers the Call Server as to whether the second media should
be stopped.

d. The Call Server sends a Stop Record request to the RCM for the second media.
Server.
Calls_CT.
interactions.

Understanding the Screen Call Recording Data Flow

Figure 13-4 shows a screen call recording Data flow, starting from the point where a call starts,
and details how the information appears in the log files. This enables you to monitor the call flow,
and troubleshoot the problems that may arise during the recording process, up to the point when
the call is inserted into the database. See Interactions-based Call Recording Data Flow Details
on page 358 for a description of every step mentioned in Figure 13-4.
Figure 13-4 Screen Call Recording Data Flow
Screen Call Recording Flow Details

information.

for voice and per participant.
including the Logger #, Logger channel and time information.
7. The RCM sends the Logger a Start Recording command (voice).
9. Screen: The Call Server sends a Start Record event request to the RCM for screen media.
10. Screen: The RCM returns channel allocation information to the Call Server (Synchronous
Answer).
11. Screen: The RCM calls the Capture Control DLL which calls the ScreenAgent.
12. Screen: The ScreenAgent starts recording.
16. The RCM returns an Asynchronous Response to the Call Server. In interactions-based
recording, the real call start time is returned here. The call start time is the time that RCM
received a response from the Logger that it has started recording, normalized to the Logger
local time.
18. Screen: The ScreenAgent returns an Asynchronous Response to the RCM.
19. Screen: The RCM returns an Asynchronous Response to the Call Server.
20. The Call Server sends a Start Record event to the Client, for example, Monitor (the client
must be registered for these type of events according to the Agent or Station and switch).
22. Screen: If there is a problem, the ScreenAgent returns an Asynchronous Recording Error
to the RCM.
23. Screen: If the screen recording fails:
a. The RCM reports a Record Failure request to the Call Server.
b. The Call Server asks RecordingDecisions if the voice should be stopped.
c. The RecordingDecisions answers the Call Server as to whether the voice should be
stopped.

d. The Call Server sends a Stop Record request to the RCM for the voice.
Server.
29. The RCM sends a Stop Record event to the Logger (voice and screen).
31. Screen: The Call Server sends a Stop Record request to the RCM for screen recording.
32. Screen: The RCM sends a Stop Record request to the ScreenAgent.
NOTE:
Screen: If a screen wrap-up time is defined, that is, the total amount of time (in seconds)
it takes for the agent to wrap-up the interaction, the RCM waits for this period before
sending the request.
33. Screen: The ScreenAgent returns an Asynchronous Response to the RCM.
Calls_CT.
NOTE:
Screen: For screen recording with a wrap-up time defined, the Call Server does NOT
wait before sending the Insert Interaction event, it simply uses the wrap-up time to
calculate the call stop time.
This means that if a screen recording problem occurs during the wrap-up time, there will
be no indication of the problem in the database and no exceptions will be thrown. You
will simply not be able to playback the screen recording to the end.
If screen is still recording in the wrap-up time, and the agent answers a new call, the
screen wrap-up recording will be stopped. The Call Server will update the stop time in
the database with the actual wrap-up time.
interactions.

Open Case Procedure

This section tells you what information to provide if you need to send a problem to customer
support at NICE for the following:
• RecordingDecisions on page 366
• Call Server on page 367
• DB Server on page 368
• TRS on page 369
• RCM on page 369
• System Administrator Parameters (CLS Plug-ins) on page 371
RecordingDecisions
Data to be provided in RecordingDecisions case reports, when contacting customer support:
• Exact Interactions Center release and service pack
• Number of Interactions Centers on sites
• Any special site configuration, for example e.g., Multi Data Hubs, cluster
• Total/QA
• Number of rules configured
• Number of users on site
• Relevant rules defined
• Actual behavior
• Actions taken

Open Case Procedure
Extension number, Agent number).
• Complete log files, in debug level (located on the computer where the Interactions Center sits,
under NICE Systems > Interactions Center > Log)
• NiceCls.RecordingDecisions.*.log.*.txt
• Provide the following database tables in CSV format:
• nice_rule
• tblSchedule
• tblScheduleActivation
• tblRERuleondition
• tblReRuleConditionItem
• Sched_Programs_Recording_Count
• nc_general
Call Server
Data to be provided in Call Server case reports, when contacting customer support:
• Total/QA
• Actual behavior
• Actions taken

Open Case Procedure
• Provide database query results, regarding problematic time frame or problematic interaction
ID/contact ID, from the following database tables:
• tblInteraction
• tblCallInteraction
• tblContact
• tblContactInteraction
• tblParticipant
• tblRecordign
• tblExceptionXX
DB Server
Data to be provided in DB Server case reports, when contacting customer support:
• Total/QA
• Actual behavior
• Actions taken
• Provide the number and size of persistent queue files.
• DB Server.log.*.txt

Open Case Procedure
• In the case of an SQL time-out, provide a list of running times for all SQL jobs on the
Interaction database.
TRS
Data to be provided in TRS case reports, when contacting customer support:
• Total/QA
• Actual behavior
• Actions taken
• TRS.log.*.txt
• Provide the exact time of the recording that should or should not be inserted by the TRS.
• Provide the tblRecordingXX database query results, regarding the problematic time frame.
RCM
Data to be provided in RCM case reports, when contacting customer support:
• Exact Interactions Center release, service pack and NICE Updates installed.
• Details of all Loggers configured to each Interactions Center: version, capture type,
number of channels, recording method, GMT and local time of the Logger.
• Mapping details for each Logger attached to the Interactions Center:
• Total/Interaction Based

Open Case Procedure
• Recording method of Logger channels (for example, Passive VoIP, DMCC, Nortel
DMS, Digital Matrix)
• Type of Sources and their number
• Static/Dynamic Mapping
• Connection between Logger and Interactions Center (LAN/WAN). If WAN, please
indicate Bandwidth and Latency.
• N+1 chain details (when relevant)
• Details from CTI plug-in on the drivers and the type of interfaces (for example, Avaya TSAP,
Genesys, CISCO ICM etc).
• Site's SBC / Gateway topology (when relevant).
• Number of ScreenAgents reporting to the Interactions Center (when relevant)
• Call Rate on the Interactions Center Machine
• Work hours of agents mapped to the Interactions Center, including peak hours (for example,
hours of shifts, 24/7)
• Actual behavior
• Actions taken
• Provide specific examples and call details (for example, Interaction ID, Segment ID, extension
number, Agent number, time of the call, relevant DB Records for the call).
• If the issue occurs in specific call scenarios, describe the scenario.
• Does the issue occur in a specific type of call? (for example, only internal calls)
IMPORTANT
Before sending log files for RCM, trigger RCM to dump the full internal state to the log file the
logs, by running the RCM Information Dumper. See Running the RCM Information Dumper
on page 376.
under NICE Systems > Interactions Center > Log):
• RCM.log.*.txt
• Complete log files, in debug level, of drivers attached to the Interactions Center.
• Complete log files of Loggers (including capture logs) attached to the Interactions Center.
• All RCM configuration backup files found under NICE Systems > Interactions Center >
Loggers

Open Case Procedure
• Files found under NICE Systems > Interactions Center -> bin
• RCM.log.Persistency.soap
• RCM.exe.config
• Provide the following database tables in CSV format (query from before the time the problem
occurred, preferably overlapping with the time in the provided logs):
• tblInteraction
• tblRecording
• tblParticipant
• tblException
• nice_admin
• Channel mapping tables
• The following Performance Counters of the RCM.Exe process:
• Private Bytes
• Threads
• Handles
• % CPU
• All Performance Counters under the .Net Memory Category.
System Administrator Parameters (CLS Plug-ins)

Data to be provided in Interactions Center Parameters (CLS Plug-ins) case reports, when
contacting customer support:
• Exact Interactions Center release, service pack, and installed Updates
• Any special site configuration, for example , Multi Data Hubs
• Actual behavior
• Actions taken

Open Case Procedure
• Is the problem related to a specific configuration (such as a specific integration or

mapping type) or does it effect all configurations?
• Does the problem reoccur after restarting the System Administrator or browser?
• Is there a work around?
• Are there also problems with additional parameters, such as Integrations parameters,
Loggers parameters, DB Server parameters.
• Provide specific examples and details
• Complete log files, in debug level from the server (located on the computer where the
Interactions Center sits, under NICE Systems > Applications> ServerBin)
• NiceApplications.CLSPlugin.log.*.txt
• A backup copy of the nice_admin database (located on the computer where the database is
installed)
• Provide a screenshot of an error if one is displayed.
• Provide Performance Counters of SysAdmin process:
• Private Bytes
• Threads
• Handles
• % CPU
• All performance counters under the .Net Memory Category.
• In case of Out Of Memory issues, please provide dump of SysAdmin process.
• In case of Online Configuration Update issues, please also provide full Interactions Center
Logs.
• Provide any other relevant data, for example, memory usage or screen shots.

Checking the Basics
Checking the Basics

For Interactions Center problems check the following:
1. View the SNMP traps. See Checking the SNMP Traps on page 374.
2. Set the reporting File level to Debug. See Setting the Reporting Level on page 375.
3. Collect all the logs and other relevant files. See the Troubleshooting Guide, Collecting
Information Using the Log Collector.
4. Check the CLS configuration. See Checking the CLS Configuration on page 377.
5. If many calls are not recorded, restart the RCM process. See Starting/Stopping the
Interactions Center Services on page 378.
6. If connectivity or power was lost, or if restarting the RCM process does not solve the problem,
restart the NICE CLS services. See Starting/Stopping the Interactions Center Services
on page 378.
7. Troubleshoot using the Interactions Center logs. See Troubleshooting using the Interactions
Center Logs on page 381 and Troubleshooting using the Interactions Center Logs
on page 381.
8. Check the ports. See Checking Ports on page 408.
9. Check the CLS exceptions. See Checking the CLS Exceptions on page 410.
10. Fine-tuning the CLS Server configuration parameters. See Fine-tuning the CLS Server
Configuration Parameters on page 411.
NOTE: For NiceScreen Logger and ScreenAgent problems, see the NiceScreen


In the SNMP Console, view the events (polling failure messages and the SNMP traps) generated
by the monitored servers.
See the NMS User's Guide for a list of the trap messages and recommended actions.

Setting the Reporting Level
Setting the Reporting Level

Set the File reporting level to Debug for the various modules and services.
To set the reporting level:

1. In System Administrator verify that you are working in Technician mode: From the Settings
menu, select Technician Mode.
2. In the Organization tree, select CLS Definitions > CLS Server. Click the Report Level tab.
3. Expand the required component and set the reporting level.
NOTE:
Currently the Screen reporting level category is not available, although it appears on the
screen. Changing this field has no effect on the system.
4. Set the File to Debug.

Running the RCM Information Dumper
Running the RCM Information Dumper

The RCM Information Dumper triggers the RCM to dump a full internal state to the log file. You
must run the RCM Information Dumper, before sending RCM log files to NICE Customer
Support.
To run the RCM Information Dumper:

1. Navigate to NICE Systems\Interactions Center\Tools\RCMInformationDumper.exe
2. Double-click RCMInformationDumper.exe.
The RCM Information Dumper window appears.
Figure 13-5 RCM Information Dumper
3. Select Resources Dump, and click DUMP.

4. Select Requests Manager Dump, and click DUMP.

Checking the CLS Configuration
Checking the CLS Configuration

Verify that the CLS Servers are correctly defined in the System Administrator. For more details,
see the System Administrator's Guide.
To check the configuration:

1. In System Administrator, verify that you are working in Technician mode: In the Organization
tree, click Organization. Then select Technician Mode and click Save.
2. In the Organization tree, select CLS Definitions > CLS Server. Verify that the Interactions
Centers are correctly defined.

Starting/Stopping the Interactions Center Services
Starting/Stopping the Interactions Center Services

The following services run on the Interactions Center:
• NICE CLS
• NICE CLS Monitor
• NICE CLS TRS
To Start/Stop the NICE CLS Service:

1. Start/stop the NICE CLS service from Services.
-or-
2. From the Start menu, select Programs> Nice Systems > Interactions Center > Tools >
Nice Services Configuration Manager.
The NICE Services Configuration Manager window appears.
Figure 13-6 NICE Services Configuration Manager Window
3. Select the required service and click to start the service, or click to stop the service.

Checking the Status of Interactions Center Components

You can use the NICE Services Configuration Manager utility to monitor the status of the services
run on the Interactions Center. In addition, you should check the status of processes related to the
NICE CLS component which can be viewed via the Windows Task Manager.
Checking Components with the NICE Services Configuration Manager
NOTE: For all-in-one sites, before proceeding ensure that the NICE Applications
Suite’s instance of the NICE Services Configuration Manager tool is not already
running.
To check the components status with the NICE Services Configuration Manager:
• From the Start menu, select Programs> Nice Systems > Interactions Center > Tools >
Under the State column the status should state Running for the following:
• NICE CLS
• NICE CLS Monitor
• NICE CLS TRS

Checking Processes Status

Figure 13-8 Windows Task Manager
To check the NICE CLS processes status:

1. Open the Windows Task Manager.
2. Click the Processes tab.
3. Ensure that the following processes appear:
• CallsrvrHost.exe
• DBSrvr.exe
• DISPATCH.exe
• NiceCls.RecordingDecisions.Scheduler.exe
• RCM.exe
4. Check for information, such as excessive CPU usage, excessive memory usage, and memory
leaks.

Troubleshooting using the Interactions Center Logs

Each component writes events (based on the selected reporting level) into its own log file. Log
files are located in D:\Program files\NICE Systems\Interactions Center\Log.
Also see Examples of Troubleshooting using the Logs on page 383.
NOTE: The Technical CLS API Programmer's Reference Guide may be useful when
analyzing the log files.
Figure 13-9 Interactions Center Log Files
There are log files for:

• Call Server - Call Server.log.xxx.txt files
• CLS Monitor - CLS Monitor.log.xxx.txt files
• CLS Manager - CLSManager.log.xxx.txt files
• DBServer - DBServer.log.xxx.txt files
• Dispatch - Dispatch.log.xxx.txt files
• RCM - RCM.log.xxx.txt files
• NiceCls.RecordingDecisions.Scheduler -
NiceCls.RecordingDecisions.Scheduler.log.xxx.txt files
• Snmp - snmp.log.xxx.txt files
• TRS - TRS.log.xxx.txt files
To troubleshoot using the logs:

1. Check the logs. From the logs it can be seen that an attempt was been made to record both
voice and screen using a rule (CLS_INITIATOR_QA (16)). The Call Server sent RCM two
requests, one to record voice and one to record screen, but both failed.

2. Check the rules. If necessary delete and recreate the Scheduler rules.
3. The Logger is not mapped correctly (switch 2, trunk 202, trunk group 3), and therefore the
voice recording fails.
4. Check the mapping.

Examples of Troubleshooting using the Logs

Example 1
For example, exception exp016 is reported for some of the calls which were not recorded. The
following logs appear:
Request to record voice:
01/08/2006 11:34:08.500 DEBUG [7928]: RCM Request data :
RecordStartRequest: requestID=110331462; interactionID=110331462;
callStartTimeInTicks=632900090480000000; clsDriverTimeDiff...=0; callStartTime=8/1/2006
6:04:08 AM; previousCallID=0; PBXCallID=14678; switchID=2; urid=0; type=33620224
(02010100); timeReceived=632900288485000000; timeUpdatedCounter=0;
RequestInternalParticipant: deviceID=0; deviceType=0; station=38053; agentID=39268;
type=REQUEST_INTERNAL_PARTICIPANT (02010000);
recordingFlags=CHANS_REC_FLAG_FULL_VOICE (4); chosenInitiator=0 (0);
timeDifference=0; recordingStartTim...=0; recordingStartTime=1/1/0001 12:00:00 AM;
recordingStopTime...=0; recordingStopTime=1/1/0001 12:00:00 AM; recordingChannel=0;
recordingLogger=0; requestInitiators=CLS_INITIATOR_QA (16);
requestMediaType=VOICE_MEDIA (2); requestWrapUpTime=0; requestWrapUpTime...=0;
summationType=SUMMATION_TYPE_SUMMED (3); participantID=1; ipPort=0;
uniqueDeviceID=38053;
ExternalParticipant: trunkNumber=202; trunkGroup=3; type=EXTERNAL_PARTICIPANT
(03000000); recordingFlags=CHANS_REC_FLAG_FULL_VOICE (4); chosenInitiator=0 (0);
requestMediaType=VOICE_MEDIA (2); requestWrapUpTime=0; requestWrapUpTime...=0;
summationType=SUMMATION_TYPE_SUMMED (3); participantID=2; ipPort=0;
(LogWriter.LOGWrite2OldLog)
Request to record screen:
01/08/2006 11:34:08.531 DEBUG [7928]: RCM Request data :
RecordStartRequest: requestID=109733342; interactionID=110331462;
callStartTimeInTicks=632900090480000000; clsDriverTimeDiff...=0; callStartTime=8/1/2006
6:04:08 AM; previousCallID=0; PBXCallID=14678; switchID=2; urid=0; type=33620224
(02010100); timeReceived=632900288485000000; timeUpdatedCounter=0;
RequestInternalParticipant: deviceID=0; deviceType=0; station=38053; agentID=39268;
type=REQUEST_INTERNAL_PARTICIPANT (02010000);
recordingFlags=CHANS_REC_FLAG_FULL_VOICE (4); chosenInitiator=0 (0);
requestMediaType=SCREEN_MEDIA (1); requestWrapUpTime=60;
requestWrapUpTime...=60000; summationType=SUMMATION_TYPE_SUMMED (3);
participantID=1; ipPort=0; (LogWriter.LOGWrite2OldLog)

Record voice response:

01/08/2006 11:34:08.593 DEBUG [7928]: RCM Start Record Request(Voice) response data :
ResourceAlocatedResponse: requestID=110331462; interactionID=110331462;
eventType=CHANS_EVENT_REC_FAIL (4);
eventReason=CHANS_REC_VOICE_LOGGER_GENERAL_FAILURE (1402); urid=0;
type=RESOURCE_ALLOCATED_RMO_RESPONSE (03010400);
timeReceived=632900288485156250; timeUpdatedCounter=0; ResponseInternalParticipant:
recordingLogger=10959703; recordingChannel=44;
Example 2
























Checking Ports
Checking Ports
Check that the required ports are open.
To check connectivity:
Check that the following ports are open.
Table 13-1: Required Ports
Proto Destination
Port Source(Client) Service Description
col (Server)
TCP 62051 Call Server on ScreenSense Server .NET Control of system

NICE Interactions Remoting recording.
Applications Server
Center (HTTP)
Playback Server
CTI (drivers)
TCP 62050 Call Server on CAPI integrations CLS API Control of system
NICE Interactions recording.
Center
TCP 62055 DB Server on NICE Call Server on NICE .NET Passing requests to
Interactions Center Interactions Center Remoting insert/update
(HTTP) interactions and
contacts to/in the
database.
TCP 62059 CLS SNMP Agent All CLS processes on .NET Passing notifications
(part of SNMP.exe the NICE Interactions Remoting for SNMP traps from
service) on NICE Center (HTTP) CLS processes to
Interactions Center CLS SNMP Agent.
TCP 62069 RCM on NICE Call Server on NICE .NET Handling recording
Interactions Center Interactions Center Remoting requests from Call
(HTTP) Server.
TCP 2010 Screen Capture Agents desktops .NET Control

Control on NICE (screen recording) Remoting ScreenAgent
Interactions Center (HTTP) recording for
NAT/PAT
environments.
TCP 62065 Call Server CAPI Client The Call Server

uses this port for
keep alive with CAPI
Client.

Checking Ports
Table 13-1: Required Ports (Continued)
Proto Destination
Port Source(Client) Service Description
col (Server)
UDP 162 NMS SNMP Service on SNMPv1 Unsolicited

Applications Server operations (traps)
sent by the SNMP
SNMP Service on
Service for alarming.
NICE Interactions
Center
SNMP Service on
Playback Server
SNMP Service on
Voice/VoIP Logger
SNMP Service on
NiceScreen Logger
SNMP Service on
Storage Center
VoIP Recording
Gateway


Query the Database
interaction ID). See the Recording Troubleshooting workflow for a list of CLS exceptions. For
more details, see the Interaction Data Reference Guide.
To check the CLS Exceptions:

1. Query the tblInteractionCatalog table to check over what period the calls occur.
2. Get all the exceptions relating to interactions:
WHERE
3. Get all exceptions relating to recordings:
WHERE
4. Check the vcExceptionDetail field in tblException.
In the Business Analyzer
To check the CLS exceptions in Business Analyzer:

• In Business Analyzer, select the interaction and click on the Exceptions tab, in the Details
pane.
Figure 13-10 Business Analyzer Exceptions Tab

Fine-tuning the CLS Server Configuration Parameters
Fine-tuning the CLS Server Configuration Parameters

You can modify the CLS Server configuration parameters.
To fine tune the CLS Server configuration:

1. Verify that you are working in Technician mode: From the Settings menu, select Technician
Mode.
Figure 13-11 Technician Mode
2. In the Organization tree, select CLS Definitions > CLS Server.

3. Select the required CLS Server, and click the Configuration tab.
Figure 13-12 CLS Server Configuration Tab
4. Modify the required parameter. For example, if you get an Internal Call Server Table is Full
error:
a. Click next to the Call Server.

b. Increase the values for SegmentTableMaxEntries, LoginTableMaxEntries and
CompoundTableMaxEntries.
See the Interactions Center Installation Guide for a complete list of CLS Server configuration
parameters.

Isolating a Screen Problem Further
Isolating a Screen Problem Further

Try to isolate the problem and identify the faulty component.
To isolate the problem further:

• Try and isolate the problem by checking the logs to see:
• If the StartRecord command was called by the RCM. See the NiceScreen Troubleshooting
Workflow.
• If the ScreenAgent received the StartRecord command. See the NiceScreen
Troubleshooting Workflow.
The following is a sample StartRecord in the ScreenAgent log:
INFO: 02/01/08 10:56:31:015 IncCommandsHandler: StartRecording received from IC
IpAddr(screen-env3:2010). capture handle(10000001),
DestinationAddr(screen-env3:30100), DestinationResourceId(93); Context = (676)
• If NiceScreen received the StartRecord command. See the NiceScreen Troubleshooting
Workflow.

To shut down the Interactions Center:

5. From the Start menu, select Programs> Nice Systems > Interactions Center > Tools >
6. Click .
-or-
Stop the NICE CLS service.
7. Shut down Windows normally.
To start the Interactions Center:

1. Power up the machine and log in via the console or PCAnywhere using the admin account.
This should launch all services, startup apps, the CLS Dispatch process, and all CLS daughter
processes. If auto-login is configured or the CLS is running as a service, even a Windows login
is unnecessary.
2. To start the NICE CLS service, press on the NICE Services Configuration Manager
window (see Figure 13-13),
-or-
3. Start the NICE CLS service.


without any options. DO NOT use text wrap as this will corrupt your configuration file.
IMPORTANT
Any change of configuration files should be done ONLY in consultation with NICE Support.
• For NiceScreen Logger and ScreenAgent problems, see the NiceScreen Troubleshooting
Workflow.
• During a NICE Perform installation of CLS 9.13.04.11, the following message appears after
entering the CLS login credentials, “Input string was not in a correct format”. The installation
fails. Set the regional settings of the CLS to English.
• The NICE Perform application receives messages with "NICE CLS Error Code 20" when
trying to start ROD under monitor modules. The system stops recording audio or screens.
Check that there is a sufficient number of TSAPI licenses.
• When running RemonitorApplication.exe, an error appears “Failed To create remonitor
event”. This utility is designed for use in an Avaya Integration only.
• If you are doing your mapping in System Administrator (and not manually using a file):
• In CLS Definitions > CLS Servers, select a CLS Server.
• Click the Configuration tab and click RCM.
• Validate that the Read Mapping from File parameter value is set to No.
• The MaxHistoryLengthHours parameter can also be used (see Fine-tuning the CLS Server
Configuration Parameters on page 411):
• If the customer wants to insert older calls this parameter can be increased.
• When retrieving calls from a tape backup to a Logger and then using TRS to insert the
calls into the database.
• A problem may arise from a feature called "silent whisper". The whisper is a sound bite that is
played to the agent to make them aware what type of call is coming through for example,
sales, fraud, and so on, before they are connected to the customer. If this feature is not fully
configured and there is silence for several seconds at the beginning of the call when the call is
connected, that is, there is no sound bite configured, no audio packets are sent through to the
NICE system and recording fails. If you ROD the call after the silenced part of the call, the call
is recorded. In order to compensate this gap of silence change the CheckforAudioDelay
RCM parameter to 7000 ms. For RTP sniffing, this is the period in milliseconds that the RCM
waits after starting a call before checking for audio availability on the Logger.

Other Useful Tools
Other Useful Tools

CAPI Tester. See Tier 4 for further information.

14
NMS Troubleshooting Flow
Contents
Workflow........................................................................................................................418
Troubleshooting NMS...................................................................................................420
Client-Side Troubleshooting........................................................................................421
Server-Side Troubleshooting.......................................................................................427
Changing the System Administrator Server Name ...................................................445
Checking for a CAClient Serialization Issue ..............................................................445
Using the NMSUtil Tool ................................................................................................446
Chapter 14: NMS Troubleshooting Flow 417

Workflow
Workflow
If you experience a problem with the NMS follow the steps below to isolate the problem.
Figure 14-1 Troubleshooting Workflow

• Is this a new installation?
If yes, when was the installation performed?
• Were any changes done to the environment that might have influenced the system? If yes,

Troubleshooting NMS
Troubleshooting NMS
NOTE: Some of the procedures include instructions that can also be executed using
NMSUtil. This tool is available on the Tier 4 Team Site. See Using the NMSUtil Tool
on page 446.
For NMS problems see the following:
1. Client-Side Troubleshooting:
a. Pinging the Monitored Server on page 421
b. Checking the SNMP Service Configuration on page 421
c. Verifying that SNMP Agent is Sending Traps on page 425
d. Repairing the SNMP Service on page 425
e. Reinstalling the NICE Agents on page 426 (not recommended for production sites)
2. Server-Side Troubleshooting
a. Collecting the System Log Files on page 427
b. Checking the NMS Services on page 428
c. Checking the Trap Mechanism on page 435
d. Checking the MIB File on page 439
3. Checking Polling Timeout Errors on page 440
4. Troubleshooting Scenarios
• Scenario 1: Monitored Server Does Not Appear in the 2D Map on page 442
• Scenario 2: Drilldown Fails in the 2D Map on page 442
• Scenario 3: Only Some Agents are Reporting on page 442
• Scenario 4: There are No Traps in the Console on page 442
• Scenario 5: No Email Notifications Are Received on page 443
5. Changing the System Administrator Server Name on page 445
6. Checking for a CAClient Serialization Issue on page 445

Client-Side Troubleshooting
In cases where specific agents or monitored servers do not send traps, the problem most likely is a
client-side problem. See the following:
• Pinging the Monitored Server on page 421
• Checking the SNMP Service Configuration on page 421
• Verifying that SNMP Agent is Sending Traps on page 425
• Repairing the SNMP Service on page 425
Pinging the Monitored Server

Check that the NMS Server can ping the monitored server.
To ping the monitored server:

• From the NMS Server, ping the monitored server.
Optionally, you can use the NMSUtil (where available) to ping the monitored server. See
Using the NMSUtil Tool on page 446.
Checking the SNMP Service Configuration

Verify that the SNMP Service on the monitored server is running, and that it is configured
correctly. Ensure that the latest DLL files are installed.
To check the SNMP Service configuration:

1. On the client machine, verify that the status of SNMP Service is Started.
2. If the SNMP Service is not started, start the service and verify that there are no service errors
in Event Viewer.
Figure 14-2 SNMP Service
3. Right-click SNMP Service and select Properties.

4. Click the Traps tab.

Figure 14-3 SNMP Service Properties - Traps Tab
5. Verify the Community name is public (in lower case letters).

6. Verify the Trap destinations by checking the IP addresses (or computer names) of the NMS
servers where trap messages will be sent.


Figure 14-4 SNMP Service Properties - Security Tab
8. Confirm that:
• The Send authentication trap checkbox is not selected.
• In the Accepted community names area, public appears with READ ONLY privileges
and admin appears with READ-WRITE privileges.
IMPORTANT
If your site uses customized community names, you must change public and admin to the
appropriate community name.
9. Ensure that Accept SNMP packets from any host is selected (localhost appears in the list.)

10. In the registry, open the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SNMP\Parameters\E
xtensionAgents
Figure 14-5 Registry Editor - Extension Agents
This lists the various agents that are loaded when the SNMP service loads, and the Data
column indicates the location of the agent key registry.
11. Navigate to the agent key to view the location of the agent DLL.
Figure 14-6 Registry Editor - Agent Key
12. Verify that the DLL is in this folder, that it is the latest version and that it is not corrupt. If
necessary, consult with NICE Support.

Verifying that SNMP Agent is Sending Traps

Verify that the Monitored Server's SNMP agent has initialized correctly and is sending traps.
To verify that the SNMP agent is sending traps:

1. Copy Nice Systems\Applications\Tools\SNMP Tools\dbwin32.exe from the Applications
Server to the Monitored Server.
2. On the Monitored Server, double-click .dbwin32.exe.
3. Check whether the SNMP agent is alive. If there are no messages, or no indication that the
SNMP agents are up or no errors this may indicate a problem.
Figure 14-7 Debug Messages
Repairing the SNMP Service

The SNMP Service may need to be repaired.
To repair the SNMP Service:

1. In the registry, open the following key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SNMP\Parameters\E
xtensionAgents
2. Backup the registry key.
3. Remove the SNMP Service using the Windows Add or Remove Programs.
4. Reinstall the SNMP Service from Add/Remove Windows Components option.
5. Ensure that you have the Windows installation disk or have access to the I386 folder.
6. Reload the registry key.

Reinstalling the NICE Agents

You can reinstall some of the NICE Agents if there is a problem during installation.
IMPORTANT
Reinstalling the NICE Agents must be done ONLY in consultation with NICE Support. This is
NOT recommended for production sites.
To reinstall the NICE Agents:

After installing the SNMP Service, install the NICE Agents. Several of the components have a
separate installation for the SNMP agents while others install the SNMP agent during the software
installation.
Component Where/How to Reinstall Nice SNMP Agents
Screen Logger Screen Logger has a separate SNMP Agent

installation which can be removed and reinstalled
without harming the Screen Logger installation.
Accusys RAID Install separately from the NiceCall Focus III software.
OS Agent to monitor the See Installing the OS/SQL Agent (Optional)

System Administrator service Appendix, in the NMS Installation Guide, for further
details.
SQL Agent to monitor the SQL See Installing the OS/SQL Agent (Optional)
Appendix, in the NMS Installation Guide, for further
details.
Telephony Services Server and Telephony Services Server and Stream Server have
Stream Server separate SNMP Agents that can be removed and
reinstalled without harming the Telephony Services
Server and Stream Server installations.
VRG Installed as part of the VRG. The SNMP agent is part

of the image prepared and delivered with the VRG.
Logger Installed as part of the Logger software. The SNMP

agent is part of the image prepared and delivered with
the Logger.
Interactions Center Installed as part of the Interactions Center.
Integration (Driver) Installed as part of the Integration.
Applications Server Installed as part of the Applications Server.
Storage Center Installed as part of the Storage Center.
NMS Script Agent See Installing the SNMP Script Agent, in the NMS
Installation Guide, for further details.

Server-Side Troubleshooting
If none of the SNMP Agents or monitored servers are able to send traps, the problem may be
located on the NMS server. See the following:
• Collecting the System Log Files on page 427
• Checking the NMS Services on page 428
• Checking the Trap Mechanism on page 435
• Checking the MIB File on page 439
Collecting the System Log Files

There are log files for the various system services and processes running on the NMS Server. Note
that these log files are different from the trap messages logs.
To collect the system log files:

1. Navigate to <Installation Drive>:\TND\SERVICES\VAR\LOG.
Figure 14-8 Services Log
2. Open the required log file, for example, aws_sadmin.log.

Figure 14-9 Opening the Log File
3. Send a zip file of the log files when logging a case.
Checking the NMS Services

The following services run on the NMS Server:
• CA-Unicenter: One of the main services of the Enterprise Manager (Console), this service is
responsible for displaying new messages on the NMS Console. This service starts up several
sub-services.
• Unicenter TND Agent Technology: The main service for all agent technology-related
services (DSM, 2-D Map). This service starts up several sub-services.
• SNMP Trap Service: This service is responsible for the polling of SNMP agents that send
traps to the server.
• SNMP Nice Trap Formatter: This service receives traps after basic processing, and
"translates" the traps into the format in which they are displayed in the Console.
NOTE:
You can also use the NMSUtil for some of these checks. See Using the NMSUtil Tool
(where available).
Checking the CA-Unicenter Service and Sub-Services

One of the main services of the Enterprise Manager (Console), this service is responsible for
displaying new messages on the NMS Console. This service starts up several sub-services.

To check the CA-Unicenter service:

1. Check that the CA-Unicenter service is started. If the service is stopped no new events are
displayed.
Figure 14-10 CA-Unicenter Service
2. If necessary, restart the CA-Unicenter service. The following message may appear:
Figure 14-11 CA-Unicenter Services Error Message
3. If this message appears, the caunint user password must be updated. The CA-Unicenter
service is run under the caunint user which is created automatically during the NMS
installation.
Update the caunint user password as follows:
a. Right-click My Computer and select Manage. The Computer Management window
appears.

Figure 14-12 Computer Management Window
b. Right-click caunint and select Set Password. The Set Password for caunint warning
message appears.
Figure 14-13 Set Password for caunint Warning Message
c. Click Proceed. The Set Password for caunint window appears.

Figure 14-14 Set Password for caunint Window.
d. Enter a new password in the New password field.

e. Enter the new password again in the Confirm password field, and click OK.

4. Update the password for the CA-Unicenter Service:

a. In the Service window, right-click CA-Unicenter service.
b. Select Properties.
c. Click the Log On tab.
d. Update the password.
5. When the CA-Unicenter service starts up, it starts up a number of other services:
a. Open the command prompt, and type unifstat.
Figure 14-15 CA-Unicenter Sub-Services
b. Check the number of relevant sub-services.

c. For each sub-service, check that current status is 500 - ACTIVE.
d. If any of the sub-services are not active, restart the CA-Unicenter service using Windows
services, as previously seen or by typing:
unicntrl stop all
unicntrl start all

Checking the Unicenter TND Agent Technology Service

The main service for all agent technology related services (DSM, 2-D Map). This service starts up
several sub-services (AWServices).
To check the Unicenter TND Agent Technology service:

1. Check that the Unicenter TND Agent Technology service is started.
Figure 14-16 Unicenter TND Agent Technology Service
2. When the Unicenter TND Agent Technology service starts up, it starts up a number of other
AWServices. Check the status of the AWServices:
a. Open the command prompt.

b. Type awservices status and press <Enter>.

Figure 14-17 AWServices Status
3. If any of the sub-services are not active, stop and restart the AWServices as follows:
a. In the command prompt, type awservices stop and press <Enter>.
b. Type awservices start and press <Enter>.

Checking the SNMP Trap Service

The SNMP Trap service is responsible for the polling of SNMP agents that send traps to the server.
This service is an OS service (in addition to the familiar SNMP Service) and is usually only
relevant on the NMS Server.
To check the SNMP Trap service:

• Check that the SNMP Trap service startup type is set to Manual (any application that needs
this service will start the service) and that the service is started.
Figure 14-18 SNMP Trap Service
Checking the SNMP Nice Trap Formatter Service

The SNMP Nice Trap Formatter service receives traps after basic processing, and "translates" the
traps into the format in which they are displayed in the Console.
To check the SNMP Nice Trap Formatter service:

1. Check that the SNMP Nice Trap Formatter service startup type is started.
2. Check that the ConsoleMessageHandlerService has started.

Figure 14-19 SNMP Nice Trap Formatter Service
Checking the Trap Mechanism

Verify that the traps mechanism is working properly on the server side.
Figure 14-20 Trap Mechanism
Raw Trap
Basic
Formatting
Formatted Based
on NICE MIB File
Raw Trap:
Basic Formatting Trap:

MIB File Translation Trap:
To check the trap mechanism:

1. To check whether traps are being sent (or not) by the agent:
a. Open the command prompt and type awm_catch POLL_EVENT TRAP: and press
<Enter>.
Figure 14-21 Checking the Trap Mechanism
b. Either wait a few moments, or cause a malfunction in the system to initiate the sending of
a trap. To cancel the capture operation, press <Ctrl> + C.
c. If a trap is sent from the SNMP agent it is displayed (this is the raw and unformatted trap
before it is captured by the DSM). If nothing is displayed, no trap was received on the
server-side.
d. If the raw trap is displayed, but the trap is not visible in the Console, it means that the
DSM is not handling the trap, or that it has not been formatted properly. This may be due
to a NICE Trap Formatting service problem (See Checking the SNMP Nice Trap
Formatter Service on page 434), or due to a problem with the MIB file resulting in the
raw event not being processed.
2. To check whether the traps are being formatted by the DSM (indicated by the NICE_TRAP
prefix), enable the NICE_TRAP filter in the EM Console.
a. In the EM Console, select Config > Settings.
b. Click the Filters tab.
c. Disable all the filters.

Figure 14-22 EM Console Configuration (CA-Server)
d. Click OK.
e. Check if the traps arrive in pairs, once with the NICE_TRAP prefix, and once with a
TrapId.
Figure 14-23 Paired Trap IDs
f. If the NICE_TRAP trap arrives without a matching TrapId trap, check the SNMP Nice
Trap Formatter service (see Checking the SNMP Nice Trap Formatter Service
on page 434).

3. To view the OID nodes (all NICE objects start with: 1.3.6.1.4.1.3167) and verify they exist on
the client, use the following commands:
awwalk -h [computer name/IP] -o OID(node)
awget - h [computer name/IP] -o OID(leaf)
Figure 14-24 Viewing the OID Nodes
Interactions Center - 1.3.6.1.4.1.3167.1.2

Logger - 1.3.6.1.4.1.3167.1.3
Integration - 1.3.6.1.4.1.3167.1.4
Applications Server - 1.3.6.1.4.1.3167.1.5
Storage Center - 1.3.6.1.4.1.3167.1.6
Accusys Raid - 1.3.6.1.4.1.3167.1.11
Drivers - 1.3.6.1.4.1.3167.1.12
VRG - 1.3.6.1.4.1.3167.1.13
Accusys Raid 2 - 1.3.6.1.4.1.3167.1.14
4. Run the SNMPUtil tool. This tool listens to any incoming traps on the NMS Server and
indicates if there is a problem receiving the traps, or if there is a problem with formatting and
translating the traps after they are received.
a. Open the command prompt.
b. Change to the appropriate directory (Nice Systems\Applications\Tools\SNMP
Tools\ on the Applications Server or <Installation Drive>:\TND\Tools on the
NMS Server).

c. Type Snmputil.exe trap and press <Enter>.

Figure 14-25 SNMPUtil Tool Command Prompt
Checking the MIB File

When new traps are added to existing components, the MIB file may also be updated. The MIB
file is maintained by NICE and is updated with every new NMS SP.
To check the MIB file:

1. Check that the NICE-MIB-II.mib and NICE-MIB-II.txt files are in the following folders:
<Installation Drive>:\TND\SCHEMA\INCLUDED.
<Installation Drive>:\TND\SERVICES\CONFIG\MIBS
2. Check that the MEGARAID.txt file is in:
<Installation Drive>:\TND\SERVICES\CONFIG\MIBS
NOTE: The MIB file should NOT be updated onsite as this may cause the loss of traps
onsite.

Checking Polling Timeout Errors

Critical timeout errors may cause problems for the SNMP agents.
To check polling timeout errors:

1. In the Unicenter - Node View, right-click the SNMP agent and select Event Browser.
Figure 14-26 Event Browser
2. Extend the timeout parameter:

a. Right-click the <Installation
Drive>:\TND\services\CONFIG\AWS_NSM\SYS\NiceEmDef.atph file.
b. Select Properties. Ensure that Read-only is not selected.
c. Open the file with Notepad.
d. Increase the pollTimeout value.
e. Save the file and change the properties back to Read-only.
Figure 14-27 Increasing the pollTimeout Value

3. Reset the services:

a. In the command prompt, type resetdsm skipwv and press <Enter>. All managed
nodes will be dropped, and then rediscovered with the new values.
The time taken depends on the number of managed nodes and the new timeout parameter
is global for all managed nodes.
b. After the command has finished, in the command prompt type awservices start and
press <Enter>.
c. To verify that are the services have been started and are running, in the command prompt
type awservices status and press <Enter>.
Figure 14-28 Resetting the Services

Scenario 1: Monitored Server Does Not Appear in the 2D Map
1. In the User Administrator, verify that you are working in Technician mode: From the Settings
menu, select Technician Mode.
2. In the Organization tree, select Network Management System .

3. Select the required NMS Server and click the Monitored Servers tab.
4. Verify that the monitored server appears in the Attached Servers list.
5. On the NMS Server, restart the Nice CAClient service.
6. Check the NiceApplications.CACLient.Host.exe.log file.
7. Check the server operating system. See Checking for a CAClient Serialization Issue
on page 445.
8. Check for Updates.
Scenario 2: Drilldown Fails in the 2D Map

1. Ping the monitored server.
2. Check the SNMP Service Configuration on the client (see Checking the SNMP Service
Configuration on page 421).
3. Run the AWSERVICES STATUS command. See Checking the Unicenter TND Agent
Technology Service on page 432 or Using the NMSUtil Tool on page 446.
4. Use the AWWALK command to query the SNMP Agent. See Using the NMSUtil Tool
on page 446 or Checking the Trap Mechanism on page 435.
Scenario 3: Only Some Agents are Reporting

1. Check the SNMP Service Configuration on the client. See Checking the SNMP Service
Configuration on page 421.
2. Use the AWWALK command to query the SNMP Agent. See Checking the Trap
Mechanism on page 435.
3. Use the SnmpUtil.exe Tool and simulate a trap by restarting any of the monitored services.
4. Use the NMSUtil Tool.
Scenario 4: There are No Traps in the Console

1. Check that the CA Unicenter service is up and running. See Checking the CA-Unicenter
Service and Sub-Services on page 428.

2. Check that the ConsoleMessageHandlerService service is up.

3. Disable the NICE_TRAP filter in the EM Console to check for SNMP Nice Trap Formatter
problems.
4. Check that the SNMP Nice Trap Formatter service is up. See Checking the SNMP Nice Trap
Formatter Service on page 434.
5. Use the SnmpUtil.exe Tool.
Scenario 5: No Email Notifications Are Received

1. Check that the CA Nice SMTP Service is up.
2. Double-click <Installation Drive>\TND\Bin\NiceEmailMgr.exe to configure the email
address. If there is more than one email address per field, separate them with semicolons.
Figure 14-30 Nice Emails Manager Window
3. Check the details in <Installation Drive>\TND\Bin\NiceUsers.txt.

4. Log in to the System Administrator on the NMS Server (not from another computer), and
check that the email is working:
a. Verify that you are working in Technician mode: From the Settings menu, select
Technician Mode.
b. In System Administrator, in the Organization tree, under Network Management System,

select the relevant NMS Server.

c. Click the Logging tab.

d. Verify the Smtp Server and the e-mail address, and click Check Email. This checks that
the email can be sent from the NMS Server.
Figure 14-32 Log Mail

Changing the System Administrator Server Name
Changing the System Administrator Server Name

The NMS communicates with the Applications Server where the System Administrator resides. If
the host name changes for the Applications Server where the System Administrator resides, this
information must be updated via a configuration file.
To change the host name of the server where the System Administrator resides:
1. On the NMS Server, open Windows Explorer.
2. Navigate to <Installation Drive>\TND\BIN.
3. Open the file NiceApplications.CAClient.Host.exe.config.
4. In the following line, change the server name to the new name.
5. Save the file.
Checking for a CAClient Serialization Issue

If all the servers are defined in the System Administrator, but one or more of them do not appear in
the 2D map, there may be a problem with the server operating systems. The CAClient requires
operating system parity, that is, all the servers must be on the same operating system and service
pack. This occurs when the servers have been updated to different SPs, for example, the
Applications Server is Windows 2003 SP2 and the NMS Server is Windows 2003 SP1.
To check for a CAClient serialization issue:

• Ensure that all the servers are running the same operating system and SP.

without any options. DO NOT use text wrap as this will corrupt your configuration file.
IMPORTANT
Any change of configuration files should be done ONLY in consultation with NICE Support.

Using the NMSUtil Tool

NMSUtil is a tool which automates the troubleshooting of NMS. This tool is available on the Tier
4 Team Site.
To use the NMSUtil tool:

1. Copy the following 5 files to a directory on the server where the NMS is installed.
• LongNameServiceStart.bat
• LongNameServiceStop.bat
• NMSUtil.exe
• run.bat
• Snmputil.exe
2. To start the utility, double-click NMSUtil.exe.
Figure 14-33 Using the NMSUtil Tool
3. The tool includes the following options:
Table 14-1: Tool Options
# Option Description When To Use
1. Run Snmp Util After selecting this option another Check if the NMS server is
window opens with the SNMP receiving any traps from the
Utility. If you see activity on the monitored servers.
window, the NMS server is
receiving SNMP messages.

Table 14-1: Tool Options (Continued)
2. Check awservices Checks if the NMS services are If none of the SNMP Agents or
status up and running. If the services monitored servers are able to
are not running, select option 3 to send traps, check the status of
restart the services. the Unicenter TND Agent
Technology Service.
3. Start NMS Starts the NMS services. Stop and start the services when
services the Unicenter TND Agent
Technology Service sub-services
are not active.
4. Stop NMS Stops the NMS services.

services
5. Run awwalk Checks if the SNMP agents on a To check the trap mechanism
monitored server are running or and view the OID nodes.
not. Enter either the host name or
IP address.
6. Ping Monitored Allows you to ping the monitored Check when specific agents or
Server server and check communication monitored servers do not send
between the NMS and monitored traps.
servers. Enter either the host
name or IP address.
7. Reset DSM Resets the NMS engine. When you have critical timeout
Engine errors and you modify the
pollTimeout and need to restart
the services.
8. Clean Sadmin Cleans the admin settings for This should be done ONLY in
NMS. consultation with NICE
Support!!!!
9. Exit Closes the tool.

15
Telephony Services Server
Troubleshooting Flow
Contents
Troubleshooting Workflow ..........................................................................................450

Checking the Telephony Services Server Configuration .........................................452
Checking the Dialogic Service ....................................................................................453
Troubleshooting the Dialogic Interface ......................................................................454
Troubleshooting using the NICE Playback Telephony Layer Tester .......................474
Checking the Telephony Services Server Service ....................................................478
Checking the Applications Server Service ................................................................478
Checking the Logs .......................................................................................................479
Checking Ports .............................................................................................................482
Shutting Down and Starting the Telephony Services Server ...................................485
Troubleshooting Dial Plans .........................................................................................486
Chapter 15: Telephony Services Server Troubleshooting Flow 449

If you encountered any problems with your Telephony Services Server, following the steps
described in this workflow will enable you to isolate and properly identify the problem.
NOTE: To troubleshoot Customer Feedback issues, see the Customer Feedback

Guide.


Checking the Telephony Services Server Configuration
Checking the Telephony Services Server Configuration

Verify that the Telephony Services Server is correctly defined in the System Administrator.
To check the Telephony Services Server configuration:

1. Log in to the NICE Perform Application Suite. From the Accessories menu, select System
Administrator.
3. In the Organization tree, expand a site and select Playback > Telephony Services.
Figure 15-1
4. Select your Telephony Services Server and verify that it is correctly defined.

Checking the Dialogic Service
Checking the Dialogic Service
IMPORTANT
This procedure is not relevant to the Diva analog interface.
To check the Dialogic service:

1. From the Start menu, select Programs > Dialogic Systems > Configuration Manager -
DCM.
The Dialogic Product Configuration Manager appears.
Figure 15-2 Dialogic Configuration Manager
2. Ensure that the Dialogic service is running.

Troubleshooting the Dialogic Interface

This section includes the following topics:
• Troubleshooting the Dialogic Analog Interface (below).
• Troubleshooting the Dialogic Digital Interface. See page 457.
• Troubleshooting the Dialogic IP Interface. See page 460.
• Using the ISDIAG Utility. See page 461.
• Testing the Extension Using the Dialogic Voice Demo. See page 464.
• Using the Universal Dialogic Diagnostics Utility. See page 470.
Troubleshooting the Dialogic Analog Interface
IMPORTANT
This procedure is not relevant to the Diva analog interface.
To troubleshoot the Dialogic analog interface:

1. Check that the Dialogic service is running. See Checking the Dialogic Service (above).
2. Ensure that the installed Dialogic software is the latest released. In the Dialogic Configuration
Manager, from the Help menu, select About.
Check that the version is System Release 6.0 Build 171.
Figure 15-3 Dialogic Configuration Manager - About System Release 6.0
3. Check that the extensions are connected to the Dialogic board on the correct ports.

4. Dialogic JCT PCIExpress board only : There may be a Power Budgeting issue.
Certain PCI Express boards include a built-in power management feature. If the installed
Dialogic SR 6 software does not support the power management PCI Express board feature,
the board may not start correctly, or may start but will fail to recognize the star (*) key, and
errors will appear. Upgrading the software will not remedy this problem.
Solution
Power Budgeting Jumper P3 is comprised of a three-pin jumper which determines the way in
which the board responds to the system power budgeting feature.
The Power Budgeting Jumper P3 can be configured in one of the two following ways:
• P3 Jumper in Pins 2-3 (default): The Dialogic SR 6 board adheres to power

budgeting values set by the system.
• P3 Jumper in Pins 1-2: The Dialogic SR 6 board ignores the power
Figure 15-4 Dialogic Board - Physical Layout

Check the Event Viewer.

Figure 15-5 Event Properties Window - Error Message Example
If Power Budgeting is not implemented: The Dialogic JCT PCIExpress board must be
plugged into a x4 or higher slot with the P3 jumper in position 1-2 (power budgeting ignored).
If Power Budgeting is implemented: A Dialogic JCT PCIExpress board can be plugged
into a x1 slot but the P3 jumper must be in position pins 2-3 (power budgeting adhered to).
TIP:
• When using the HP DL380 G5 Dialogic board, set the jumper on pins 1 and 2,
causing power budgeting to be ignored. When this jumper is set on the default pins
(2,3) the board may not have power, preventing the board from starting.
• Ensure that the latest software is installed.
For more information about this issue, see P3 Jumper Settings for Dialogic® PCI Express Boards.
5. For each Dialogic board, check that the SW100 switch is set to the required ID number.
6. If more than one Dialogic board is installed, check the H100 cable connections. The first
connector on the H100 cable must be connected to board ID 0, and the last connector on the
H100 cable must be connected to the last Dialogic board.
7. Check that the Dialogic board is placed correctly into the slot.
8. Move the Dialogic board to a different slot.
9. Use the ISDIAG Utility to check the E1 or T1 trunks. See Using the ISDIAG Utility
on page 461.
10. Check that the Dialogic boards have been configured correctly according to the type of board.

11. Use the Dialogic Voice Demo to test the channel, check that you can record and playback
WAV and VOX files, and test that you can get digits from the extension. See Testing the
Extension Using the Dialogic Voice Demo on page 464.
12. Test the lines from the patch panel with an analog phone and from the Y-Cable with a
telephone butt set.
13. Try to configure and test one port at a time.
14. Verify the pinouts.
15. Check the Dialogic board using the Universal Dialogic Diagnostics Utility. See Using the
Universal Dialogic Diagnostics Utility on page 470.
Troubleshooting the Dialogic Digital Interface
To troubleshoot the Dialogic digital interface:

1. Check that the Dialogic service is running. See Checking the Dialogic Service on page 453.
Digital HMP Interface: Check that the version is HMP Release 3.0 Service Update 214.
Digital SR 6.0 Interface: Check that the version is System Release 6.0 Build 171.
3. Check the cables that are connected to the Dialogic boards.

4. Dialogic JCT PCIExpress board only : There may be a Power Budgeting issue.

Certain PCI Express boards include a built-in power management feature. If the installed
Dialogic SR 6 software does not support the power management PCI Express board feature,
the board may not start correctly, or may start but will fail to recognize the star (*) key, and
errors will appear. Upgrading the software will not remedy this problem.
Solution
Power Budgeting Jumper P3 is comprised of a three-pin jumper which determines the way in
which the board responds to the system power budgeting feature.
The Power Budgeting Jumper P3 can be configured in one of the two following ways:
• P3 Jumper in Pins 2-3 (default): The Dialogic SR 6 board adheres to power

• P3 Jumper in Pins 1-2: The Dialogic SR 6 board ignores the power
Figure 15-7 Dialogic Board - Physical Layout

Check the Event Viewer.

Figure 15-8 Event Properties Window - Error Message Example
If Power Budgeting is not implemented: The Dialogic JCT PCIExpress board must be
plugged into a x4 or higher slot with the P3 jumper in position 1-2 (power budgeting ignored).
If Power Budgeting is implemented: A Dialogic JCT PCIExpress board can be plugged
into a x1 slot but the P3 jumper must be in position pins 2-3 (power budgeting adhered to).
TIP:
• When using the HP DL380 G5 Dialogic board, set the jumper on pins 1 and 2,
causing power budgeting to be ignored. When this jumper is set on the default pins
(2,3) the board may not have power, preventing the board from starting.
• Ensure that the latest software is installed.
For more information about this issue, see P3 Jumper Settings for Dialogic® PCI Express Boards.
5. For each Dialogic board, check that the SW100 switch is set to the required ID number.
6. If more than one Dialogic board is installed, check the H100 cable connections. The first
connector on the H100 cable must be connected to board ID 0, and the last connector on the
H100 cable must be connected to the last Dialogic board.
7. Check that the Dialogic board is placed correctly into the slot.
8. Move the Dialogic board to a different slot.
9. Use the ISDIAG Utility to check the E1 or T1 trunks. See Using the ISDIAG Utility
on page 461.
10. Check that the Dialogic boards have been configured correctly according to the type of board,
that is, Digital T1 or Digital E1.

11. For Digital E1 or T1 ISDN PRI trunks, check the switch configuration. The configuration
varies according to the type of switch integration.
12. Test the lines from the patch panel with an analog phone and from the Y-Cable with a
telephone butt set.
13. Try to configure and test one port at a time.
14. Verify the pinouts.
15. Check the Dialogic board using the Universal Dialogic Diagnostics Utility. See Using the
Universal Dialogic Diagnostics Utility on page 470.
Troubleshooting the Dialogic IP Interface
To troubleshoot the Dialogic board for the IP interface:

1. Check that the Dialogic service is running. See Checking the Dialogic Service (above).
Check that the version is HMP Release 3.0 Service Update 214.

Using the ISDIAG Utility
IMPORTANT
This procedure troubleshoots Dialogic digital trunk boards only.
This procedure confirms that the Dialogic board and the trunk are properly connected.
In this procedure, you enter a telephone extension from your site. The program dials the telephone
extension and the telephone rings.
Before you begin, you will need the following:
• The location of the isdiag.exe file.
• A telephone extension connected to your site.
• The trunk connected from the switch to the Logger.
To test the Dialogic SR 6 board using the ISDIAG utility:

1. Verify that a green LED is lit on the board panel.
2. Open a command window and type isdiag.exe.
3. On the command line, type the following:
• Digital T1 environment: Type isdiag 1 1 t s.
• Digital E1 environment: Type isdiag 1 1 e s.
Figure 15-10 Cmd.exe Window
Digital E1 environment: isdiag 1 1 e s
Digital T1 environment: isdiag 1 1 t s
The Main Menu appears.

Figure 15-11 Cmd.exe - isdiag 1 1 e s Window
Verify that the

value for LOS
is 0
4. Verify that all values are set to zero (0).
WARNING
If any of the values are not set to zero, check that both the Dialogic board and the switch
are configured correctly.
• Ericsson only: Proceed to Step 5.
• All switches except for Ericsson: Proceed to Step 6.

5. Ericsson only:
a. At the prompt, type 1 (set call parameters).
b. Type 9 (change user info layer1 protocol).
c. Set 03 to A-law.
Figure 15-12 Cmd.exe - isdiag 1 1 e s Window
6. Return to the main menu and at the prompt, type 6.

The Call prompt appears.

Figure 15-13 Call Prompt Example
Extension number
7. Type an extension number and press <Enter>.

The entered telephone extension rings.
8. Close the command window.

Testing the Extension Using the Dialogic Voice Demo

The Dialogic Voice Demo confirms system health between Dialogic and the telephone system
installed at your site. Use the Dialogic Voice Demo to test the following:
• Connectivity between Dialogic board and the extension
• Recording and playing back WAV and VOX files
• Getting digits from the extension.
This section describes:
• Testing an Analog Extension Using the Dialogic Voice Demo
• Testing a Digital Extension Using the Dialogic Voice Demo
Testing an Analog Extension Using the Dialogic Voice Demo
To test an analog extension using the Dialogic Voice Demo:

1. When you install the Dialogic software, ensure that Demos is selected.
2. After the installation is complete, in the Dialogic Configuration Manager, check that the
Dialogic services are running and that the board is functioning properly.
3. Start the VoiceDemo application by selecting Start > Programs > Dialogic Service >
Demos > Voice > Voice Demo.
The Dialogic Voice Demo Application appears.

Figure 15-15 Dialogic VoiceDemo Application
4. To select an extension to use for your test:

a. Select File > Open.
The Select (Voice) Channel window appears.
Figure 15-16 Select (Voice) Channel Window
b. Select a channel (extension) and click OK.

The Device opened message appears.
Figure 15-17 Device Opened Message
5. To test the extension:

a. Select Function > Dial.
The Dial window appears.

Figure 15-18 Dial Window
b. Enter the extension number you need to test. Click OK.

c. Ensure that the extension rings.
6. To close the call, select Function > On Hook.
7. To test recording and playing a WAV file:
a. Record to a WAV file, by selecting Function > Record WAV. Talk into your phone
handset. A WAV file is automatically created.
b. To play back the WAV file, select Function > Play WAV.
The WAV file is played back to your extension.
Figure 15-19 Playing WAV File Message
8. To test recording and playing a VOX file:

a. Record to a VOX file, by selecting Function > Record VOX. Talk into your phone
handset. A VOX file is automatically created.
b. To play back the VOX file, select Function > Play VOX.
The VOX file is played back to your extension.
9. To verify that the phone digits are received correctly (IVR):
a. Select Function > Getdigits.
b. Enter 5 digits.
A message appears informing you that the digits were received.
NOTE: To verify phone digits, the extension must be off-hook.

Figure 15-20 Received Digits Message
Testing a Digital Extension Using the Dialogic Voice Demo
To test a digital extension using the Dialogic Voice Demo:

1. When you install the Dialogic software, ensure that Demos is selected.
2. After the installation is complete, in the Dialogic Configuration Manager, check that the
Dialogic services are running and that the board is functioning properly.
3. Start the VoiceDemo application by selecting Start > Programs > Dialogic Service >
Demos > Voice > Voice Demo.
The Dialogic Voice Demo Application appears.

Figure 15-22 Dialogic VoiceDemo Application
4. To select an extension to use for your test:

a. Select File > Open.
The Select (Voice) Channel window appears.
Figure 15-23 Select (Voice) Channel Window
b. Select a channel (extension) and click OK.

The Select (Digital) Channel window appears.
Figure 15-24 Select (Digital) Channel Window

c. Select a channel and click OK.

The Device opened message appears.
Figure 15-25 Device Opened Message
5. To test the extension:

a. Select Function > Make Call.
The Dial window appears.
Figure 15-26 Dial Window
b. Enter the extension number you need to test. Click OK.

c. Ensure that the extension rings.
6. To close the call, select Function > Release Call.
7. To test recording and playing a WAV file:
a. Record to a WAV file, by selecting Function > Record WAV. Talk into your phone
handset. A WAV file is automatically created.
b. To play back the WAV file, select Function > Play WAV.
The WAV file is played back to your extension.
Figure 15-27 Playing WAV File Message
8. To test recording and playing a VOX file:

a. Record to a VOX file, by selecting Function > Record VOX. Talk into your phone
handset. A VOX file is automatically created.
b. To play back the VOX file, select Function > Play VOX.
The VOX file is played back to your extension.
9. To verify that the phone digits are received correctly (IVR):
a. Select Function > Getdigits.
b. Enter 5 digits.

A message appears informing you that the digits were received.
NOTE: To verify phone digits, the extension must be off-hook.
Figure 15-28 Received Digits Message
Using the Universal Dialogic Diagnostics Utility

You use the Universal Dialogic Diagnostics Utility to check the Dialogic board.
To use the Universal Dialogic Diagnostics Utility:

1. Stop all active applications on all Dialogic boards.
2. From the Start menu, select Programs > Intel Dialogic System Software > Universal
Dialogic Diagnostics Utility.
The UDD Initialization window appears.
Figure 15-29 UDD Initialization
3. Click Continue.
A warning message appears.

Figure 15-30 Warning Message
4. Click Yes.
The Universal Intel Dialogic Diagnostics window appears.
Figure 15-31 Universal Intel Dialogic Diagnostics Window
5. Select the board you need to test and click Specify Tests.
The Specify Tests window appears.

Figure 15-32 Universal Intel Dialogic Diagnostics - Specify Tests
6. Click Select All Tests and then click OK.

The Test Progress window appears.

Figure 15-33 Universal Intel Dialogic Diagnostics - Test Progress Window
7. If any failure occurs, a new screen appears displaying information about the failed board. To
view the details about the failed board, click Details.
8. Click Done.

Troubleshooting using the NICE Playback Telephony Layer Tester
Troubleshooting using the NICE Playback Telephony Layer

Tester
The NICE Playback Telephony Layer Tester tests the connection between the Telephony Services
Server and your phone lines. The NICE Playback Telephony Layer Tester can be used for all line
types except for SIP extension.
NOTE: The NICE Playback Telephony Layer Tester is automatically installed with the
Telephony Services Server.
To troubleshoot line connections using the NICE Playback Telephony Layer Tester:
1. Navigate to ...\NICE Telephony Services\bin and double-click
NiceApplications.Playback.TelephonyServices.TelLayer.GuiTester.exe.
The NICE Playback Telephony Layer Tester appears.
Figure 15-34 NICE Playback Telephony Layer Tester Main Window
2. Click Detect Resources.

The NICE Playback Telephony Layer Tester displays the detected resources.

Figure 15-35 NICE Playback Telephony Layer Tester - Detected Resources
Detected resources
3. IP lines only: If you need to test IP lines, in the Telephony Configuration and IP
Configuration areas, enter the relevant information.
4. Click Init Telephony.
The available lines are displayed.
Figure 15-36 NICE Playback Telephony Layer Tester - Available Lines
Available lines

5. Select one or more lines or click Create All Lines. The selected lines become available for
testing.
NOTE: When testing IP lines, to confirm connectivity, you only need to test one line.
When testing analog lines, we recommend testing all available lines.
6. Click the Lines tab and select a line.

Figure 15-37 NICE Playback Telephony Layer Tester - Lines Tab
7. Enter a phone number and click Make Call.

The phone will ring.
8. Pick up the phone receiver.
• If you need to test voice, proceed to Step 9.
• If you do not need to perform further tests on the line, close the call.
A message appears indicating that the call started and ended successfully.

Figure 15-38 NICE Playback Telephony Layer Tester - Information Messages
9. To play a recorded call, in the Play area, click Browse and select a WAV file to play. Then
click Play.
The selected file plays. The NICE Playback Telephony Layer Tester displays a message
indicating that the file played successfully.
Figure 15-39 NICE Playback Telephony Layer Tester - File Played Successfully
10. Click End Call.

Checking the Telephony Services Server Service
Checking the Telephony Services Server Service

After the Telephony Services Server has been configured in the System Administrator and the
Telephony Services Server software is completely installed, you can verify that the NICE
Telephony service is operating properly.
NOTE: If the Telephony Services Server is installed but System Administrator is not yet
configured, the Telephony Services Server service may fail to start.
To check the NICE Telephony Server service:

• Open the Services window and verify that the service Status is Started and its Startup Type is
set to Automatic.
Figure 15-40 Services Window
Checking the Applications Server Service

Check that following service is running on the Applications Server.
• NICE System Administrator: This is the main Applications Server service that controls the
login process to NICE applications.

Checking the Logs
Checking the Logs

This section describes:
• Setting the Logs Reporting Level (below).
• Viewing the Telephony Services Server Logs. See page 480.
• Viewing the NICE Playback Streaming Services Logs. See page 481.
• Viewing the NICE Playback Administration Service Logs. See page 481.
• Viewing the NICE Player Logs. See page 481.
For information about the Log Collector, see the Troubleshooting Guide, Collecting Information
Using the Log Collector.
Setting the Logs Reporting Level

Set the log level to DEBUG.
To set the logs reporting level:

Administrator.
3. In the Organization tree, expand your site and select Playback > Telephony Services.
Figure 15-42
4. Select a Telephony Services Server, select the Log Level tab, and from the Log Level
dropdown, select Debug.

Checking the Logs
Figure 15-43 Log Level - Debug
5. Click Save .
Viewing the Telephony Services Server Logs

The Telephony Services Server log files are located in the following folder:
...\Program files\NICE Systems\NICE Telephony Services\log
To view the Telephony Services Server Logs:

• Check the TelephonyServicesLog.txt log files for the Telephony Services Server service.
Figure 15-44 Telephony Services Server Log Files

Checking the Logs
Viewing the NICE Playback Streaming Services Logs

The NICE Playback Streaming Services run on the Telephony Services Server and receive
streamed packets from the Stream Server and synchronize them for playback to the Player.
The log files are located in the following location:
...\Program files\NICE Systems\NICE Telephony Services\log
To view the NICE Playback Streaming Services Logs:

• Check the MediaServices.txt log files.
Viewing the NICE Playback Administration Service Logs

The NICE Playback Administration service runs on the Applications Server and manages the state
of each Playback to Extension resource, port and so on. If no resources are available, the NICE
Playback Administration service displays an error.
The log files are located in the following location:
...\NICE Systems\Applications\ServerBin
To view the NICE Playback Administration Service Logs:

• Check the PlaybackAdministration.txt log files.
Viewing the NICE Player Logs

The log files are located on the Client Workstation in the following location:
C:\Documents and Settings\[user]\Local Settings\Application
Data\Release3\NicePlayer.
To view the NICE Player Logs:

• On the Client Workstation, check the log files for the NICE Player.

Checking Ports
Checking Ports
Check the required ports are open.
Administrator.
3. In the Organization tree, expand your site and select Playback > Telephony Services.
4. Select a Telephony Services Server, select the General tab, and check the HTTP and TCP
ports:
Troubleshooting Playback to Extension Problems
To troubleshoot Playback to Extension Problems:

• No free resources error message:
• Restart the NICE Playback Administration service on the Applications Server.
• Check the port configuration in System Administrator. See Checking Ports on page 482.
• The phone is not ringing (a phone is not responding and an error message appears):
• Search for a dial command in the log. See Viewing the Telephony Services Server Logs
on page 480 or see the Troubleshooting Guide: Collecting Information Using the Log
Collector.
• Check that the extension is connected to the Dialogic board on the correct port. See
Troubleshooting the Dialogic Interface on page 454.
• Pressed star (*) but nothing happens or an error message appears:
• Check for a network problem, where the server to client event connectivity is blocked. See
Checking Ports Using Port Tester on page 483.
• Check that the NICE Playback Streaming service is running on the Applications Server.
See Checking the Telephony Services Server Service on page 478 and Checking the
Applications Server Service on page 478.
• Check the Player log for errors. See Troubleshooting Guide: Collecting Information Using
the Log Collector.
• Server down message but the Telephony Services Server service is running:
• Check for a network problem.

Checking Ports
Checking Ports Using Port Tester

Use the Port Tester to check the ports between the Player and the Telephony Services Server.
To check ports using port tester:

Administrator.
2. In the Organization tree, expand your site and select Applications > Configurations >
General. Then click the General tab and check whether the system is configured to work
with random ports or specific ports range.
Figure 15-45 General Settings Tab
3. On the workstation:
a. Copy RemotingClientTester.exe and ClientRemotingPortTester.dll to the
workstation.
b. Double-click RemotingClientTester.exe.
c. In the From field, enter the lower port range, and in the To field, enter the upper port
range. This can be done in batches of 500 ports, in the range of 1024 to 5000. If the
system is configured to use specific port ranges (as shown in step 1), use the relevant port
ranges.
d. Click Run.

Checking Ports
Figure 15-46 Client Tester Form Window
4. On the Telephony Services Server:

a. Copy RemotingServerTester.exe and ServerRemotingPortTester.dll to the
workstation.
b. Double-click RemotingServerTester.exe.
c. In the Client Name field, enter the client workstation name.
d. In the From field, enter the lower port range, and in the To field, enter the upper port
range. This can be done in batches of 500 ports, in the range of 1024 to 5000. If the
system is configured to use specific port ranges. If the system is configured to use specific
port ranges (as shown in step 1), use the relevant port ranges.
e. Click Run.

Shutting Down and Starting the Telephony Services Server
Figure 15-47 Server Remoting Port Tester Window
5. If a number of ports fail, this indicates a communication problem between the server and
client.
Shutting Down and Starting the Telephony Services Server
To shut down the Telephony Services Server:

• Open the Services window, select the NICE Telephony Services service and change its service
Status to Stopped.
-or-
• Open the Services window, select the NICE Telephony Services service and change its service
Status to Started.
Figure 15-48 Services Window

Troubleshooting Dial Plans
Troubleshooting Dial Plans
IMPORTANT
If a switch does not require a dial plan, select Empty Prefix when defining it. Do not leave dial
plan definitions as Disabled!

without any options. Do not use text wrap as this will corrupt your configuration file.
• In any event, any change of configuration files should be done ONLY in consultation with
NICE Support!!!!
• Do not use a loopback IP address (such as 127.0.0.1) for the Telephony Services Server
address, even if the Telephony Services Server co-resides on the Applications Server. This
may prevent the Telephony Services Server from starting properly.
• When picking up the phone, the prompt that is heard is Please press star (*). This may be
confused with Please press Start. Make sure you press * to begin playing the call.
• If you discover unidentified problems when running the Telephony Services Server, you may
need to install the latest Update Pack. Check if an Update Pack is available.
• If the Playback to Extension option appears grayed out, this may be due to a problem with the
resource managers. In this same scenario, Playback to Speaker works fine. See below.
Figure 15-49 Playback to Extension - Select Output Window

If Playback to Extension appears grayed out, do the following:

a. In System Administrator, expand the Master Site and select Playback > Telephony
Services > Dial Plan. The following message may appear.
Figure 15-50 Dial Plan Updates Window
b. Click OK and restart the NICE Playback Administration Service.

• Customer Feedback: If you have correlation issues between your surveys and the agents’
interactions, resulting in interaction playback failure, this may be due to one of the following:
• A faulty UCID configuration. Check your switch configuration. See Samples of Avaya’s
UCID Configuration on page 323.
• The port-extension pairs in System Administrator may have been defined incorrectly.
See Installing Dialogic SR 6.0 Boards for the Analog/Digital Interface on page 28 and
the Customer Feedback Setup Guide.
• The extensions may not have been defined as monitored devices. See the integration guide
relevant to your site.

16
NiceScreen Troubleshooting Flow
Installation of NiceScreen components is performed using an installation wizard designed with

easy-to-follow steps. This section lists some of the most common problems you may encounter
and simple solutions. If you continue to need help, contact your local supplier or the nearest NICE
System Customer Service Center. Contact information is located in the inside cover of this
manual.
NOTE: Any references made in this chapter to values which must be changed, refers
directly to the Agent workstation registry unless stated otherwise. Page numbers are
the location of the description of each parameter.
Contents
Overview........................................................................................................................491
Isolating the Problem ...................................................................................................492
Gathering Information ..............................................................................................492
Checking the NiceScreen Logger Service Status ....................................................492
Checking the ScreenAgent Interactions Center Registration...................................493
Checking the Configuration......................................................................................494
Checking SAMS for Errors .......................................................................................494
Isolating the Problem Further...................................................................................496
Checking the NiceScreen Logs................................................................................496
Checking the ScreenAgent Logs..............................................................................496
Checking the RCM Logs ..........................................................................................497
Checking the Call Server Logs.................................................................................497
Checking the Call Status in the Database ................................................................497
Checking the Rule....................................................................................................497
Checking the Naming Resolution.............................................................................498
Checking Connectivity..............................................................................................498
Verifying the ScreenAgent Configuration .................................................................500
Testing the ScreenAgent ..........................................................................................500
Chapter 16: NiceScreen Troubleshooting Flow 489

Other Useful Tools....................................................................................................501
NICE ScreenAgent Installation/Operational Problems..............................................503
NICE ScreenAgent Optimization and Performance Issues.......................................505
NiceScreen Logger Configuration Parameters..........................................................507

Overview
Overview
If you experience a problem with Screen recording follow the steps below to isolate the problem:
Figure 16-1 Workflow Diagram

Isolating the Problem

Checking the NiceScreen Logger Service Status

When to check the NiceScreen Logger service status:
• If all screens are not being recorded, check whether the NiceScreen Logger service is active.
• If some agents screens are being recorded, you can skip this check.
To check the NiceScreen Logger service status:
1. On the computer where the NiceScreen Logger is installed, check that the NiceScreen Logger
service is running. If the service is not running select Start > Settings > Control Panel >
Administrative Tools > Services, and start the service.
2. Point your cursor to the icon in the tray, a tooltip displays information about NiceScreen
Logger. When you click on the icon, status details are displayed.
Figure 16-2 Logger Status Details
3. On the computer where the NiceScreen Logger is installed check that mmloggersvc.exe is
an active process running on the computer. If not, the installation may have failed.

Checking the ScreenAgent Interactions Center Registration

When to check the ScreenAgent Interactions Center registration:
• If all or some screens are not being recorded.
To check the ScreenAgent Interactions Center registration:

1. On the Interactions Center, run Tester.exe from the D:\program files\Nice systems\Nice
CLS\BIN folder.
Figure 16-3 Tester Application
2. Click Connect . The Connect to CLS window appears.

3. Click OK. A SUCCESS message appears.
4. Select Maintenance > Universal Agent Table.
Figure 16-4 Universal Agent Table
5. Click Refresh. The ScreenAgents that were registered to the Interactions Center appear in the
list. If a ScreenAgent is not registered, it will not appear in this list and will not be recorded.
6. Perform the following checks based on the outcome:
• If there are no agents logged in, there may be a network problem. Check the network
connectivity and the ports. See Checking Connectivity on page 498.

• If the agents that are not being recorded do not appear in the list:
• Check the agent workstation where ScreenAgent is installed and screenagent.exe is
running.
• Check that the particular workstation is configured correctly. See Checking the
Configuration on page 494.
• Check the particular workstation for a port blockage See Checking Connectivity
on page 498.
7. Check the SAMS logs. See Checking SAMS for Errors on page 494.
8. Search the ScreenAgent logs in order to identify problems with regards to registration to the
Interactions Center. See Checking the ScreenAgent Logs on page 496.
Checking the Configuration

Verify that:
• The NiceScreen Logger, ScreenAgents, Channel Pools and Sources Pool are correctly defined
in the System Administrator.
• The Agent is configured correctly in User Administrator.
To check the configuration:

Organization tree, click Organization. Then select Technician Mode and click Save.
2. Expand the Master Site > Logger Servers > NiceScreen Logger and check the details for
the required NiceScreen Logger. Verify that these details correspond with any criteria in the
Rules Manager > Scheduler > Filter.
3. In the Organization tree, select CLS Definitions > Channel Mapping > 2. Sources
Definition. Verify that the ScreenAgents are correctly defined.
4. In the Organization tree, select CLS Definitions > Channel Mapping > 4. Dynamic
Mapping. Check the following tabs:
a. Channels Pool
b. Sources Pool
c. Attach/Detach
5. Check that the Agent is correctly configured in User Administrator, including the Operating
System login name.
Checking SAMS for Errors

SAMS monitors ScreenAgents; debug information and logs that are sent from ScreenAgents are
received by the SAMS. Every time a ScreenAgent activates, stops, or fails to function properly,
status information is displayed in a console window called SAMS Report. The status is displayed

according to severity. From least to most severe, severity is as follows: Debug, Info, Warn, Fatal,
Bug, and Event. By default, the SAMS threshold is set to Fatal. This means that status with a
severity of Fatal, Bug, or Event will be displayed in the SAMS Report console.
To check SAMS:
1. Check that the SAMS service is running. If the service is not running select Start > Settings
> Control Panel > Administrative Tools > Services, and start the service.
2. Ensure that SAMS has been installed properly.
3. Once the SAMS service is running, the SAMS Server icon is displayed in the computer's
system tray .
4. To start the SAMS Report, right-click the SAMS Server icon and select Start.
5. Messages are sent to the SAMS Report console window.
Figure 16-5 SAMS Report
All messages sent to the SAMS Report console window display the following parameters:
• Date and time message was sent
• Severity level - From least to most severe: Debug, Info, Warn, Fatal, Bug, Event
• Name of the component that sent the message (for example, SA - for ScreenAgent)
• IP address and port number or host name of the machine from which the message was sent
• Report type:
• Install - message report success/failure
• Uninstall - message report success/failure
• Startup - information report
• Shutdown - information report
• Online - message sent while ScreenAgent is running

• Message content
6. Check the SAMSReport.log file to review the status information that is no longer displayed
in the SAMS Report console. The file is in the SAMS installation folder:
C:\Program Files\Nice Systems\SAMS Server\SAMSServerLog.
Isolating the Problem Further

Try to isolate the problem and identify the faulty component.
To isolate the problem further:

• Try and isolate the problem by checking the logs to see:
• If the StartRecord command was called by the RCM. See Checking the RCM Logs
on page 497.
• If the ScreenAgent received the StartRecord command. See Checking the ScreenAgent
Logs on page 496.
The following is a sample StartRecord in the ScreenAgent log:
INFO: 02/01/08 10:56:31:015 IncCommandsHandler:
StartRecording received from IC IpAddr(screen-env3:2010).
capture handle(10000001),
DestinationAddr(screen-env3:30100),
DestinationResourceId(93); Context = (676)
• If NiceScreen received the StartRecord command. See Checking the ScreenAgent Logs
on page 496.
Checking the NiceScreen Logs

Settings for your NiceScreen Logger are stored in the MMLogger.cfg configuration file located in
the NiceScreen Logger installation directory. LogFile_Path indicates the location of the
NiceScreen log files.
The logs are located in the installation folder, by default in:
C:\Programs\Nice Systems\Multimedia Logger.
To check the NiceScreen Logs:

• Review the NiceScreen Logger logs (logfileXX.log).
Checking the ScreenAgent Logs

In NICE Perform Release 3 Service Pack 3 the logs are located in C:\Program Files\Nice Agent.

In NICE Perform Release 3 Service Pack 4 and above, the logs are located in different folders
depending on the Operating System. For Windows 2000, Windows XP, Windows 2003 Server the
logs are in C:\Documents and Settings\All Users\Application Data\Nice\Screen Agent. For
Windows Vista the logs are in C:\ProgramData\Nice\Screen Agent.
To check the ScreenAgent Logs:

• Review the ScreenAgent logs (LogfileXX.dat).
Checking the RCM Logs

The log files are located in the D:\Program files\NICE Systems\NICE CLS\Bin\Log.
To check the RCM Logs:

• On the NICE Interactions Center, open the RCM process (Resource Manager console), or
review the RCM.log.xxx.txt files.
Checking the Call Server Logs

The log files are located in the D:\Program files\NICE Systems\NICE CLS\Bin\Log.
To check the Call Server Logs:

• Review the Call Server log files (Call Server.log.xxx.txt files).
Checking the Call Status in the Database

Check the status of the calls that failed to record the screen in the database.
To check the call status in the database:

• Check tblExceptionXX in the nice_interaction table (query on iInteractionID). For more
details, see the Interaction Data Reference Guide.
Checking the Rule

In the Rules Manager verify that there is a valid rule for recording screens. See the Rules Manager
Guide for more information and the Interaction Data Reference Guide for details on the fields in
the Filter Expressions.
To check the rule:

1. In the Audit Trail, check if the rule has been changed recently. See the Audit Trail Guide for
more information.
2. In the NICE Perform applications suite, open Rules Manager > Scheduler.

3. Select the relevant rule, and click Edit Rule.
4. Click next to Filter.

Figure 16-6 Rule Filter
5. Verify that Screen is selected, check all the Rule Filter tabs and verify that the filter does not
exclude the required screen interactions.
Checking the Naming Resolution

Naming resolution is the ability to resolve between the computer name and IP. NICE Perform uses
the computer name.
To check the naming resolution:

• Ensure that the DNS is configured on the workstation and the Interaction Server.
Perform connectivity checks.
1. Check that port 30100 is open for incoming communication on the NiceScreen Logger server.
2. Check that port 2010 is open for incoming communication on the NICE Interactions Center
server.
3. Check that port 2001 is open for incoming communication on the NICE SAMS server.
4. Verify the NAT/PAT connectivity and configuration:

a. The services must be accessed by fully qualified machine names; a DNS service should be
available to allow names resolution.
b. For a NAT/PAT environment, verify that useMachineName is set to True in the
configuration file for each service.
IMPORTANT Any change of configuration files should be performed ONLY in consultation with
NICE Support!
5. The ScreenAgent process uses arbitrary ports to communicate with the ScreenLogger,
Interactions Center and SAMS Server. Verify that outgoing communication is not blocked
from the ScreenAgent workstations.
6. Check that the following ports are open.
Port List
Port List (cont.)

Verifying the ScreenAgent Configuration

The ScreenAgent must be configured for the correct Interaction Server. This is set during the
installation and can be verified in the ScreenAgent registry.
To verify the ScreenAgent configuration:

1. In the workstation registry, open the registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Nice Systems\Nice Screen\ClsList
2. Check the ClsAddr1 … ClsAddr5 parameters. These are the network addresses of the
associated NICE Interactions Centers.
Testing the ScreenAgent

Start a call and verify monitoring the screens.
To test the ScreenAgent:

1. Start a call on an agent workstation.
2. In the NICE Perform applications suite, open Monitor.
3. Click the Settings button. In the Settings window, select Screen as your Recording
Media. Click Save .

4. In the Folders pane, select the group that includes the agent whose activity you want to
monitor. The agent names in the selected group are displayed in the Details pane.
5. Select an agent who is logged in and on the phone. (Indicated by this icon ).
6. Click Monitor . The Screen icon should appear in the Monitor column.
7. NICE Player opens and plays the agent's screen. If there is a recording problem, the relevant
error will appear. Use this reason to resolve the recording failure.


without any options.
IMPORTANT
• DO NOT use text wrap as this will corrupt your configuration file.
• Any change of configuration files should be done ONLY in consultation with NICE Support!!!!

• Restart the Interaction Server during non-production hours.
• Open Interaction Server service manager and click Stop NICE CLS.
• Shut down Windows normally.
• Reinstall the ScreenAgent software. This should only be done in consultation with NICE
Support.
• All the files on the F: partition are managed by the NiceScreen Logger and are required. Do
not delete these files.
• If you are using Terminal Server on Citrix check the connectivity.
• If you are doing your mapping in System Administrator (and not manually using a file):
• In CLS Definitions > CLS Servers, select a CLS Server.
• Click the Configuration tab and click RCM.
• Validate that the Read Mapping from File parameter value is set to No.
• Solutions for various additional troubleshooting and performance issues are detailed in:
• NICE ScreenAgent Installation/Operational Problems on page 503
• NICE ScreenAgent Optimization and Performance Issues on page 505
• NiceScreen Logger Configuration Parameters on page 507
Other Useful Tools

• Run MML_ManagementTester.exe to view a list of the NiceScreen Loggers on the network.

Figure 16-7 MMI Management Tester
• The Expansion Tool is an internal utility which enables you to expand the number of input
channels and/or the amount of storage capacity of the NiceScreen Logger. The Expansion Tool
performs the necessary modification and adjustment to the NiceScreen Logger data system
and configuration file while preserving the existing metadata. For further details, see the
NiceScreen Expansion Tool Guide.
IMPORTANT
This tool should only be used in consultation with NICE Support!!!!
• Run LogCollector.exe on the workstation to gather the Player log, Playlist, Screen Log and
any other selected files. The files will be collected in the C:\Program Files\NICE
Systems\NICE Player\temp folder. After a full day of collecting the logs, zip the directory
and send the zip file to NICE Technical Support.

NICE ScreenAgent Installation/Operational Problems
If... Do the following...
You want to verify if a remote You need to check that no secondary capture driver was
control application is installed on installed by the remote-control application. To do so follow
the workstation the following steps:
1. Enter the Registry (use RegEdit)
2. Examine the value of the “\Device\Video0” key
located in the following hive path:
HKEY_LOCAL_MACHINE\\HARDWARE\\DEVICE
MAP\\VIDEO.
The video driver name is identified by the string that is

located before the \Device0 string. For example: If
the string is:
“\Registry\Machine\…\Services\atirage3\Device0”,
the video driver name is: atirage3.
3. Locate the video driver name (found in the above

step) in the following hive path:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentCont
rolSet\Services.
4. Enter the Device0 folder located under this folder.
5. Open the InstalledDisplayDrivers key.
The InstalledDisplayDrivers key contains a list of

the device drivers that are loaded for the Display
adapter. In a normal case the list should contain one
driver which is the driver used by the Display adapter
itself. If more than one driver is listed then there is a
good chance that a remote-control application added
the second driver.

ScreenAgent installation fails and Manually remove all ScreenAgent related files as follows:
regular uninstallation procedures 1. Make sure ScreenAgent is down (use
fail ScreenAgentTester)
2. Delete the ScreenAgent folder from the workstation’s
hard disk.
3. Open the Workstation registry and delete the
following keys:
• Before deleting this key, write down the string value of
AppGUID.
HKEY_LOCAL_MACHINE\Software\NICE
Systems\NICE Screen.
• Under the following key

HKEY_LOCAL_MACHINE\Software\Microsoft\Wi
ndows\CurrentVersion\Run - Delete the following
String keys: AGNTREC and AGENTREC (if exist).
• Open the following hive:
HKEY_LOCAL_MACHINE\Software\Microsoft\Wi
ndows\CurrentVersion\Uninstall. There are
several keys with a long string name (for example:
{23e76x3…-…-…-…-…}).
Search for the key that matches the string from the
AppGUID key. When located, delete the key.
4. Reboot the workstation and install the ScreenAgent.
ScreenAgent fails to load Change the order in which the ScreenAgent loads.
To do this:
• Increase the WaitForActivate parameter in the Agent
registry file.

NICE ScreenAgent Optimization and Performance Issues

This table provides adjustments for optimized performance.
There is a slow-down in general Make the following changes in ScreenAgent registry.

performance of the workstation • Set the CaptureMethod to Scraper in the Agent
registry file.
• Set the CompressionLevel to 1 in the Agent registry
file.
• Lower the workstation’s screen resolution/color depth.
• Set EnableAnimation=0, EnableFullDrag=0,
EnableScreenSaver=0, and EnableWallPaper=0 in
the Agent registry file.
• Obtain and install the latest display drivers.
• Scraper CaptureMethod only: Lower Thread Priority
to 0 (Below Normal).
The Data Rate is too high • Lower the workstation’s screen resolution/color depth.
• Set EnableAnimation=0, EnableScreenSaver=0,
and EnableWallPaper=0 in the Agent registry file.
• Set EnableFullDrag=1 in the Agent registry file.
• Set CompressionLevel=2 in the Agent registry file.
• Increase the value of CacheSize in the Agent registry
file.
• Set CaptureMethod=1 (Hooking) in the Agent
registry file. In most cases, the Scraper method
produces more data than the Hooking method, but
there are exceptions to this rule, therefore it is
advisable to check this per workstation.
The ScreenAgent disconnects • Make adjustments for a high data rate (see the
due to a high data rate previous item in this list)
• Increase RecBuffSize in the Agent registry file.
• Increase the interval between two consecutive
KeyFrames: increases MinTimeBetweenKF in the
Agent registry file.

The following message appears This message indicates that the Capture component did
when you start the workstation: not successfully load.
Capture layer initialization failed. • Increase the timeout of WaifForActivate in the Agent
Application is aborting. If you are registry file.
after installation restart the
• Check for conflicts with the Display Driver or other
computer.
application that is preventing ScreenAgent to load the
Capture component.
You need to change the scraping • Change CaptureMethod in the Agent registry file.
method used by the Scraper
capture method.

NiceScreen Logger Configuration Parameters

Your system settings for your NiceScreen Logger are stored in the MMLogger.cfg configuration
file located in the NiceScreen Logger installation directory. This section provides information
about the parameters contained in the MMLogger.cfg file.
WARNING
Do not edit values in the MMLogger.cfg file. Editing values directly in the MMLogger.cfg
file can cause your system to malfunction!
The following table lists the parameters contained in the MMLogger.cfg file:
Parameter Description Value
General
Logger_Id Logical ID for the NiceScreen Logger Default = 1
Max_Input_Bandwidth_KbSec Not used. Default = 6000
Periodical_Check_Resource_Hours Determines an idle time interval between Default = 1

which the Logger saves all unsaved
buffers and synchronizes time with the
client for a specific channel.
Concurrency
Num_Of_Dispatch_Threads Number of threads in the Logger engine Default = 25

thread pool.
Num_Of_JobProcessor_Threads Job processor thread pool size. Default = 5

Note: Do not define more than 20.
File
Logic_File_Size_In_Kb Same as sector in a conventional file Default = 384

system. Used once for Data System
construction. A large file size will reduce
the Data System size but could cause
ineffective disk usage.
File_Page_Size_In_Kb Part of the file which the Logger retains in Default = 128
memory, called cache quantum or page.
Usually the Logger reads/writes a whole
page from/to disk. This value should not
exceed the file size. A large page size
improves I/O operation performance but
utilizes a lot of computer memory.
Storage

NiceScreen Logger Configuration Parameters (Continued)
Storage This section lists all the storage portions used by NiceScreen
Logger.
The number on the right side of the equal sign (=) indicates
the order in which this portion is used for Logger storage.
Each portion (file and partition), must be listed here with its
ordinal number.
Example:
STORAGE_PORTION_1 = 1
STORAGE_PORTION_2 = 2
Note: You can define your own storage portion name, but
you must then use this same name to configure the Storage
Portion parameters (path+size). See below.
Storage Portion Each storage portion must be defined as a separate section

in the file using the same portion name that appears in the
Storage section (described above).
Two parameters are listed for each portion:
• File_Address - specifies portion location
If the storage portion is a file, the complete path to the file
must be defined. If the portion is a partition, the partition
letter must be defined as follows: \\.\X, where X is an
example of a partition letter.
• Size_In_Mb - size of the portion in MB
Following is an example of a file storage portion:
[STORAGE_PORTION_1]
File_Address = D:\Data\Storage_File_Name1.dat
Size_In_Mb = 5000 ; MB
Following is an example of a partition storage portion:
[STORAGE_PORTION_2]
File_Address = \\.\Z:
Size_In_Mb = 17000 ; MB
File_Address Address of file or partition. The Logger Default =

will create a file if it does not exist yet. D:\MMLData\M
ML_Data_File_1
.dat
or
\\.\Z:

Size_In_Mb Size of storage. Default = 5000

Note: Even if a file or partition is larger
than this value, the Logger will use the
amount of storage space defined here.
Data System
DB_Path Path of the Logger’s Data System files. Default =

D:\MML_Root\D
ataSystem
Num_Of_Cached_Pages_For_Table Number of Data System Table pages that Default = 200

the Logger retains in memory.
Note: We recommend retaining the
Number of Input Channels + Number of
Output Channels * 2 pages. However, if
your computer does not have enough
memory, you can define a lower value.
Average_Packet_Size_Kb Average packet that the client workstation Default = 10

sends to the Logger. Dependent on client
behavior.
Note: A smaller value may increase the
number of descriptors resulting in a larger
Data System size. A larger value may
cause ineffective storage usage.
Auto Deletion
Auto_Deletion_Enabled Enables/Disables the auto deletion Default = 1

mechanism. 1 = Yes, 0 = No
Files_High_Watermark_Percent Auto deletion starts when the percentage Default = 80

of stored files is greater than this value.
Files_Low_Watermark_Percent Auto Deletion stops when the percentage Default = 50

of stored files is less than this value.
Descriptors_High_Watermark_Percent Auto deletion starts when the percentage Default = 80

of descriptors exceeds this value.
Descriptors_Low_Watermark_Percent Auto Deletion stops when the percentage Default = 50

of used descriptors is less than this value.
Channels
Num_Of_Input_Channels Number of recording input channels. Default = 100

Used once for Data System construction.

Num_Of_Output_Channels Number of output channels used for Default = 3

playback.
Memory Cache
Cache_Confidence_Coefficient Increases memory cache size to provide Default = 3

ample memory above the memory
allocated per recording channel.
If the Logger does not have enough
memory, this value can be decreased.
Memory Allocation
Protocol_Message_Size Maximum message size buffer for Default = 1024

request and response (not data!!).
Protocol_Message_Allocator_Coeffici Allocation coefficient for static memory Default = 3

ent pool of protocol messages.
Log_Message_Size Log message buffer (response) in bytes. Default = 512

Maximum log message size that would
be sent to the remote client/log server.
Log_Messages_In_Pool Number of log messages in the static Default = 500

pool. Remaining log messages are then
rejected until space in the pool is freed.
Communication
Connection_Alive_Check_Interval_In_ Time interval (seconds) for checking Default = 60

Sec whether an idle connection is alive.
Connection_Alive_Check_Duration_In Time duration (seconds) of the check Default = 3

_Sec connection time.
Logger_Port Accept incoming connections port. Default = 30100
Max_Async_Writes_Per_Connection Maximum number of asynchronous write Default = 10

operations per connection.
-1 = no restriction
Remaining requests are queued. A
higher value improves performance for a
system with a slow disk.
Allocation_Retries_On_Full_Cache Number of retries to allocate cached Default = 10

buffer, when Memory Manager is full,
before aborting. A higher value reduces
performance in a busy system but will
give greater safety, since the number of
aborts will be reduced.

Items_Limit_In_Connection_Output_Q Watermark threshold of items in output Default = 50

ueue queue per connection, before error is
created. A higher value increases
memory usage and possibly reduces
performance, but ensures greater safety
since there are no aborts on output.
Logs to Screen
Severity_To_Screen Log severity to screen. Default = 2033
Logs to File
LogFile_Path Location of the NiceScreen log files. Default =

C:\Program
Files\NICE
Systems\Multim
edia Logger
Number_Of_LogsFiles Default = 5
Size_Of_Each_LogFile Size of each log file in Kb. Default =

1000000
Severity_To_File Log severity of log files. Default = 18417
Logs To System Log
Host_Name Name of host that appears in the Default = MML

message source in the System Log.
Severity_To_SysLog Log severity of system log. Default = 1985
Publisher General
Verbose_Mode Do not change this parameter. Default = 32

17
Storage Center Troubleshooting Flow
This chapter describes common troubleshooting issues and how to verify that the Storage Center
was properly installed. It also includes a troubleshooting workflow to solve various issues.
Contents
Common Troubleshooting Issues...............................................................................514

Installation Freezes: SNMP Dependencies..............................................................514
Archiving Failure: Multiple Network Interface Cards ................................................515
Storage Center Is Not Communicating with the NMS ..............................................515
Allowing Duplicate Archiving ....................................................................................521
Storage Center Does Not Archive Calls ...................................................................521
Other Useful Tools....................................................................................................524
Verifying the Storage Center Installation ...................................................................526
Implementing the Troubleshooting Workflow............................................................527
Verifying that the Storage Center Database is Selected ..........................................527
Verifying the Used and Free Space..........................................................................528
Gathering Storage Center Activity Statistics.............................................................528
Checking that the Archiving Rule Inserts New Calls ................................................528
Troubleshooting the Archiving Rule .........................................................................529
Checking the Storage Center Archiving ...................................................................531
Checking the Storage Center Logs ..........................................................................531
Checking the Nice Storage Streaming Service Logs................................................532
Verifying Playback from Storage Center Only ...........................................................533
Chapter 17: Storage Center Troubleshooting Flow 513

Common Troubleshooting Issues

Installation Freezes: SNMP Dependencies
During the software installation, the SNMP service is stopped. If the SNMP service has
dependencies, the service cannot be stopped. This causes the installation to freeze.
This problem occurs on HP servers.
Workaround: Stop the dependencies manually and then run the installation.
To stop SNMP service dependencies:

1. From the Control Panel, select Administrative Tools > Services. The Services window
appears.
2. Double-click the SNMP service to open its Properties window. Click the Dependencies tab.
In the example above, the SNMP service has four dependencies.

3. If the SNMP service has dependencies, do the following:
a. Close the SNMP Service Properties window.
b. In the Services window, right-click the dependent service and select Stop.
c. Repeat this procedure, for each dependent service.
4. Install the Storage Center software.
5. After the installation is completed, restart the Storage Center. (The services will then restart
automatically.)

Archiving Failure: Multiple Network Interface Cards

If interactions are not being archived, check if the Storage Center server has multiple active
network interface cards. If the Storage Center server has more than one active network interface
card, the Storage Center IP address defined in the System Administrator must be the primary
network connection.
To check the network connections of the Storage Center:

1. From the Settings menu, select Network Connections. The Network Connections window
appears.
2. From the Advanced menu, select Advanced Settings. The Advanced Settings window
appears.
3. Verify that the first connection is defined with the IP address of the Storage Center server.
4. If the first connection is not defined with the IP address of the Storage Center server, change
the order of the connections (using the Up/Down arrows), or change the IP address of the first
connection to the IP address of the Storage Center server.
Storage Center Is Not Communicating with the NMS

After the Storage Center is installed and the NMS server is configured, the Storage Center agent
appears in the NMS Unicenter 2D Map view and the Storage Center sends traps to the NMS
console.
If the Storage Center agent does not appear in the Map view and the Storage Center traps do not
appear on the NMS Console, check if the SNMP service, the NICE Storage Center service and the
DCOM agent are configured properly. These objects are configured transparently during the
installation.
Perform the following verification tests to ensure that all the objects are configured properly.
• Verify SCLoader Service User on page 516

• Verify the SnmpAgent Object on page 517

• Verify the SNMP Service Settings on page 520
Verify SCLoader Service User

This procedure verifies that the SCLoader Service is logging on to the same account (user) as the
SNMP Agent and SNMP service objects.
To verify which user will be logged on by the SCLoader:

1. From the Start menu, select Run.
2. In the Run window, enter services.msc and click OK.

The Services window appears.
3. Right-click the SCLoader Service and select Properties. The SCLoader Properties window
appears.

4. Click the Log On tab. Verify that This account is selected and that the correct username
appears in the field.
The next steps verify that the SnmpAgent object and SNMP Service both log on using the same
account.
Verify the SnmpAgent Object

This procedure verifies that the SnmpAgent object is logging on to the same account (user) as the
SCLoader and SNMP service objects.
To verify the SnmpAgent Object:

2. In the Run window, enter dcomcnfg and click OK.
The Component Services window appears.

3. Navigate to Components Services > Computers > My Computer > DCOM Config.
4. Right-click the SnmpAgent Object and select Properties.

The SnmpAgent Object Properties window appears.
5. Click the Identity tab.

6. Verify that the This user radio button is selected, and that the User is the same user as
defined in the other Storage Center objects running this application.
• If the user is properly defined, this procedure is complete. Continue with Verify the
SNMP Service Settings on page 520.
• If the user is not properly defined, continue with step 7.
7. If the This user radio button is not selected or the username is incorrect, you must do the
following before changing the agent properties:

a. Stop the Storage Center Server.

b. Stop the SNMP Service.
c. Verify that there are no StorageCenter.exe processes running in the Windows Task
Manager.
d. Select the This User radio button and define the user name.
8. Click the Security tab. Permissions areas are displayed.
9. In the Configuration Permissions area, click Edit. The Change Configuration Permissions
window appears.
10. Verify that the user belongs to the Administrators Group with Full Control Permission.

11. Click OK to close the SnmpAgent Object Properties window. Then exit the application.
Verify the SNMP Service Settings

This procedure verifies that the SNMP Service is logging on to the same account (user) as the
SCLoader and the SnmpAgent objects.
To verify the SNMP Service settings:

2. In the Run window, enter services.msc and click OK.
The SNMP Services window appears.

3. Right-click SNMP Service and select Properties.

4. Click the Log On tab. Verify that the This account field contains the same username as the
Storage Center uses for logon.
Allowing Duplicate Archiving

By default, an interaction can only be archived one time. If an interaction matches the filter for
several rules, the interaction will be archived according to the first rule that runs. It will not be
archived again when other rules run.
You can change this policy so that an interaction may be archived per rule. In this scenario, an
interaction may be archived multiple times - each time the interaction matches a running rule filter.
This is configured via the Rule Engine configuration file.
To allow multiple archiving:

1. Navigate to the following directory: Program Files\Nice Systems\Applications\Server
Bin.
2. Open the NiceApplications.RuleEngine.Service.exe.config file.
3. Navigate to the following key: AllowDuplicateArchiving
4. Change the value to True. (The default is False.)
Storage Center Does Not Archive Calls

Perform the following check sequence:
Rule Engine Inserts New Calls
Check the Storage Center tables, tblStorageCenterXX in the nice_interactions of the relevant
calls to see whether the Rule Engine inserts new calls:

• Query on iArchiveID to see if it increases, and check the last interactions Start and Stop times.
When the time of the last interaction is known, you can tell whether new interactions were
inserted.
• If the Rule Engine inserts calls, the problem is in the Storage Center.
New Rows are Not Inserted
New rows are not inserted in the tblStorageCenterXX tables:

• Check whether new tasks are added to the tblSCTasks table in the nice_admin database.
• If tasks are added, check that they match the rule filters.
Errors in the Rule Engine File
Check the Rule Engine Log file for errors.
Active Rule in the Rule Manager
Check that there is an Active Rule in the Rule Manager, and check its schedule.
Storage Center Rule Schedule
In the nice_rule database, check the next schedule time of the Storage Center rule by running the
following query:
A section of the tblREStorageCenterRule table appears below, showing the format of the
date fields.

dtLastRunTime is the local time of the Application Server.
Check that the Rule Engine database is not full.

Rule Engine Generates Actions
Check that the Rule Engine generates actions for the Storage Center Rules by running the
following query:
SELECT *
Check the Start time, Stop time and interaction ids in the results of the query.
If the table does not fill up, check that the Event Providers collect the interactions:
Run the query:
SELECT *
Check that the results contain calls that are not archived.
• If the table does not contain the calls, the error may be in the Event Collection step. Call
the NICE Customer Services.
• If the interaction is in the Event table, the problem may be in the Generation step.
The reason could be one of the following:
a. The Generation Query takes a long time and does not complete because of timeout.
b. The rule has problems with the file name mask and the post process of the Storage
Center rule, stored procedure spREStorageCenterRulePostProcessing, failed.
Check that the File name mask is not empty by running the following query:
nvcFileNameMask
• The problem could be in the Execution step:
In a Multi Data Hub environment, the secondary Rule Engine Log file may contain the
following message:
transaction coordinator. ]
OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin::JoinTransaction
It may be that the Microsoft Distribute Transaction Coordinator (MSDTC) is not
configured properly.

Storage Center rules require that the MSDCT works correctly. This can be checked by
performing the following command:
BEGIN TRANSACTION
COMMIT
If the command fails, MSDTC must be fixed. See Enabling MSDTC to Function in a Multi
Data Hub Environment.
Rules defined by the Rules Manager are relevant for all sites.
The Rules definitions are written to the Master site and are replicated to all secondary
sites.
Other Useful Tools

• Run NMFExplorer.exe. The NMF Explorer utility provides NMF file analysis and enables
you to:
• Group packets by type in tree view
• View full packet details
• Save as AUD Format

• Play a file using NICE Media Player

Verifying the Storage Center Installation
Verifying the Storage Center Installation

After installing Storage Center, you verify that it was installed properly by generating a call from a
Logger and then playing it back.
To verify Storage Center installation:

1. Configure and install Storage Center.
2. Restart the Storage Center.
3. Create a new storage group and unit in the System Administrator.
4. Create a new rule that generates a call from the Logger in .nmf format and plays the call back
in the Business Analyzer.
5. Open the log files for the Storage Center and Storage Streaming Service and verify that no
errors occurred.
6. Verify that the following services are enabled:
• LogService
• Nice Storage Streaming Service
• Nice Storage Center Service
Verify that these services

are enabled
7. If the installation was not successful, continue to Implementing the Troubleshooting

Workflow on page 527.
8. Continue to Verifying Playback from Storage Center Only on page 533.

Implementing the Troubleshooting Workflow

The diagram below describes the workflow for troubleshooting problems with Storage Center.
Verifying that the Storage Center Database is Selected

In the System Administrator, check that the Storage Center database is selected.
• In System Administrator, in the Organization tree, under Master Site > Database Servers,
General tab, check that the nice_storage_center database is selected in the Databases on
this SQL Server area.

Verifying the Used and Free Space

Check the used and free space on the Storage Center servers and storage areas.
1. In System Administrator, in the Organization tree, under System Monitoring > Storage
Center Servers branch, expand the hierarchy to view a list of all Storage Centers.
2. Expand a Storage Center to view its storage areas.
3. Select a Storage Center or one of its storage areas. The Status window appears.
Gathering Storage Center Activity Statistics

Use the Storage Center Investigator to verify that all the required files are archived on the file
system. The Storage Center Investigator is installed together with the Storage Center in the same
installation directory.
The default path is: C:\All Programs\NICE Systems\Storage Center.
Checking that the Archiving Rule Inserts New Calls

Archiving to Storage Center starts with the archiving rule. The rule determines which calls needs
to be archived and then adds the required information to the tblStorageCenter and tblSCTasks
tables in the database.
1. Check the Storage Center tables, tblStorageCenterXX in the nice_interactions of the
relevant calls to verify that the Rule Engine inserts new calls. From Storage Center
Investigator, query iArchiveID to see if the ID number increases, and check the Start and Stop
times of the last interactions. When you know the time of the last interaction, you can verify
whether new interactions were inserted.
2. Check whether new tasks are added to the tblSCTasks table in the nice_admin database. If
tasks are added, check that they match the rule filters.

If the Rule Engine inserts new calls into tblStorageCenterXX, the archiving rule is probably
correct and the problem is in the Storage Center. In this case continue with Checking the Storage
Center Archiving on page 531. If there are no new calls, continue with Troubleshooting the
Archiving Rule on page 529.
Troubleshooting the Archiving Rule

Troubleshooting the archiving rule comprises the following steps:
1. Checking the Rule Engine Log file for errors.
2. Checking that there is an Active Rule in the Rule Manager.
3. Checking the Storage Center rule schedule. See the procedure below.
4. Checking that the Rule Engine generates actions. See page 529.
To check the Storage Center rule schedule:

1. In the nice_rule database, check the next scheduled time of the Storage Center rule by
running the following query:
A section of the tblREStorageCenterRule table appears below, showing the format of the date
fields.

dtLastRunTime is the local time of the Applications Server.
2. Check that the Rule Engine database is not full.
To check that the Rule Engine generates actions:

1. Check that the Rule Engine generates actions for the Storage Center Rules by running the
following query:
SELECT *

2. In the query results, check the start time, stop time and interaction IDs.
3. If the table does not fill up, check that the Event Providers collect the interactions by running
the query:
SELECT *
4. Check that the results contain calls that are not archived.
• If the table does not contain the calls, the error can be in the Event Collection step. Call the
NICE Customer Services.
• If the interaction is in the Event table, the problem can be in the Generation step. The
reason could be one of the following:
• The Generation Query takes a lot of time and does not complete because of timeout.
• The rule has problems with the file name mask and the post process of the Storage Center
rule. The database function spREStorageCenterRulePostProcessing failed. Check
that the File name mask is not empty by running the following query:
nvcFileNameMask
• The problem could be in the Execution step. In a Multi Data Hub environment, the
secondary Rule Engine Log file can contain the following message:
transaction coordinator.]
OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin:JoinTransaction
• It is possible that the Microsoft Distribute Transaction Coordinator (MSDTC) is not
configured properly. Storage Center rules require that the MSDTC works correctly. This
can be checked by performing the following command:
BEGIN TRANSACTION
COMMIT
If the command fails, MSDTC must be fixed.

Rules defined by the Rules Manager are relevant for all sites. The Rules definitions are
written to the Master site and are replicated to all Secondary Data Hubs.
Checking the Storage Center Archiving

• Check the iFSArchiveClass column in the tblStorageCenterXX table. The values represent
the following:
• 0 - Not archived yet. Check if the Storage Center service is up and check the Storage
Center log files.
• 1 - Archiving in progress. Check the Storage Center log files.
• 2 - Archived. Everything seems to be OK.
• 5 - Archiving failed. Check the Storage Center log files.
• 7 - Call was not found in the Logger. Check why the call is not found in the Logger.
Checking the Storage Center Logs

1. Set the log level to DEBUG (the level is usually set to LOW), and change the entry in the
registry:
a. From the Start menu, select Run. The Run window appears.
b. Enter Regedit. The Registry Editor window appears.
c. From the Edit menu, select Find, and enter DebugCategoryMask.
d. Set the value to 255.
e. Restart the Storage Center service.
2. Review the Storage Center logs. By default, the log files are stored in the same directory
where the Storage Center is running. The default log file names are:
Time Event Category Description
Owner Thread ID

3. Track the task snapshots in the log files. Each task snapshot details the current tasks and the
number of elements distributed among Loggers.
Task Name Task ID Rule ID
Logger ID Number of Elements
Checking the Nice Storage Streaming Service Logs

You check the Nice Storage Streaming Service logs to verify that playback performed from
Storage Center is successful.
By default, the log files are stored in the following directory: ...\Program Files\Nice
Systems\Storage Center\Nice Playback Service SC\StorageStreaming<Process
ID>Logs. The Process ID is a value that changes each time the system is restarted.
The following procedure lists the text strings you need to find in the Nice Storage Streaming
Service logs in order to verify that playback from the Storage Center was successful.
To check the Nice Storage Streaming Service logs:

1. Search for the text ERR:, and verify that it does not appear in the log file.
2. Search for the text StartOutgoingSessionEvent, and verify that it appears.
3. Search for the text OpenSession, and verify that it appears.
4. In the same row as the OpenSession text, verify that Result=OK appears.
5. Search for the text StartStreamingPackets, and verify that it appears.
6. Search for the text StopOutgoingSessionEvent, and verify that it appears.
7. Search for the text StopStreamingPackets, and verify that it appears.

Verifying Playback from Storage Center Only

Verifying playback from the Storage Center entails the following main steps:
• Verify that the Storage Center is archiving properly.
• Configure NICE Perform to locate interactions in the Storage Center only.
• From Business Analyzer, playback an archived interaction from the Storage Center.
The procedure below comprises these steps.
To verify playback from Storage Center only:

1. Complete Verifying the Storage Center Installation on page 526.
2. In the System Administrator, change the Locate Settings to locate interactions in the Storage
Center only.
IMPORTANT
When you change the Locate Settings, playback is temporarily discontinued while the Playback
Service restarts. Only perform this procedure at times when playback can be interrupted.
a. Verify that you are working in Technician mode: From the Settings menu, select
Technician Mode.
b. Navigate to Applications > Configurations > Locate Settings.
c. Note the original settings in order to change them back in Step 4.

d. In the Locate Settings area, select Search Storage Center only.
The Storage Center list appears.

e. Click Add. The Available Storage Centers window appears listing all defined Storage
Center servers that were not yet added the Locate Order list.
f. Select one or several Storage Center servers and click OK. The servers appear in the list.
g. Click Save .
The Playback Service restarts and registers the new Locate Settings.
NOTE: When the Playback Service is restarting, playback is temporarily stopped.
3. From the Business Analyzer application, query for archived calls. See the Business Analyzer
Guide for more detailed information.
a. Edit or create a query.

The Advanced Query window appears.
b. Select Call Profile, and in the Archiving area, select Archived for Voice and Screen.
c. Click Save & Run.
The archived interactions are listed in Business Analyzer. This list includes interactions
stored in the Storage Center as well as other locations. Since we defined the Locate
Settings to occur only in the Storage Center, not all these interactions will play back.
d. Play back an interaction. If playback is not successful, select another interaction until you
find one from the Storage Center.
e. Verify in the Playback Service logs that the playback was performed from Storage Center.
See Checking the Nice Storage Streaming Service Logs on page 532.
4. To change the Locate Settings back to their original settings, repeat Step 2 but use the original
Locate Settings in Step d.

A
Collecting Information Using the Log
Collector
Each NICE Perform server and application stores vast amounts of information that proves useful
for analyzing and troubleshooting a system.
Introducing the Log Collector! The Log Collector is an easy-to-use utility that gathers the
information from all your NICE Perform servers and local workstation, in the form of reports, for
easy transfer for analysis.
The Log Collector is compatible with all versions of NICE Perform Release 3, Service Pack 4 and
higher.
The Log Collector does not require an installation process. Simply copy the Log Collector files to
your hard drive and you are ready to go!
Contents
What is the Log Collector? ..........................................................................................538

What is a Component?.............................................................................................538
What is a Report? ....................................................................................................539
FAQs ..............................................................................................................................540
Limitations.....................................................................................................................541
Getting Started ..............................................................................................................542
Collecting Information from NICE Perform Servers ..................................................543
Collecting Information from Your Local Machine......................................................550
Saving and Exporting Reports ....................................................................................555
Importing Saved Reports for Review..........................................................................556
Scheduling Collection ..................................................................................................558
Viewing Collected Reports ..........................................................................................560
Viewing and Saving Reports in other Applications...................................................562
Reports Created with the Log Collector .....................................................................563
Appendix A: Collecting Information Using the Log Collector 537

What is the Log Collector?

Your NICE Perform system is composed of many components such as Loggers, servers, and
applications. They can be installed on one or several different machines. The Log Collector has
the ability to collect all pertinent information generated by each component, in the form of
reports, to assist in evaluating and troubleshooting your system.
The Log Collector has two different collection groups:
• Server Reports - scans your NICE Perform system configuration using two different
techniques:
• Read from Database - you define the location of the SQL server where your NICE
Perform databases reside. Based on this information, the Log Collector locates all NICE
Perform components and relevant information stored on all machines.
• Add Manually - you define a server, and optionally an SQL server. You then select only
those components whose data you want to collect.
• Client Reports - collects data from your local workstation. The data collected includes
configuration files, registry files, etc.
Server and Client reports can be combined in one export file. They can also be viewed side-by-side
in Log Collector window.
The Log Collector is compatible with all versions of NICE Perform Release 3, Service Pack 4 and
higher.
your hard-drive and your are ready to go!
What is a Component?
A component is a functional unit of your NICE Perform system from which information can be
collected for troubleshooting procedures. Each component can produce several reports. Examples
of components are:
• ScreenSense
• Playback Server
• SQL Server
Figure A-1 Components
Components

What is a Report?
A report is a piece of information retrieved by the Log Collector for a specific component. A
report is the lowest level in the Log Collector Reports tree. Examples of reports are:
• System Administrator logs
• ScreenSense configuration
Figure A-2 Reports
Component
Reports

FAQs
FAQs
Can I select what information to collect?
Yes
The Log Collector has a discovery phase where all information that can potentially be
collected is listed. You can select to exclude items from your collection.
Can I specify a time frame for information collected in each report?
No
Currently, the Log Collector has a predefined time frame for each report. This is internal and
can differ for each report. This time frame cannot be modified.
Example: A report has a time frame for one month. You run the report on February 1.
Information will be collected from January 1 until February 1. If you run the report again on
February 15, information will be collected from January 15 until February 15.
Can I preschedule more than one collection?
Yes
To schedule repeated collections, or separate collections for Clients and for Servers (or for
selected components), start another instance of the Log Collector. Each instance of the Log
Collector runs as a separate application and you can specify a separate schedule in each.
Note that the first set of reports will be overwritten by the second set of reports. To avoid
losing the first set of reports, save the first set before the next run of the Log Collector.
Can I export the collected reports to be opened on a different machine?
Yes
To transport collected reports from one machine to another, create an Export file. Then import
the file on the new machine.
Where are the reports saved?
Reports are automatically saved to a temporary location on your local machine. You cannot
change this location. The reports will automatically be overwritten the next time that Log
Collector is run. To save reports, create an Export file. You can also save selected files by
opening them in another application and then saving.

Limitations
Limitations
The following limitations are currently in effect:
• The Packages report (in the Applications Suite section) fails when run on NICE Perform
Release 3, Service Pack 4.
• If the SQL is down, the Automatic Discovery (Read from Database) option does not work.
Reports must be selected manually.
• ClearSight reports are produced only when the system is configured for NT authentication.
• The Log Collector can only connect to Loggers which are installed as services. Loggers which
are installed as processes cannot be polled for reports.
• The Log Collector saves its reports in your local temp folder. You must verify that there is at
least 1 GB of free space on the local temp folder to store reports. Reports will not be saved if
enough free space is not available.
• The temporary location for all Log Collector reports cannot be changed. Reports are saved in
your local temp folder until they are exported to another location or overwritten by another run
of the Log Collector.
• Reports which are not exported to another location, will be automatically overwritten at the
time of the next run (scheduled or manual).
• When you use the Open With Viewer option to open a report, Log Collector opens the report
from its source (from the server), not from the copy it saved locally. Thus the contents of the
report seen might differ from the contents of the saved report.

Getting Started
Getting Started
Verify that there is at least 1 GB of free space on the local temp folder to store reports. Reports will
not be saved if enough free space is not available. The temporary location for reports cannot be
changed.
your hard drive and you are ready to go!
Figure A-3 Log Collector Application Files
To start the Log Collector, double-click NiceApplications.LogCollector.exe.

Continue with:
• Collecting Information from NICE Perform Servers on page 543
-or-
• Collecting Information from Your Local Machine on page 550

Collecting Information from NICE Perform Servers

Server Reports scans your NICE Perform system configuration using two different techniques:
• Read from Database - you define the location of the SQL server where your NICE Perform
databases reside. Based on this information, the Log Collector locates all NICE Perform
components and relevant information stored on all machines.
• Add Manually - you define a server and select only those components whose data you want to
collect.
When you create an export file, all reports that are currently in the Log Collector (Server reports
and Client reports) will be joined together in the export file.
Server and Client reports can be viewed side-by-side in the Log Collector window.
To collect information from all NICE Perform Servers:

1. If you have already collected Client reports, do not open the Log Collector again. The Server
reports can be added to the Client reports so that they can be included in the same export file.
Continue with Step 2.
-or-
To open the Log Collector, do the following:
a. Copy the Log Collector folder to your local hard drive.

b. Double-click NiceApplications.LogCollector.exe to start the Log Collector. This can

take a few seconds.
The NICE Perform Log Collector appears.
Figure A-5 NICE Perform Log Collector
Selection
Area
Reports
Collection
Log
Progress
Bar
2. If Server Reports is not the current view, select Server Reports.
3. You can connect to a server to allow automatic discovery of all components. Continue with
Step 4.
-or-
You can manually add only selected components. Continue with Step 5.
-or-
You can do both.
4. To connect to a server for automatic discovery, do the following:
a. Click Read From Database . The SQL Server Credentials window appears.

Figure A-6 SQL Server Credentials Window
b. Enter the Hostname or IP address for the SQL Server where the NICE Perform
databases reside.
c. Select SQL Server connection credentials.
d. Click OK.
Log Collector runs a discovery task and fills the Server Reports list with all the
information components that can be collected from your NICE Perform servers and
connected Data Hubs.
Figure A-7 Server Reports
e. Expand a branch to see all potential reports. Clear the checkbox of any report not currently
needed.

If you leave an item selected that is not installed (i.e., you leave ScreenSense selected and
ScreenSense is not installed), it will appear as an error after the collection is run.
Figure A-8 Server Reports - Selection
Component
Reports
Component
Reports
5. To manually add only selected components, do the following:
a. Click Add Manually .

The Choose Component window appears.
Figure A-9 Choose Component window
b. Enter the Host Name or IP Address of the Server where the components reside.
c. If the SQL server is installed on a different server, select Use dedicated SQL Server.
Then enter its host name or IP address.
d. Select components from the list. Then click OK.

The selected components appear under the new branch, Data Hub:Undefined.
Figure A-10 Server Reports - Undefined
Data Hub:Undefined
6. Click Start to start collection now or click Schedule Collection to start collection at
a later time (see Scheduling Collection on page 558).
The Server Credentials window appears.
Figure A-11 Server Credentials Window
Click here to fill the entire table

with the same values

7. In the Machine Credentials area, select one of the following types of Authentication:
• Current Windows User
• Different Windows User (enter a Password and Domain (optional) for the user)
TIP: You can enter the information (Authentication or Password and Domain) in one
row then click Copy Credentials to fill the entire table with the same values.
8. In the SQL Server Credentials area, select one of the following types of Authentication:
• SQL User (enter a Password and Domain (optional) for the user)
9. Click Collect. The Collection begins.

Figure A-12 Server Reports Results
10. When the collection is complete, you will see the following results:
a. Successful collection is indicated by a green checkmark
b. Unsuccessful collection is indicated by a red X .

c. The log area shows the status of collected reports. To clear the log area, select File >
Clear Log.
d. The Progress bar indicates Collection Finished.

11. Double-click a report that completed successfully in the Server Reports list to see its
contents in the right pane. Each report appears in a separate tabbed page. For tips on viewing
more than one report at a time, see Viewing Collected Reports on page 560.
NOTE: If you double-click a report that was unsuccessful or was not selected for
collection, the Log Collector will run again, for that report only, and try to locate its
information and create a report.
Figure A-13 Server Tabbed Page Output
12. To save reports or to prepare them for shipping, export them as follows:
a. Click File > Export. The Save As window appears.
b. Enter a name for the zipped file and browse to a location. Then click Save.
All Client and Server reports are saved in one zipped file.
The new zipped file can be imported into the Log Collector on a different machine for
viewing.

Collecting Information from Your Local Machine

Client Reports collect information from your local workstation.
When you create an export file, all reports that are currently in the Log Collector (Server reports
and Client reports) will be joined together in the export file.
Server and Client reports can be viewed side-by-side in the Log Collector window.
To collect information from your local machine:

1. If you have already collected Server reports, do not open the Log Collector again. The Client
reports can be added to the Server reports so that they can be included in the same export file.
Continue with Step 4.
-or-
To open the Log Collector, proceed with the following steps.
2. Copy the Log Collector folder to your local hard-drive.
3. Double-click NiceApplications.LogCollector.exe to start the Log Collector. This can take a

few seconds.
The NICE Perform Log Collector appears.

Figure A-15 NICE Perform Log Collector
Selection
Area
Reports
Collection
Log
Progress
Bar
4. Select Client Reports.
All information that the Log Collector can potentially collect from your local machine appears
under the Machine LOCALHOST branch. This includes machine settings and NICE
applications.

Figure A-16 Client Reports
5. Expand the Machine LOCALHOST branch to see all potential reports. Clear the checkbox of
any report not currently needed.
If you leave an item selected that is not on your machine (i.e., you leave Player selected and
Player is not installed), it will appear as an error after the collection is run. The information
found on your machine will be collected and saved in the form of separate reports.

Figure A-17 Client Reports - Selection
Component
Reports
Component
Reports
6. Click Start to start collection now or click Schedule Collection to start collection at
a later time (see Scheduling Collection on page 558).
Figure A-18 Client Reports Results

7. When the collection is complete, you will see the following results:
a. Successful collection is indicated by a green checkmark
b. Unsuccessful collection is indicated by a red X .

c. The log area shows the status of collected reports. To clear the log area, select File >
Clear Log.
d. The Progress bar indicates Collection Finished.
8. Double-click a report that completed successfully in the Client Reports list to see its contents
in the right pane. Each report appears in a separate tabbed page. For tips on viewing more than
one report at a time, see Viewing Collected Reports on page 560.
NOTE: If you double-click a report that was unsuccessful or was not selected for
collection, the Log Collector will run again, for that report only, and try to locate its
information and create a report.
Figure A-19 Client Tabbed Page Output
9. To save reports or to prepare them for shipping, export them as follows:

a. Click File > Export. The Save As window appears.
b. Enter a name for the zipped file and browse to a location. Then click Save.
All Client and Server reports are saved in one zipped file.
The new zipped file can be imported into the Log Collector on a different machine for
viewing.

Saving and Exporting Reports
Saving and Exporting Reports

When reports are created, they are stored in a temporary location on your workstation. The Log
Collector provides the ability to save all reports (Server reports and Client reports) in one zipped
file. This zipped file can be opened using the Log Collector on another machine or unzipped and
the reports opened one-by-one.
NOTE: Some types of reports, such as log files and configuration files, can also be
opened using other applications and then saved. See Viewing and Saving Reports in
other Applications on page 562.
All Successful reports will be exported. Failed reports will be copied to maintain the Reports tree
structure. Reports not marked to run will not be copied to the export file.
Figure A-20 Reports Exported
Create Export file from: Export file after reimporting:
To create a export file:

1. In the Log Collector window, select File > Export. The Save As window appears. All reports
are saved in one zipped file.
2. Enter a name for the zipped file and browse to a location. Then click Save.
All reports (Client reports and Server reports) are zipped together and saved.
The new zipped file can be imported into the Log Collector on another machine for viewing or
unzipped and viewed file-by-file.

Importing Saved Reports for Review

Reports that were zipped using the Export feature of the Log Collector can be imported on another
machine and viewed in the Log Collector window. The machine to which you import the reports,
does not have to be connected to the network. Reports that are imported cannot be updated or
rerun.
When you import a Log Collector file, the file structure of all reports is maintained making the
information you need quickly accessible.
To import exported files:

1. In the Log Collector window, select File > Import. The Open window appears.
2. Browse to the location of the saved zip file. Then click Open.
All reports (Client reports and Server reports) appear together in the Imported Reports list.
Figure A-21 Imported Reports window
Server Reports
Client Reports

3. Expand the branches in the Imported Reports list. Then double-click a report to view it in the
right-hand pane. Each report appears in a separate tabbed page. For tips on viewing more than
one report at a time, see Viewing Collected Reports on page 560.
NOTE: Reports that were unsuccessful cannot be rerun from the Import Reports list.

Scheduling Collection
The Log Collector can be prescheduled to create reports at a predetermined time. Each instance of
the Log Collector can be scheduled for one run. To schedule repeated collections, or separate
collections for Clients and for Servers (or for selected components), start the Log Collector
application again.
To schedule regular collection:

1. Open the Log Collector and select reports (Server and/or Client).
2. Click Schedule Collection . The Schedule Collection window opens.

Figure A-22 Schedule Collection window
Click here to fill the

entire table with the
same values
3. Select a Start Time.

4. In the Machine Credentials area, select one of the following types of Authentication:
5. In the SQL Server Credentials area, select one of the following types of Authentication:
• SQL User (enter a Password and Domain (optional) for the user)
row, then click Copy Credentials to fill the entire table with the same values.
6. Click Schedule.

7. The Log Collector window is minimized and a tray icon appears in the Notification area of
your desktop until the time when the Log Collector runs.
Figure A-23 Notification Area with Log Collector Icon
When the scheduled time arrives, Log Collector runs and the Log Collector window opens
with the results.
TIP: To cancel the collection before it runs, right-click the Log Collector icon and select
Cancel Scheduling.

Viewing Collected Reports

Collected reports can be viewed from within the Log Collector window. You can open and view
several reports side-by-side. Server reports and Client reports can be opened simultaneously.
Some types of reports, such as log files and configuration files, can be opened using other
applicable applications such as Notepad or Internet Explorer, etc. These files can then be searched
or saved using these applications. See Viewing and Saving Reports in other Applications
on page 562.
To view collected reports:

1. In the Log Collector window, fill the left pane of the window with either Server Reports,
Client Reports, or Imported Reports. For instructions, see the relevant sections.
• Collecting Information from Your Local Machine on page 550
• Collecting Information from NICE Perform Servers on page 543
• Importing Saved Reports for Review on page 556
2. Expand a branch and double-click a report. Each report appears in a separate tabbed page in
the right-hand pane. You can select reports from the Server, Client, and Imported reports list
simultaneously.
Figure A-24 Tabbed Pages
If you click a report tab, the report will be highlighted in the Reports tree.

3. To split the viewing screen:

Figure A-25 Tabbed Pages - Splitting Screen
a. Click a tab and hold down the left mouse button until the directional indicator appears.
b. Drag the mouse to the directional indicator and select a location for the tabbed page.
The split screen appears.
Figure A-26 Tabbed Pages Split Screen
4. If you have many tabbed pages opened, use the dropdown list to switch between pages.
Figure A-27 Tabbed Pages Dropdown List

Viewing and Saving Reports in other Applications

Some types of reports, such as log files and configuration files, can be opened using other
applications such as Notepad or Internet Explorer, etc. These files can then be searched or saved
using these applications.
To open a report using another application:

• In the left pane of the Log Collector window, right-click a report and select Open With
Viewer. This option only appears for reports which can be opened in another application.
Figure A-28 Open With Viewer
The file opens in an appropriate application.

Figure A-29 Sample Report opened in Notepad
The file can then be saved, or searched as a standard file.

Reports Created with the Log Collector

The Log Collector produces many reports. A current list of available reports is accessible from the
Log Collector application.
To view a list of available reports created with the Log Collector:

• In the Log Collector window, select Help > Reports List. The list of Log Collector reports
appears.
Server-Side Reports
The following sets of server-side reports are available from each of your Data Hubs. You can
select a whole group of reports or individual reports.
General Server-Side Reports
Applications Suite Reports
Playback Telephony Services Server Reports
Logger Reports
SQL Server Reports
Interactions Center Reports
Storage Center Reports
Reporter Reports
NiceScreen Logger Reports
ScreenSense Reports
Stream Server Reports
Business Services Framework (BSF) Server Reports
CTI Integrations Reports
Media Library Reports
Audio Analysis Reports
VRG Reports
MPCM (Media Provider Controller Manager) Reports
RTS Reports
Windows Services Collects Windows Services details on the machine

General Server-Side Reports (Continued)
NICE Registry Collects the registry under HKLM\SOFTWARE\NICE

SYSTEMS
Time Settings General
Computer Configuration Collects system information data report
Application Events Collects Application events
System Events Collects System events
Security Events Collects Security events
Retention Configuration Collects Retention Service configuration file
Retention Logs Collects Retention Service log files
Locate Logs Collects Locate log files
Audio Analysis Logs Collects Audio Analysis log files
Business Analyzer Collects Business Analyzer configuration file

Configuration
ClearSight Logs Collects ClearSight log files
Packages Collects the packages charts
Evaluation Charts Collects the charts from Evaluation reports
Locate Database Settings Collects Locate setting from the

nice_admin.dbo.tblApplicationsSystemSettings table
Summary Page Logs Collects Summary Page log files
Audio Analysis Configuration Collects Audio Analysis configuration file
My Universe logs Collects My Universe log files
Administrative Database Data Collects administrative data nice_admin database
System Administrator Collects System Administrator configuration file

Configuration
System Administrator Logs Collects System Administrator log files
Rules Manager Logs Collects Rules Manager log files
Rules Manager Configuration Collects Rules Manager configuration file
Business Analyzer Logs Collects Business Analyzer log files

Applications Suite Reports (Continued)
Investigator Logs Collects Investigator log files
Locate Configuration Collects Locate configuration file
Server Bin Files Collects versions of files in Applications\ServerBin directory
Client Bin Files Collects versions of files in Applications\ClientBin directory
Lexicon Manager Logs Collects Lexicon Manager log files
Insight Manager Collects Insight Manager categorization status

Categorization
Insight Manager Tasks Collects Insight Manager Reports
Coaching Logs Collects Coaching log files
Monitor Logs Collects Monitor log files
Lexicon Manager Words Collects Lexicon words groups
ClearSight Categories Collects number of transcribed and not transcribed calls for
each category
ClearSight Topics Collects information about Top 20 topics for each category
ClearSight Categories Collects information about ClearSight categories correlation

Correlation
Users Administrator Logs Collects Users Administrator log files
Rule Engine Logs Collects Rule Engine log files
Rule Engine Configuration Collects Rule Engine configuration file
Query Configuration Collects Query configuration
Monitor Configuration Collects Monitor configuration file
Evaluation Logs Collects Evaluation log files
Importer Logs Collects Importer log files
IIS Logs Collects Internet Information Services (IIS) log files
IIS Configuration Collects IIS configuration
IP Phone Logs Collects IP Phone Applications log files
IP Phone Configuration Collects IP Phone configuration file
Branches Logs Collects Branches log files
Applications Registry Collects registry under HKLM\SOFTWARE\NICE Applications

Files Versions Collects file versions of Playback Telephony Services Server
PBSLink Configuration Collects PBSLink configuration file
Playback GUI configuration Collects Playback GUI configuration file
Playback Administrator Collects Playback Administrator configuration file

Configuration
Resource Manager Collects Resource Manager configuration file

Configuration
General Logs Collects Playback Telephony Services Server log files
Playback Telephony Services Collects the chart from Playback Telephony Services Server
Report Charts reports
Configuration Collects Playback Telephony Services Server configuration
Logger Reports
Logger Reports
File Versions Collects Voice Logger files versions
Voice Logger Configuration Collects Resource Manager configuration files
Voice Logger SNMP Registry Collects Voice SNMP Agent registry under
HKLM\SOFTWARE\NICE Systems\VoiceLoggerSnmpAgent
Voice Logger SCSI Registry Collects Voice SCSI registry under

HKLM\HARDWARE\DEVICEMAP\Scsi
Voice Logger Registry Collects Voice Logger registry under HKLM\SOFTWARE\NICE

Systems\Setup\NICE NiceLog
VoIP Firmware Files Collects VoIP firmware files
Voice Logger Logs Collects Voice Logger log files
VoIP CIM Service Registry Collects VoIP Logger CIMService registry under
HKLM\SYSTEM\CurrentControlSet\SERVICES\CIMService
Voice SCSI Registry Collects VoIP SCSI registry under

HKLM\HARDWARE\DEVICEMAP\Scsi
VoIP Registry Collects VoIP Logger registry from

HKLM\SYSTEM\CurrentControlSet\SERVICES\Nicelog
VoIP %APPDATA% Logs Collects VoIP Logger %APPDATA% log files
VoIP Logs Collects VoIP Logger log files

SQL Server Reports
SQL Server Reports
Logs Collects SQL error and log files
Calls Volume Collects volume of inserted calls
Database Information Collects Database disk size information data report
Database Server Collects Database Server performance counters values

Performance Counters
Call Server Performance Collects Call Server performance counters values

Counters
.NET Memory Performance Collects .NET Memory performance counters values

Counters
Process Performance Collects Process performance counters values

Counters
Configuration Collects Interactions Center configuration
XML data Collects Interactions Center .XML files
SOAP files Collects Interactions Center .SOAP files
Files Versions Collects Interactions Center file versions
NiceCTI Registry Collects Interactions Center registry under

HKLM\SOFTWARE\NiceCTI
Interactions Center Logs Collects Interactions Center log files
Logs Collects Storage Center log files
Configuration Collects Storage Center configuration files
Process Performance Collects Storage Center process performance counters values

Counters

Reporter Reports
Reporter Reports
Logs Collects Reporter log files
Reports List Collects a list of existing reports from the database
Registry Collects NiceScreen Logger registry under

HKLM\SOFTWARE\Nice Systems Ltd.\Nice Multimedia
Logger
File Versions Collects NiceScreen Logger file versions
Logs Collects NiceScreen Logger log files
Configuration Collects NiceScreen Logger configuration file
ScreenSense Reports
ScreenSense Reports
Process Performance Collects ScreenSense process performance counters values

Counters
Server Performance Collects ScreenSense server performance counters values

Counters
Logs Collects ScreenSense log files
Configuration Collects ScreenSense configuration files
Database Configuration Collects ScreenSense database configuration from

nice_screen_sense.dbo.tblConfiguration table
Logs Collects Stream Server log files
Playback Streaming Logs Collects Playback Streaming log files
Encoder Configuration Collects Streaming Encoder configuration file
Encoder Logs Collects Playback Stream Encoder log files
Streaming Startup Collects Streaming Startup configuration file

Configuration

Stream Server Reports (Continued)
Files Versions Collects Stream Server files versions
Configuration Collects Business Services Framework configuration file
Logs Collects Business Services Framework log files
NiceCTI Registry Collects NICECTI registry under HKLM\SOFTWARE\NICECTI
Logs Collects CTI Server log files
File Versions Collects Media Library file versions
Database Logs Collects logs from the nice_interactions.tblMLLog table
Configuration Collects Media Library configuration files
Registry Collects Media Library Framework registry under

HKLM\SOFTWARE\Nice Systems\NICE Media Library
Framework
Database Registry Collects Media Library Server DataBase registry under

HKLM\SOFTWARE\Nice systems\Setup\NICE
Applications\Media Library Server\DataBase
Installation Registry Collects Media Library installation registry under

HKLM\SSOFTWARE\Microsoft\Windows\CurrentVersion\Unin
stall
Compure Registry Collects Compure registry under HKLM\SOFTWARE\NICE

Applications\Audio Analysis\Compure
File Versions Collects file versions
Playback Administrator Logs Collects Playback Administrator logs files

Audio Analysis Reports (Continued)
Configuration Collects Audio Analysis configuration file
Analysis Data Charts Collects analysis graphic data
Process Performance Collects Audio Analysis process performance counters values

Counters
Logs Collects Audio Analysis log and text files
Machine Information Collects machine information
Analysis Data Collects the analysis data
VRG Reports
VRG Reports
Database Configuration Collects VRG database configuration from nice_admin VRG

tables
Logs Collects MPCM log files
RTS Reports
RTS Reports
RTS Logs Collects RTS log files
RTS File Versions Collects RTS file versions

Client-Side Reports
The following sets of client-side reports are available from your local machine. You can select a
whole group of reports or individual reports.
General Client-Side Reports
Player Reports
ScreenAgent Reports
ScreenSense Agent Reports
IE Local Machine Settings Gets Internet Explorer settings for Local Machine zone
IE Trusted Sites Settings Gets Internet Explorer settings for Trusted Sites zone
NICE Registry Collects registry under HKLM\SOFTWARE\NICE SYSTEMS
Computer Configuration Collects system information data report
Player Reports
Player Reports
Configuration Collects Player configuration files from the

%APPS_DATA%\NicePlayer\Release3 path
Logs Collects Player log files from the

%APPS_DATA%\NicePlayer\Release3 path
ScreenAgent Reports
ScreenAgent Reports
ScreenAgent Registry Collects Voice ScreenAgent registry under

HKLM\SOFTWARE\Nice Systems\Nice Screen
Logs Collects ScreenAgent log files from the \nice\Screen Agent

path
Logs Collects ScreenSense Agent log files from the

%APPS_DATA%\Nice systems\ScreenSense log path
Configuration Collects ScreenSense Agent configuration file

B
Enabling Display of NICE System
Information in System Administrator
Contents
Overview........................................................................................................................574
Common Terms.............................................................................................................575
Verifying the System Administrator User...................................................................576
Adding the System Administrator User to Computers not in an Active Directory
Environment..................................................................................................................578
Editing DCOM Permissions .........................................................................................582
Setting Security Permissions through the WMI Control...........................................588
Appendix B: Enabling Display of NICE System Information in System Administrator 573

Overview
Overview
The System Administrator includes a summary page that displays the NICE product version and
the installed Updates. See the System Administrator Guide for more information.
Figure B-1 Summary Page in the System Administrator
Version
Update
This summary page is generated by accessing the product information from the computers in the
NICE system. In order to access this product information, the System Administrator user needs to
be included in the DCOM and WMI security settings on each computer in the NICE system. The
System Administrator user is the user associated with the Nice SystemAdministrator service.
This association is defined when NICE Perform applications are installed on the Applications
Server. To verify the identity of this user, see Verifying the System Administrator User
on page 576.
In this appendix, we describe how to change the security settings to enable the System
Administrator user to access information on these computers.
Workflow
The workflow differs slightly depending on whether or not you are working in an Active Directory
environment.
1. Verifying the System Administrator User. To verify the identity of the System
Administrator user, you check which user is associated with the Nice SystemAdministrator
service.
2. (Not Active Directory Environment Only) Adding the System Administrator User to
Computers not in an Active Directory Environment. The System Administrator user needs
to be included on all computers in the NICE system. If you are not working in an Active
Directory environment, you need to add this user through the Control Panel, if necessary.
3. Editing DCOM Permissions. You edit the DCOM permissions to enable the remote launch
and remote activation of WMI queries.
4. Setting Security Permissions through the WMI Control. By means of the WMI Control,
you set security permissions that enable the System Administrator user to remotely access
NICE system information.

Common Terms
Common Terms
Some common Microsoft terms used in this appendix are listed in the table below.
Table B-1: Common Microsoft Terms
Term Meaning
COM Component Object Model. COM is a platform-independent, distributed,

object-oriented system for creating binary software components that can
interact.
DCOM Distributed Component Object Model. DCOM is a protocol that enables

software components to communicate directly over a network.
WMI Windows Management Instrumentation. WMI is the Microsoft

implementation of Web-based Enterprise Management (WBEM), which is
an industry initiative to develop a standard technology for accessing
management information in an enterprise environment. Remote WMI
connections are made through DCOM.
WMI Namespace Each WMI namespace has a security descriptor, which allows each
namespace to have unique security settings that determine who has
access to the namespace data and methods.

Verifying the System Administrator User

The System Administrator user is the user associated with the Nice SystemAdministrator
service. This association is defined when NICE Perform applications are installed on the
Applications Server. If you need to verify the identity of the System Administrator user, you check
which user is associated with the Nice SystemAdministrator service.
NOTE: In an Active Directory environment, verify that the domain name of the System
Administrator user is the same domain as that of the computer, or is a trusted domain.
To verify the System Administrator user:

1. On the Applications Server, click Start, and then select Run.
The Run window appears.
2. In the Open field, enter services.msc, and click OK.

The Services window appears.
3. In the Name column, find Nice SystemAdministrator.

4. Verify the name in the Log On As column.
User Name
This name is the user name of the System Administrator.
IMPORTANT
The System Administrator user cannot be Local System. The Local System user cannot be
granted the correct permissions for all the machines in the system. If the System Administrator
user is Local System, you need to change it to another user.

Adding the System Administrator User to Computers not in an Active Directory Environment
Adding the System Administrator User to Computers not in

an Active Directory Environment
The System Administrator user needs to be included on all computers in the NICE system. If you
are not working in an Active Directory environment, you need to add this user through the Control
Panel, if necessary.
IMPORTANT
Before starting this procedure, obtain the user name and password of the System Administrator
user. See Verifying the System Administrator User on page 576 to obtain the user name.
To add the System Administrator user to a computer:

1. From the Start menu, select Control Panel.
The Control Panel appears.
2. In the Name column, double-click Administrative Tools.

The Administrative Tools window appears.
3. Double-click Computer Management.

The Computer Management window appears.
4. Expand System Tools > Local Users and Groups > Users.
A list of existing local users appears in the right pane.
5. Verify that the System Administrator user appears in the list. If the user does not appear,
proceed to Step 6.

6. Right-click the Users directory, and then select New User.
The New User window appears.
7. Complete the fields as follows:

• User name: Enter the user name exactly as it appears in the Services window on the
Applications Server. See Verifying the System Administrator User on page 576.
• Full name: Enter the first name and last name of the user.
• Description: (Optional) Enter text describing the System Administrator user.
• Password and Confirm Password: Enter the password for the System Administrator
user.
8. Clear the User must change password at next logon check box.
9. Click Create.

The System Administrator user appears in the list in the left pane.
10. Proceed to Editing DCOM Permissions on page 582.

Editing DCOM Permissions

You edit the DCOM permissions to enable the remote launch and remote activation of WMI
queries.
IMPORTANT
Before you begin this procedure, you need the System Administrator user name.
To edit DCOM permissions:

2. In the Open field, enter DCOMCNFG, and then click OK.

The Component Services window appears.

3. In the Console Root tree, expand Component Services > Computers.
Right-click
My Computer
4. Right-click My Computer and select Properties.

The My Computer Properties window appears.

5. Click the COM Security tab.
6. In the Launch and Activation Permissions area, click Edit Limits.

The Launch Permission window appears.
7. In the Group or user names area, verify that the name of the System Administrator user
appears, and proceed to Step 9.
8. If the name of the System Administrator user does not appear in the Groups or user names
area, complete the following steps:
a. Click Add.

The Select Users, Computer, or Groups window appears.
b. In the Enter the object names to select area, enter the name of the System
Administrator user.
c. Click Check Names.
The user name and domain name appear.
d. Click OK.
The Launch Permission window reappears.
9. In the Group or user names area, select the System Administrator user.
10. In the Permissions for <User> area, in the Allow column, select Remote Launch and
Remote Activation, and then click OK.
The COM Security tab reappears.

11. In the Access Permissions area, click Edit Limits.

The Access Permission window appears.
12. In the Group or user names area, verify that the name of the System Administrator user
a. Click Add.
Administrator user.
d. Click OK.
The Access Permission window reappears.
15. In the Permissions for <User> area, do the following:

• Verify that Local Access is selected.

• Select Remote Access.
Verify Local Access Selected

Select Remote Access
16. Click OK.

The COM Security tab reappears.
17. Click OK to close the My Computer Properties window.
18. Proceed to Setting Security Permissions through the WMI Control on page 588.

Setting Security Permissions through the WMI Control

By means of the WMI Control, you set security permissions that enable the System Administrator
user to remotely access NICE system information.
To set security permissions through the WMI Control:

2. In the Open field, enter wmimgmt.msc, and then click OK.

The Windows Management Infrastructure (WMI) window appears.
Right-click WMI Control
3. Right-click the WMI Control (Local) icon, and then click Properties.

The WMI Control (Local) Properties window appears.
4. Expand Root, and select the DEFAULT namespace.
5. Click Security.

The Security for ROOT\DEFAULT window appears.
6. In the Group or user name area, verify that the name of the System Administrator user
a. Click Add.
Administrator user.
d. Click OK.
The Security for ROOT\DEFAULT window reappears.

9. In the Permissions for <User> area, in the Allow column, select Execute Methods and
Remote Enable.
10. Click OK, and then close the Windows Management Infrastructure (WMI) window.

Nice Perform Troubleshooting Guide - Rev. A2

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Nice Perform Troubleshooting Guide - Rev. A2

Încărcat de

Drepturi de autor:

Formate disponibile

NICE Perform®

APAC Region: (Asia/Pacific)

The Americas Region: (North, Central, South America)

NICE invites you to join the NICE User Group (NUG).

International Headquarters-Israel North America

All queries, comments, and suggestions are welcome!

For more information about NICE, visit www.nice.com

Revision Modification Date Description

A1 December 2008 • Added the following cross-server troubleshooting

A2 February • Added cross references and small changes to the

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

About this Troubleshooting Guide................................................................................ 24

Chapter 1: Introducing NICE Perform Troubleshooting 23

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

About this Troubleshooting Guide

Chapter 1: Introducing NICE Perform Troubleshooting 24

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Chapter 1: Introducing NICE Perform Troubleshooting 25

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Chapter 1: Introducing NICE Perform Troubleshooting 26

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Finding the Technical and User Guides

Chapter 1: Introducing NICE Perform Troubleshooting 27

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Figure 1-2 ExtraNICE Portal Home Page

5. Expand the Content Type: Technical and User Guides.

Chapter 1: Introducing NICE Perform Troubleshooting 28

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Figure 1-4 List of Manuals

Chapter 1: Introducing NICE Perform Troubleshooting 29

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Setting an Alert for any Changes

2. Click on NICE Perform Release 3.1. The following window appears.

Chapter 1: Introducing NICE Perform Troubleshooting 30

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Figure 1-6 NICE Perform Release 3.1 - Actions

3. Select the Actions drop-down menu.

4. Select Alert Me. The following window appears.

Chapter 1: Introducing NICE Perform Troubleshooting 31

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Figure 1-8 Alerts Page

5. Select Send a daily summary.

Chapter 1: Introducing NICE Perform Troubleshooting 32

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Synchronizing the Guides to Your Local Machine

4. In the URL field, add http://www.extranice.com/EIS/Products/NICEPerform.

Chapter 1: Introducing NICE Perform Troubleshooting 33

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Indexing and Searching the Guides