08 OS90538EN15GLA0 Mediations Troubleshooting

Mediations Troubleshooting
Contents
1 General Troubleshooting Tasks 3
1.1 Verifying and Correcting status of Mediations 4
1.1.1 General Procedure 4
1.1.2 NetAct Status Script 7
1.1.3 Mediation Framework Console 13
1.1.4 Managing Service Assemblies with MF Console 15
1.1.5 Mediations Logging and Tracing 21
2 Troubleshooting the Core Network 29
2.1 NE3S/WS Troubleshooting Cases 30
2.1.1 Network Element integration is not working properly 30
2.1.2 Alarm upload failure 32
2.1.3 PM Data not available 34
2.2 SNMP Troubleshooting Cases 35
2.2.1 Trap delivery is not working 35
2.2.2 Network element is not polled by Agent Supervision 38
2.2.3 SNMP AGENT NOT RESPONDING alarm is generated 38
2.3 XoH Troubleshooting Cases 39
2.3.1 Alarms are uploaded but NetAct Monitor shows a time-out error 39
2.3.2 XoH mediation reports alarms in Monitor 41
3 Troubleshooting the Radio Network 45
3.1 NWI3 Mediation 46
3.1.1 Integration data upload is not working 46
3.1.2 Alarms are not received 49
3.1.3 PM data is not received from OMS 51
3.2 Q3/MML Mediation 53
OS90538EN15GLA0
1
© Nokia Solutions and Networks 2015
3.2.1 No alarm or event come from Q3 based NEs like BSC and SGSN 53
3.2.2 PM data is not received from BSC/SGSN 54
3.2.3 Generating upload file failed 56
3.2.4 Upload or file based plan provisioning ongoing 56
4 Exercises 57
OS90538EN15GLA0
2
1 General Troubleshooting Tasks
OS90538EN15GLA0
3
1.1 Verifying and Correcting status of Mediations

You may need to verify and correct the working status of the NetAct southbound
mediation in many error cases.
Follow the instructions below for checking the status of the following mediations:
 Q3 Mediations
 NWI3
 NE3S/WS
 XoH
 NX2S
 SNMP Northbound Interface
 FM ASCII Alarm Forwarding Northbound Interface
 3GGP CORBA NBI - Common Services
 3GPP CORBA PM NBI
1.1.1 General Procedure

1. Log in to any NetAct VM as the omc user.
2. Mediations run as services in the virtual machine, and they are managed by
netact_status.sh.
[omc]$ /opt/cpf/sbin/netact_status.sh status
The following is part of the command output:
[root@n85vm16 ~]# /opt/cpf/sbin/netact_status.sh status

Executing NetAct cluster status script
Status of NetAct
Node Service Status
n85vm03
ssh-n85vm03 started
dns-master started
netfs-n85vm03 started
dirsrv started
nslcd-n85vm03 started
omagentaf-n85vm03 started
ntp-n85vm03 started
cpfnx2s-n85vm03 started
tomcat_vcentplg started
OS90538EN15GLA0
4
3. Depending on the results of the netact_status.sh command, follow the first

or the secondchoice shown below:
 If the status of the service is started, then go to step 4.
Or
 If the status of the service is not started, then restart the service by
switching to the root user and entering:
[root]# smanager.pl start service <sm service name>
TIP
If the service is still not started, delete data folder and execute the below
commands:
• [root]# rm –rf /opt/oss/NSN-<sm service name>/smx/data/*
• [root]# smanager.pl start service <sm service name>
4. Even if the netact_status.sh command shows that the service is running, it

is still possible that a particular service assembly inside a service in the cluster is
not running. You can use the MF Console (Mediation Framework Console) to
check if a service assembly is in started state. Details about Mediation
Framework Console will be explained in next section.
5. If the status is Stopped, start the service assembly.
6. If the problem persists, send the mediation-specific log files
oss_activity*.log,oss_trace*.log, and oss_error*.log to the NSN
Technical Assistance Center. Note that you must collect the log files from each
VM node where the problematic service (services) is running.
OS90538EN15GLA0
5
General Troubleshooting Tasks

Correcting the status of a NetAct Mediation Service
You may need to verify and correct the working status of the NetAct southbound
mediation in many error cases. The main tasks are:
Verify If status is not If the status is If the status is

mediation “started” “started” but still “failed”
Service status restart the the service is remove the
• netact_status corresponding failing manage data
script service the status of directories of
• smanager.pl the SMX SAs. of the failing
script • Mediation mediation and
Framework restart the
console service
• Mediation
directories
• smanager.pl
script
1 © Nokia Solutions and Networks 2015
Fig. 1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
6
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.2 NetAct Status Script

The NetAct Status script provides the status of the services running in the NetAct
cluster.

Status of NetAct
Node Service Status
n85vm03
ssh-n85vm03 started
dns-master started
dirsrv started
ntp-n85vm03 started
n85vm04
ssh-n85vm04 started
db started
ntp-n85vm04 started
n85vm05
ssh-n85vm05 started
nfs started
dirsrv-secondary started
ntp-n85vm05 started
n85vm06
ssh-n85vm06 started
ntp-n85vm06 started
lb-unify-backup started
n85vm07
ssh-n85vm07 started
omagent started
ntp-n85vm07 started
vcenterselfmon started
hpsim started
n85vm08
ssh-n85vm08 started
OS90538EN15GLA0
7
ftp-n85vm08 started
ntp-n85vm08 started
pingserver-n85vm08 started
passwordchangews-n85vm08 started
tnameserver-n85vm08 started
nbi3gc started
nbiim started
rac3gp started
n85vm09
ssh-n85vm09 started
ftp-n85vm09 started
ntp-n85vm09 started
nwi3 started
nwi3-http started
jacorb started
saucnt started
n85vm10
ssh-n85vm10 started
ftp-n85vm10 started
ntp-n85vm10 started
osi started
q3user started
n85vm11
ssh-n85vm11 started
ntp-n85vm11 started
lb-unify-primary started
socks started
n85vm12
ssh-n85vm12 started
ftp-n85vm12 started
ntp-n85vm12 started
nx2s started
xoh started
OS90538EN15GLA0
8
ne3sws_dynamicadaptation started
generic_mediations started
n85vm13
ssh-n85vm13 started
ntp-n85vm13 started
fmascii started
nbisnmp started
nbine3s started
n85vm14
ssh-n85vm14 started
ftp-n85vm14 started
ntp-n85vm14 started
nbi3gcpm-n85vm14 started
n85vm15
ssh-n85vm15 started
ftp-n85vm15 started
ntp-n85vm15 started
nbi3gcpm-n85vm15 started
n85vm16
ssh-n85vm16 started
dns-slave started
ntp-n85vm16 started
common_mediations-n85vm16 started
httpd-n85vm16 started
n85vm17
ssh-n85vm17 started
ntp-n85vm17 started
common_mediations-n85vm17 started
OS90538EN15GLA0
9
n85vm18
ssh-n85vm18 started
ntp-n85vm18 started
fm_pipe-n85vm18 started
n85vm19
ssh-n85vm19 started
ntp-n85vm19 started
fm_pipe-n85vm19 started
n85vm20
ssh-n85vm20 started
ftp-n85vm20 started
ntp-n85vm20 started
was-n85vm20 started
dmgr-n85vm20 started
ihs-n85vm20 started
jetty-n85vm20 started
n85vm21
ssh-n85vm21 started
ftp-n85vm21 started
ntp-n85vm21 started
was-n85vm21 started
ihs-n85vm21 started
n85vm22
ssh-n85vm22 started
ftp-n85vm22 started
ntp-n85vm22 started
was-n85vm22 started
ihs-n85vm22 started
n85vm23
ssh-n85vm23 started
ftp-n85vm23 started
OS90538EN15GLA0
10
ntp-n85vm23 started
was-n85vm23 started
ihs-n85vm23 started
n85vm26
nfs-n85vm26 started
system started
lb-n85vm26 started
db_crons started
n85vm27
lb-n85vm27 started
message started
n85vm28
base-n85vm28 started
cmessage started
osscore started
osscore2 started
nww started
mvi started
dns-namesurfer started
n85vm29
base-n85vm29 started
rep-n85vm29 started
n85vm30
sqm_eem-n85vm30 started
sqm_jboss-n85vm30 started
sqm_collector_npm started
dmgr_linas-n85vm30 started
nodeagent_linas-n85vm30 started
ihs_linas-n85vm30 started
was_linas-n85vm30 started
n85vm31
sqm_eem-n85vm31 started
sqm_jboss-n85vm31 started
nodeagent_linas-n85vm31 started
ihs_linas-n85vm31 started
was_linas-n85vm31 started
OS90538EN15GLA0
11

NetAct cluster status

Status of NetAct
Node Service Status
n85vm03
ssh-n85vm03 started
dns-master started
dirsrv started
ntp-n85vm03 started
n85vm04
ssh-n85vm04 started
db started
ntp-n85vm04 started
Fig. 2
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.3 Mediation Framework Console

MF Console is a application intended to monitor, troubleshoot and administer MF
runtime (Apache Servicemix instance).
The purpose of the MF Console is to get a consolidated view of the runtime
information to troubleshoot the system. It is integrated with tracer to trace the logs for
particular use-cases. It monitors the performance of the different components in the
servicemix runtime.
The Scope of MF console is to monitor and troubleshoot the apache servicemix
instance that includes:
 interfacing to start/stop existing mediations
 viewing mediation information
 viewing mediations running on the given MF instance
 viewing different components within an mediation
Usage
The MFC.sh script will be available under the mf-console folder within the servicemix
folder.
Use the command
#./MF.sh <command> [args]
to use particular operations.
Commands
Get Displays all available static information for the Servicemix

instance (list of Service Assemblies, list of Service Units and list
of End Points)
dispSA Retrieves the list of service assemblies deployed in the
Servicemix instance along with their status
dispMI Retrieves the mediation information (list of service units and list
of endpoints) for the given service assembly
start Starts the service assembly
stop Stops the service assembly
shutDown Shuts down the service assembly
OS90538EN15GLA0
13

Mediation Framework Console
MF Console is an application to monitor, troubleshoot and administer MF runtime
(Apache Servicemix instance). The MFC.sh script is available under the mf-
console folder within the servicemix instance folder located in the VM running the
corresponding mediation. Use the command
#./MF.sh <command> [args]
Command Description
Displays all available static information for the Servicemix instance
Get
(list of Service Assemblies, list of Service Units and list of End Points)
Retrieves the list of service assemblies deployed in the Servicemix
dispSA
instance along with their status
Retrieves the mediation information (list of service units and list of
dispMI
endpoints) for the given service assembly
start Starts the service assembly
stop Stops the service assembly
shutDown Shuts down the service assembly
Fig. 3
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
14
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.4 Managing Service Assemblies with MF Console

To see the correct status of the components of a specific Mediation, execute the
commands below:
1. Log on as root user in the VM node that is currently running the mediation
(NetAct service) in question
2. To get a list of the service assemblies deployed in one specific Mediation, enter
the following commands:
[root]# cd /opt/oss/<Mediation Folder>/smx/mf-console
[root]# ./MFC.sh dispSA
The Mediation folders are as follows:
 Q3 Mediation : /opt/oss/NSN-q3_mediations/smx/mf-console
 NWI3: /opt/oss/NSN-nwi3/smx/mf-console
 NE3S/WS: /opt/oss/NSN-common_mediations/smx/mf-console
 XoH: /opt/oss/NSN-xoh/smx/mf-console
 NX2S: /opt/oss/NSN-nx2s/smx/mf-console
 SNMP Northbound Interface: /opt/oss/NSN-nbisnmp/smx/mf-console
 FM ASCII NBI: /opt/oss/NSN-fmascii/smx/mf-console
 3GGP CORBA NBI - Common: /opt/oss/NSN-nbi3gc/smx/mf-
console
 3GPP CORBA PM NBI: /opt/oss/NSN-nbi3gcpm/smx/mf-console
The output of the MFConsole commands indicates the status of each service
assembly of the service in question in the following way:
INFO - DispSACommand - <service_assembly> -- Started
3. If the status of the service assembly is "stopped" or "shutdown" start the service
assembly by entering:
[root]# ./MFC.sh start -sa <service_assembly
OS90538EN15GLA0
15
Examples of Service Assemblies in different Mediations:
NWI3
[root@n85vm09 mf-console]# cd /opt/oss/NSN-nwi3/smx/mf-console
[root@n85vm09 mf-console]# ./MFC.sh dispSA
INFO - DispSACommand - Service Assemblies available are:
INFO - DispSACommand - nwi3-mediation-pm-file-transfer-sa -- Started
INFO - DispSACommand - nwi3-mediation-sa -- Started
INFO - DispSACommand - nas-nw3_cm-jbi-sa -- Started
INFO - DispSACommand - nwi3-mediation-logUpload-sa -- Started
INFO - DispSACommand - nwi3-mediation-audit-trail-sa -- Started
INFO - DispSACommand - nwi3-lic-mediation-sa -- Started
INFO - DispSACommand - nwi3-mediation-fm-sa -- Started
INFO - DispSACommand - nwi3-mediation-um-sa -- Started
INFO - DispSACommand - nwi3-mediation-certificate-management-sa --
Started
INFO - DispSACommand - nwi3-mediation-pm-file-query-sa -- Started
Q3/MML
[root@n85vm10 mf-console]# cd /opt/oss/NSN-q3_mediations/smx/mf-console
INFO - DispSACommand - q3nebr-sa -- Started
INFO - DispSACommand - q3hwm-sa -- Started
INFO - DispSACommand - q3stack-sa -- Started
INFO - DispSACommand - q3slt-sa -- Started
INFO - DispSACommand - q3common-sa -- Started
INFO - DispSACommand - q3neum-sa -- Started
INFO - DispSACommand - sa-SmixHeartbeat -- Started
INFO - DispSACommand - q3pm-sa -- Started
INFO - DispSACommand - q3aom-sa -- Started
INFO - DispSACommand - q3swm-sa -- Started
INFO - DispSACommand - q3tm-sa -- Started
INFO - DispSACommand - q3fm-sa -- Started
INFO - DispSACommand - q3lm-sa -- Started
INFO - DispSACommand - q3atl-sa -- Started
OS90538EN15GLA0
16
XoH
[root@n85vm12 mf-console]# cd /opt/oss/NSN-xoh/smx/mf-console
INFO - DispSACommand - xohslt-sa -- Started
INFO - DispSACommand - xohpm-sa -- Started
INFO - DispSACommand - xohfm-sa -- Started
INFO - DispSACommand - common-sa -- Started
Common Mediations: NE3S/WS – 3GPP Corba Southbound – NE3S/SNMP

Mediation
[root@n85vm16 mf-console]# cd /opt/oss/NSN-common_mediations/smx/mf-console
INFO - DispSACommand - mf-3gpp-corba-smi-bc-sa -- Started
INFO - DispSACommand - m3gc-sbm-seq-push-consumer-sa -- Started
INFO - DispSACommand - com.nsn.cms9000.active -- Started
INFO - DispSACommand - atlc-mf-sa -- Started
INFO - DispSACommand - m3gcsbm-fm-adapter-oes-sa -- Started
INFO - DispSACommand - com.nsn.juniper.base -- Started
INFO - DispSACommand - mf-3gpp-corba-sbm-jbi4corba-v3-sa -- Started
INFO - DispSACommand - ne3sws-manager-oes-sa -- Started
INFO - DispSACommand - ne3s-snmp-manager-sa -- Started
INFO - DispSACommand - com.nsn.pm.ftp.mediation-sa -- Started
INFO - DispSACommand - m3gcsbm-cs-adapter-oes-sa -- Started
INFO - DispSACommand - m3gcsbm-bcm-adapter-oes-sa -- Started
INFO - DispSACommand - m3gc-sbm-fm-jbi4corba-v3-sa -- Started
INFO - DispSACommand - mf-3gpp-corba-sbm-cm-sa -- Started
INFO - DispSACommand - ne3sws-manager-https-sa -- Started
INFO - DispSACommand - m3gcsbm-pm-adapter-oes-sa -- Started
INFO - DispSACommand - m3gc-sbm-fm-jbi4corba-sa -- Started
INFO - DispSACommand - m3gcs-cs-jbi4corba-sa -- Started
INFO - DispSACommand - sa-SmixHeartbeat -- Started
INFO - DispSACommand - com.nsn.fns -- Started
INFO - DispSACommand - com.nsn.hp.hphw -- Started
INFO - DispSACommand - mf-3gpp-corba-sbm-jbi4corba-sa -- Started
INFO - DispSACommand - mf-3gpp-corba-mediations-common-sa -- Started
INFO - DispSACommand - mf-corba-notification-sa -- Started
OS90538EN15GLA0
17
INFO - DispSACommand - m3gcsbm-fpe-adapter-oes-sa -- Started

INFO - DispSACommand - mf-3gpp-corba-sbm-fm-sa -- Started
INFO - DispSACommand - bss-ivm-med-sa -- Started
INFO - DispSACommand - ne3sws-manager-sa -- Started
[root@n85vm16 mf-console]#
SNMP Northbound Interface

[root@n85vm08 mf-console]# cd /opt/oss/NSN-nbisnmp/smx/mf-console
INFO - DispSACommand - AlarmIRP-Operation-SA -- Started
INFO - DispSACommand - nbisnmp-agent-sa -- Started
INFO - DispSACommand - nbitrap-sa -- Started
INFO - DispSACommand - nbisnmp-access-sa -- Started
3GPP Corba NBI Common Services (nbi3gc service):

[root@n85vm08 mf-console]# cd /opt/oss/NSN-nbi3gc/smx/mf-console
INFO - DispSACommand - AlarmIRP-Notification-SA --
Started
INFO - DispSACommand - Common-SA -- Started
INFO - DispSACommand - FTIRP-Notification-SA -- Started
INFO - DispSACommand - NotificationIRP-SA -- Started
INFO - DispSACommand - Epirp-SA -- Started
INFO - DispSACommand - BasicCMAccessIRP-SA -- Started
INFO - DispSACommand - AlarmIRP-Operation-SA -- Started
INFO - DispSACommand - Csirp-SA -- Started
INFO - DispSACommand - KernelCmIRP-SA -- Started
INFO - DispSACommand - FTIRP-Operation-SA -- Started
OS90538EN15GLA0
18

Mediation Framework Console: Service Assemblies
[root@n85vm09 mf-console]# cd /opt/oss/NSN-nwi3/smx/mf-console

INFO - DispSACom - nwi3-mediation-pm-file-transfer-sa -- Started
INFO - DispSACommand - nwi3-mediation-logUpload-sa -- Started
INFO - DispSACommand - nwi3-mediation-certificate-management-sa --
Started
Fig. 4
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90538EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mediation Framework Console: Starting / stopping Service Assemblies

INFO - DispSACommand - nwi3-mediation-pm-file-transfer-sa -- Started
INFO - DispSACommand - nwi3-mediation-logUpload-sa -- Stopped
INFO - DispSACommand - nwi3-mediation-certificate-management-sa -- Started
[root@n85vm09 mf-console]# ./MFC.sh start -sa nwi3-mediation-
logUpload-sa
INFO - StartCommand - nwi3-mediation-logUpload-sa service
Assembly started successfully
Fig. 5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90538EN15GLA0
20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.5 Mediations Logging and Tracing

Logging and tracing is the process of automatically recording information about the
behavior of the system, components and individual workflows to assist in:
• Debugging
• Troubleshooting
• Auditing activities
The following table lists the differences between logging and tracing:
Logging` Tracing
Used to make a note of important events Used to record any information that helps
occurring in the system. in debugging problems with the code.
Used to capture state changes, to Used to indicate which methods are
indicate warnings, or to indicate errors. called, what data is passed into a
method, what values are returned out of
a method, and what data is returned from
calls to methods beyond the boundary of
a component.
Log messages are easy to understand. Trace messages are detailed and
technical in nature, so they are of use
mainly to the developer community who
wrote the application. With tracing turned
on, it should be possible to understand
most of the problems that occur in the
system.
By default, logging is turned on and By default, tracing is turned off. The
consumes less space. messages are of high volume and
consume system resources.
OS90538EN15GLA0
21

Mediations Logging and Tracing
Logging Tracing
Used to make a note of important events Used to record any information that helps in
occurring in the system. debugging problems with the code.
Used to indicate which methods are called,

what data is passed into a method, what values
Used to capture state changes, to
are returned out of a method, and what data is
indicate warnings, or to indicate errors.
returned from calls to methods beyond the
boundary of a component.
Trace messages are detailed and technical in

nature, so they are of use mainly to the
developer community who wrote the
Log messages are easy to understand.
application. With tracing turned on, it should be
possible to understand most of the problems
that occur in the system.
By default, tracing is turned off. The messages
By default, logging is turned on and
are of high volume and consume system
consumes less space.
resources.
Fig. 6
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
22
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Log files provide information about events or errors in the system. They are
automatically created and maintained by the system. The administrator uses the log
files to monitor and debug problems.
NetAct system requirements defines the content that each component must provide
into logs and traces for different scopes under different circumstances.
The following table lists the NetAct system requirements for logging and tracing,
highlighting the contents of log/trace. It also shows the log or trace types provided by
NetAct system logging service. This requirement applies to all applications and
mediations
Logging Type Description Log Name Log Level

Event Logging It shows the activity in the Activity Log INFO
component at a broader level.
Service Mix Startup and shutdown of the Servicemix-Log INFO
Event Logging component.
Event Tracing Start and stop of the workflow Trace Log CONFIG
within the component, including FINE
parameters.
FINER
Request to another component,
FINEST
including parameters and results.
Request from another component,
including parameters.
Component is not able to recover
from an error situation.
Error Logging Functional problems that are Error Log SEVERE
caused by users, run-time
FATAL
environment or faults in the
component. ERROR
Component is not able to recover WARNING
from an error situation. WARN
CONFIG
OS90538EN15GLA0
23

Mediations Logging and Tracing
Logging Type Description Log Name Log Level

It shows the activity in the component at a
Event Logging Activity Log INFO
broader level.
Activity log in
Service Mix Event
Startup and shutdown of the component. Servicemix-Log INFO
Logging
directory
• Start and stop of the workflow within the
component, including parameters.
CONFIG
• Request to another component, including
parameters and results. Request from FINE
Event Tracing Trace Log
another component, including FINER
parameters.
FINEST
• Component is not able to recover from
an error situation.
SEVERE
• Functional problems that are caused by FATAL
users, run-time environment or faults in ERROR
Error Logging the component. Error Log
• Component is not able to recover from WARNING
an error situation. WARN
CONFIG
Fig. 7
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
24
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. . and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Each mediation component defines the default logging. The common guideline is that
the basic activity and error logs are named same way.
For example, in the case of XoH-mediation the logging directory is

/var/opt/oss/log/xoh
[root@n85vm12 xoh]# ll
total 24248
drwxr-x--- 2 xohuser sysop 4096 Sep 15 12:35 NSN-mf_component
-rw-r--r-- 1 xohuser sysop 3437559 Oct 5 03:15 oss_activity0_0.log
-rw-r--r-- 1 xohuser sysop 10485818 Sep 28 16:20 oss_activity0_1.log
-rw-r--r-- 1 xohuser sysop 10485863 Sep 5 03:54 oss_activity0_2.log
-rw-r--r-- 1 xohuser sysop 380392 Oct 3 10:21 oss_error0_0.log
-rw-r--r-- 1 xohuser sysop 0 Aug 16 10:33 oss_trace0_0.log
drwx------ 2 xohuser sysop 4096 Aug 16 10:33 Performance-Log
drwx------ 2 xohuser sysop 4096 Aug 16 10:33 Servicemix-Log
[root@n85vm12 xoh]#
Mediation Log Directories Trace Script
NWI3 SBI /var/opt/oss/log/ nwi3 /opt/oss/NSN-

nwi3/bin/nwi3_setLoggingLevel.sh <level>
change switches logging between
PRODUCTION and TROUBLESHOOT
Q3/MML SBI /var/opt/oss/log/q3user /opt/oss/NSN-q3med-

/var/opt/oss/log/q3mml common/bin/set_q3user_logger_level.sh –
fragment LM –level <level>
Available values for command line parameter –
fragment: NEBR , LM , SLT , ATL , UM , PM , FM
, SWM , AoM ,HWM/
Available values for parameter – level: FATAL ,
ERROR , WARN , INFO , DEBUG , TRACE ,
OFF ,ALL
NE3S/WS /var/opt/oss/log/ Manually modify the file: /opt/oss/NSN-

SBI common_mediations common_mediations/smx/conf/log4j.xml:
Change the threshold value from OFF to
DEBUG for mf-trace parameter.
Change the root level of the log from INFO
to DEBUG
OS90538EN15GLA0
25
XoH SBI /var/opt/oss/log/ xoh #/opt/oss/NSN-

jbi_swp/bin/set_logger_level.sh -instance
xoh -logger com.nsn.oss.xoh -level <level>
The possible values for level are INFO
(default), TRACE, DEBUG, WARN, ERROR
and FATAL
NX2S SBI /var/opt/oss/log/ nx2s #/opt/oss/NSN-jbi_swp/bin/

set_logger_level.sh –instance nx2s –logger
com.nsn.oss.nx2s –level <level>
The log levels include FATAL, ERROR,
WARN, INFO, DEBUG and TRACE
FMASCII /var/opt/oss/log/ fmascii # /etc/init.d/fmascii log <level>

NBI The possible values of <level> are ERROR,
INFO and DEBUG.
SNMP NBI /var/opt/oss/log/ # nbisnmp log <level>

nbisnmp The possible values of <level> are INFO,
DEBUG and TRACE.
3GPP Corba /var/opt/oss/log/ nbigc # smx-clt log <level>

Common The possible values of <level> are INFO,
NBI DEBUG and TRACE.
3GPP Corba /var/opt/oss/log/ # nbi3gcpm log <level>

PM NBI nbi3gcpm The possible values of <level> are INFO,
DEBUG and TRACE.
3GPP Corba  /var/opt/oss/log/ Change the log level by changing the value
Bulk CM NBI rac/rac3gp in of <level value> in rac3gp_log4j.xml file in
rac3gp node rac3gp node:
The possible values of <level value> are
 /var/opt/oss/log/
INFO, DEBUG, WARN, ERROR and FATAL.
rac in WAS
node Modify the trace level in WAS node as
"omc"user:
[omc]$ /opt/nokia/oss/bin/set_trace_level.sh
-logger rac.rrcsvc -level <levelName> –
create
The possible values of <levelName>, from
higher level to lower level, are SEVERE,
WARNING, INFO, CONFIG, FINE , FINER
and FINEST. And there are two special
values for <level value>, which are ALL and
OFF. ALL means all the messages are
logged and OFF means no message is
logged
OS90538EN15GLA0
26

Mediations Logging and Tracing (1/2)
Mediation Log Directories Trace Script / Configuration File
#/opt/oss/NSN-nwi3/bin/nwi3_setLoggingLevel.sh <level>
NWI3 SBI /var/opt/oss/log/ nwi3
<level> can be PRODUCTION or TROUBLESHOOT
# /opt/oss/NSN-q3med-common/bin/set_q3user_logger_level.sh –
fragment LM –level DEBUG
/var/opt/oss/log/q3user
Q3/MML SBI Available values for fragment: NEBR , LM , SLT , ATL , UM , PM ,
/var/opt/oss/log/q3mml FM, SWM , AoM ,HWM
Available values for command line parameter – level: FATAL , ERROR,
WARN , INFO , DEBUG , TRACE , OFF ,ALL
Manually modify the file: /opt/oss/NSN-

common_mediations/smx/conf/log4j.xml:
/var/opt/oss/log/
NE3S/WS SBI Change the threshold value from OFF to DEBUG for mf-trace
common_mediations
parameter.
Change the root level of the log from INFO to DEBUG
#/opt/oss/NSN-jbi_swp/bin/
set_logger_level.sh -instance xoh -logger com.nsn.oss.xoh -level
XoH SBI /var/opt/oss/log/ xoh <level>
The possible values for level are INFO (default), TRACE, DEBUG,
WARN, ERROR and FATAL
# /opt/oss/NSN-jbi_swp/bin/
set_logger_level.sh –instance nx2s –logger com.nsn.oss.nx2s –
NX2S SBI /var/opt/oss/log/ nx2s level DEBUG
1 © Nokia Solutions and Networks 2014 The log levels include FATAL, ERROR, WARN, INFO, DEBUG and
TRACE
Fig. 8
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90538EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
. Solutions
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mediations Logging and Tracing (2/2)
Mediation Log Directories Trace Script / Configuration File
FMASCII NBI /var/opt/oss/log/ fmascii # /etc/init.d/fmascii log <level>

The possible values of <level> are ERROR, INFO and DEBUG.
SNMP NBI /var/opt/oss/log/ # nbisnmp log <level>
nbisnmp The possible values of <level> are INFO, DEBUG and TRACE.
3GPP Corba /var/opt/oss/log/ nbigc #smx-clt log <level>
Common NBI The possible values of <level> are INFO, DEBUG and TRACE.
3GPP Corba /var/opt/oss/log/ # nbi3gcpm log <level>
PM NBI nbi3gcpm The possible values of <level> are INFO, DEBUG and TRACE.
3GPP Corba • /var/opt/oss/log/rac/ • Change the log level by changing the value of <level value> in
Bulk CM NBI rac3gp in rac3gp node rac3gp_log4j.xml file in rac3gp node:
The possible values of <level value> are INFO, DEBUG, WARN,
• /var/opt/oss/log/rac in ERROR and FATAL.
WAS node • Modify the trace level in WAS node as "omc"user:
[omc]$ /opt/nokia/oss/bin/set_trace_level.sh -logger rac.rrcsvc -
level <levelName> –create
The possible values of <levelName>, from higher level to lower level,
are SEVERE, WARNING, INFO, CONFIG, FINE , FINER and
FINEST. And there are two special values for <level value>, which are
ALL and OFF. ALL means all the messages are logged and OFF
means no message is logged
Fig. 9
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
28
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Troubleshooting the Core Network
OS90538EN15GLA0
29
2.1 NE3S/WS Troubleshooting Cases

2.1.1 Network Element integration is not working properly
The possible cause may be the agent registration data is lost or corrupted in the
mediation while performing operations such as upgrade.
Solution
Re-register the network element:
1. Execute the following SQL query by providing GID of the agent managed object:
Delete from COMMON_MEDIATIONS.JBOSS_CACHE_DATA where
FQN='/jboss/NE3SWSMOFAgentInfo-"GID_OF_AGENT_OBJECT"';
Registration entries of the agent in the mediation are cleared.
2. Perform upload operation which registers the agent in the mediation.
OS90538EN15GLA0
30
Troubleshooting Core Network

NE3S/WS Mediation To check connectivity issues
ping the NE3S/WS agent:
Network Element integration is not working properly # wget http:://<agent
ip/hostname>:port
The possible cause may be the agent registration data is
lost or corrupted in the mediation while performing operations such as upgrade.
Solution: Re-register the network element:
• Execute the following SQL query by providing GID of the agent managed object:
Delete from COMMON_MEDIATIONS.JBOSS_CACHE_DATA where

FQN='/jboss/NE3SWSMOFAgentInfo-"GID_OF_AGENT_OBJECT"';
Registration entries of the agent in the mediation are cleared.
• Perform integration data upload operation which registers the agent in the
mediation.
Please review the chapter document for more NE3S/WS Troubleshooting cases
Fig. 10
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90538EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
. Solutions
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.2 Alarm upload failure

Possible Causes
1. Manager is not registered and the com.nsn.oss.mf.mediation.ne3sws
service is not available.
Solution
Check the Service Assembly (SA) deployment:
a) Log in to the common mediations VM and check the status of Service
Assemblies via MF Console
b) Correct the status of stopped service Assemblies
c) If problem continues, service asemblies status can also be verified by
navigating to ./opt/oss/NSN-
common_mediations/smx/data/smx/serviceassemblies
Check all the NE3SWS related SA's deployment in the respective folders, the
value of the state.xml file must be started. Else, restart the affected
service assembly or restart the whole common_mediation service through
smanager.pl
2. Registration failure and 9047 Alarm is raised due to registration failure.

Solution
Check if the request has been sent to agent:
a) See Mediation logs in the common mediations VM to check if upload was
triggered.
Navigate to /var/opt/oss/log/common_mediations, and in the
oss_activity*.log and oss_trace*.log files, search for the Object
DN, for example, PLMN-XX/NTHLRFE-XX.
b) Check JOMA logs if the request has been acknowledged. Navigate to
.../R_ESYMAC34_8.0_snmp4j/extras/esymacstarter/logEvents.l
og.0.
c) Check the Instructions and Supplementary information field of the Alarm to
check why 9047 alarm is raised.
d) Verify that the HTTP port provided while creating NE3SWS Interface is correct.
e) Verify the NEAC credentials.
f) Verify that the NE3SWS interface object instance is 1 (for example, NE3SWS-
1).
g) Verify the Host name that is provided in MO properties of the Monitor.
OS90538EN15GLA0
32
3. Check lock manager issues:

Navigate to /var/opt/oss/log/common_mediations/ and check the
oss_error0_0.log file.
4. Request has not reached the agent.
Solution
a) Check connectivity issue:
Execute the following command to ping agent:
wget http:://<agent ip/hostname>:port
If the connection is not successful, contact Network Administrator.
b) Check the LB configuration and Notification URL in the mf-
ne3soap.properties file present in /opt/oss/NSN-
common_mediations/smx/mf-conf in common mediations node.
5. OFaS response has not reached mediations:
Solution
To check if OFaS response has not reached mediations:
a) Check JOMA logs to verify if request has been sent wrong notification URL.
Navigate to
.../R_ESYMAC34_8.0_snmp4j/extras/esymacstarter/logEvents.l
og.0.
b) Check if JOMA response is forwarded to NE3SWS mediation consumer URL
c) Check firewall to verify connectivity problems between agent and notification
URL in NetAct mediation.
OS90538EN15GLA0
33
2.1.3 PM Data not available

The PM data does not reach the NetAct Performance Management database or
reporting tools or both.
Possible causes
1. Registration failure and 9047 Alarm is raised due to registration failure.
Solution
Check if the request has been sent to agent:
a) See Mediation logs in the common mediations VM to check if upload was
triggered.
Navigate to /var/opt/oss/log/common_mediations, and in the
oss_activity*.log and oss_trace*.log files, search for the Object
DN, for example, PLMN-XX/NTHLRFE-XX.
b) Check JOMA logs if the request has been acknowledged. Navigate to
.../R_ESYMAC34_8.0_snmp4j/extras/esymacstarter/logEvents.log.0.
c) Check the Instructions and Supplementary information field of the Alarm to
check why 9047 alarm is raised.
d) Verify that the HTTP port provided while creating NE3SWS Interface is correct.
e) Verify the NEAC credentials.
f) Verify that the NE3SWS interface object instance is 1 (for example, NE3SWS-
1).
g) Verify the Host name that is provided in MO properties of the Monitor
2. Check lock manager issues: Navigate to
/var/opt/oss/log/common_mediations/ and check
theoss_error0_0.log file
3. The request file is not being picked up by file poller.
Check if file poller is configured for the wrong directory:
Navigate to /opt/oss/NSN-common_mediations/smx/mf-conf and find
the response and request directory in the mf-ne3soap-oes.properties
4. Request has not reached agent.
Check connectivity issue:
a) Execute the following command to ping Agent: wget http:://<agent
ip/hostname>:port
b) Check the LB configuration and Notification URL in the mf-
ne3soap.properties file in /opt/oss/NSN-
common_mediations/smx/mf-conf in common mediations node.
OS90538EN15GLA0
34
2.2 SNMP Troubleshooting Cases

2.2.1 Trap delivery is not working
Problem
When trap delivery is not working, the alarms are not present in FM DB.
The mediation receives trap from load balancer port 262 and load balancer port 162.
Possible causes
 Traps are not received
 Traps are received, but the mediation is not listening to the traps
 The mediation is up and running, and is listening to the traps, but the traps
are not auto-forwarded to the correct port
Solution
1. Check that traps are received
a) Log in to any NetAct VM, and switch to root user.
b) To see if the traps are received, execute tcpdump command.
#tcpdump -i eth0 'udp port 262'
When traps are received, tcpdump command displays the following:
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode listening on eth0, link-type EN10MB (Ethernet), capture size
96 bytes 10:33:11.011232 IP karadhcp18947.ntc.nokia.com.2564
>urho.ntc.nokia.com.snmptrap: Trap(35) E:nokia.7 1.1.1.1
enterpriseSpecifics=1 0 .iso.org.dod.internet.private=[|snmp]
2. Check that the mediation is listening to traps

b) Check that the mediation is up and running and listening to traps.
#netstat -ulp | grep 16262
When the mediation is up and running and listening to traps, the following
message appears:
udp 0 0 *:16262 *:*3421/java
OS90538EN15GLA0
35
3. Check trap auto-forwarding

b) Check that the traps are auto-forwarded from port 262 and load balancer port
162 to port 16262.
#service iptables status
When the autoforwarding works, the following message appears:
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
1 REDIRECT udp -- 0.0.0.0/0 0.0.0.0/0 udp
dpt:262 redir ports 16262
2 REDIRECT udp -- 0.0.0.0/0 0.0.0.0/0 udp
dpt:162 redir ports 16262
4. Check trap by mediation

a) Open SNMP mediation logs. The location is:
/var/opt/oss/log/snmp-mus/oss_trace0_0.log
The log is available on WAS VM node.
b) Ensure that the log contains a list of message bindings and values.
OS90538EN15GLA0
36

SNMP Mediation
Trap delivery is not working

When trap delivery is not working, the alarms are not present in FM DB. The mediation receives trap from
load balancer port 262 and load balancer port 162.
Possible causes
• Traps are not received, traps are received, but the mediation is not listening to the traps OR the
mediation is up and running, and is listening to the traps, but the traps are not auto -forwarded to the
correct port.
Check that traps are received

1. Log in to any NetAct VM, and switch to root user.
2. To see if the traps are received, execute tcpdump command.
# tcpdump -i eth0 'udp port 262'
When traps are received, tcpdump command displays the following:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
10:33:11.011232 IP karadhcp18947.ntc.nokia.com.2564
>urho.ntc.nokia.com.snmptrap: Trap(35) E:nokia.7 1.1.1.1
enterpriseSpecifics=1 0 .iso.org.dod.internet.private=[|snmp]
Please review the chapter document for more SNMP Troubleshooting cases
Fig. 11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90538EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
. Solutions
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 Network element is not polled by Agent Supervision

The network element is not polled by agent supervision.
Agent supervision is enabled in the adaptation fragment, however the network
element is not monitored.
Solution
Perform the following to enable agent supervision on the network element:
1. In the SNMP FM Mediator Application Select Agent Supervision Configuration.
2. Select the Agent Polling Active check box.
3. Click Update.
SNMP Mediation Agent Polling Configuration has been updated
message appears and the configuration is successfully updated.
2.2.3 SNMP AGENT NOT RESPONDING alarm is generated

Although SNMP agent is up at the managed element, the related alarm is generated
because of a network routing problem, wrong parameter configuration, or firewall
blocking. SNMP AGENT NOT RESPONDING alarm is generated, but the SNMP
agent is up and running on the related host.
Solution
1. Check IP routing configuration and Firewall configuration
The firewall must allow SNMP (UDP) traffic from the virtual machine to the
managed element (SNMPGET) and from the managed element to the virtual
machine (SNMP trap). The firewall must allow ICMP (ping) traffic from the virtual
machine to the managed element as well.
2. Check SNMP Read Access configuration.
In Network Element Access Control, check the SNMP Read Access service
type of corresponding object. The SNMP Read Access must be the same as
informed by the managed element administrator.
3. Check adaptation.
Recheck adaptation (AgentConfiguration) to verify if the configured OID and
SNMP version is in accordance with the managed element MIB.
Go to
/var/opt/oss/global/adaptations/archive/<adaptation_name>/co
ntent and open the SNMP FM fragment. Verify the details under
<agentSupervision> tag.
OS90538EN15GLA0
38
2.3 XoH Troubleshooting Cases

2.3.1 Alarms are uploaded but NetAct Monitor shows a time-
out error
When alarms are uploaded, there is an Alarm Upload Failure error, and NetAct
Monitor shows timeout.
Solution
Change the time-out value accordingly by performing the following steps:
1. Log in to the fm_pipe VM as the root user:
2. Modify the file:
$PRODUCTROOT/conf/Pref_system.interface.fmSouthMediation.xoh
.xml
3. On the node synchronization-as, edit the value of initialResponseTimeout
and responseTimeout to an appropriate value.
4. Save the changes and wait for several minutes to take effect.
OS90538EN15GLA0
39

XoH Mediation
Alarms are uploaded but NetAct Monitor shows a time-out error

When alarms are uploaded, there is an Alarm Upload Failure error, and NetAct
Monitor shows timeout.
Solution
Change the time-out value accordingly by performing the following steps:
1. Log in to the fm_pipe VM as the root user:

2. Modify the file:
$PRODUCTROOT/conf/Pref_system.interface.fmSouthMediation.xoh.xml
3. On the node synchronization-as, edit the value of initialResponseTimeout
and responseTimeout to an appropriate value.
4. Save the changes and wait for several minutes to take effect
Fig. 12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
40
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.2 XoH mediation reports alarms in Monitor

If Monitor is showing the following alarms coming from PLMN-PLMN/NETACT-
<cluster_ID>/VM-<XOH_NODENAME>/JBISERVER-xoh/MEDIATION-xoh,
please check the possible reason respectively as explained in the table below:.
Alarm Problem Solution

20002-Incorrect XoH mediation gets Check if the credentials are changed on
credentials FTP/SFTP credentials from the network element and the credentials in
NEAC service to connect to NEAC are synchronized.
a network element. But if
the credential is wrong, XoH
mediation sends this alarm
for the specific network
element.
20003- XoH mediation uses SFTP Check:
Certificate error and HTTPS protocols to 1. if the network element’s public RSA
communicate with a key is added in
network element if the /opt/oss/NSNxoh/smx/mf-
network element is conf/xoh-certs/known_hosts.
configured to use security
mode. But if the certificates 2. if the network element’s SSH
are not correct, XoH certificate and private key are
mediation sends this alarm installed on the network element.
for the specific network
3. if the network element’s SSH
element.
certificate or CA is imported to NetAct
under /opt/oss/NSNxoh/smx/mf-
conf/xoh-
certs/truststore.jks.
4. if the network element’s SSH
certificate or CA expires.
5. if the NetAct’s SSH certificate and
private key is imported to
/opt/oss/NSN-xoh/smx/mf-
conf/xoh-certs/keystore.jks
and its alias in keystore.jks is
XoH.
6. if the NetAct’s certificate is trustedon
by the network element if the NetAct’s
SSH certificate or CA expires.
20005- XoH mediation Check:
Configuration communicates with a 1. if the object of a network element
error network element only when is created in Object Explorer.
the integration is correctly
done. But if the integration 2. if the XoH object and FTP object
is not completed, XoH are created under the network
mediation sends this alarm
OS90538EN15GLA0
41
for the specific network element

element
3. object in Object Explorer.
4. if the properties of host name, port
and security mode for XoH and
FTP objects are properly filled in
Object Explorer.
5. if the network element’s
credentials are created in NEAC
20006-Unable XoH mediation connects to Check if the network element is properly
to connect NE a network element to integrated or the network is disconnected.
upload files and send SOAP
message. But if it is unable
to connect to a network
element, XoH mediation
sends this alarm for the
specific network element
20007-Internal XoH mediation gets data Check if the internal services such as
service failure from internal service. But if NASDA, WAS and NEAC work normally.
some of these internal
services are unavailable,
XoH mediation sends this
alarm for the specific
service.
OS90538EN15GLA0
42
Troubleshooting Core Networks

XoH Mediation
Alarm Problem Solution
XoH mediation gets FTP/SFTP credentials from NEAC
Check if the credentials are changed on
20002-Incorrect service to connect to a network element. But if the
the network element and the credentials
credentials credential is wrong, XoH mediation sends this alarm for
in NEAC are synchronized.
the specific network element.
Check:
1. if the object of a network element is
created in Object Explorer.
2. if the XoH object and FTP object are
XoH mediation communicates with a network element created under the network element
20005-Configuration only when the integration is correctly done. But if the 3. object in Object Explorer.
error integration is not completed, XoH mediation sends this 4. if the properties of host name, port
alarm for the specific network element and security mode for XoH and FTP
objects are properly filled in Object
Explorer.
5. if the network element’s credentials
are created in NEAC
XoH mediation connects to a network element to upload
Check if the network element is
20006-Unable to files and send SOAP message. But if it is unable to
properly integrated or the network is
connect NE connect to a network element, XoH mediation sends this
disconnected.
alarm for the specific network element
XoH mediation gets data from internal service. But if Check if the internal services such as
20007-Internal service
some of these internal services are unavailable, XoH NASDA, WAS and NEAC work
failure
mediation sends this alarm for the specific service. normally.
Fig. 13
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90538EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
. Solutions
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90538EN15GLA0
44
3 Troubleshooting the Radio Network
OS90538EN15GLA0
45
3.1 NWI3 Mediation

3.1.1 Integration data upload is not working
The following objects do not appear automatically in NetAct Monitor within two
minutes after starting the integration data upload from the integration agent object
(OMS/AXC) context menu:
 NWI3-1 interface under OMS/AXC with correct nwi3systemid attribute
 The NE agent objects (RNC, WBTS, MRBTS, LNBTS) connected via the
integration agent
TIP
You can force a complete reintegration of a previously integrated OMS/AXC by
removing the NWI3-1 object from NetAct Monitor, and then start the integration data
upload from OMS/ AXC context menu.
If the NWI3-1 object exists, then the mediation skips the rest of the integration steps
and proceeds to the integration data upload.
Embedded, not standalone, AXC is a child of the WBTS object so upload the WBTS
via OMS before starting the AXC integration data upload.
Solution
1. Verify that the integration agent object (OMS/AXC) is correctly created in
NetAct Monitor.
a) Check if the correct instance and version are used.
Note:
The menu item for Integration Data Upload may be missing if the wrong version
is provided.
b) Check if the value in directIntegration is True.
2. Verify that the HTTP-1 object in NetAct Monitor is created as a child of the
integration agent (OMS/AXC).
a) • Check if the instance is 1.
b) • Check if the hostname has the correct IP value. For example,
10.11.12.13.
c) • Check if the security mode is HTTPS for OMS and HTTP for AXC.
3. Check if Mediation is running: Verify status in smanager.pl and verify the
status of Service Assemblies.
OS90538EN15GLA0
46
4. Check if the OMS/AXC is registered correctly.

Run nwi3_db_tool.pl in the VM node holding the nwi3 service:
[root@n83-vm09 bin]# /opt/oss/bin/nwi3_db_tool.pl -list services
SYSTEM ID | NODE | SERVICE |
TIME STAMP | STATE
------------------------------------------------------------------------
--------------------------------------
NOTIFICATION_SERVICE | n83-vm09.n83.obs.wa-slab.nsn-rdnet.net |
IDL:NWI3/NotifyPublish_V1:1.0 | 2014-04-17 19:39:33 | ACTIVE
NE-OMS-4 | n83-vm09.n83.obs.wa-slab.nsn-rdnet.net |
IDL:NWI3/SessionManager_V1:1.0 | 2014-04-23 03:51:01 | ACTIVE
------------------------------------------------------------------------
--------------------------------------
Found 2 service(s)
Check if an entry is in active state and has a timestamp equivalent to less than
24 hours.
 If the element is not registered, then the issue could be with the Network
Element Access Control(NEAC) credentials, enabling of the Centralized NE
User Management (CNUM), or with NE registration credentials
 If the element is registered but the objects do not appear in Monitor, there
might be a resource shortage on the side of the network element, or a
primary key violation in NASDA
5. Verify that NEAC contains the proper credentials.
a) In Network Element Access Control application .check if NWI3 Access
credentials have been created for the Maintenance Region (MR) or NE Type.
b) Check if the credentials have been associated to the sysop user group and
to the group of the user logged into NetAct Monitor.
c) If MR credentials are used, then check if OMS/AXC is attached to that MR.
6. Make sure that CNUM is not enabled in OMS/AXC.

To disable CNUM in OMS:
 Open the Parameter tool, and then select fsFragmentId=OMS→
omsFragmentId=System → omsFragmentId=RUIM
→omsParameterId=RUIMAutomaticActivation.
 Change the OMS Parameter Value from 1 to 0.
 Log in to the OMS or GOMS as the root user.
 Deactivate CNUM in OMS with the following command:
# fscontrolRUIM --disable
OS90538EN15GLA0
47
To disable CNUM in AXC:

 Log in to AXC using ftp.
 Delete the following files:
/cfg/neac.db
/var/backup/neac.db.backup
 Reboot the AXC node
7. Check that the NE registration credentials are correct.

The valid registration credentials for the network element are system/system
or nwi3system/<password>.
To acquire the nwi3system password from SCA, log in as root user in the VM
running the LDAP service with the following command:
[root@node ~]# /opt/nokia/oss/bin/syscredacc.sh -user
nwi3system -type appserv -instance appserv
 Check AXC registration credentials:

a) Open the AXC Manager.
b) Select Configuration → Management Protocol.
c) Select Configuration in Active State and click Modify.
d) Check if the username and password under Initial Registration are
valid.
 Check OMS registration credentials:
a) Open the Parameter tool using the Application Launcher.
b) Select fsClusterId=ClusterRoot → fsFragmentId=NWI3 →
fsFragmentId=mediator.
c) Select the active configuration fsnwi3N3CFId, that is, the one with
Attribute Namefsnwi3configurationActive and has an Active Value of
true.
For example, the fsnwi3N3CFId=1 object has the attribute name
fsnwi3configurationActive that has the Active Value of true.
d) Check that the values of fsnwi3initialRegistrationPassword and
fsnwi3initialRegistrationUsername are valid.
OS90538EN15GLA0
48
3.1.2 Alarms are not received

The active alarms in NetAct database or Monitoring tools or both do not match the
active alarms in network elements.
Solution
1. Check if the NWI3 mediation is running.
Revise nwi3 service status with smanager.pl and revise the status of the nwi3
mediation service assemblies using MF Console. Restart service or restart
service assemblies accordingly.
2. Check if the network element is properly integrated.
 Check if the NWI3-1 interface object appears with the correct nwi3stemid
attribute under the OMS/AXC object in NetAct Monitor.
If the NWI3-1 object does not appear, then start the integration data upload
from the OMS/ AXC context menu.
 Check if the network element agent objects (RNC, WBTS, MRBTS, LNBTS)
that are connected via OMS appear in NetAct Monitor.
If these do not appear, then start the integration data upload from the
OMS/AXC context menu.
 Check if the network element is correctly registered.
For more information, see Step 4 in the previous case for Integration data
upload is not working.
OS90538EN15GLA0
49
Troubleshooting Radio Networks

NWI3 Mediation
More cases information
Alarms are not received
The active alarms in NetAct database or Monitoring tools in the course material
or both do not match the active alarms in NE.
Solution
•Check if the NWI3 mediation is running.
Revise nwi3 service status with smanager.pl and revise the status of the nwi3 mediation
service assemblies using MF Console. Restart service or restart service assemblies
accordingly.
•Check if the network element is properly integrated.
 Check if the NWI3-1 interface object appears with the correct nwi3stemid attribute
under the OMS/AXC object in NetAct Monitor.
 If the NWI3-1 object does not appear, then start the integration data upload from the
OMS/ AXC context menu.
 Check if the network element agent objects (RNC, WBTS, MRBTS, LNBTS) that are
connected via OMS appear in NetAct Monitor.
If these do not appear, then start the integration data upload from the OMS/AXC
context menu.
 Check if the network element is correctly registered.
1 Use #©/opt/oss/bin/nwi3_db_tool.pl
Nokia Solutions and Networks 2014 -list services tool to check registration of NE
Fig. 14
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.3 PM data is not received from OMS

The OMS PM data does not reach the NetAct Performance Management database or
reporting tools or both.
Solution
1. Check if the NWI3 mediation is running.
2. Check if the network element is properly integrated.
 Check if the NWI3-1 interface object appears with the correct nwi3stemid
attribute under the OMS/AXC object in NetAct Monitor.
If the NWI3-1 object does not appear, then start the integration data upload
from the OMS/ AXC context menu.
 Check if the network element agent objects (RNC, W BTS, MRBTS, LNBTS)
that are connected via OMS appear in NetAct Monitor.
If these do not appear, then start the integration data upload from the
OMS/AXC context menu.
 Check if the network element is correctly registered.
For more information, see Step 4 in the previous case for Integration data
upload is not working
3. .Check if some measurements are active. Use Administration of Measurements
application or Element Manager accordingly to the case.
4. If the certificates are not configured for secure file transfer, then ensure that the
OMS transfer mode is HTTP instead of HTTPS.
If needed, disable secure file transfer from OMS:
a) Open the RNC OMS Parameter Tool from the Application Launcher.
b) To open the TLSMode parameter, select fsClusterId=ClusterRoot →
fsFragmentId=OMS →omsFragmentId=System → omsFragmentId=Network
→ omsFragmentId=TLS
c) Set the TLSMode(FT) parameter value to “Off”.
 Select omsFragment=TLS → omsParameterID=TLSModeFT
 Select omsParameterValue, then double-click the item under the Active
Value column to
 select Off from the drop-down menu.
5. Check the NWI3 mediation PM activity logs for any PM file transfer activity.
Check the PMEventActivity.log* and PMEventFiles.log* files in
/var/opt/oss/log/nwi3directory
OS90538EN15GLA0
51
Troubleshooting Radio Network

NWI3 Mediation: Checking Network Element Registration
[root@n83-vm09 bin]# /opt/oss/bin/nwi3_db_tool.pl -list services

SYSTEM ID | NODE | SERVICE
| TIME STAMP | STATE
------------------------------------------------------------------
--------------------------------------------
NOTIFICATION_SERVICE | n83-vm09.n83.obs.wa-slab.nsn-rdnet.net |
IDL:NWI3/NotifyPublish_V1:1.0 | 2014-08-17 19:39:33 | ACTIVE
NE-OMS-4 | n83-vm09.n83.obs.wa-slab.nsn-rdnet.net |
IDL:NWI3/SessionManager_V1:1.0 | 2014-08-21 03:51:01 | ACTIVE
------------------------------------------------------------------
--------------------------------------------
Found 2 service(s)
Fig. 15
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90538EN15GLA0
. . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . © Nokia
. .Solutions
. .and.Networks
. . 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Q3/MML Mediation

3.2.1 No alarm or event come from Q3 based NEs like BSC and
SGSN
When the network element is correctly integrated, yet the following operations still
fail:
 There are no NE specific alarm coming from BSC and SGSN.
 Alarm Upload, it fails.
 There is no PM data coming.
 No HWM event comes.
Solution
1. Check status of NetAct services q3user and osi using smanager.pl. Start
services if stopped.
2. Log in to theVM node where osi and q3user services are located as root user.
3. Restart the osi and q3user services like below:
[root]# /opt/cpf/bin/smanager.pl stop service osi
Service osi stopped
[root]# /opt/cpf/bin/smanager.pl start service osi
Service osi started on node <node name>
[root]# /opt/cpf/bin/smanager.pl stop service q3user
Service q3user stopped
[root]# /opt/cpf/bin/smanager.pl start service q3user
Service q3user started on node <node name>
4. If problem continues, verify that Service Assemblies in q3user mediation are
started using MF Console located in /opt/oss/NSN-nwi3/smx/mf-console
OS90538EN15GLA0
53
3.2.2 PM data is not received from BSC/SGSN

Solution
1. Check if the Q3 mediation is running.
 Check q3service and osi service via smanager.
 If services are not started, use smanager.pl tool to start them
 If problem continues, verify that the Service Assemblies of q3user JBI are
started. If any found stopped, start it using MF Console.
2. Check if the network element is properly integrated. Check if the OSI-1 interface
object exists with the correct attribute under the BSC/SGSN object in NetAct
Monitor. For more information, refer to both BSC and SGSN integration
documents.
3. Check if the measurements are active using Administration of Measurements
application.
4. Check if the measurement file ready event has been received by NetAct.
a) Query the following SQL statement “select * from
Q3_COMMON.Q3_PM_EVENTS where DN has the format of %BSC-xxxxxx%.
b) The event should be in the database. If not, check the
com.nsn.oss.q3.common.duplicate_event_interval value in
$Q3USER_HOME/smx/mf-conf/com.nsn.oss.q3.common.properties.
The default value is 120 minutes, which means the same file from the same
network element will be dropped within 120 minutes.
c) Check if the credential exists and is correct for “Q3 Access” for the network
element using NEAC application.
5. Check if the file has been transferred to NetAct.
a) Open download capacity log /var/tmp/NSN-q3med-
common/capacity/CL_Common_FileUpload_<date>.csv.
b) Check if the file in question has been recorded in the log with status 0
(successful status).
c) Check if the credential exists and is correct for given access type (FTP Access
or FTAM Access) in NEAC application.
d) Check if you can manually transfer the file in question by FTP client or
/opt/oss/NSNq3med-common/bin/ftam_client.sh. If the file can’t be
transferred manually, check the DCN configuration.
6. Check if the file has been decoded into OMeS.
a) open the decoding capacity log /var/tmp/NSN-q3med-
pm/capacity/CL_Pm_<bsc|sgsn>_<meas|obs>_dec_cap_<date>.csv
b) Check if the file is decoded successfully. All measurements should be
decoded into OMeS files.
OS90538EN15GLA0
54
Troubleshooting Radio Network

Q3/MML Mediation
PM Data not received

1. Check if the Q3 mediation is running. Use smanager.pl and verify SAs using MF Console.
2. Check if the NE is properly integrated. Check if the OSI-1 interface object exists with the correct attribute
under the BSC/SGSN object in NetAct Monitor.
3. Check if the measurements are active using Administration of Measurements application.
4. Check if the measurement file ready event has been received by NetAct.
• Execute the query “select * from Q3_COMMON.Q3_PM_EVENTS where DN has the format of %BSC-xxxxxx%.
• The event should be in the database. If not, check the com.nsn.oss.q3.common.duplicate_event_interval value in
$Q3USER_HOME/smx/mf-conf/com.nsn.oss.q3.common.properties. The default value is 120 minutes, which means the
same file from the same network element will be dropped within 120 minutes.
• Check if the credential exists and is correct for “Q3 Access” for the network element using NEAC application.
5. Check if the file has been transferred to NetAct.
• Open download capacity log /var/tmp/NSN-q3med-common/capacity/CL_Common_FileUpload_<date>.csv.
• Check if the file in question has been recorded in the log with status 0 (successful status).
• Check if the credential exists and is correct for given access type (FTP Access or FTAM Access) in
NEAC application.
• Check if you can manually transfer the file in question by FTP client or /opt/oss/NSNq3med-
common/bin/ftam_client.sh. If the file can’t be transferred manually, check the DCN configuration.
6. Check if the file has been decoded into OMeS.
• open the decoding capacity log /var/tmp/NSN-q3med-
pm/capacity/CL_Pm_<bsc|sgsn>_<meas|obs>_dec_cap_<date>.csv
• Check if the file is decoded successfully. All measurements should be decoded into OMeS files
Fig. 16
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90538EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
. Solutions
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.3 Generating upload file failed

The Upload operation from CM Operations application fails with the following error:
Generating upload file failed
Solution
1. Check the real network element version.
2. Telnet to the network element and check version by executing
ZDFD:OMU:88E;.
3. If there is difference between the versions, then modify network element version
by executing:
/ opt/oss/NSN-q3med-cm/bin/q3versMx.sh VersionChange <dn>
<version>.
For more information on q3versMx.sh script description, refer to Q3/MML
stand-alone components data flow in Administering Q3/MML Southbound
Mediation.
3.2.4 Upload or file based plan provisioning ongoing

When Upload or Provision operations are executed from CM Operations Manager
application, it fails with the following error:
UPLOAD OR FILE BASED PLAN PROVISIONING ONGOING
Solution
Directory name provided in upload request should have write-permissions for the
FTAM/FTP Unix credential user.
OS90538EN15GLA0
56
4 Exercises
OS90538EN15GLA0
57
Exercise 1
Title: General Troubleshooting Tasks
Objectives: Use the general procedures for verifying the status of

Southbound and Northbound mediations
Pre-requisite: none
Task
Please execute the following exercises in the testing system using the Credentials
provided by your trainer. Answer the corresponding questions.
Query
1. From one of the mediations below, verify the corresponding status of the service
and revise the status of the corresponding Service Assemblies.
a) What commands can you use to check the status of the mediation Service?
______________________________________________________________
______________________________________________________________
______________________________________________________________
b) How can you display the Service Assemblies of your selected mediation? List
the Service Assemblies. Are any of them in a status different from "started"?
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
c) Practice stopping /starting Service Assemblies. Select from your mediation

one Service Assembly and stop it. Check the impact of this in the status of the
corresponding service for the mediation. What can you see? Restart the
Service Assembly.
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
OS90538EN15GLA0
58
d) Stop your selected mediation service. After the command is successful, check
the status of the mediation Service Assemblies. What can you see? Restart
the service after verfication.
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
e) In which log can you find the startup/ stop of the mediation? List below the
corresponding location and related entries of the log.
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
_____________________________________________________________
f) List the logs for your selected mediation. Where are they located? What
information can you find there? Open the log files and compare with the
descriptions in Operating Documentation.
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
OS90538EN15GLA0
59
g) Start tracing for your selected mediation. List the steps you need to execute
this task. Where can you find the tracer results? From the tracer result file,
what kind information is recorded by the tracer? In case of Q3/MML mediation,
select only one fragment to trace.
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
h) Stop the trace. Verify in the log file timestamp that tracer is actually stopped.
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
_____________________________________________________________
OS90538EN15GLA0
60
Exercise 2
Title: Troubleshooting Radio and Core Networks
Objectives: Practice different procedures to troubleshoot specific failures

cases
Pre-requisite: none
Task
Please execute the following exercises in the testing system using the Credentials
provided by your trainer. Answer the corresponding questions.
Query
Execute the tasks for troubleshooting the case defined by your trainer. List the steps
and results of those tasks.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
___________________________________________________________________
OS90538EN15GLA0
61
OS90538EN15GLA0
62

08 OS90538EN15GLA0 Mediations Troubleshooting

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

08 OS90538EN15GLA0 Mediations Troubleshooting

Încărcat de

Drepturi de autor:

Formate disponibile

Mediations Troubleshooting

1 General Troubleshooting Tasks

1.1 Verifying and Correcting status of Mediations

1.1.1 General Procedure

[root@n85vm16 ~]# /opt/cpf/sbin/netact_status.sh status

3. Depending on the results of the netact_status.sh command, follow the first

4. Even if the netact_status.sh command shows that the service is running, it

General Troubleshooting Tasks

Verify If status is not If the status is If the status is

1 © Nokia Solutions and Networks 2015

1.1.2 NetAct Status Script

[root@n85vm16 ~]# /opt/cpf/sbin/netact_status.sh status

General Troubleshooting Tasks

[root@n85vm16 ~]# /opt/cpf/sbin/netact_status.sh status

1.1.3 Mediation Framework Console

Get Displays all available static information for the Servicemix

General Troubleshooting Tasks

1.1.4 Managing Service Assemblies with MF Console

Examples of Service Assemblies in different Mediations:

Common Mediations: NE3S/WS – 3GPP Corba Southbound – NE3S/SNMP

INFO - DispSACommand - m3gcsbm-fpe-adapter-oes-sa -- Started

SNMP Northbound Interface

3GPP Corba NBI Common Services (nbi3gc service):

General Troubleshooting Tasks

[root@n85vm09 mf-console]# cd /opt/oss/NSN-nwi3/smx/mf-console

1 © Nokia Solutions and Networks 2015

General Troubleshooting Tasks

[root@n85vm09 mf-console]# ./MFC.sh dispSA

1.1.5 Mediations Logging and Tracing

General Troubleshooting Tasks

Used to indicate which methods are called,

Trace messages are detailed and technical in

Logging Type Description Log Name Log Level

General Troubleshooting Tasks

Logging Type Description Log Name Log Level

For example, in the case of XoH-mediation the logging directory is

Mediation Log Directories Trace Script

NWI3 SBI /var/opt/oss/log/ nwi3 /opt/oss/NSN-

Q3/MML SBI /var/opt/oss/log/q3user /opt/oss/NSN-q3med-

NE3S/WS /var/opt/oss/log/ Manually modify the file: /opt/oss/NSN-

XoH SBI /var/opt/oss/log/ xoh #/opt/oss/NSN-

NX2S SBI /var/opt/oss/log/ nx2s #/opt/oss/NSN-jbi_swp/bin/

FMASCII /var/opt/oss/log/ fmascii # /etc/init.d/fmascii log <level>

SNMP NBI /var/opt/oss/log/ # nbisnmp log <level>

3GPP Corba /var/opt/oss/log/ nbigc # smx-clt log <level>

3GPP Corba /var/opt/oss/log/ # nbi3gcpm log <level>

General Troubleshooting Tasks

Manually modify the file: /opt/oss/NSN-

General Troubleshooting Tasks

Mediation Log Directories Trace Script / Configuration File

FMASCII NBI /var/opt/oss/log/ fmascii # /etc/init.d/fmascii log <level>

2 Troubleshooting the Core Network

2.1 NE3S/WS Troubleshooting Cases

Troubleshooting Core Network

Solution: Re-register the network element:

Delete from COMMON_MEDIATIONS.JBOSS_CACHE_DATA where

2.1.2 Alarm upload failure

2. Registration failure and 9047 Alarm is raised due to registration failure.

3. Check lock manager issues:

2.1.3 PM Data not available

2.2 SNMP Troubleshooting Cases

2. Check that the mediation is listening to traps

3. Check trap auto-forwarding

4. Check trap by mediation