Troubleshooting error message: Naviagent respawning too fast; disabled...

1 of 2

http://knowledgebase.emc.com/emcice/resultDisplay2.do?result=-1&clus...

EMC Knowledgebase

"Troubleshooting error message: Naviagent respawning too fast; disabled for 5 minutes."
ID:

emc257445

Usage:

69

Date Created:

12/07/2010

Last Modified:

01/11/2011

STATUS:

Approved

Audience:

Customer

Knowledgebase Solution
Question:

Troubleshooting error message: Naviagent respawning too fast; disabled for 5 minutes.

Environment:

Product: Celerra File Server (CFS)

Environment:

Product: Celerra Network Server (CNS)

Problem:

CS_PLATFORM:MasterControl:EMERGENCY:6::::1290663970:Daemon Naviagent unexpectedly exited (status = 1536); ifexit=1, exitstatus=6,

ifsignal=0, termsig=0, ifstop=0, stopsig=0, ifdump=0.

Problem:

CS_PLATFORM:MasterControl:EMERGENCY:15::::1290663979:Naviagent respawning too fast; disabled for 5 minutes.

Problem:

Signal ABRT received

Problem:

Naviagent respawning too fast; disabled for 5 minutes

Problem:

The following set of messages is seen streaming in the log files every five minutes. [nasadmin@LosAngeles_CS ~]$ nas_logviewer /nas/log
/sys_log Dec 7 10:23:15 2010:CS_PLATFORM:MasterControl:EMERGENCY:6::::1290663970:Daemon Naviagent unexpectedly exited (status =
1536); ifexit=1, exitstatus=6, ifsignal=0, termsig=0, ifstop=0, stopsig=0, ifdump=0.
Dec 7 10:23:15 2010:CS_PLATFORM:MasterControl:EMERGENCY:15::::1290663979:Naviagent respawning too fast; disabled for 5 minutes.
[nasadmin@LosAngeles_CS ~]$ cat /var/log/messages
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: Agent initializing with pid 22384
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: Agent daemon process created, pid 22384
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: Agent has started up.
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: EV_HBASPPortState::_setParentHBAPort not NULL
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: EV_HBAPort::_setParentHost not NULL
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: EV_Device::DeviceBusIDConvert: 255,8
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: Signal ABRT received
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: EV_MultiHostRAIDXX::RemoveHost() : EV_Server Missing for host
The first step is to confirm the Control Station name and IP are valid. This can be completed by ensuring that the following five outputs are
consistent. The parts that are required to match are in bold.
1. [root@LosAngeles_CS ~]$ uname -a
Linux LosAngeles_CS 2.6.9-67.0.4.5611.EMC #1 Fri Apr 4 12:30:56 EST 2008 i686 i686 i386 GNU/Linux
2. [root@LosAngeles_CS ~]$ cat /etc/hosts |grep -i LosAngeles_CS
10.241.183.100 LosAngeles_CS.EMC.local
LosAngeles_CS
3. [root@LosAngeles_CS nasadmin]# cat /nas/site/cshosts
0:LosAngeles_CS:0:10.241.183.100:::APM000815016330000:
4. [root@LosAngeles_CS nasadmin]# tail -1 /var/log/messages
Dec 7 10:23:15 LosAngeles_CS Navisphere Agent[22384]: Signal ABRT received
5. [nasadmin@LosAngeles_CS ~]$ /sbin/ifconfig eth3

Fix:
eth3
Link encap:Ethernet HWaddr 00:1B:21:13:A3:E0
inet addr:10.241.183.100 Bcast:10.241.183.127 Mask:255.255.255.128
inet6 addr: fe80::21b:21ff:fe13:a3e0/64 Scope:Link
UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2954088 errors:0 dropped:0 overruns:0 frame:0
TX packets:2177836 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:392851044 (374.6 MiB) TX bytes:451237250 (430.3 MiB)
Base address:0xc800 Memory:fc5c0000-fc5e0000
If any of the above outputs do not match, proactively reboot the Control Station. After the Control Station reboots, if they still do not match, some
manual editing may be required. If the above outputs all match properly, check the "/var/log/messages" file to see if the problem still occurs. If the
problem persists, confirm that the Control Station and CLARiiON storage processors (SPs) are linked at 100FD on the public network. This command
must be run as root account on the Control Station. [root@LosAngeles_CS sbin]# /sbin/mii-tool -v eth3
eth3: negotiated 100baseTx-FD flow-control, link ok
product info: vendor 00:50:43, model 2 rev 3
basic mode:
autonegotiation enabled

2/13/2012 1:35 PM

Troubleshooting error message: Naviagent respawning too fast; disabled...

http://knowledgebase.emc.com/emcice/resultDisplay2.do?result=-1&clus...

basic status: autonegotiation complete, link ok

capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control If the link is showing less than "negotiated
100baseTx-FD," the network administrator must manually down and up the port on the associated network switch for the Control Station. Note: A
reboot of the Control Station will NOT reset this link speed. While the network administrator is logged in to the switch, have the administrator confirm
that the CLARiiON SP ports are set to auto/auto on the switch side for the CLARiiON SP A and SP B. Hard coding the ports has been known to cause
communication problems. If the network switch does not allow for auto/auto configurations, set the CLARiiON SP ports to auto / FD Here is a basic
diagram to show the above:
Control Station | Network Switch
|
CLARiiON SPs
(CS)
| (Customer side) |
(Internal to the SPs)
100-FD --> 100-FD
auto/auto --> auto/auto
(SP A)
|
auto/auto --> auto/auto
(SP B)
|
|
If the problem persists after resolving the network port link speeds. Obtain permission to
restart the Management Server on the SP. [root@LosAngeles_CS sbin]# cat /etc/hosts |grep -i sp
10.241.183.105 A_APM00081501633
SPA # CLARiiON SP
10.241.183.106 B_APM00081501633
SPB # CLARiiON SP Then log in to the CLARiiON SPs directly via the following web links:
http: //10.241.183.105/setup Then repeat this step for the peer SP. http: //10.241.183.106/setup If either of the CLARiiON SPs are
non-responsive and are not able to be logged in to to remotely, engage CLARiiON Support to address the CLARiiON Management problem. Once the
CLARiiON Management problem is resolve then confirm the naviagent restarting problem is resolved on the Celerra Control Station.

2 of 2

2/13/2012 1:35 PM