Getting SP Collects

(What are SP Collects?)

 SP collects SP Logs
Each SP is a Windows machine underneath
They have a perl script installed on all SPs that
gather and generate a log file in c:\dumps
The time it takes to generate logs depends on
how big the array is, how busy it is, and how
eventful its life may have been.
Were going to step through how to kick off SP
collects, monitor them while building and
retrieval
 Starting SP collects remotely
From the Control Station
/nas/tools/.get_spcollect
/nas/sbin/navicli
/nas/tools/collect_support_materials -getsp
RemotelyAnywhere
From the Windows Console
ssh session to the block on VNX
Computer Management command prompt
Unisphere
Unisphere Service Manager
RCM_Unisphere
naviseccli over ESRS
 Please note
It doesnt make any difference which method
you use to start or retrieve the logs its the
exact same script (spcollect.pl) running on the
SPs
For example, you can start SPA SP collect using
naviseccli from the desktop, SPB from
Unisphere and retrieve SPAs SP logs using
RemotelyAnywhere and SPBs logs using
navicli
 Control Station
/nas/sbin/navicli
Get the IP addresses of the SP:
grep SP /etc/hosts
Test SP connection by getagent on both SPs
/nas/sbin/navicli h SPA_IP getagent
/nas/sbin/navicli h SPB_IP getagent
Start SP collect script on both SPs
/nas/sbin/navicli h SPA_IP spcollect
/nas/sbin/navicli h SPB_IP spcollect
Get the SPs IP addresses

Test connectivity and the

management server on both
SPs using getagent

Start the SP collects on both SPs

 Control Station
/nas/sbin/navicli
To monitor:
/nas/sbin/navicli h SPA_IP managefiles list | grep
runlog
/nas/sbin/navicli h SPA_IP managefiles list | grep
runlog
To retrieve
/nas/sbin/navicli h SPA_IP managefiles -retrieve
/nas/sbin/navicli h SPB_IP managefiles -retrieve
SPA SP Logs gathering
SPB SP Logs gathering
SPA SP Logs gathering
SPB SP Logs gathering
Both SPs Logs
gathered
 Control Station
/nas/tools/.get_spcollect
It operates serially :
Start SP collects on SPA
Monitor SPA log gathering until logs gathered
Transfer SPA log to /nas/var/log
Start SP collects on SPB
Monitor SPB log gathering until logs gathered
Transfer SPB log to /nas/var/log
Combine SPA & SPB logs into one zip file
/nas/var/log/SPCOLLECT.zip
 Starting

Monitoring SPA
Retrieval
Starting
Monitoring SPB
Retrieval

Combine

Pros
Does the whole thing for you!

Cons
Slower as done serially.
Can time out if it takes too long.
Sometimes retrieves an incorrect file instead of the expected one.
 Control Station
/nas/tools/collect_support_materials -getsp
This tool gathers File logs and optionally get
SP collects. It start SP collects, then gathers
file logs and when done waits for the SP
Collects to finish gather before amalgamating
them in one zip file in /nas/var/emcsupport
Pros
Does everything for you

Cons
Also gathers file logs which we are less interested in.
 RemotelyAnywhere
From the Windows console
Login to RemotelyAnywhere on each SP
(clariion1992/clariion1992)
Remote Control
Login to the SP (clariion/clariion!)
Open a console window (cmd)
Run spcollect.pl
RemotelyAnwhere
 RemotelyAnwhere
To retrieve the SP collect file once the script
has finished running
Logout* of the console (Windows GUI)
Select File Manager
Use the File Manager to transfer the file back to
your desktop from c:\DUMPS on the SP.
Its easiest to select the file by ordering by most
recently modified. If theres a runlog file, then
wait and refresh.

* It is vitally important not to select Shutdown instead of Logout you would shutdown the SP
RemotelyAnwhere

Transfer to:
(Local)

C:\DUMPS

SPCollect
 RemotelyAnwhere
ssh session to the block on VNX
Login as clariion1992/clariion1992
Run spcollect.pl
We need to collect the SP Collects using a
different method, such as RemotelyAnywhere File
Manager
 RemotelyAnwhere
Computer Management command prompt
Login to Remotely Anywhere, Computer
Management, Command Prompt.
Run spcollect.pl
 Unisphere Client
Ensure you are on the correct system using
the Dashboard
 Unisphere Client
System, Diagnostic Files, Generate Diagnostic
files for both SPA & SPB separately.
 Unisphere Client
To monitor, Get Diagnostics Files

Order by Date

Transfer to here

runlog: still gathering

Refresh periodicially
until runlog is gone
 Unisphere Client
To retrieve the files, Select the Log and
Transfer
Order by Date Select Log Transfer Confirm Monitor Transfer

Ignore RP files
 Unisphere Service Manager
Verify were on the correct system (S/N)
Diagnostics, Capture Diagnostic Data
 Unisphere Service Manager
Save the SP Collects once gathered

When Logs
are gathered
click Save As

Choose your
location to
save to

Save
 RCM_Unisphere
ftp://ftp.emc.com/pub/rcm/vnx/tools/rcm_unisphere/RCM_Unisphere_1.25.exe

Login to the array

 RCM_Unisphere
Update your
default log
location setting
(Only needs to be done once)
 RCM_Unisphere
Start SP Collects
Health checks
Generate
Diagnostics for
both SPs
 RCM_Unisphere
To Monitor
Get Diagnostic
Files (SP A & B)
Look for runlog
If runlog exists,
logs are still
gathering
Close and
reload the page
to get an
update.
 RCM_Unisphere
To Retrieve
Select the
SP Collect
Select the
type,
pre/post.
Click
transfer
 naviseccli
Using cmd.exe
To start
Verify
connectivity
getagent
Start SP collects
on both SP
spcollect
 naviseccli
Using cmd.exe
To monitor
Look for the runlog file

The grep utility is very useful copy it from \\corpeufs9\Techsupp\GTS

RCM\_Daniel\tools\grep.exe into C:\Program Files (x86)\EMC\NGTRiiAGE
 naviseccli
Using cmd.exe
To retrieve
managefiles -
retrieve
 Summary
There are many ways of starting SP Collects
There are many ways of retrieving SP Collects
All yield the exact same log files.
Running in parallel is faster and more reliable,
but takes more real time.
 Recommendations
Block only
Unisphere client
Unified
/nas/tools/.get_spcollect
parallel
 Troubleshooting
Triiage is blank for an SP

File was transferred correctly. File may not be transferred correctly.

Restart the management Yes No Check the file size on the SP against
Zip Valid?
server on that SP and take the file on your PC and retransfer if
fresh logs. different

PCE: Schedule an SP reboot

UE: Proactive SP reboot
No Customer permission is required.
OK now?
If possible verify host connectivity
to the other SP via triiage or
Yes Unisphere client beforehand.

Done
Questions
 Restarting Management Agent
http://SP_IP/setup
Login with Unisphere credentials
 Restarting Management Agent
Select Restart Management Agent
Confirm Management agent restart
Note: There is no production impact to management agent restart
 Restarting Management Agent
If on a control station and getagent is
working
navicli networkadmin -restartcimom