iDataPlex dx360

EBG Server Education
Servicing the Lenovo iDataPlex dx360

M2 and M3 servers
November 2010
Study guide
XW5181
This course is owned and published by
EBG Server Education.
___________________________________________________________________________________
Copyright 2014, Lenovo Corporation.

Lenovo
8001 Development Drive
Morrisville, North Carolina, 27560
Lenovo reserves the right to change product information and specifications at any time
without notice. This publication might include technical inaccuracies or typographical
errors. References herein to Lenovo products and services do not imply that Lenovo
intends to make them available in all countries. Lenovo provides this publication as is,
without warranty of any kind, either expressed or implied, including the implied
warranties of merchantability or fitness for a particular purpose. Some jurisdictions do
not allow disclaimer of expressed or implied warranties. Therefore, this disclaimer might
not apply to you.
Data on competitive products is obtained from publicly obtained information and is
subject to change without notice. Contact the manufacturer for the most recent
information.
Lenovo and the Lenovo logo is a trademark or registered trademark of Lenovo
Corporation or its subsidiaries in the United States, other countries, or both. Intel and
the Intel logo is a trademark or registered trademark of Intel Corporation or its
subsidiaries in the United States, other countries, or both. Other names and brands are
the property of their respective owners.
The following terms are trademarks, registered trademarks, or service marks of Lenovo:
Access Connections, Active Protection System, Automated Solutions, Easy Eject Utility,
Easy-Open Cover, IdeaCentre, IdeaPad, ImageUltra, Lenovo Care, MaxBright, NetVista,
New World. New Thinking, OneKey, PC As A Service, Rapid Restore, Remote
Deployment Manager, Rescue and Recovery, ScrollPoint, Secure Data Disposal,
Skylight, Software Delivery Center, System Information Gatherer, System Information
Reporter, System Migration Assistant, System x, Think Pad, ThinkAccessories,
ThinkCenter, ThinkCentre, ThinkDisk, ThinkDrive, ThinkLight, ThinkPad, ThinkPlus,
ThinkScribe, ThinkServer, ThinkStation, ThinkStore, ThinkVantage, ThinkVision,
ThinkWorld, TopSeller, TrackPoint, TransNote, UltraBase, UltraBay, UltraConnect,
UltraNav, VeriFace.
For more information, go to: http://www.lenovo.com/legal/copytrade.html.
The terms listed for the following partners are the property of their respective owners:
AMD
Intel
IBM
Microsoft
NVIDIA
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Table of contents
Table of contents
Table of contents........................................................................................................... 3
Objectives .................................................................................................................. 5
Servicing the IBM iDataPlex dx360 M2 and M3 servers ............................................. 6
Overview .................................................................................................................... 6
Features and specifications ....................................................................................... 7
Production description ............................................................................................... 9
System configurations and architecture ................................................................... 11
Mechanical system-board layouts ............................................................................ 13
System-board mechanical description ................................................................ 14
Light path ............................................................................................................ 16
Memory and SCSI Subsystems ............................................................................... 18
Memory Subsystem ............................................................................................ 18
Important memory subsystem conecpt ............................................................... 19
Memory mirroring rules ....................................................................................... 19
Serial Attached SCSI (SAS) subsystem ............................................................. 21
Intel microprocessor subsystem overview ............................................................... 22
Thermal profile ......................................................................................................... 24
Unified Extensible Firmware Interface (UEFI) .......................................................... 25
Integrated Management Module (IMM) .................................................................... 29
Preboot Dynamic System Analysis (pDSA) and Online DSA ................................... 31
Component installation ............................................................................................ 33
Installing and uninstalling the storage riser card assembly, the system- board
assembly and the ServeRAID Battery ................................................................ 33
Remove the storage riser card assembly ........................................................... 34
System board tray assembly .............................................................................. 36
Remove the battery holder and ServeRAID Li-Ion battery.................................. 39
NVIDIA graphics and compute devices.................................................................... 43
Proper installation of NVIDIA devices ...................................................................... 45
Overview ............................................................................................................ 45
PCI Express Slot ................................................................................................ 45
Auxiliary power ................................................................................................... 46
Troubleshooting and problem determination ............................................................ 48
Problem determination pathway for Intel products.............................................. 48
iDataPlex warranty call flow ................................................................................ 48
The iDataPlex dx360 M2 and M3 - Important common problems ....................... 49
Six system states................................................................................................ 51
Problem determination Logs ............................................................................ 52
System event log ................................................................................................ 53
Online DSA ......................................................................................................... 54
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Table of contents
Helpful links ................................................................................................................. 55

Summary ...................................................................................................................... 56
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Objectives
Objectives
After completing this course, you will be able to:
1. Provide an overview of the iDataPlex dx360 M2 and M3 servers.
2. Understand the main differences between the iDataPlex dx360, the dx360 M2
and the dx360 M3 servers.
3. Describe the system architecture of the iDataPlex dx360 M2 and M3 servers
including the memory and SAS subsystems.
4. Describe the mechanical and electrical systems found in the iDataPlex dx360 M2
and M3 servers.
5. Understand how the iDataPlex dx360 M2 and M3 servers use IMM, UEFI and
Preboot DSA.
6. Describe the thermal component of the iDataPlex dx360 M2 and M3 servers.
7. Understand the different NVIDIA GPU offerings and component installation.
8. Perform problem determination activities and troubleshoot the iDataPlex dx360
M2 and M3 servers.
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Overview
Servicing the IBM iDataPlex dx360 M2

and M3 servers
Overview
The IBM System x iDataPlex dx360 M2 type 7321 and 7323 system-board tray is suited
for data-center environments that require high performance, energy-efficient, cost
effective hardware. The modular design of the IBM iDataPlex components makes it
possible for the customer to order customized server solutions that meet the specific
needs of their current environment.
The iDataPlex dx360 M2 is supported in the 2U Chassis (type 6385) using one or two
compute nodes. The dx360 M2 is supported in the 3U Storage Rich Chassis (type 7834)
using one compute node only.
The iDataPlex dx360 M3 type 6313, 6385, 6386 and 6391 are updated system-board
tray with slightly enhanced capabilities; otherwise, it serves the same purpose and has
the same configuration options as the M2. The dx360 M3 is discussed in more detail in
the following section of this course.
Customers can configure the dx360 M2 and M3 to meet a large variety of business
needs. By combining compute nodes and storage or I/O trays, there are four possible
configurations for the system:
Compute intensive: maximum processing 2U, two compute nodes
Compute + storage: balanced storage and processing 2U, one node slot and drive
hray hard disk drive: up to five (3.5-inch) or six (3.5-inch x4 + 2.5-inch x2)
Compute + I/O: maximum component flexibility 2U, one node slot I/O: up to two
PCIe. hard disk drive: up to eight (2.5-inch)
3U storage chassis: maximize storage density 3U, one node slot & triple drive tray
hard disk drive: 12 (3.5-inch Drives) up to 12TB
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Features and specifications
Features and specifications

Processor
Two-socket system
Intel Nehalem-EP DP Quad Core Processors with QPI (up to 95w)
Tylersburg chipset B3
Memory
DDR-3 registered DIMMs,800, 1066, or 1333MHz
16 DIMMs, 8 DIMMs per CPU
1GB, 2GB, 4GB
Mirroring: supports single rank, dual rank and quad rank Gigabit Ethernet (two ports)
I/O slots
One PCI-E x16 riser, for 2U 2x compute nodes configuration, 2U storage tray
configuration and 3U storage configuration, or two PCI-E x16 riser for 2U PCI tray
configuration.
One PCI-E x4 (Mini-PCI connector)
One PCI-E x8, storage back slot for 2nd PCIe slot enablement on 2Ux1 server with
storage tray configuration and 3U storage (supports IBM ServeRAIDBR10i
SAS/SATA Controller and SAS/SATA 2 only)
Storage bays
2 x SS 3.5-inch SAS/SATA or
4 x SS 3.5-inch SAS/SATA or
4 x SS 2.5-inch GEN-2 solid state drive or hard disk drive SATA
5 x SS 3.5-inch SATA or
4 x SS 3.5-inch SATA with 2x SS 2.5-inch GEN-2 solid state drive or hard disk drive
SATA
8 x HS 2.5-inch GEN-1 SAS or 12 x HS 3.5-inch SAS/SATA (for 3U only)
RAID
Hardware RAID 0/1 through mini-PCI paddle adapter.
Optional HW RAID 0/1 through PCI-E adapter.
Optional HW RAID 0,1E, 5, 6, 10 through PCI-E adapter.
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Features and specifications
Power supply unit
Dual ac 750W power supply
Modify domain A Power paddle card to support dual ac power supply

o Provides direct access storage device power
o Regular 5V and 3.3V power
o Interfaces to SSI standard boards with standard AVX connections
o Thermistor connections
o Support AEM function
o Support dual ac with throttle function
Modify Domain B power paddle card to support dual ac Power supply

o Provides direct access storage device power
o Provides graphic adapter power
o Support dual ac without throttle function
o For PCI tray configuration and 3Y storage configuration
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Production description
Production description
Figure 1 shows the iDataPlex dx360 M2 and M3 front view.
Figure 1: iDataplex dx360 M2 and M3 front view
Figure 2 shows the iDataPlex dx360 M2 and M3 inside view.
Figure 2: iDataplex dx360 M2 and M3 inside view
December 2014
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Production description
Figure 3 shows the iDataPlex dx360 M2 and M3 rear view.
Figure 3: iDataplex dx360 M2 and M3 rear view
Figure 4 shows the iDataPlex dx360 M2 and M3 power supply unit.
Figure 4: iDataplex dx360 M2 and M3 power supply unit
December 2014
10
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers System configurations and architecture
System configurations and architecture

iDataPlex dx360 M2 and M3 2U and 3U system configurations
Depending on the customers requirements, the dx360 servers can be configured in
many different ways. Refer to Figure 5 below for all 2U and 3U configurations:
Figure 5: dx360 M2 and M3 system configurations for 2U and 3U
December 2014
11
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers System configurations and architecture
Figure 6 shows the dx360 M2 and M3 system block diagram
Figure 6: dx360 M2 and M3 system block diagram
The dx360 M2 and M3 processor board design supports up to two Intel Nehalem-EP
Dual Core (DC) or Quad Core (QC) processors. The Nehalem processor supports high
frequencies up 2.93GHz.
The Tylersburg input and out hub (IOH) provides the interface between the processors,
and PCI Express buses that interface to the Intel Controller Hub 10 (ICH10), the high
speed daughter card connector, and the Blade Storage Expansion (BSE) connector.
December 2014
12
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Mechanical system-board layouts
Mechanical system-board layouts

The following illustrations show the LEDs, connectors, and jumpers on the iDataPlex
dx360 M2 and M3 system board.
Figure 7 shows the locations of the LEDs and controls on the system board.
Figure 7: LEDs and controls on the system board
The Power-on LED flashes 1/2Hz before the IMM is ready. It is solid on while the
system is turned on, and flashing 1Hz while system is in standby mode.
December 2014
13
XW5181
System-board mechanical description

The system board used in the M2 and M3 servers is an off the shelf board that supports
baseboard management controller (BMC) and intelligent platform management interface
(IPMI). It uses UEFI instead of the legacy BIOS. UEFI is discussed later in this course.
Figure 8 shows the connectors on the system board.
Figure 8: Connectors locations on the system board
December 2014
14
XW5181
Figure 9 shows the locations of the jumpers on the system board.
Figure 9: Jumpers location
Table 1 defines the function of each jumper on the system board.

Table 1: Jumper description
Jumper
Description
Clear CMOS (JP1)
Pins 1 and 2: normal (default)

Pins 2 and 3: Clear the CMOS data
UEFI boot recovery (JP2)
Pins 1 and 2: Normal (default)

Pins 2 and 3: Use the secondary page during startup
December 2014
15
XW5181
Light path
When the customer has a failed component in a server, one of the first indications they
receive is a lit error LED on the front of the iDataplex system board. To determine
exactly which component is faulty, the customer must remove the system board tray,
and then locate the error LED that is lit next to the failing component. The components
in the dx360 M2 and M3 system board which have corresponding error LED are
indicated in Figure 10 below.
Figure 10: LED location
December 2014
16
XW5181
Light path LED descriptions:
Error LED: When an error LED is lit, it indicates that the associated component has
failed.
H8 heartbeat: When this LED is flashing, it indicates that the power management
controller is functioning normally.
IMM heartbeat: When this LED is flashing at a constant rate of every other second,
it indicates normal operation of the IMM controller. When this LED is flashing at a
constant rate of every other half-second, it indicates that the IMM controller is
initializing or is not functional.
Microprocessor mismatch: When this LED is lit, it indicates that microprocessor 1

is not installed, or the microprocessors do not have the same cache size and type,
and clock speed.
Standby power: When this LED is lit, it indicates that the server is connected to an
ac power source and that the power supply has supplied the 5volt standby voltage
to the system board.
You can light any error LED after ac power has been removed from the system- board
tray so that you can isolate a problem. After ac power has been removed from the
system-board tray, power remains available to these LED for up to 90 seconds. To view
the error LED, press and hold the reminder button on the system board to light the error
LED. The error LED that were lit while the system-board tray was running will be lit
again while the button is being pressed.
December 2014
17
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Memory and SCSI Subsystems
Memory and SCSI Subsystems

Memory Subsystem
The DDR3 channel speed depends on the DIMM population. The system DDR3 speed
is the lowest of the maximum speed of all channels between two CPUs. Table 2
explains the correct single, double and quad-rank DIMM installation sequence for
processors 1 and 2.
Table 2: DIMM installation seqence
DIMM type
Processor
DIMM connector sequence
Only single-rank and

double-rank
Microprocessor 1
Install the DIMMs in the

following sequence: 3, 6, 8,
2, 5, 7, 1, 4
Microprocessor 2

following sequence: 11,14,
16, 10, 13, 15, 9 ,12
Microprocessor 1

following sequence: 3, 6, 8,
2, 5, 7
Microprocessor 2

following sequence: 11, 14,
16, 10, 13,15
Quad-rank only, or
combination of quad-rank,
single rank, double-rank
December 2014
18
XW5181
Figure 11 shows the installation sequence for installing single-rank and double-rank
DIMMs in non-memory-mirroring mode.
Figure 11: DIMM installation sequence for installing single-rank and double-rank DIMMs in nonmemory-mirroring mode.
Important memory subsystem conecpt
ECC memory corrects single bit memory errors and detects 2 bit errors.
Chipkill detects full DRAM memory errors and increases system availability.
Memory Mirroring reduces system downtime by keeping a real-time copy of data in

memory, and benefits businesses that do not have replacement memory on hand,
or cannot power down the server for service until a scheduled downtime.
The online spare is disabled. The system does not support online spare.
Memory mirroring rules

Memory-mirroring mode replicates and stores data on two pairs of DIMMs within two
channels simultaneously. If a failure occurs, the memory controller switches from the
primary pair of DIMMs to the backup pair of DIMMs. You must enable memory mirroring
through the Setup utility. See Using the Setup utility in the iDataPlex dx360 M2
Problem Determination and Service Guide.
December 2014
19
XW5181
Table 3 shows the installation sequence for installing DIMMs in memory-mirroring mode.
Table 3: DIMM installation seqence
Microprocessor
number
Pair
DIMM connector sequence
3, 6
2, 5
1, 4
14, 11
13, 10
12, 9
Figure 11 graphically shows the installation sequence for installing DIMM pairs in
memory-mirroring mode.
Figure 11: Installation sequence for installing DIMM pairs in memory-mirroring mode
December 2014
20
XW5181
When you use the memory-mirroring mode, you must consider the following information:
To support memory mirroring, the DIMMs in each pair must match but the pairs can
be different from each other. For example, the first pair of DIMMs must match and
the second pair of DIMMs must match; however, the first and second pair of DIMMs
can be different from each other.
Channel two DIMM connectors are not used in memory-mirroring mode. The
maximum available memory is reduced by half when memory mirroring is enabled.
For example, if you install 64 GB of memory, only 32 GB of addressable memory is
available when you use memory mirroring.
When you install or remove DIMMs, the system-board tray configuration information
changes. When you restart the system-board tray, the system displays a message
that indicates that the memory configuration has changed.
Serial Attached SCSI (SAS) subsystem
Storage is enabled using a LSI1064 based mini-PCIe daughter adapter that plugs
into the mini-PCI connector. Supports RAID 0,1, 1E.
Storage is enabled using a LSI1068 based PCIe adapter that plugs into the front
slot for 2U system. Supports RAID 0,1, 1E.
Storage is enabled using a LSI1078 based PCIe adapter that plugs into the front
slot for 2U system. Supports RAID 0,1,5,6,10,50,60.
A hard disk drive expander card is required to support 12 Drives on 3U system.

The SAS Adapter is located in the rear PCIE slots.
December 2014
21
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Intel microprocessor subsystem overview
Intel microprocessor subsystem overview

The dx360 M2 ships with two Intel Nehalem processors, both of which are one of the
following processor types:
Nehelam EP 2.00/800-4M QC (80W)
The dx360 M3 ships with either two Westmere 4- or 6-core processors or two Nehalem
quad-core processors. Processors can be one of the following types:
Nehalem Intel Xeon Quad Core X5500 Series (95W)
Intel Xeon Quad Core E5500 Series (80W)F
Intel Xeon Quad Core L5500 Series (60W)
Westmere Intel Xeon Six Core X5600 Series (95W)
Intel Xeon Quad Core E5600 Series (80W)
December 2014
22
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Intel microprocessor subsystem overview
The socket used by the Nehalem microprocessor is a Land Grid Array (LGA) 1366
Socket B. It supersedes the LGA 775 Socket T. There are no holes on these sockets.
Instead there are pins which touch contacts on the underside of the microprocessor.
Figure 12 shows the LGA 1366 Socket B.
Figure 12: LGA 1366 socket B
December 2014
23
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Thermal profile
Thermal profile
There are nine thermal profile values for the system that are set based on the system
configuration (CPU power and highest capacity memory DIMM).
In the dx360 M2 and M3 system, the UEFI automatically detects the system
configuration and applies the proper thermal profile without any other input required.
Since the thermal profile is set automatically, this should enable for the possibility of
field upgrades to customer nodes.
Notes: Nodes in the same 2U or the same 3U chassis must be configured identically to
prevent airflow imbalance that could lead to insufficient cooling or wasted power for the
fans.
It is critical that if a system is serviced that the thermistors (and clips) not be damaged
and they must be reattached to the rear face of the heat sink base when servicing is
complete. Failure to reattach the thermistor securely can result in CPU throttling and
possibly even shutdowns based on CPU overheating.
Soft shutdown is implemented based on ambient temperature thresholds that are
measured at the front panel and also the CPU platform environment control interface
(PECI) thermal values.
Only the bottom node can initiate soft shutdown due to lack of communication path
between the power supply unit (PSU) and top node.
DIMMs can use the onboard temperature sensor to throttle DIMM bandwidth similar to
the way that CPUs are able to throttle. A warning is generated when CPU or DIMM
throttling is activated.
If DIMM overheating occurs, ensure that the customer did not discard any DIMM fillers
or air baffles. If so, have the customer replace the missing fillers or baffles immediately.
The removal of baffles and fillers potentially interrupts the optimal airflow pattern and
might cause unwanted overheating incidents.
Notes: If a memory throttling event takes place, a message is displayed in the system
event log which looks similar to the message used for the CPU throttling message.
December 2014
24
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Unified Extensible Firmware Interface (UEFI)
Unified Extensible Firmware Interface (UEFI)

The UEFI specification defines a new model for the interface between personalcomputer operating systems and platform firmware. The interface consists of data
tables that contain platform-related information, plus boot and runtime service calls that
are available to the operating system and its loader. Together, these provide a standard
environment for starting an operating system and running pre-boot applications.
Pure UEFI does not support legacy operating systems. Legacy compatibility is provided
by the Compatibility Support Module (CSM). The CSM provides all the required
resources to start a legacy OS such as legacy BIOS calls & interrupts, memory maps &
tables and legacy boot mechanisms. To ensure the highest level of compatibility, IBMs
CSM is based off of the IBM BIOS we have been shipping for years. The CSM is
integrated into the UEFI code and does not require any interaction by the user to invoke.
To access the UEFI setup utility, perform the following steps:
1. During the system Power-On Self Test (POST), press <F1> Setup when
prompted as shown in Figure 13.
Figure 13: System POST screen
December 2014
25
XW5181
2. The System Configuration and Boot Management screen is displayed as

shown in Figure14. From this screen you can access and configure system
settings, set the server date and time, select or add start options, change the
boot order, access the system event log (SEL), configure user security, restore
settings, load default settings and save the settings.
Figure 14: System configuration and Boot Management screen
December 2014
26
XW5181
3. Select the System Information option on the System Configuration and Boot
Management screen to view the System Information screen as shown in Figure
15. Select the System Summary option.
Figure 15: System Information screen
December 2014
27
XW5181
4. The System Summary screen is displayed as shown in Figure 16. This screen
summarizes the basic details of the dx360 M2 or M3 server. Here you can see
microprocessor(s) speeds and the amount of memory installed as well as other
information that can be used for problem determination purposes.
Figure 16: System summary
December 2014
28
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Integrated Management Module (IMM)
Integrated Management Module (IMM)

As with all new System x products released in 2009 and beyond, the dx360 M2 and M3
servers use IMM, which is a logical or physical architecture that consolidates the BMC
and RSA-II functionality, video controller, digital video compression, remote control, and
SUPER I/O controls.
If there is a system failure or issue, the user can obtain several different types of
information from the IMM that assist in determining specific service actions to resolve
various types of problems. The primary information available to service from the IMM is
listed in Table 4. The IMM does not provide a complete list of all service information
from the system, as there are other more complete methods to obtain service data.
However, when specific issues arise, the information captured by IMM can lead to an
appropriate service action.
Table 4: Common system symptoms detected by IMM
Symptom
IMM captures data from:
Fan issues
Light path, fan speed, event log
System temperature issues
Light path, ambient temperature, event log
Voltage and power issues
Voltage data, light path, event log
CPU, DASD, power supply, VRM,

memory, system board, riser card,
battery, SAS, RAID failures
Light path, event log
NMI (PCI bus and option adapters)

issues
Light path log
System unexpected restart
IMM configuration, light path, event log
Unable to start the system
CPU configuration issues
Memory mirroring issues
Memory configuration issues
IMM or UEFI firmware corruption issue,

unsuccessful update issues
Event log
Boot configuration issues
Event log
December 2014
29
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Integrated Management Module (IMM)
Table 4: Common system symptoms detected by IMM (continued)

Symptom
IMM captures data from:
Energy management issues
Event log
IMM communication issues
Event log
When the powering on the dx360 M2 or M3 for the first time or after power has been
removed it takes two to three minutes for the system to reach the operating system
desktop and display information on the screen. This is because the system has to
initialize the code for the IMM. The power LED flashes rapidly while the IMM code is first
being loaded. When the IMM code is almost loaded, the Power LED flashes more
slowly. When the power LED is steady, IMM is fully loaded and you should see
information displayed on the screen.
December 2014
30
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Preboot Dynamic System Analysis (pDSA)
and Online DSA
Preboot Dynamic System Analysis (pDSA) and Online

DSA
Dynamic System Analysis (DSA) is a strategic problem determination tool for data
collection, fault detection and remediation. DSA collects and transmits system
information to the IBM service and support organization regarding system configuration,
status, logs, and diagnostics.
DSA Preboot (pDSA) enables customers to view the collected data before sending it to
IBM. It produces a compressed CIM-XML file to send to Lenovo. Customers can also
use the collected data as input to their internal tools.
pDSA is accessed by pressing the <F2> (Diagnostics) key when prompted during the
Power-On Self Test (POST) of the system.
The diagnostic programs create a merged log that includes events from all collected
logs. The information is collected into a file that you can send to Lenovo service and
support. Additionally, you can view the server information locally through a generated
text report file. You can also copy the log to removable media and view the log from a
Web browser.
The dx360 M2 and M3 support both Online Portable/Installable DSA as well as preboot
DSA. Preboot DSA is embedded in the flash ROM, so no USB key or CD image is
required.
Table 5 shows the pDSA diagnostic capabilities.
Table 5: pDSA diagnostic capabilities
Component
Support
Processors
CPU stress test
Memory DIMM
DIMM isolation test and stress test
Intel Ethernet tests
EEPROM
FIFO
Adapter register
Adapter interrupt
Loopback
SAS or SATA drive
Hard disk drive self-check
LSI SAS MegaRAID controllers
MegaRAID self-check
December 2014
31
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Preboot Dynamic System Analysis (pDSA)
and Online DSA
Table 5: pDSA diagnostic capabilities (continued)

Component
Support
iBMC
I2C bus test
The DIMM isolation test cannot run inside of the Mini Control Program (MCP) so it runs
prior to loading of DSA in a standalone environment.
No hard disk drive test is available for MegaRAID attached SAS/SATA drives, LSI IR
attached drives and drives controlled through the ICH9 or ICH10 chipset are supported.
DSA does not support diagnostics for MegaRAID.
December 2014
32
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Component installation
Component installation
Installing and uninstalling the storage riser card assembly, the systemboard assembly and the ServeRAID Battery
Note: These instructions apply to the 3U chassis only. Two people are required to
remove a 3U chassis from a rack.
In order to access certain crucial components inside a 3U chassis, other major
components must be removed sequentially in this manner: 1) Storage Riser Card, 2)
Fan Assembly, 3) System-Board Tray Assembly, 4) ServeRAID battery assembly.
Figure 17 shows the 3U Chassis with the top cover assembly removed.
Figure 17: System
Before you begin any uninstall process, read the safety statements and Installation
guidelines in the Problem Determination and Service Guide for either the IBM System x
iDataPlex dx360 M2 or the System x iDataPlex dx360 M3.
You must shut down the operating system and turn off the system-board tray before you
remove the system-board tray from a chassis or before you install simple-swap hard
disk drives.
December 2014
33
XW5181
Remove the storage riser card assembly

To remove the storage riser card assembly, perform the following steps:
1. Remove the top cover assembly from the chassis.
2. Note the cable routing and connection locations; then, disconnect any cables
from the assembly.
3. When you move the storage-riser-card assembly, do not enable it to touch any
components or structures inside the 3U chassis.
4. Lift up on the storage-riser-card assembly and remove the assembly from the
bracket and the 3U chassis as shown in Figure 18.
Figure 18: Lift up on the storage-riser-card assembly
December 2014
34
XW5181
5. If the storage riser card is being replaced, slide the riser card out of the assembly
bracket as shown in Figure 19.
Figure 19: Slide the riser card out of the assembly
6. Disconnect both cables from the riser card, then remove the conversion card
(highlighted in Figure 19 above). Place the conversion card aside.
7. Connect the conversion card to the new riser card, then reconnect the cables
and replace the riser card in the metal assembly.
December 2014
35
XW5181
System board tray assembly

To remove the system board tray assembly, follow the steps below:
1. While facing the rear of the 3U Chassis, slide the fan assembly retention bracket
to the right to remove it as shown in Figure 20. Place the bracket aside.
Figure 20: Remove the fan assembly retention bracket
December 2014
36
XW5181
2. As you lift the fan assembly by the handle, be mindful of the retention bracket
clips (highlighted in Figure 21). Angle the fan assembly toward the front of the
chassis in order to get it past the clips.
Figure 21: Fan retention bracket clips
3. Detach the ServeRAID cable from the rear of the system board tray as shown in
Figure 22.
Figure 22: Detach the ServeRAID cable
December 2014
37
XW5181
4. Push the system-board tray assembly out of the front of the chassis by pressing
against the rear of the system board tray assembly as shown in Figure 23.
Figure 23: Push the system-board tray assembly out of the front of the chassis
5. Carefully set the system-board tray on a flat, static-protective surface, with the
cover side up.
December 2014
38
XW5181
6. If you need to access the system-board tray itself, pull the cover release on each
side of the system-board tray outward; then, lift the cover open as shown in
Figure 24.
Figure 24: Lift the cover open
Remove the battery holder and ServeRAID Li-Ion battery

To remove the battery holder and ServeRAID Li-Ion battery, follow the steps below:
1. Follow the steps listed above to remove the storage riser card, the fan assembly,
and the system board tray.
2. Enable that the battery cable can move freely inside the system-board tray.
December 2014
39
XW5181
3. Remove the filler panel from the hard disk drive bay as shown in Figure 25.
Figure 25: Remove the filler panel
4. Press the release button on the battery holder and pull the battery holder from
the front hard disk drive bay until the battery cable is out of the hard disk drive
bay as shown in Figure 26 and Figure 27.
Figure 26: Presse the release button on the battery holder
December 2014
40
XW5181
Figure 27: Pull the battery holder from the front of hard disk drive bay
5. Press the tray retention tab on the bottom of the battery holder to release the
ServeRAID Li-Ion battery and charge circuit assembly. Figure 28 shows a
ServeRAID Li-Ion battery and charge circuit assembly that has been removed
from the battery holder.
Figure 28: ServeRAID Li-Ion battery and charge circuit assembly
December 2014
41
XW5181
6. Disconnect the battery cable from the ServeRAID Li-Ion battery and charge
circuit assembly.
7. Press both retention tabs on the tray to release the ServeRAID Li-Ion battery and
charger circuit assembly as shown in Figure 29.
Figure 29: Release the ServeRAID Li-Ion battery and charger circuit assembly
8. Press the charge circuit retention tab to release the ServeRAID Li-Ion battery
from the charge circuit assembly; then, disconnect the cable from the charge
circuit.
For more information about removing and replacing server components, access the IBM
System x infocenter to download the system dx360 M2 and M3 problem determination
and service guide (PDSG) document.
December 2014
42
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers NVIDIA graphics and compute devices
NVIDIA graphics and compute devices

The following pictures represent NVIDIA devices that can be used within the dx360 M2
and M3.
Figure 30: NVIDIA Tesla M1060
December 2014
43
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers NVIDIA graphics and compute devices
Figure 32: NVIDIA Tesla M2070 and M2070Q
December 2014
44
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Proper installation of NVIDIA devices
Proper installation of NVIDIA devices

Overview
One of the first things to check when diagnosing a video related issue is to ensure the
video card or PCI Express interface card is installed and seated correctly and that all
cables are attached correctly. The following describes important information related to
the hardware installation of NVIDIA devices.
PCI Express Slot

All of the NVIDIA add-in PCI Express cards with on-board GPU require x16 mechanical
PCI Express slots (See Figure 33 below). The NVIDIA add-in interface cards that are
used to connect external GPU enclosures to the system come in both x16 and x8 PCI
Express cards although the x16 PCI Express interface card is preferred due to
performance. The NVIDIA add-in card should be completely and evenly inserted into
the PCI Express slot and all of the retention mechanisms of the system should be used
and verified to ensure the card is held in place firmly.
Figure 33: PCI express slot sizes
December 2014
45
XW5181
Auxiliary power
The NVIDIA add-in cards with on-board GPU can require as much as 150W 300W of
power in order to run at full performance. A single x16 PCI Express slot is capable of
providing 75W of power, but if the card requires more than 75W then an auxiliary power
cable is required to provide the additional power required to the card. This auxiliary
power and the connectors and cables involved have been defined by the PCISIG
consortium and specs for these connectors and cables can be found on their Website,
www.pcisig.org. The NVIDIA cards that require this additional power all have auxiliary
power connectors located on the rear or top of the card (See Figures 34 and 35 below
that show the 6pin connector capable of providing an additional 75W and the 8pin
connector capable of providing an additional 150W to the card).
An NVIDIA card that requires between 75W and 150W of total power have one of the 6
pin power connectors whereas a card that requires between 150W and 225W will either
have two 6pin connectors, one 8pin connector, or one 6pin and one 8pin connector. For
example, the NVIDIA Quadro FX 5800 card requires just under 225W and this card has
one 6pin connector and one 8pin connector and to provide the additional 150W of
power required over and above the 75W provided by the PCI Express slot, either two
6pin auxiliary power cables can be used (The 8pin connector accepts both 6pin and
8pin auxiliary power cables) or one 8pin auxiliary power cable can be used. If the
NVIDIA add-in card has either a 6pin or 8pin auxiliary power connector then an auxiliary
power cable must be properly attached to the card in order for it to function correctly.
Figure 34: 6pin auxiliary power connector (75W)
December 2014
46
XW5181
Figure 35: 8pin auxiliary power connector (150W)
December 2014
47
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Troubleshooting and problem determination
Troubleshooting and problem determination

Problem determination pathway for Intel products
Whenever you encounter a memory problem in the Intel Nehalem processors that have
their memory controllers embedded in CPUs, you are required to follow the specific
problem determination path listed below. Perform these steps in the following order until
the issue is resolved:
1. Assure that there is not a known UEFI code issue, check RETAIN tips or check
with next level of support.
1. Assure that all memory modules are installed in the correct slots.
2. Check that all memory is correctly seated.
3. Replace memory.
4. Replace the CPU.
5. Replace system board (if you decide that the memory socket is damaged)
iDataPlex warranty call flow

Figure 36 shows the iDataPlex servers warranty call flow.
Figure 36: iDataPlex servers warranty call flow
December 2014
48
XW5181
The nodes are individual servers and should be treated as such when problems arise.
You enter the maintenance package just as you would a normal server. When directed
you should pull logs and use all normal troubleshooting procedures as outlined in the
PDSG for the system type you are working with.
The iDataPlex dx360 M2 and M3 - Important common problems

The following problems are presented as examples of problem determination on the
dx360 M2 and M3 server. These problems are identified as being likely to occur but
they are not the only problems that the customer might encounter.
Problem #1: The Integrated Management Module (IMM) takes as much as five minutes
to become ready after applying ac power or resetting the IMM.
Action plan for problem #1: This behavior was corrected with the release of the IMM
1.07 going forward. Be sure to upgrade dx360 M2 and M3 servers to IMM 1.07 or higher
to mitigate this problem.
Problem #2: Drive letter assigned to the primary hard disk is not correct.
Because the Integrated Management Module (IMM) is capable of presenting virtual
removable media drives to the operating system, users can experience a known
Windows 2003 issue where the drive letter that is assigned to the primary hard disk is
not correct.
Action plan for problem #2: This Microsoft Windows issue is documented at the
following URL: http://support.microsoft.com/kb/896536
Problem #3: UEFI does not start after load default or deployment
Boot Order entries for UEFI installed Operating Systems (OS) might be required to
actually start that OS correctly. Load default, by design, deletes these entries.
Deployment images might also use this information to automatically start the first time.
Details: This behavior is by design. UEFI OS's are aware of the problem. When you
boot your UEFI OS using use the Boot From File method, the OS will recreate the
UEFI Boot Order Entry if needed. You might also copy the OS' boot manager to
\EFI\BOOT\BOOTX64.EFI in the GPT System Partition. This enables the EFI OS to
boot automatically in the future if the Boot Order entry is deleted again.
The boot manager for WS08 is: \EFI\BOOT\Microsoft\Bootmgrfw.efi
December 2014
49
XW5181
Copying the boot manager to \EFI\BOOT\BOOTX64.EFI prior to cloning the disk

displays the new disk to start in a system that does not have the UEFI Boot Order entry
created.
Workaround for problem #4:
The specific workaround is dependent upon the actual UEFI OS. Please refer to the
UEFI OS information for the actual file name and path used to start that particular OS.
For Windows Server 2008 (WS08), the steps are:
1. Start your system, and when prompted, press F1 to enter setup.
2. Select Boot Manager
3. Select Boot From File
4. Select the GPT System Partition where you installed WS08
5. Select EFI
6. Select BOOT
7. Select Microsoft
8. Select bootmgfw.efi
Problem #5: There is no video output.
Action plan for problem #5: Power the system off, then move the Clear CMOS jumper
from Pins 1 and 2 to Pins 2 and 3 to clear the CMOS data. Wait ten (10) seconds and
power the system on again.
Problem #6: The two PM Serial Attached SCSI (SAS) to 4x cables, Field Replaceable
Unit (FRU) 44T0680, on the iDataPlex dx360 M3 Server, type 6391, chassis might get
pinched while closing the top cover of the iDataPlex chassis I/O enclosure (FRU
69Y1025).
Action plan for problem #6: If the two PM SAS to 4x cables are already damaged,
then they should be replaced.
December 2014
50
XW5181
Six system states

A server is in one of six system states when you are called to provide service. The
system states are based around what happened when the system was last started.
Each system state has tools available to diagnose problems. These diagnostic tools are
carried forward from the previous state to the next state. So that the diagnostic tools in
system state one are available in system states two, three, four, five and six. System
state two has additional tools that are available in system states three, four, five and six.
The six system states are:
System state #1 There is no ac power.
System state #2 There is ac power but there is no dc output.
System state #3 There is both ac and dc power but the system fails to complete
POST.
System state #4 There is both ac and dc power, the system completes POST but the
OS fails to start loading.
System state #5 There is both ac and dc power, the system completes POST but the
OS fails to complete loading.
System state #6 There is both ac and dc power, the system completes POST and the
OS completes loading but stops during operation.
December 2014
51
XW5181
Problem determination Logs

When the customer has a problem with a dx360 M2 or M3 system the first indication
they receive is a fault LED light on the chassis. The customer then finds the system with
the fault LED lit. Inside the system, an error LED is lit next to the failing component,
such as a single DIMM or microprocessor.
The error LED is not the only indication of a problem. An error message is also
generated during the Power-On Self Test (POST) of the system. You need to find the
error message or messages generated by the failing component in the logs that are
available for problem determination.
There are several ways of gathering information for problem determination purposes.
Table 6 shows the methods of viewing event logs.
Table 6: Methods of viewing event logs
Condition
Action
The system board tray is responding

and is connected to a network.
Run portable or installable DSA to view the

diagnostic event log or create an output file that
you can send to Lenovo service and support.
The system board tray is not

responding and is not connected to a
network.
Use IPMI tool locally to view the system event

log.
The system board tray is responding

and has IMM running.
In a Web browser, type the IP address of IMM

and check the event log.
The system board tray is not

responding.
If the DSA Preboot is installed, restart the

system and press F2 to start DSA Preboot. If
the DSA Preboot is not installed, insert the DSA
Preboot CD or USB flash drive and restart the
system to start the DSA Preboot to collect DSA
logs.
Alternatively, you can restart the system and
press F1 to enter UEFI and view the POST logs
or system event logs
December 2014
52
XW5181
System event log

You can view the system event log in the <F1> setup utility to see the error messages
generated during POST.
To access the system event log, perform the following steps in Figure 37 video:
(SWF video in PDF format only)
Figure 37: Accessing the system event log
December 2014
53
XW5181
Online DSA
Online Dynamic System Analysis (DSA) and pDSA pull the system status messages
from the Integrated Management Module (IMM).
If the customer cannot turn off the system they can still look at the logs. They can use
Online Dynamic System Analysis (DSA) if they have it installed. The logs are displayed
by Online DSA in the same manner as they are in pDSA. Figure 38 shows the Online
DSA Integrated Management Module (IMM) log and the logs that comprise it.
Figure 38: Online DSA logs
December 2014
54
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Helpful links
Helpful links
Links to other documents can be included here, as applicable. Include the link and the
title in the following table. Remove this paragraph if there is no introduction required.
Table 3: Helpful links
Title
Link to materials
IBM Fix
Central
http://www-933.ibm.com/support/fixcentral/
IBM
ServerProven
http://www03.ibm.com/systems/info/x86servers/serverproven/compat/us/flexsyste
ms.html
IBM
ToolsCenter
http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=toolcenter
IBM Support
Portal
http://www.ibm.com/supportportal
December 2014
55
XW5181
Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Summary
Summary
This course enabled you to:
1. Provide an overview of the iDataPlex dx360 M2 and M3 servers.
2. Understand the main differences between the iDataPlex dx360, the dx360 M2
and the dx360 M3 servers.
3. Describe the system architecture of the iDataPlex dx360 M2 and M3 servers
including the memory and SAS subsystems.
4. Describe the mechanical and electrical systems found in the iDataPlex dx360 M2
and M3 servers.
5. Understand how the iDataPlex dx360 M2 and M3 servers use IMM, UEFI and
Preboot DSA.
6. Describe the thermal component of the iDataPlex dx360 M2 and M3 servers.
7. Understand the different NVIDIA GPU offerings and component installation.
8. Perform problem determination activities and troubleshoot the iDataPlex dx360
M2 and M3 servers.
December 2014
56
XW5181

iDataPlex dx360

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

iDataPlex dx360

Încărcat de

Drepturi de autor:

Formate disponibile

EBG Server Education

Servicing the Lenovo iDataPlex dx360

Copyright 2014, Lenovo Corporation.

Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Table of contents

Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Table of contents

Helpful links ................................................................................................................. 55

Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Objectives

Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Overview

Servicing the IBM iDataPlex dx360 M2

Compute intensive: maximum processing 2U, two compute nodes

Features and specifications

Intel Nehalem-EP DP Quad Core Processors with QPI (up to 95w)

DDR-3 registered DIMMs,800, 1066, or 1333MHz

16 DIMMs, 8 DIMMs per CPU

1GB, 2GB, 4GB

One PCI-E x4 (Mini-PCI connector)

4 x SS 2.5-inch GEN-2 solid state drive or hard disk drive SATA

8 x HS 2.5-inch GEN-1 SAS or 12 x HS 3.5-inch SAS/SATA (for 3U only)

Hardware RAID 0/1 through mini-PCI paddle adapter.

Optional HW RAID 0/1 through PCI-E adapter.

Optional HW RAID 0,1E, 5, 6, 10 through PCI-E adapter.

Power supply unit

Dual ac 750W power supply

Modify domain A Power paddle card to support dual ac power supply

Modify Domain B power paddle card to support dual ac Power supply

Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Production description

Figure 1: iDataplex dx360 M2 and M3 front view

Figure 2 shows the iDataPlex dx360 M2 and M3 inside view.

Figure 2: iDataplex dx360 M2 and M3 inside view

Servicing the Lenovo iDataPlex dx360 M2 and M3 servers Production description

Figure 3 shows the iDataPlex dx360 M2 and M3 rear view.

Figure 3: iDataplex dx360 M2 and M3 rear view

Figure 4 shows the iDataPlex dx360 M2 and M3 power supply unit.

Figure 4: iDataplex dx360 M2 and M3 power supply unit

System configurations and architecture

Figure 5: dx360 M2 and M3 system configurations for 2U and 3U

Figure 6 shows the dx360 M2 and M3 system block diagram

Figure 6: dx360 M2 and M3 system block diagram

Mechanical system-board layouts

Figure 7: LEDs and controls on the system board

System-board mechanical description

Figure 8: Connectors locations on the system board

Figure 9 shows the locations of the jumpers on the system board.

Figure 9: Jumpers location

Table 1 defines the function of each jumper on the system board.

Clear CMOS (JP1)

Pins 1 and 2: normal (default)

UEFI boot recovery (JP2)

Pins 1 and 2: Normal (default)

Figure 10: LED location

Light path LED descriptions:

Microprocessor mismatch: When this LED is lit, it indicates that microprocessor 1

Memory and SCSI Subsystems

DIMM connector sequence

Only single-rank and

Install the DIMMs in the

Install the DIMMs in the

Install the DIMMs in the

Install the DIMMs in the

Important memory subsystem conecpt

Memory Mirroring reduces system downtime by keeping a real-time copy of data in

Memory mirroring rules

DIMM connector sequence

Serial Attached SCSI (SAS) subsystem