Documente Academic
Documente Profesional
Documente Cultură
Infrastructure
Configuration and Deployment Best Practices
September 2018
H17317
Validation Guide
Abstract
This validation guide provides best-practice storage and configuration guidelines for a
Dell EMC VxRail hyper-converged appliance for SAP HANA deployment. The solution
incorporates 14th generation Dell EMC PowerEdge servers, VMware vSphere
infrastructure, and VMware vSAN storage.
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect
to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular
purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other
trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks
of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners.
Published in the USA 09/18 Design Guide H17317.
Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change
without notice.
Contents
Introduction ...................................................................................................................................4
References ...................................................................................................................................21
Introduction
Executive Dell EMC VxRail Appliance, the exclusive hyper-converged infrastructure appliance from
summary Dell EMC and VMware, is the easiest and fastest way to extend and simplify a VMware
environment. Powered by VMware vSAN and managed through the VMware vCenter
interface, the VxRail Appliance gives existing VMware customers an experience with
which they are already familiar. Seamless integration with existing VMware tools also
enables customers to use and extend their current IT tools and processes.
In 2018, SAP extended their HANA platform support with a certification and validation
process for a hyper-converged platform (a software stack comprising VMware vSphere
and vSAN) and infrastructure (a VxRail hardware stack). This process complements
existing hardware certifications for SAP HANA appliances and enterprise storage.
SAP has certified the VxRail appliance in the configuration that this validation guide
describes for running SAP HANA workloads.
Solution This Dell EMC hyper-converged solution for SAP HANA based on VxRail infrastructure
overview incorporates Dell EMC PowerEdge 14th generation servers with vSphere and vSAN
virtualization software. The solution encompasses design configurations and deployment
options. Customers use SAP sizing tools for SAP systems on VMware virtualized
infrastructure to determine the requirements of the deployment and work with Dell EMC
representatives to configure and deploy the solution.
Key benefits With the VxRail appliance, Dell EMC delivers a fully engineered, hyper-converged
infrastructure solution that enables businesses to innovate faster and accelerate their IT
operations. Customers implementing the solution can expect the following benefits:
Agility―A modern SAP landscape management experience that delivers
automated provisioning capabilities for SAP applications including SAP HANA and
provides a faster time-to-value.
Engineering―Compute, networking, and storage components that are integrated
with the specified prerequisites, and tested dependencies that deliver a seamless
solution experience.
Optimization―Design and deployment guides that highlight proven performance,
automation, and resiliency best practices for SAP landscapes including SAP HANA.
Document This guide describes how to plan, prepare, and configure a VxRail hyper-converged
purpose infrastructure for SAP HANA deployments. The guide provides validated best practices for
the configuration and design of virtualized SAP HANA deployments on VxRail based on
vSphere and vSAN technologies.
This guide does not replace the requirement for VxRail implementation services by Dell
EMC Professional Services. Detailed VxRail installation and implementation
documentation is available for Dell EMC service personnel only.
Audience This guide is for SAP Basis administrators, system administrators, storage administrators,
and presales architects who design mission-critical SAP HANA systems to be deployed
on VxRail hyper-converged appliances. You should have some knowledge of VxRail
appliances, PowerEdge servers, and VMware virtualization technologies including
vSphere and vSAN.
We value your Dell EMC and the authors of this guide welcome your feedback on the solution and the
feedback solution documentation. Contact the Dell EMC Solutions team with your comments.
Dell EMC VxRail SAP has certified SAP HANA on a VxRail P series All Flash (P570F) hyper-converged
model infrastructure. The dual-socket VxRail P570F is the only model to support SAP HANA
workloads. Hybrid models (the P570) and other models such as the VxRail E series,
V series, or S series are not supported in a SAP HANA environment.
The following figure shows the system specifications for the VxRail P series:
Table 1. VxRail P series system specifications
Sizing With SAP HANA Tailored Datacenter Integration (TDI) phase 5, SAP introduced
considerations customer-workload-driven SAP HANA system sizing, in which SAP application
performance standard (SAPS) requirements for specific customer workloads are used to
determine the type and number of processors required to run SAP HANA. Customers use
the SAP HANA Quick Sizer tool and sizing reports and share the results with Dell EMC to
determine the optimal number of VxRail nodes, CPU types, and memory sizes for their
SAP HANA environment.
It is important to understand the impact and limitations of vSphere virtual machine (VM)
and vSAN. The References section of this guide provides links to relevant information
from VMware and SAP.
CPU and While VxRail is generally available with single or dual CPU sockets, SAP HANA is
memory certified only on dual-socket VxRail P-Series appliances with Intel Xeon-SP platinum or
considerations gold CPUs and a minimum of 12 cores per socket.
When configuring memory for VxRail nodes used for SAP HANA, ensure that you comply
with the SAP requirement for a symmetric homogeneous DIMM assembly and that all
memory channels (six with Intel Xeon-SP CPU architecture) are used.
Mixed DIMM configurations are not supported on VxRail nodes. The following memory
configurations are available for VxRail P570F nodes:
384 GB (12 x 32 GB DIMMs1)
Note: 128 GB DIMMs are only supported with Intel Xeon-SP M-type CPUs.
Because of this strict SAP requirement for a symmetric homogeneous DIMM assembly,
memory sizes such as 1,024 GB or 2,048 GB are not possible with the Intel Xeon-SP
CPU architecture.
The following table shows the CPU type and DIMM assembly combinations that are
available in the two-socket (2S) VxRail nodes for SAP HANA:
6140M(18C, 2.3GHz) 2S 2S 2S 2S 2S 2S
6142M(16C, 2.6GHz) 2S 2S 2S 2S 2S 2S
8160M(24C, 2.1GHz) 2S 2S 2S 2S 2S 2S
8170M(26C, 2.1GHz) 2S 2S 2S 2S 2S 2S
8176M(28C, 2.1GHz) 2S 2S 2S 2S 2S 2S
8180M(28C, 2.5GHz) 2S 2S 2S 2S 2S 2S
Sharing CPU sockets between SAP HANA VMs (NUMA2 node sharing) is not supported
on dual-socket VxRail appliances and therefore only one SAP HANA production VM per
CPU socket using all available cores can be deployed. However, it is possible to deploy
SAP HANA VMs which use both CPU sockets and all cores.
With the NUMA architecture, each processor has its local memory and can also access
the non-local memory of the other processors over an advanced memory controller. Even
though this is a high-performance connection, access to local memory is always faster
than access to the non-local memory of the other processors. Take this into consideration
when designing and configuring SAP HANA VMs under vSphere on VxRail appliances.
2. When specifying the memory size, select Reserve all guest memory, as shown
in the following figure:
3. Repeat these steps for all SAP HANA production VMs. It is not necessary to
perform these steps for non-production VMs.
Example 2: SAP HANA VM requiring 2,048 GB memory and 40 vCPUs
In this example, the required memory is more than what is available on a single processor
(NUMA node). Follow these steps:
1. Use the Cores per Socket value to distribute the required vCPUs equally across
the two available sockets, as shown in the following figure:
2. Specify the memory size you require, as shown in the following figure:
Important: If a production VM partially uses resources from both NUMA nodes, current SAP and
VMware rules do not allow the use of unused CPU and memory resources for SAP HANA or non-
SAP HANA VMs. In our second example, the SAP HANA VM uses only 2,048 GB of the 3,072 GB
available memory capacity. The remaining 1,024 GB cannot be used for other workloads.
Therefore, when an SAP HANA VM spans NUMA resources (memory and CPU), it is a best
practice to allocate all available resources to the VM and not waste available resources.
For more information about the SAPS-based sizing of the Intel Xeon-SP CPU models that
are available for VxRail P-Series configurations, see Appendix: SAPS values for
PowerEdge R740xd (VxRail P570F).
Number of A VxRail cluster can have a minimum of three and as many as 64 nodes. When running
VxRail nodes SAP HANA in production on a VxRail cluster, Dell EMC strongly recommends that you
use a minimum of four nodes to avoid an availability and performance impact when nodes
become unavailable, for example, during software upgrades (maintenance mode).
When sizing a VxRail cluster for SAP HANA, consider having enough free resources
(standby nodes) for smooth upgrades or situations where VMware HA needs to migrate
VMs to available nodes if a VxRail node fails.
The following table lists the sizing recommendations for a VxRail cluster for SAP HANA:
Up to 7 1
Up to 14 2
Up to 21 3
Up to 28 4
Up to 35 5
Up to 42 6
Up to 49 7
Up to 56 8
SAP HANA VMs The SAP certification for SAP HANA on VxRail P-Series hyper-converged infrastructure
per node and per limits the deployment to two production SAP HANA VMs per physical VxRail node. This
total limitation does not apply to non-production SAP HANA VMs.
While it is possible to deploy two production SAP HANA VMs on a physical host (one per
CPU socket), Dell EMC recommends that you consider having enough free resources to
allow vSphere HA to perform an automatic restart of the SAP HANA VMs on another host
if one host fails due to a hardware problem. See Table 3 for the VxRail standby node
sizing recommendations.
Non-SAP HANA If up to two SAP HANA VMs that do not span NUMA nodes are deployed on a physical
VMs host and free resources such as CPU and memory are available, deploy the non-SAP
HANA VMs on the same physical host.
Disks The VxRail P-Series P570F nodes support up to 24 disks per node. Only all-flash
configurations are supported for SAP HANA, with a minimum of two vSAN disk groups per
node and a minimum of three capacity disks per disk group.
Based on capacity requirements, you can configure up to four disk groups with a
maximum of five capacity disks per disk group. SAS or NVMe disks are required for the
vSAN cache tier. All SSD types are supported for the capacity tier.
The following table provides details about the disks that are available in a VxRail node:
The RAID-1 failure tolerance method must be assigned in the storage policy to the SAP
HANA persistence (data and log of each SAP HANA VM), as explained in VM storage
policies for SAP HANA data and log. That means that twice the capacity of the data and
log devices is required as SSD capacity.
Although RAID-1 is the default for all vSAN devices, you can create a new capacity-
optimized storage policy with RAID-5/6 and assign it to other non-persistent devices.
RAID-5/6 consumes 1.33 times the usable device capacity on disk.
Network To support the SAP HANA bandwidth requirements, Dell EMC recommends configuring
the VxRail nodes with a two 25 GbE SFP28 network daughter cards (NDCs) and an
additional two 25 GbE SFP28 connectivity cards for SAP HANA-specific application
network traffic (the SAP HANA internode). Although 10 GbE network configurations are
also supported, they provide limited network and storage performance, which might result
in longer SAP HANA database restart times.
In addition to the system networks required for vSphere and vSAN software, configure the
following networks for SAP HANA when the corresponding function (SAP HANA system
replication) is used:
App Server
Clients
System replication
Internode
Backup
We recommend creating a new vSphere distributed switch (VDS) for the networks
required for SAP HANA and assigning the additional 2 x 25 GbE (or 2 x 10 GbE) network
cards as uplinks to this new distributed switch. To meet the performance requirements of
the SAP HANA internode network communication, set an MTU size of 9,000 on this
distributed switch and in the operating system network settings of the SAP HANA VM for
the internode network. Follow VMware and SAP HANA requirements and best practices
when configuring networks on the VxRail distributed switch.
For detailed information and considerations for planning logical and physical VxRail
networks, see the Dell EMC VxRail Network Guide.
SAP HANA Only scale-up (single-node) SAP HANA deployments are allowed on VxRail infrastructure.
deployment In accordance with SAP requirements, a scale-out deployment requires four-socket
options systems at a minimum.The VxRail on 14G Dell servers is currently available only on two-
socket systems.
CPU and Use SAP and VMware sizing guidelines to size CPU and memory for the SAP HANA
memory VMs. Over-commitment of CPU and memory resources is not allowed.
SAP HANA Special configuration requirements apply when you configure the SAP HANA persistence
persistence (data and log) of an SAP HANA VM.
SCSI Controller
A VMware best practice is to use dedicated SCSI controllers for the SAP HANA data and
log devices. Configure SCSI controllers as VMware Paravirtual. The example in the
following figure uses SCSI Controller 0 for the system device, SCSI Controller 1 for the
data device, and SCS Controller 2 for the log device.
Dell EMC engineers performed the VxRail HCI certification tests with four capacity
disks per disk group. They found that setting 12 disk stripes per object provided the
best overall results.
You can assign the VM storage policy to the storage device using the Edit Settings
screen, as shown in the following figure:
SAP HANA persistent devices can be thin provisioned because the storage policy has the
Object space reservation (%) = 100 setting, which means that the device is thick-
provisioned by default.
SAP HANA SAP HANA installations require a file system to store the SAP HANA binaries, trace, and
shared file configuration files. This file system is mounted under the /hana/shared mount point.
system
In SAP HANA scale-up (single-node) deployments, this mount point can reside on the
local system device. SAP HANA requires approximately 1x the RAM memory capacity for
the /hana/shared file system.
Note that SAP HANA scale-out deployments are only supported on four-socket servers
and therefore are not supported on two-socket VxRail hyper-converged infrastructures.
The minimum vSphere HA settings must be in place, as shown in the following figure:
The VMware HA VM monitoring feature,“Guest not heartbeating,” which restarts the guest
OS of the VM as well as SAP HANA on the same host, handles operating system crashes
provided the SAP HANA autostart options are enabled.
Enable the heartbeat monitoring feature when vSphere HA is enabled. Dell EMC
recommends setting Heartbeat monitoring sensitivity to High, as shown in the following
figure:
VMware Tools must be installed and running in the VM. You can install VMware Tools as
part of the operating system (open-vm-tools) or by using the vSphere web client and
selecting Guest OS > Install VMware Tools in the VM context menu.
Optimizing file You can configure the SAP HANA file I/O layer with certain parameters to optimize file
I/Os after the I/Os for a given storage solution and file system. The Linux XFS file system is used on all
SAP HANA storage volumes for the SAP HANA persistence.
installation
Setting file I/O layer parameters in SAP HANA 1.0
For SAP HANA 1.0 installations, use the SAP HANA hdbparam command as <sid>adm
in the Linux shell:
$ su - <sid>adm
$ hdbparam –p # lists current parameter setting
$ hdbparam –-paramset fileio.max_parallel_io_requests[Data]=128
$ hdbparam –-paramset fileio.max_parallel_io_requests[Log]=256
For more information, see SAP Note 2399079: Elimination of hdbparam in HANA 2
(access requires an SAP username and password).
In addition, VMware developed and supports a script that must be deployed on every
VxRail node. The script checks if a VM has a device on the vSAN datastore. If the device
is found, the following parameters are added to the VM’s configuration file:
"guestinfo.vsan.enabled": True/False,
"guestinfo.SDS.solution": ESXi_version, which is also the
vSAN release version
"guestinfo.vm_on_vsan": True/False
This information is used by the SAP CIM provider and support tools such as
sapsysinfo.sh.
To find out how to obtain and deploy the script, visit the SAP HANA on VMware vSphere
website.
Health check with VxRail Manager
Before reporting a problem to SAP, check the physical health of the VxRail hardware. The
health function in the VxRail Manager application provides details of any problems, as
shown in the following figure.
Ensure that all hardware-related issues are fixed before reporting an SAP HANA problem
to SAP.
Dell EMC Dell EMC Online Support provides a single point of support for all VxRail hardware issues
support calls as well as vSphere Hypervisor and vSAN issues.
To collect VxRail log information and create a support bundle for Dell EMC support, log in
to VxRail Manager and select Config > General. Generate New Log Bundle is located
under Log Collection, as shown in the following figure.
References
Dell EMC The following documentation on Dell EMC.com and Dell EMC Online Support provides
documentation additional, relevant information. Access to documents on Online Support depends on your
login credentials. If you do not have access to a document, contact your Dell EMC
representative.
Dell EMC VxRail Appliance Version 4.5 Administration Guide
Dell EMC VxRail Appliance Operations Guide
Dell EMC VxRail Appliance Techbook
Dell EMC VxRail Appliances on 14th Generation PowerEdge Servers
Dell EMC VxRail Network Guide
Dell EMC Switch Configuration Guide for VxRail
Note: Deploying VxRail requires Dell EMC implementation services. Detailed installation
documentation is only available to Dell EMC service personnel.
SAP SAP Note 618104: “sapsysinfo” – Compiling system information on Linux provides
documentation additional, relevant information.
For proper sizing of a SAP HANA environment, look for the latest version of the SAP
HANA Quick Sizer tool on the Quick Sizer website.
The SAP HANA on VMware vSphere website provides information about running SAP
HANA on VMware vSphere and points to several related documents.
Visit the SAP Help Portal for all other SAP HANA documentation.
Overview Dell EMC performed internal testing with the SD benchmarks to validate the extrapolated
SAPS values across the platinum, gold, silver, and bronze CPU range. The results
showed that the extrapolations were accurate to within a 5 to 10 percent range in our
laboratory environment. ”M” editions of a CPU model have the same extrapolated SAPS
values as non-M editions of that model.
The following table shows the SAPS values for the Dell EMC PowerEdge R740/R740xd
servers on which the VxRail P series (P570F) models are based and which are supported
for SAP HANA installations.
Table 5. PowerEdge R740 systems values with SAPS and SAPS per core
Number of
Number of Number of SAPS
PowerEdge server cores per SAPS
cores sockets per core
socket
PowerEdge R740 (Intel Xeon Gold 5115, 2.40 GHz) 20 2 10 62850 3143
PowerEdge R740 (Intel Xeon Gold 5118, 2.30 GHz) 24 2 12 73713 3071
PowerEdge R740 (Intel Xeon Gold 5120, 2.20 GHz) 28 2 14 82765 2956
PowerEdge R740 (Intel Xeon Gold 6126, 2.60 GHz) 24 2 12 82765 3449
PowerEdge R740 (Intel Xeon Gold 6130, 2.10 GHz) 32 2 16 97637 3051
PowerEdge R740 (Intel Xeon Gold 6132, 2.60 GHz) 28 2 14 97637 3487
PowerEdge R740 (Intel Xeon Gold 6136, 3.00 GHz) 24 2 12 93758 3907
PowerEdge R740 (Intel Xeon Gold 6138, 2.00 GHz) 40 2 20 113803 2845
PowerEdge R740 (Intel Xeon Gold 6140, 2.30 GHz) 36 2 18 113803 3161
PowerEdge R740 (Intel Xeon Gold 6142, 2.60 GHz) 32 2 16 109276 3415
PowerEdge R740 (Intel Xeon Gold 6148, 2.40 GHz) 40 2 20 126088 3152
PowerEdge R740 (Intel Xeon Gold 6150, 2.70 GHz) 36 2 18 124148 3449
PowerEdge R740 (Intel Xeon Gold 6152, 2.10 GHz) 44 2 22 127381 2895
PowerEdge R740 (Intel Xeon Gold 6154, 3.00 GHz) 36 2 18 132554 3682
Number of
Number of Number of SAPS
PowerEdge server cores per SAPS
cores sockets per core
socket
PowerEdge R740 (Intel Xeon Platinum 8180, 2.50 GHz) 56 2 28 175230 3129
PowerEdge R740 (Intel Xeon Platinum 8180M, 2.50 GHz) 56 2 28 175230 3129