Documente Academic
Documente Profesional
Documente Cultură
Virtualizing Oracle:
Oracle RAC on VMware vSphere 4
Executive Summary become increasingly well-established with newer
releases of VMware vSphere. Advances in the
While most databases running in a VMware envi-
capabilities and overall performance of VMware
ronment benefit from the increased reliability
have put to rest the arguments about running
of a virtual environment, complete failover still
high-performance applications as VMs.
requires database clustering. Yet, traditional
clustering methods that use Raw Disk Mappings
However, the current release of VMware vSphere
(RDMs) have generally achieved redundancy at
can only provide continuous availability through
the expense of the many benefits that result from
VMware Fault Tolerance for single vCPU systems,
running in virtual environments.
and then only in limited configurations. vSphere
Recent advances in the capabilities of VMware is not yet able to provide fault tolerance for
vSphere have opened the door to new cluster- multi-CPU systems, which are often needed to
ing methods. These methods meet the demands of high-performance databases
and other Tier 1 platforms. Thus, concerns remain
Traditional enable individual virtual machines around enabling high availability on virtual
(VMs) in a database cluster to
database clustering be migrated via VMware vMotion machines with more than one virtual CPU, along
is still required for from one ESX host to another, cre- with other properties that are not yet supported
by VMware fault tolerance. Organizations with
mission-critical, ating an opportunity to synergis- enterprise-class database platforms that require
tically combine the natural resil-
high-availability and iency of database clusters with mission-critical availability or carrier-grade
high-performance the high-availability and load- stability must find other ways to meet this need in
a virtual environment.
compute capacity. balancing properties of VMware
virtual environments.
As a result, traditional database clustering is
The net result is a high-performance database still required for mission-critical, high availabil-
system with greater reliability than what could ity and high-performance compute capacity. Yet,
otherwise be achieved through traditional clus- when using traditional methods, clustering virtual
tering methods on either a physical or virtual machines in VMware leads to another limitation.
infrastructure. We have delivered Oracle RAC on The individual nodes in a typical cluster — whether
VMware to a leading entertainment/communica- or not these nodes are running on physical, virtual
tions provider, which is an industry first. or even mixed architectures — require access to
shared data. These shared drives are used for
Database Virtualization: Getting Started storing information common to all systems, as well
The fundamentals of running database systems as for keeping all of the nodes in a given cluster
such as Oracle in a virtual environment have coordinated (voting and quorum drives).
The primary difference between Fibre Channel For those using an iSCSI gateway, the VM configu-
and IP-based storage solutions is solely that a ration might look something like this:
“gateway” VM is required in Fibre Channel SAN
environments. It provides the same benefits in
• Two vCPUs.
both storage configurations. • 4-GB RAM.
Since both configurations allow clustering without
• 10-GB primary disk (can be thin-provisioned).
the need for SCSI bus sharing, all of the VMs — • 100-GB secondary disk, thick-provisioned
including iSCSI or NFS “gateway” VMs — can be (to be shared via iSCSI).
moved between the various ESX hosts in a DRS • Two vNICs (VMXNET3 driver): one for adminis-
cluster via vMotion. This enables clusters to be tration and one for the iSCSI network.
freely configured such that the benefits of HA and
DRS can be synergistically added to the failover
• Current Linux distribution (CentOS, Ubuntu,
and Fedora have been successfully tested. Red
capabilities inherent in Oracle RAC clusters. Hat Enterprise Linux, SUSE Linux and Oracle
Enterprise Linux have been used in similar
Virtual Machine Architecture
solutions and should work, as well).
The virtual machine configuration for the
individual Oracle RAC nodes relies on in-guest Further, the example VMs as configured in
iSCSI or in-guest NFS protocol for all shared this document make use of a 10-GB Ethernet
drives. This means that each virtual machine converged network for both network and storage
connects directly to an iSCSI or NFS shared drive access. When configuring for Gigabit Ethernet
for all data that must be held in common. This networks, additional dedicated network ports and
connection uses the same protocols and security interfaces at the physical layer will be required.
mechanisms that would be used if these VMs were
instead servers in a purely physical environment. The above example configuration is intended to
support up to a medium-sized Oracle database
With an appropriate underlying infrastructure, for development, small-scale production, and
iSCSI and NFS deliver similar performance, secondary support for enterprise-class, large-
with unique benefits and drawbacks that are scale database solutions, such as Oracle Exadata.
well known among storage administrators and This configuration should be modified as
architects. The decision of which to choose is necessary to support alternate use cases.
driven by available skills layout of the underlying
infrastructure, company security policies and iSCSI Tuning
even personal tastes and preferences. As such, There are a variety of options for an appropriate
the examples used in this document are based on iSCSI gateway VM, most of which are some variant
iSCSI, but they can also be readily applied to NFS of Linux. These include Red Hat Enterprise Linux,
configurations. Ubuntu, SUSE, Fedora and FreeNAS, to name a
few. All have an iSCSI target capability built into
Configuring a Virtualized Oracle System
them. The two most common iSCSI target applica-
Properly sizing the VMs that make up a RAC tions found on Linux variants are:
cluster and the gateway VM (if implemented) is
critical to maximizing performance. An example • iSCSI Enterprise Target
VM configuration for Oracle RAC nodes might • TGT
have the following characteristics:
The iSCSI Enterprise Target is arguably the more
• Four vCPUs. mature of the two. TGT is included “in the box”
with Red Hat and its derivatives, and is more
• 12-GB RAM. than capable as a iSCSI platform. However, the
• 50-GB primary disk (can be thin-provisioned). default settings for both iSCSI systems are too
• Two vNICs (VMXNET3 driver): one public and conservative for the level of performance needed
one private. for Oracle RAC. Tuning is required to achieve a
• Current Linux distribution (CentOS, Ubuntu desirable level of performance. There are several
and Fedora have been successfully tested. Red online resources for tuning and configuring an
Hat Enterprise Linux, SUSE Linux and Oracle iSCSI target on Linux for Oracle.
High Availability and DRS Configuration vSphere DRS will ensure that all virtual Oracle
One of the primary drivers for deploying Oracle RAC nodes receive the resources they require
RAC is the high availability provided by a RAC by dynamically load-balancing the nodes across
cluster. This cluster failover carries forward into a the vSphere HA/DRS cluster. In the event any
VMware vSphere environment and — with cluster ESX host in the cluster fails (or RAC node when
nodes that can be migrated via vMotion — can now HA heartbeat monitoring is used), vSphere HA
be configured to take advantage of these capabili- will automatically restart all failed RAC nodes
ties. Remember that VMware HA will, in the event on another available ESX host. The process of
a physical ESX host fails, automatically restart all restarting these nodes will follow all HA and DRS
of the failed host’s VMs on surviving ESX hosts. rules in place to ensure that the failed nodes are
VMs do experience downtime when this happens. placed on a host where no other nodes in the
For this reason, allowing more than one virtual- same RAC cluster are running. With this combi-
ized RAC server node in a given RAC cluster to nation, Oracle RAC will automatically manage the
run on a single ESX host needlessly exposes the loss of a failed node from an application perspec-
RAC cluster to failure scenarios from which it tive, and vSphere will then automatically recover
potentially may not recover gracefully. the failed RAC node, restoring the Oracle RAC
cluster’s state to normal. All of this occurs with
As such, it is important to set a series of DRS no human intervention required.
anti-affinity policies between all nodes in a given
RAC cluster. A typical virtualized Oracle RAC envi- The end result is that by using in-guest, iSCSI
ronment will consist of three server nodes. Since (and/or NFS) storage for shared data, virtual-
anti-affinity DRS policies can currently only be set ized Oracle RAC database clusters can achieve
between two specific VMs, multiple policies are improved levels of redundancy and — on appropri-
required to keep three or more nodes in a RAC ate hardware infrastructure — enhanced levels of
cluster properly separated. Be sure to name the performance that cannot be achieved on physical
DRS policies such that they can be easily identified infrastructure alone.
About Cognizant
Cognizant (Nasdaq: CTSH) is a leading provider of information technology, consulting,and business process outsourc-
ing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered in Teaneck,
N.J., Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process
expertise and a global, collaborative workforce that embodies the future of work. With over 50 delivery centers world-
wide and approximately 104,000 employees as of December 31, 2010, Cognizant is a member of the NASDAQ-100, the
S&P 500, the Forbes Global 2000, and the Fortune 1000 and is ranked among the top performing and fastest growing
companies in the world.
© Copyright 2011, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any
means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is
subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.