whitEPAPEr

Data Protection for Virtual Server Environments

E x P lO r i n g O P t i O n s A n d t E c h n O lO g i E s f O r b Ac k u P A n d r E c Ov E ry O f v i rt uA l M Ac h i n E s
by k E n t c h r i s t E n s E n , v i rt u A l i z At i O n P r Ac t i c E M A n Ag E r , dAtA l i n k

AbstrAct
A dizzying range of technology choices awaits IT organizations trying to master the art of data protection for virtual server environments. Those choices range from whether to use familiar technologies, something new, or a combination thereof in order to effectively meet the needs of the organization. Based on Datalinks insights gained from our experience protecting enterprise data centers, this white paper provides practical advice and considerations surrounding backup and recovery of virtual server infrastructures. We offer a glimpse at new data protection techniques and approaches that can help unlock the ultimate potential of an organizations optimized virtual environment. We also provide details on how offloading much of the work from the virtualized application infrastructure can help eliminate many virtual server backup and recovery challenges. Given the current prevalence of VMware in many installations, this paper focuses primarily on backup and recovery considerations in VMware environments.

October 2009 | Page 1

Table of Contents

Abstract Introduction Early successes are not always easy to repeat Making good decisions for virtual server data protection Common data protection challenges for virtual environments Typical setting now includes hundreds of VMs Confusing options Evaluating the options Option 1: Backup agents and the use of third-party backup software Option 2: Server-based snapshots with VMware Consolidated Backup (VCB) High-level guidance on VCB operations Option 3: Storage array-based snapshots with array-based remote replication Other data protection considerations VMwares Site Recovery Manager (SRM) for remote replication NAS (NFS) or SAN (Fibre Channel) for VMware backup/performance Use of deduplication to reduce backup data footprints Datalinks Enhanced Data Recovery (EDR) framework Applicable in a VM setting Conclusion Datalink can help

1 3 3 3 4 4 5 6 6 7 8 8 9 9 10 10 10 10 12 12

October 2009 | Page 2

Introduction

intrOductiOn
Early successes are not always easy to repeat Todays data centers are in the midst of a massive revolution originating from the move to virtual server architectures powered by solutions such as VMware. Such growth is easy to understand. The benefits of server consolidation via virtualization can be significant: adoption and growth can prove somewhat elusive. To ensure the return on investment from virtualization is retained across the data center, its important to look at the underlying network and shared storage infrastructure supporting the move to virtual servers. The way IT organizations choose to use other parts of their virtual server ecosystem for production-level storage, archival, data protection and replication can significantly impact overall savings. Such decisions can also impact how easy or hard it is to manage and protect the virtual infrastructure. The area of data protection offers a prime example of some of the complexities that can arise as a result of trying to scale out the virtual infrastructure. Applying what have long been considered accepted data protection practices during this accelerated growth phase can present significant challenges for growing virtual server environments. Such challenges include concentrated bottlenecks, performance slowdowns, and excessive data protection overhead.

Dramatic savings in power/cooling costs and data center

floor space

Cost savings from reuse and extension of existing assets Simplified application management, rapid resource
provisioning and the ability to achieve new levels of application availability and portability Given the widespread early success of virtual servers in organizations, many have since chosen to enter the next wave of virtualization: Scaling the solution end-to-end througout the IT infrastructure and including enterprise applications. However, as virtual server environments grow in size, the design and day-to-day operation of such environments can become more complex. Repeating success during the next phase of accelerated virtual server

Making good decisions for virtual server data protection A dizzying range of technology choices awaits IT organizations trying to master the art of data protection for virtual server environments:

The area of data protection offers a prime example of some of the complexities that can arise as a result of trying to scale out the virtual infrastructure.

Should you use whats familiar or adopt something

different?

Datalinks depth of experience protecting enterprise data centers allows us to offer real-world insights for these types of questions. Based on many such insights, this white paper provides practical advice and considerations surrounding backup and recovery of virtual server infrastructures. We offer a glimpse at new data protection
continued on next page

When does it make sense to combine multiple

technologies?

What are some of the pros and cons of each approach?

October 2009 | Page 3

Common Data Protection Challenges for Virtual Environments

Making good decisions for virtual server data protection (continued) techniques and approaches that can help unlock the ultimate potential of your organizations optimized virtual environment and ultimately enable you to scale to an end-to-end solution with higher demand enterprise applications. We also provide details on how offloading much of the work from the virtualized application infrastructure can help you avoid many virtual server backup and recovery challenges. Given the current prevalence of VMware in many installations, this paper focuses primarily on backup and recovery considerations in VMware environments.

cOMMOn dAtA PrOtEctiOn chAllEngEs fOr virtuAl EnvirOnMEnts

Typical setting now includes hundreds of VMs At the start of many early VMware deployments and pilot projects, IT organizations only needed to concern themselves with protecting a handful of virtual machines (VMs) on one or two ESX Servers. This proved a fairly simple task with test/development and lower impact applications often involving the same software-based backup solutions used when the applications existed on separate physical servers. However, as the number of virtual machines in production has grown to support an ESX Server farm and up to several hundred virtual machines, data protection challenges have emerged. These include:

Backup management issues surrounding lengthy backup

processes and increasingly complex job scheduling

Storage bottlenecks when many guests access the same

storage devices at the same time

Lengthy restores now involving more layers of

abstraction and complexity and the need to manage different technologies working together Avoiding such backup bottlenecks and overuse of ESX Server resources is one key to efficient data protection. Just as essential to success is applying the right skills and knowledge to develop a comprehensive approach to data protection. Such an approach should successfully address the virtual environments many-layered recovery needs from logical and physical recovery to local recovery and remote recovery. All of these data protection needs must now be considered and incorporated in todays nowmaturing virtual world.

Data I/O bottlenecks as multiple application backup

jobs and processes become consolidated onto fewer physical servers

Backups consuming too much of an ESX Servers

resources (CPU and memory)

Backups consuming too much network bandwidth Potential data I/O bottlenecks on the network as backup
data moves to and from each VM Guest OS, each ESX Server, the VMware Consolidated Backup (VCB) proxy server, a third-party backup server and its underlying shared storage target
As the number of virtual machines in production increased to support an ESX Server farm and up to several hundred virtual machines, data protection challenges emerged.

October 2009 | Page 4

Common Data Protection Challenges for Virtual Environments

Confusing options As VMware popularity has grown, a number of data protection options have become available. As a result, the varied technologies and objectives surrounding virtual server data protection and remote replication can resemble a confusing maze for those still getting comfortable with the day-to-day operation of virtual machines. Even once an organization decides on a specific technology path, the various offerings available from one or more vendors can be complicated. Figure 1 demonstrates some of this confusion. Beyond this, just as there is a proliferation of technology choices for virtual server protection and recovery, there also exist as many goals and objectives trying to be achieved within each VMware-enabled data center or line of business. Deciding which data protection objectives are most important, in order of priority, is not always easy. Organizations must first sift through multiple options like those identified in Figure 2.

figure 1: there are several data protection options from which to choose in a virtualized environment

VIRTUAL SERVER TOOLBOX pRESEnTS MULTITUDE OF OpTIOnS

backup Agent in guest Os? backup Agent in vMware service console? storage Array-based snapshots? server-based snapshots? via vMware consolidated backup (vcb) site recovery Manager? storage Array-based replication? vcb Pr p software? backup with deduplication?
OTHER TECHnOLOGY CHOICES IMpACT BACKUp/RECOVERY EFFECTIVEnESS
pOTEnTIAL BACKUp TARGETS

disk virtual tape library (vtl) tape

nETWORK STORAGE pROTOCOL sAn (iscsi / fc) vMfs rdM (raw disk Mapping) nAs (nfs) ndMP

figure 2: A first step in developing a data protection strategy is to clearly define your goals

WHAT ARE YOUR GOALS

full image-level backups (vMdk)? Application Aware? local restore? remote restore? granular file restore? crash-consistent? bare Metal restore?
OTHER COnSIDERATIOnS
How often? frequency of backup/rPO, expected frequency of restores How fast? backup window, rtO How easy? Management oversight, time needed for backup/restore

October 2009 | Page 5

Evaluating the Options

EvAluAting thE OPtiOns

A myriad of options exist for backing up and recovering data in a virtualized environment, three of which are summarized below. OpTIOn 1: Backup agents and the use of third-party backup software In many ways, traditional backup performed via backup software agents has been a popular choice in emerging virtual server environments. VMware environments opting for this approach to data protection have tended to deploy this option in one of three different ways: 1. Backup agent installed individually in each VMs Guest OS 2. Backup agent installed in the ESX Servers Service Console 3. Backup software integrated with VCB Table 1 below discusses the pros and cons of the first two of these options. The third option is discussed in the next section describing use of VCB.
table 1: the merits of using backup agents to protect virtual server data

Approach: backup Agent in the guest Os (best used for file-level restores) PrOs
One of the most popular methods Often perceived as an easier data protection solution for smaller VMware installations as it requires little change to pre-VMware backup processes Can result in more targeted, application-specific backups and restores Can be implemented with some form of deduplication in the guest OS to minimize strain on bandwidth and server CPU
(Potential caveats for deduplication: restore process may be significantly more complex with client-side deduplication. Other intermediate hardware may also be required.)

cOns
May unduly increase the backup window as the number of VMs and backup agents proliferate Added stress on physical server resourcesCPU cycles, memory, and bandwidthcan lead to potential performance and latency issues as the number of backup agents and VMs grow per ESX Server (ESX Servers tend to require more server resources to support the added number of VMs and operating systems installed) Use of multiple backup agents and jobs can tax the IP network used to transfer backup data Can be costly to license multiple backup agents as VMs multiply Backup scheduling issues and management complexity can grow as administrators struggle to keep track of mounting backup jobs that must share resources on the ESX Server and network May not scale easily or support centralized, standard data protection processes to accommodate a wider range of applications especially for applications that operate both inside and outside of VMware

Approach: backup Agent in the service console (vMdk-level restore only) PrOs
Takes advantage of VM file encapsulation to back up VMDK files directly from the service console Does not require installation of multiple backup agents per each VM

cOns
Is not VM-aware or application-aware Difficult to manage and only provides VMDK-level resources While not as significant as each VM having a backup client, there is still the added processing overhead on ESX Server resources

October 2009 | Page 6

Evaluating the Options

OpTIOn 2: Server-based snapshots with VMware Consolidated Backup (VCB) Recognizing some of the ESX Server resource contention issues that began to appear with backup agents used inside each Guest OS, VMware released VCB as an offhost backup alternative. Installed on a Microsoft Windowsbased physical or virtual machine, VCB is responsible for properly quiescing and restarting VM operations and the applications in the guest OS using VMware Tools in order to take server-based, crash-consistent snapshots of each VMs contents. VCB provides the following methods of accessing virtual machine disk data: backups that do not require exposing SAN LUNS to the proxy server. The VCB proxy reads data through the IO stack of the ESX host.

LAN Mode (NBD Mode) Best used when environment

does not permit the use of the SAN or hotadd modes. Leverages over-the-network protocol to access the virtual disk and send the data to the VCB proxy. For best results with backup and restores of NFS, organizations should consider using NDMP. Third party backup software integrated with VCB is responsible for scheduling and initiating VCB snapshots. Once the snapshots reside on the VCB proxy server, the backup software performs its routine backup to disk, a virtual tape library or physical tape. Table 2 describes benefits and disadvantages of this approach.

SAN Mode Used with Fibre Channel and iSCSI storage

to completely offload backups to a physical VCB proxy.

HotAdd Mode Utilizes VM as a VCB proxy with any

type of storage visible to the ESX Server to perform
table 2: the merits of using vcb

Approach: using vcb PrOs

Developed and supported by VMware with its own built-in API Supports both file-level backup/restore for Windows VMs or full VM image backup Offers an off-host backup method with direct access to shared storage via the VMware file system (VMFS) Can significantly reduce backup processing overhead for ESX Server resources Integrates with many popular third-party backup applications No need to mount volumes in order to perform single file restores for Windows files Uses VSS as part of VMware Tools for quiescing on ESX Server 3.5 update 2 hosts, which allows application-level quiescing on Windows Server 2003 and file system-level quiescing on Windows Vista and Windows 2008 virtual machines

cOns
Backup software integration with VCB can vary greatly and may require specific licensing structure, added scripting and extra knowledge of the VCB command line interface Single VCB proxy is limited in scalability by the host bus architecture and simultaneous backup sessions, so for environments with hundreds of VMs, it is costly to scale across multiple physical VCB proxy servers VCB proxy only runs on Windows, so file-level restores are only supported on the Windows Guest OS and other VM environments (such as Linux and Solaris) only have the option to leverage full VM backups using VCB Only supports RDMs in virtual compatibility mode, not physical compatibility mode Image-level backup is a two-step process (from VCB staging area to backup destination), so full VM backup increases backup processing overhead and disk resources on the VCB proxy and also complicates restores Cannot scale to other data center environments (such as critical Oracle implementations) that remain outside of VMware. Does not integrate with VMware Site Recovery Manager (SRM) for remote restores

October 2009 | Page 7

Evaluating the Options

High-level guidance on VCB operations For organizations that use VCB, Datalink recommends the following:

Plan extra space for the staging area needed for full
VMDK backups

Determine backup levels and retention (SLAs, RTOs,

RPOs)

Minimize the number of simultaneous backup sessions to

no more than eight

Carefully consider the total amount of data to be

backed up per backup windows

Dedicate different HBAs for reading data (in VMFS),

writing data (to the holding area), and subsequent offload to tape or disk

Size the backup environment for peak workload Determine whether or not you are performing
VMDK-level or Guest OS file-level restores

Leverage third party backup integration modules to

simplify backup management with VCB

OpTIOn 3: Storage array-based snapshots with array-based remote replication Just as with growing environments that operate outside of VMware, organizations have begun to leverage the replication and snapshot capabilities available within their underlying intelligent shared storage systems for VMWare-related backup, restore and remote replication. The array-based snapshot option allows system administrators to perform low latency block-level snapshots within the storage array, resulting in zero impact on the ESX Server resources and production network. Because array-based point-in-time (PIT) snapshots from some storage vendors provide near instantaneous backup and restores, the freeze time for virtual disks until the VMware snapshot can be deleted is greatly reduced compared to other backup approaches. A few different snapshot approaches are possible: 1. Snapshot copies with VMFS and RDM (FCP/iSCSI): Crash-consistent VMDK datastore-level snapshots provide immediate PIT state. VMs are grouped based on backup policy making recovery of the entire datastore very easy. Since this is a LUN, individual VMDK file recovery requires the LUN to be mounted. 2. Snapshot copies with VMDKs on NFS: Snapshot copies are still taken on the entire volume, but because the virtual disks are not abstracted within a LUN, individual files can be easily extracted. 3. Hot or warm application-aware snapshots: Snapshot process is integrated with VMware for guest-to-VMDK consistency. When distilled down to its simplest mechanism, this approach uses array-based integration modules or custom scripting to communicate with the VMware infrastructure API to enumerate all VMs on a datastore and execute pre-freeze and post-thaw operations to provide application-aware consistency. This method requires more effort but poses less risk. Table 3 describes pros and cons surrounding use of the methods above for storage array-based snapshot and replication for VMware.
Organizations have begun to leverage the replication and snapshot capabilities available within their underlying intelligent shared storage systems for VMware-related backup, restore and remote replication.

October 2009 | Page 8

Other Data Protection Considerations

table 3: the merits of using storage array-based snapshots with replication

Approach: using storage Array-based snapshots with replication PrOs

Depending on the storage system in use, can leverage resources from the physical storage array to provide rapid, often highly efficient low-latency snapshots, significantly minimizing the impact of the backup process on growing VMware installations Allows backups and restores to be applied data center-wide, instead of only in the VMware environment If performed with the NFS file system, file-level restores are a faster, more straightforward process for all OS environments (Microsoft Windows, Linux, Unix, etc.), not just Windows environments Backup proxy no longer needs to be a Windows server, thus enabling a heterogeneous backup and recovery environment Can closely integrate snapshots and remote replication functionality (with some storage systems) to offer efficient local and remote replication, often from within a single toolset

cOns
More complexity may be introduced when working with VMFS and Fibre Channel block storage layers Application-level recovery may still require more than one approach, depending on storage vendor features with regards to virtual infrastructure-aware and application-aware integration modules Levels of integration can vary by storage vendor and may still require custom scripting and integration Backup retention policy for VMs combined with datastore layout needs to be carefully planned and considered before implementation

OthEr dAtA PrOtEctiOn cOnsidErAtiOns

Other technology decisions can also impact the effectiveness and efficiency of data protection and recovery for VMware environments. VMwares Site Recovery Manager (SRM) for remote replication

Site Recovery Manager (SRM) is meant to automate, ease and more effectively monitor and test disaster recovery, with little to no impact on production operations. SRM is designed to integrate storage-based replication functionality with VMwares Virtual Center and a centralized management interface and policy engine. It initiates functions like VM failover and rewriting and restoring VMs remotely in the proper sequence and then coordinating this with storage replication.

Note that SRM is currently only supported by use of the Fibre Channel SAN and iSCSI protocols. It is also not integrated with VCB.

Site Recovery Manager (SRM) is meant to automate, ease and more effectively monitor and test disaster recovery, with little to no impact on production operations.

October 2009 | Page 9

Other Data Protection Considerations

nAS (nFS) or SAn (Fibre Channel) for VMware backup/performance Many VMware environments began their foray into shared storage using a storage area network (SAN) based on Fibre Channel network connectivity. This is understandable, given the wide support for Fibre Channel offered by VMwares operating system (VMFS), and VMware features such as VCB and SRM. As the VM footprint grew in size and complexity, organizations began to revisit whether or not a network-attached storage (NAS) architecture, based on NFS, might offer other benefits, such as: More streamlined VM performance. NFS installations with VMware often report the ability to support a greater number of VMs per ESX Server. They also report the ability to avoid more common FC-based disk queuing bottlenecks and SCSI reservation locks that would otherwise decrease performance. Greater simplicity and faster backup/restores when combined with snapshot functionality and the straightforward directory structure of NFS. For a closer look at the merits and disadvantages of using FC, iSCSI or NFS protocols with VMware, contact a Datalink representative.

Use of deduplication to reduce backup data footprints Datalink has assisted VMware customers with deploying ancillary technologies such as data deduplication in their efforts to reduce the capacity of VMware-related backup data transmitted and stored, both locally and remotely. There are a number of options for deduplication in the market. Some offer client-side deduplication that is performed at the ESX Server (or within the Guest OS) prior to backup data transmission. Others are appliances that operate in the network. Still others offer the ability to deduplicate primary VMDK file data, while also being able to deduplicate secondary snapshot backup sets and archives.

October 2009 | Page 10

Datalinks Enhanced Data Recovery (EDR) Framework

dAtAlinks EnhAncEd dAtA rEcOvEry (Edr) frAMEwOrk

Applicable in a VM setting Datalink has deployed its enhanced data recovery (EDR) model for years to support the recovery and availability needs of mission critical applications in several customer environments. This framework focuses on data center growth, simplification of IT administration and the ability to successfully blur the lines between data protection, disaster recovery and business continuity. With the advent of VMware, the EDR model has also been refined for use in virtual server environments. Still, its core tenets remain, including: Reliance on disk-based backup and tape-based archiving, which has been enabled by falling SATAbased disk prices and the use of mature, advanced software functionality Leveraging the performance, reliability and the random access nature of RAID Augmenting the abilities of tape Data protection technologies and strategies underpinning the EDR framework are shown in Figures 3 and 4.

EnHAnCED DATA RECOVERY COnTInUUM

Recovery Point
days $ $$$ hours seconds $$$ $ seconds

Recovery Time
hours days $ $$$

Cost of Protection Cost of Lost Data

Failure Occurs

$$$ $

Cost of Recovery Cost of Time

Conventional Disk

Virtual Tape Library

Point-in-Time Copy
REPLICATION

Continuous Data Protection

figure 3: Edr is a unique datalink framework that incorporates disk-based technologies into data protection strategies

DATA DEDUPLICATION

SRM

figure 4: Edr with arraybased snapshots and remote replication

Storage Network Replication WAN Snapshot Copies Storage Network

October 2009 | Page 11

Conclusion

cOnclusiOn
Based on these tenets, Datalink has deployed a wide range of data protection architectures for virtual server environments that incorporate various options outlined in the previous sections of this paper. Some of the most successful implementations, as shown in Figure 4, tend to incorporate use of the following: Array-based snapshots Array-based remote replication VMware SRM

Datalink can help A complete data center solutions and services provider for Fortune 500 and mid-tier enterprises, Datalink transforms data centers so they become more efficient, manageable and responsive to changing business needs. Datalink helps leverage and protect storage, server, and network investments with a focus on long-term value, offering a full lifecycle of services, from consulting and design to implementation, management and support. Datalink solutions span virtualization and consolidation, data storage and protection, advanced networks, and business continuity. Each delivers measurable performance gains and maximizes the business value of IT. With a strong base of technical resources, Datalink is known for unbiased and balanced insights, field-tested best practices, and practical advice and support that help todays data centers make the best use of technology to meet business and IT needs. To learn more about how Datalink can help your organization sort through its options for data protection with virtual servers, contact Datalink at (800) 448-6314 or visit www.datalink.com. VMware-based data centers are benefiting from Datalinks needs-driven data protection practice and gaining ultimate reassurance of the merits of their planned solutions. Working with Datalink, organizations have access to current proof points from Datalinks in-depth testing and interoperability lab, where we incorporate VMwarebased and backup software provider technologies as well as disk-based snapshot and replication technologies from underlying storage solution vendors.

October 2009 | Page 12