Nondisruptive Storage Relocation:

Planned Events with EMC VPLEX

Best Practices Planning

Abstract

This white paper covers best practices for using EMC VPLEX to migrate block storage data, for purposes
of a tech refresh or lease rollover, or for ongoing data mobility as performance and availability requirements
change.
May 2010

Copyright 2010 EMC Corporation. All rights reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is
subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION
MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com
All other trademarks used herein are the property of their respective owners.
Part Number h7065
Nondisruptive Storage Relocation: Planned Events with EMC VPLEX
Best Practices Planning

Table of Contents
Executive summary ............................................................................................ 4
Data migration challenge ............................................................................................................. 4

Introduction ......................................................................................................... 5
Audience ...................................................................................................................................... 5

Technology refresh............................................................................................. 5
EMC VPLEX................................................................................................................................. 6
EMC VPLEX clustering architecture ........................................................................................ 7

EMC VPLEX-assisted data relocation ............................................................. 10

Data migration operation............................................................................................................ 10
Post-migration considerations.................................................................................................... 15

Other considerations........................................................................................ 16
Conclusion ........................................................................................................ 17
References ........................................................................................................ 17

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

Executive summary
Customers have been demanding ever-increasing service levels while looking for reduction in capital and
operating costs. Data centers undergo continuous technology transformation in order to meet these
changing requirements. In recent years, consolidation of server and storage resources has proven to be
instrumental in reducing costs, improving utilization, and providing greater flexibility. The challenge is to
leverage new technologies, while at the same time not compromise the level of service to the end users.
A cloud analogy is often used to represent the vision for a fluid and dynamic IT infrastructures. One
definition of a cloud is the disassociation of IT services provided to users from the physical infrastructure.
An instantiation of a cloud is where servers are virtualized and services are optimized on any server in the
environment, connected to the appropriate storage, accessed from anywhere within the IT infrastructure.
Additionally, the cloud analogy describes an environment where services may be seamlessly relocated as
requirements change. Figure 1 illustrates a private cloud infrastructure where users and the services
provided by IT are loosely coupled and the relationships between services, compute, and storage are
dynamic and flexible.

Figure 1. IT services provided by any server, connected to any storage, anywhere

The technology exists today to build a cloud-like IT infrastructure. EMC, the industry leader in enterprise
storage, recently released EMC VPLEX, a solution that delivers local and distributed storage federation,
allowing full read and write LUN access and transparent movement of data within the data center and
between data centers. In addition, VPLEXs storage federation allows for the seamless integration of new
technologies such as the Symmetrix VMAX with Enginuity or CLARiiON storage arrays while
critical applications continue to provide uncompromised levels of service to users.

Data migration challenge

Ask any storage professional for a list of their biggest challenges and data migration will be at, or near, the
top of the list. Often, data migration is required during technology refresh initiatives where newer storage
systems such as the Symmetrix VMAX replace legacy storage.
Nondisruptive Storage Relocation: Planned Events with EMC VPLEX
Best Practices Planning

Data relocation may also be required to move data between storage tiers that reside on different storage
systems as performance and availability requirements change. During the information lifespan, data may
have different value to the organization, and changes in availability and performance may require data
movement up or down tiers. For example, when an application is in development, performance and
availability may not be nearly as critical as they are when in full production, therefore development and test
environments can leverage more cost-effective storage. When an application moves into production, data
availability and performance increase, and likely require data movement to a higher tier. As production
ramps up, even higher service levels may be required and thus some or all of the data may require the
highest-performance storage, such as what Enterprise Flash Drives deliver. If this ultra-high performance
storage is within the same storage system, capabilities such as Virtual LUN technologies may be leveraged
to seamlessly move data within the array. If this ultra-high performance storage is located in a different
storage system, data migration between systems is required.
Many different tools are available to simplify data movement between storage systems. Some are hostbased such as EMC Open Replicator or PPME Copy, or are capabilities built into host Logical Volume
Managers. While these tools are appropriate in some instances, they do use host resources and require the
involvement of server and application administrators. Others tools are array-based such as SAN Copy
and Open Migrator. EMC VPLEX offers a third option that is SAN-based, where data can nondisruptively
move between arrays without the direct involvement of the host or the storage system.

Introduction
This white paper discusses the best practices for using EMC VPLEX to nondisruptively move data for
purposes of a technology refresh or lease rollover, or to do ongoing data relocation as part of normal
operations and data lifecycle management.

Audience
This white paper is intended for storage architects and administrators who are responsible for planning and
implementing the EMC VPLEX for the purpose of data mobility and migration as part of a technology
refresh. The audience has general knowledge and experience working with:

EMC CLARiiON and Symmetrix storage systems

Fibre Channel SAN design and operations

VPLEX concepts and use of the CLI for basic management

Technology refresh
Many data centers have a process referred to as evergreening where hardware and software resources are
periodically replaced. However, this is not simply a practice of replacing the old with new. Capabilities
exist today that were not available four years ago, the typical time for technology replacement. Today we
have storage capabilities such as ultra-high performance storage provided by Enterprise Flash Drives, lowcost storage provided by high-capacity SATA drives, efficiency techniques such as Virtual Provisioning,
and the ability to set policies that move data between types as business requirements and workloads change.
Leveraging these capabilities allows us to offer the highest service levels for our demanding clients, and to
do so at the lowest possible capital and operating costs. Furthermore, data centers need to position
themselves to rapidly respond to the changes in business requirements and technology development cycles,
which are now measured in months rather than in years.
Virtualization of servers and federation of storage resources have proven instrumental to achieving these
goals. Inserting a virtualization layer between the operating systems and server hardware, as with VMware
vSphere, provides proven value in managing compute resources and changing service levels. Similarly,
storage federation provides a layer between the server and storage that enables movement of data between
the underlying storage systems, completely transparent to the host environment. EMC has recently
released the next-generation storage federation, the EMC VPLEX, that delivers local and distributed
Nondisruptive Storage Relocation: Planned Events with EMC VPLEX
Best Practices Planning

federation. This federation layer enables seamless migration between storage systems and allows storage
administrators to leverage new storage capabilities that provide greater levels of efficiency and lower cost,
without compromising service levels to end users.

EMC VPLEX
The EMC VPLEX family is the next-generation solution for information mobility and access within,
across, and between data centers. It is the first platform in the world that delivers both local and distributed
federation.
Local federation provides the transparent cooperation of physical elements within a site. Distributed
federation extends access between two locations across distance. VPLEX is a solution for federating both
EMC and non-EMC storage.
EMC VPLEX introduces a new architecture, which incorporates learnings from EMCs 20-plus years of
expertise in designing, implementing, and perfecting enterprise-class intelligent cache and distributed data
protection solutions.
Built on a foundation of scalable and highly available processor engines, EMC VPLEX is designed to
seamlessly scale from small to large configurations. VPLEX resides between the servers and
heterogeneous storage assets and uses a unique clustering architecture that allows servers at multiple data
centers to have read/write access to shared block storage devices. Unique characteristics of this new
architecture include:

Scale-out clustering hardware that lets you start small and grow big with predictable service levels

Advanced data caching, utilizing large-scale SDRAM cache to improve performance and reduce I/O
latency and array contention

Distributed cache coherence for automatic sharing, balancing, and failover of I/O across the cluster

A consistent view of one or more LUNs across VPLEX clusters separated either by a few feet with a
data center or across synchronous distances, enabling new models of high availability and workload
relocation

Figure 2. Capability of the EMC VPLEX system to federate heterogeneous storage

AccessAnywhere, available with VPLEX, is EMCs breakthrough technology that enables a single copy
of data to be shared, accessed, and relocated over distance. EMC GeoSynchrony is the VPLEX
operating system.
VPLEX family consists of two products: VPLEX Local and VPLEX Metro.

VPLEX Local provides simplified management and nondisruptive data mobility across heterogeneous
arrays.

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

VPLEX Metro provides data access and mobility between two VPLEX clusters within synchronous
distances.

Figure 3. EMC VPLEX family offering with architectural limits

With a unique scale-up and scale-out architecture, VPLEX's advanced data caching and distributed cache
coherency provide workload resiliency, automatic sharing, and balancing and failover of storage domains,
and enables both local and remote data access with predictable service levels.
VPLEX Local supports local federation today. VPLEX Metro delivers distributed federation capabilities
and extends access between two locations at synchronous distances. VPLEX Metro leverages
AccessAnywhere to enable a single copy of data to be shared, accessed, and relocated over distance.
The combination of a virtualized data center and EMC VPLEX provides customers entirely new ways to
solve IT problems and introduce new models of computing. Specifically, customers can:

Move virtualized applications across data centers

Enable workload balancing and relocation across sites

Aggregate data centers and deliver IT services 24 x forever

EMC VPLEX clustering architecture

VPLEX uses a unique clustering architecture to help customers remove the physical boundaries of the data
center and allow servers at multiple data centers to have read/write access to shared block storage devices.
A VPLEX Local configuration is defined by up to four VPLEX Engines, which are integrated into a single
cluster image through their fully redundant inter-engine fabric interconnections. This cluster interconnect
functionality allows for the online addition of VPLEX Engines, providing the exceptional scalability for
both VPLEX Local and VPLEX Metro configurations. All connectivity between VPLEX cluster nodes and
across VPLEX Metro configurations is fully redundant, ensuring protection against single points of failure.
A VPLEX Cluster can scale up through the addition of more engines, and scale out by connecting clusters
into a Metro-Plex (two VPLEX Metro clusters connected within metro distances). VPLEX Metro helps
transparently move and share workloads, including virtualized hosts, consolidates data centers, and
optimizes resource utilization across data centers. In addition, it provides nondisruptive data mobility,
heterogeneous storage management, and improved application availability. VPLEX Metro supports up to
two clusters, which can be in the same data center at two different sites within synchronous distances
(approximately up to 60 miles or 100 kilometers apart).

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

Figure 4. Local and distributed federation with EMC VPLEX Local and VPLEX Metro
EMC VPLEX maintains customer expectations for high-end storage in terms of availability. High-end
availability is more than just redundancy; it means nondisruptive operations and upgrades, and being
always online. EMC VPLEX provides:

AccessAnywhere, with full connectivity of resources across clusters and Metro-Plex configurations

Data mobility and migration options across heterogeneous storage arrays

The power to maintain service levels and functionality as consolidation grows

Simplified control for provisioning in complex environments

Dynamic load balancing of data between storage array assets

A VPLEX cluster is composed of one, two, or four engines. The engine is responsible for federating the I/O
stream, and connects to hosts and storage using Fibre Channel connections as the data transport. A single
VPLEX Cluster consists of an engine with the following major components:

Two directors, which run the GeoSynchrony software and connect to storage, hosts, and other directors
in the cluster with Fibre Channel and gigabit Ethernet connections

One Standby Power Supply, which provides backup power to sustain the engine through transient
power loss

Two management modules, which contain interfaces for remote management of a VPLEX Engine

Each cluster also consists of:

A management server, which manages the cluster and provides an interface from a remote
management station

An EMC standard 40U cabinet to hold all of the equipment of the cluster

Additionally, clusters containing more than one engine also have:

A pair of Fibre Channel switches used for inter-director communication between various engines

A pair of Universal Power Supplies that provide backup power for the Fibre Channel switches and
allow the system to ride through transient power loss

Logically, VPLEX is similar to other storage systems, with front-end ports that connect to host and backend ports that connect to storage systems. Storage presented to the back end is provisioned to hosts through
the front end. Advanced provisioning options allow devices to be striped, mirrored, and concatenated as
required by the host and application environment. The following figure and terms illustrate the
configuration objects used in configuring a VPLEX and the relationships between them.
Nondisruptive Storage Relocation: Planned Events with EMC VPLEX
Best Practices Planning

Storage volumes: A storage volume is a device or LUN on an attached storage system that is visible to
the VPLEX. The available capacity on a storage volume is used to create extents, devices, and virtual
volumes.

Extent: An extent is any subset (including all) of the capacity on a storage volume.

Device: Devices are configured from one or more extents in a RAID 1, RAID 0, or concatenated
(RAID-C) configuration.

Virtual volume: Virtual volumes are devices that can be provisioned to a host.

VPLEX ports: VPLEX ports are front-end ports used by a host to access virtual volumes. Ports are
added when a storage view is created to define the connection between initiators and virtual volumes.

Registered initiator: A registered initiator is a host bus adapter (HBA) that has been identified on the
VPLEX.

Storage view: A storage view is a logical grouping of ports, initiators, and virtual volumes for LUN
mapping and masking purposes. A storage view is similar to storage groups used on the CLARiiON
and to masking views used in Symmetrix VMAX.

Figure 5. VPLEX configuration objects

When a VPLEX is added between servers and storage systems, existing data volumes are encapsulated
while preserving all prior data. When using the VPLEX for migration during a technology refresh, source
and target storage volumes are presented to the VPLEX and a migration session is created. The actual data
movement is from the source array, through the VPLEX to the target array, completely transparent to
attached servers. The source and target devices can be of different RAID protection types and storage tiers,
and may consist of EMC and non-EMC storage arrays. In addition, the target device can be larger in
capacity than the source device, allowing changes in volume sizes as part of the migration. VPLEX
supports data movement within the data center or between local and remote data centers when the VPLEX
is configured as a Metro-Plex environment. In a multi-cluster configuration, the VPLEX can present the
same volume to hosts at both the local and remote sites while maintaining write order fidelity and cache
Nondisruptive Storage Relocation: Planned Events with EMC VPLEX
Best Practices Planning

coherency. This allows the movement of not only the storage but also the compute resources between data
centers.
The VPLEX offers many capabilities, including advanced provisioning, that allow volumes to divided and
aggregated to provide appropriate capacity and performance. Migrations are seldom a one-time event.
With the VPLEX as an integral part of the IT infrastructure, data movement between tiers and between data
centers as business and workload requirements change is seamless.
The management interface for the VPLEX includes a robust CLI and an intuitive web-based graphical user
interface.

EMC VPLEX-assisted data relocation

With EMC VPLEX, storage presented to hosts can be nondisruptively moved to different storage tiers
within and between back-end storage arrays. To illustrate the process, an example will be used where the
source volumes for the migration reside on a Symmetrix storage system, and the target volumes reside on a
CLARiiON CX array. The host environment consists of a pair of ESX servers in a cluster configuration.
Figure 6 shows the environment used in this example.

Figure 6. VPLEX migration scenario

For this example, the VPLEX system is installed, configured, and connected to both the Symmetrix and
CLARiiON arrays. The ESX servers and both storage systems are connected to the Fibre Channel fabric as
indicated above. While the scenario used to illustrate the process is based on an ESX environment, a
similar process would be followed for other host environments.

Data migration operation

For this example, we will initiate the migration of the data on the source devices that reside on the
Symmetrix to the target devices on the CLARiiON. The migration operation is transparent to the ESX
servers and the virtual machines that reside on the datastores.
Nondisruptive Storage Relocation: Planned Events with EMC VPLEX
Best Practices Planning

10

The actual data movement from source to target is performed in the following manner:
Convert the source volume from a RAID 0 to a RAID 1 configuration
Add the target device as a mirror
Synchronize the mirror
When fully synchronized, promote the target device to be the primary copy
Remove the source device to convert it back to a RAID 0 configuration
This procedure may be performed manually; however, the best practice is to use the batch migration
procedure that automates the steps.
1.

On the VPLEX console, create a batch migration plan. A plan is a file that identifies the source
and target devices and other attributes. When the session is created, it is assigned a session name.
A session can be created manually by creating and editing the plan file; however, for this example
we will use the following command to create the plan file automatically. Note in this example, the
wildcard character * was used to specify the source and target devices.
cd /clusters/cluster-2/devices
batch-migrate create-plan
f Datastore* -t Target_Dev* /tmp/batch_mig_plan
The following is an example of the command dialog.

View the file created by the above command. This file defines the migration session details and is
used by subsequent commands. In a root window on the SMS console, execute the following
command.
vi /tmp/batch_mig_plan
The following is an example of the file that was created.

2.

Check the plan and then start the migration session. The check-plan command validates the
migration plan. Starting the migration will convert the source device to a RAID 1 device, add the
target device as a mirror, and start the synchronization process.
batch-migrate check-plan /tmp/batch_mig_plan

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

11

batch-migrate start /tmp/batch_mig_plan

The following is an example of the command dialog.

3.

Verify the status of the migration. The following command will show the status of all active
device migration sessions.
cd /data-migrations/device-migrations
ll
The following is an example of the dialog. Here we see the migration is in progress and 7 percent
complete.

Display the device configuration. You will see that the migration process converts the source
device to a RAID 1 configuration and adds the target as a mirror. Use the following command to
view the configuration of the device. Note the source devices are renamed MIGRATE_<Session
name>. Also, note the status of the device shows a degraded state during synchronization. This is
normal until the mirror is fully synchronized.
cd clusters/cluster-2/devices
ll
The following is an example of the command dialog.

Determine how long the migration will take. Execute the following command to display details
about the source device that was renamed MIGRATE_BR0_0.

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

12

cd /cluster/cluster-2/devices/MIGRATE_BR0_0
ll

4.

Verify that the migration has completed. When the migration completes the PercentageDone will
show 100. Execute the following command.
cd /data-migrations/device-migrations
ll
The following is an example of what you will see when the synchronization is complete.

Display details about the operational state of the device. It should now show OK, indicating that
the RAID 1 mirrors are synchronized.
cd /clusters/cluster-2/devices
ll
The following is an example of what you would see when the mirrors have completed
synchronization.

5.

Once the synchronization completes, the migration session can be committed. The following
command will commit the migration.

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

13

batch-migrate commit /tmp/batch_mig

After committing, both the source and target devices are now visible with the target device being
associated with the virtual volume that is presented to the ESX cluster.

Also note the migration session will show as committed.

cd /data-migrations/device-migrations
ll

6.

The next step of the batch migration is the clean. This dismantles the source device down to the
storage volume and the source storage device is changed to an unclaimed state. Using the rename
option will cause the target volume to assume the name of the source volume. The following
command will clean up the migration.
batch-migrate clean -rename-targets /tmp/batch_mig
In the following example note that the migration target device assumed the name of the source
device and it is now associated with the virtual device. Also note that the source device was
removed.

Verify the source device is now unclaimed.

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

14

cd /clusters/cluster-2/storage-elements/storage-volumes
ll

7.

The final step of the batch migration is to remove all information about the migration session from
the VPLEX. The following command will remove all information about the completed migration
session from the VPLEX.
batch-migrate remove /tmp/batch_mig
The following is an example of the command dialog. Note that the session information is now
gone.

Best Practice: Schedule data migration during off-hours to minimize the impact of an increased workload
on the back end.
Best Practice: Stop the migration during hours of production and resume during off-hours.
The data migration is now complete. The final step is post-migration considerations.

Post-migration considerations
If the source storage system will no longer be part of the VPLEX environment, it can be removed by
performing the necessary masking, zoning, and other configuration changes. If the source devices will be
redeployed for other uses in the VPLEX environment, the following steps will not be necessary.
1.

On a management server connected to the source Symmetrix, execute the following commands to
remove the masking view and associated initiator and storage groups for the source devices.

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

15

symaccess sid <98>

symaccess sid <98>
symaccess sid <98>

delete view name zephyr32 unmap

delete name zephyr32 type storage
delete name zephyr32 type initiator

2.

Remove the zones that define the connection between the source Symmetrix and the VPLEX from
the zone configuration.

3.

Remove the devices from the VPLEX configuration. Use the following command to remove the
source devices form the configuration.
cd /clusters/cluster-2/storage-volumes
storage-volume forget

To prevent access to data the source device may be deleted on the source storage array. Optionally the
Symmetrix Secure Erasure Service may be used to thoroughly cleanse the data from the array.

Other considerations
Up to 25 migration sessions can run concurrently on a VPLEX system. Additional sessions can be defined
and queued for execution. When a session completes, a queued session will begin.
Best Practice: Migrate one server or cluster at a time.
Performance impacts during migration and the time it takes for a migration to complete are highly
dependent on workload during the migration, storage back-end configuration, and SAN and WAN link
configurations. One consideration for performance is the transfer size. The default value is 2 MB but is
configurable for 4 KB to 32 MB. The transfer size defines the size of a region on the source device that is
temporarily locked, read, and then written to the target. When the transfer size is set large, migration will
be faster but potentially could impact performance on the front end, especially when migrating across
clusters in a Metro-Plex configuration. Smaller transfer size will result in less front-end impact but
migrations will take longer.
Best Practice: Schedule migrations during periods of light workloads.

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

16

Conclusion
Changing requirements is inevitable in all data centers. The EMC VPLEX provides capabilities not
previously available by adding a storage federation layer between the server and storage systems.
Federation enables nondisruptive data relocation between arrays as capacity and performance requirements
change, and allows you to seamlessly integrate new storage technology meet stringent service level
requirements.
The migration procedures outlined in this paper used a VMware vSphere environment with Symmetrix
VMAX as the source array and a CLARiiON CX3-20 as the target array. However, as can be clearly seen,
while the details of the specific commands executed may change (depending on the environment), the same
process can be used with other combinations of host environments and source and target arrays.

References
For more specific information, reference the following on Powerlink:

EMC VPLEX CLI Guide

Implementation and Planning Best Practices for EMC VPLEX Technical Notes

Nondisruptive Storage Relocation: Planned Events with EMC VPLEX

Best Practices Planning

17