Configuring Citrix XenServer Application Servers with DataCore Storage Servers Technical Bulletin #15b

June 7, 2010 Applicable to:

SANsymphony 6.0 PSP2 Update 3 or greater SANmelody 2.0 PSP4 Update 2 or greater
Overview This Technical Bulletin provides the steps to configure DataCore Storage Servers to present iSCSI or Fibre Channel virtual volumes to Citrix XenServer Host application servers. The configuration changes documented here have been tested and have proven to provide multi-pathing capability within DataCore test labs and many field implementations. DataCore will provide support for its products (SANsymphony / SANmelody) in production environments when implemented in the documented releases of both DataCore and Citrix products using the procedures described in this document. Any questions regarding support of multi-pathing within the XenServer product should be discussed with the appropriate Citrix / XenServer support resources.
Technical Bulletin #15b Cumulative Change Summary Technical Bulletin re-published for XenServer 5.0 Update 1 only as TB #15b. Added disclaimer and new logo and reformatted to accommodate removal of XenServer 4.1 references. Added new edit for rc.local file to avoid a known issue for iSCSI disk Pools being unavailable if XenServer Host is rebooted with Storage Server unavailable Step 17. Added Note to Step # 2 on Page 5 for XenServer 5.5 to Edit /etc/iscsi/iscsid-mpath.conf vs. /etc/iscsi/iscsid.conf. Added Additional Steps 1 on page 8, and Steps 9 through 11 page 10 for XenServer 5.5 for Software iSCSI. Added Note before Step 17 on Page 11 as a temporary solution for lack of multiple IQN support in XenServer 5.5 which will be addressed in a future XenServer release. Adjusted Steps on pages 8 12 added cli output of xe commands Added reboot after creation of the Dummy SRs to properly complete their configuration (step 11 on page 12). Adjusted notes and referenced sections to execute to clarify the sequence of procedures needed for software iSCSI. Removed reference to specific releases in the Requirements Section when giving guidance for Disable Port While Stopped. Clarified Notes at step 5 on page 13. Revised process (too many changes to list) document should be reviewed in its entirety. Revised Control SR naming convention to ensure uniqueness in the first 12 characters. Software iSCSI section: added new step to run a Citrix-provided 'DC-srprobe.sh' script prior to discovering new storage in XenCenter. Added XenServer 5.5 Updates 1 and 2 to qualified versions - qualified versions are now explicitly stated inside the document. Also changed SANsymphony and SANmelody versions to match the current supported as the previous versions stated were 'end of life'. Other, minor layout and formatting changes. Updated sr-probe command. Removed an outdated statement. Added a reference to TB#16 for limitation reclaiming SAUs with XenServer 5.5 COPYRIGHT Copyright 2010 by DataCore Software Corporation. All rights reserved. DataCore, the DataCore logo, SANsymphony, and SANmelody are trademarks of DataCore Software Corporation. Other DataCore product or service names or logos referenced herein are trademarks of DataCore Software Corporation. All other products, services and company names mentioned herein may be trademarks of their respective owners. ALTHOUGH THE MATERIAL PRESENTED IN THIS DOCUMENT IS BELIEVED TO BE ACCURATE, IT IS PROVIDED AS IS AND USERS MUST TAKE ALL RESPONSIBILITY FOR THE USE OR APPLICATION OF THE PRODUCTS DESCRIBED AND THE INFORMATION CONTAINED IN THIS DOCUMENT. NEITHER DATACORE NOR ITS SUPPLIERS MAKE ANY EXPRESS OR IMPLIED REPRESENTATION, WARRANTY OR ENDORSEMENT REGARDING, AND SHALL HAVE NO LIABILITY FOR, THE USE OR APPLICATION OF ANY DATACORE OR THIRD PARTY PRODUCTS OR THE OTHER INFORMATION REFERRED TO IN THIS DOCUMENT. ALL SUCH WARRANTIES (INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, FITNESS FOR A PARTICULAR PURPOSE AND AGAINST HIDDEN DEFECTS) AND LIABILITY ARE HEREBY DISCLAIMED TO THE FULLEST EXTENT PERMITTED BY LAW. No part of this document may be copied, reproduced, translated or reduced to any electronic medium or machine-readable form without the prior written consent of DataCore Software Corporation. Date

January 27, 2009

August 18, 2009

August 20, 2009

December 4, 2009

March 5, 2010 March 26, 2010

May 7, 2010

June 7, 2010

DataCore Storage Server requirements:

General: SANsymphony 6.0 PSP3 Update 2 or greater, SANmelody 2.0 PSP4 Updates 2 or greater. Important: The above versions are minimum requirements and users should always refer to the latest release notes for SANsymphony and SANmelody for any new fixes or enhancements specific to XenServer. Do not assume that all features of XenServer can be used with these minimum requirements and that you may need to upgrade your existing SANsymphony or SANmelody installations. Use the Citrix XenServer Application Server type. Use Third Party AP Mirror type if using XenServer Multi-pathing and make sure the mirror is healthy before configuring the Virtual Volume on the XenServer Hosts for the first time. Virtual Volumes must only be mapped once to any XenServer. You should not map the same virtual volume twice to the same application server. The LUN number used for a Virtual Volume for any XenServer Host Mappings, if multi-mapped for multi-pathing, must be identical. The first 12 characters of each Virtual Volume mapped to any XenServer Hosts must not match any other Virtual Volume. Note: Volume names are not used by the XenServer Host, only the Virtual Volume name. Refer to Technical Bulletin #8 If using Network Managed Volume Pools for the recommended SAU size for XenServer Hosts

If Using DataCore iSCSI Targets Using the DataCore iSCSI Manager GUI, make sure that the Disable Port While Stopped option is checked.

If Using DataCore Fibre Channel Targets Using the DataCore Fibre Channel Manager GUI, change all DataCore Storage Server Fibre Channel ports that will be used as targets for the XenServers to be Target only - the default is Target and Initiator. Target-only configured HBAs can also be used as Target HBAs for mirroring between DataCore Storage Servers. Using the DataCore Fibre Channel Manager GUI, make sure that the Disable Port While Stopped option is checked.

XenServer Host requirements:

General: Citrix XenServer 5.5 with no update or updates 1, 2 or 3, XenServer 5.0 with updates 1, 2 or 3 Important: Citrix XenServer 4.x is no longer supported. Multi-pathing for XenServer 5.x is not compatible with earlier versions of XenServer. If upgrading, carefully plan maintenance activities to allow for necessary configuration changes described in this document and begin with the Master Server before upgrading other XenServer Hosts in server pools sharing resources. Having too many running Virtual Machines on the same LUN can result in performance degradation. Consult your Citrix documentation for recommendations on the maximum number of VMs per SAN disk device. DataCore recommends no more than 15 VM per LUN; however this is only a recommendation and should not be taken as a limitation of either SANmelody or SANsymphony as each customers requirement will be different. If you do find that you are experiencing performance issues then try to reduce the amount of VMs on the LUN by moving them to another Virtual Volume. Please refer to Technical Bulletin #11 for setting the disk timeout values on the VMs. Refer to Technical Bulletin #16 for limitations with XenServer 5.5 and reclaiming unused storage units

If using QLogic QLA405x/406x iSCSI HBAs: Use QLogics Firmware revision 3.01.49 or greater

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Upgrade Consideration:
If using XenServer Multipathing and upgrading from XenServer 5.0 Update 1 or 2 to a later release: Upgrading from already-configured Application Servers running Citrix XenServer 5.0 to later releases will overwrite changes made to the /etc/multipath-enabled.conf file. It is not enough to simply save a copy of this file and re-apply it after the upgrade. Therefore, the steps to configure multipathing will need to be re-applied and the XenServer Hosts rebooted or the multipathd service restarted using the command:

service multipathd restart

Upgrading the XenServer 5.0 LVM storage to XenServer 5.5 is a one-way operation so Citrix recommends only performing the upgrade when you are certain the storage will no longer need to be attached to a pool running an older software version. Before upgrading form XenServer 5.0 to XenServer 5.5 and using software iSCSI, new Control SR LUNs must be created and assigned to the XenServers. Please refer to page 6 of this document for more information. Before upgrading the Storage Repository from 5.0 to the 5.5 LVHD, verify that the volume group has at least 4mb available in order for the upgrade to succeed.

CLI Command Notation:

General: This document describes CLI commands used in the XenServer console and follows the documentation standard for UNIX man pages. Items between less than and greater than symbols are mandatory and variable depending on your specific implementation,

For example:

<mandatory>,<program_name>
In this document, a shorthand description of the user specific item that must be used as the argument will be between the delimiters. Lines that are continuous have a backslash (\) character at the end of the line to indicate that there is no carriage return and the command continues on the next line.

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Configure the XenServer Host for Multipathing

1. From the XenCenter GUI configure the XenServer Pool and add the XenServer Hosts.

Configure multi-pathing (skip this step if not implementing multi-pathing)

2.

Using the XenConsole, edit the /etc/multipath-enabled.conf file on each XenServer Host

Note: /etc/mutipath.conf is a soft link that points to /etc/multipath-enabled.conf

Find the first defaults section in the conf file. It will be the uncommented defaults section at the beginning of the file. Within this section, add the polling_interval line shown below. Note: The second defaults section which is commented out with hash (#) symbols at the beginning of the each line should not be used.

For example, when you are finished, it will look like this:

defaults

{ user_friendly_names polling_interval } no 10

Important Note: Adding polling_interval in the DataCore device section will not work. Please make sure it remains in the defaults section. You should only have one uncommented defaults section and the first defaults section should contain all the settings you want active. The second defaults section should stay commented out.

Enable the Multi-pathing support on each XenServer Host from the XenCenter GUI: 3. 4. 5. 6. Put the XenServer Host in maintenance mode Click on General > Properties > Multipathing Select Enable multipathing on this server Exit maintenance mode

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Configuring Fibre Channel HBAs to Login to the DataCore Storage Server Fibre Channel Targets
For Emulex HBAs use /usr/sbin/hbanyware For QLogic HBAs use /opt/QLogic_Corporation/SANsurferCLI For more information on Fibre Channel configuration settings, please refer to the HBA Vendors own websites. 1. Using the DataCore GUI, assign the appropriate Fibre Channel HBAs to the XenServer application server(s). For more information, refer to DataCores online help under section Adding Application Servers to the SAN 2. Apply the configuration

Configuring Hardware iSCSI HBAs to Login to the DataCore Storage Server iSCSI Targets
1. 2. From the XenCenter GUI, click on the XenServer Host name then click on the console tab and login as root. Use the HBA Command Line Interfaces (CLIs) included in the XenServer Host to configure the HBA:

For QLogic iSCSI HBAs use /opt/QLogic_Corporation/SANsurferiCLI For more information on iSCSI HBA configuration settings, please refer to the iSCSI HBA Vendors own websites. Below is an example of configuring a QLogic iSCSI HBA Configuration: Set the IP networking configuration for the HBA. This example assumes DHCP and HBA port 0. Specify the appropriate values if using static IP addressing or a multi-port HBA.

/opt/QLogic_Corporation/SANsurferiCLI/iscli ipdhcp 0
Add a persistent iSCSI target to port 0 of the HBA.

/opt/QLogic_Corporation/SANsurferiCLI/iscli -pa 0 <iSCSI target IP \ address> -INAME <iSCSI target IQN>

3. 4. 5. Repeat the above command line to add a persistent target for every Storage Server target that will serve storage to the XenServer Host pool. Repeat the above steps for every XenServer Host in the Pool. Using the DataCore GUI, assign the appropriate iSCSI HBA channels to the XenServer application servers.

For more information, refer to DataCores online help under section Adding Application Servers to the SAN 6. Apply the configuration

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Configuring iSCSI Software Initiators to Login to the DataCore Storage Server iSCSI Targets
1. 2. 3. From the XenCenter GUI, click on the XenServer Host name then click on the General tab Highlight the iSCSI IQN and copy the value In the Storage Server GUI, manually add a new iSCSI channel to the application server using the copied IQN from step 2 For instructions, refer to the DataCores online help under section Adding a New Channel Manually 4. Create a virtual volume from each DataCore Storage Server that will map disks to the XenServer Host. The virtual volume(s) should be a linear volume of 128MB in size. These will be used as Control SRs required by Xen software iSCSI in order to keep the iSCSI login session active. Change the virtual volume name to conform to this naming convention: CtrlSR1 on the first Storage Server and CtrlSR2 on the other. Using the DataCore GUI, map the ControlSR virtual volume to every XenServer Host that will receive disks from the DataCore Storage Server. Do this for each ControlSR created.

5. 6.

Important Note: Because the software iSCSI initiator has not yet logged into the DataCore Storage Server target(s) the mapping paths for the ControlSRs must be verified before performing an apply in the DataCore GUI. Modify the mappings as needed. 7. If you have either multiple iSCSI initiator ports or multiple iSCSI target ports that the XenServer will connect to, then you will need to map the ControlSR for each individual initiator/target combination. The ControlSR must be seen by each initiator/target combination or it will not login automatically after a XenServer reboot. This is because the XenServer 5.5 iSCSI initiator does not recognize multiple target IQNs for the same LUN. This will be addressed by Citrix in a future XenServer release. Apply the configuration in the DataCore GUI From the XenCenter GUI, select the XenServer host (select the XenServer Pool master if using a Server Pool) choose > Storage > New Storage Repository Note: Please use the following naming convention for the new SR name

8. 9.

ControlSR_<DataCore Storage Server name>

replacing the <DataCore Server.

Storage Server name> with the DNS name or IP address of the DataCore Storage

10. Choose iSCSI and Click Next. 11. Target Host: Enter the IP address of the Storage Server iSCSI Target 12. Click Discover IQNs (i.e. the IQN of the iSCSI Target on the first Storage Server) 13. Click Discover LUNs (The DataCore ControlSR Virtual Volume should appear in the box). 14. Click Finish to create the SR and Yes to format. 15. Repeat Steps 10 through 15 above using the IP address of the each Storage Server iSCSI target that will map virtual volumes to the XenServer.

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Configuring the Startup Sequence of Control SRs (iSCSI Software Initiator only)
You now have a Control SR corresponding to each DataCore Storage Server target. The following set of steps ensures that the Control SRs are always attached at host boot time with the highest priority to ensure the correct number of paths are activated: 1. From the Console of the XenServer pool master host create a single VDI on each of the Control SRs created above using the following commands: For each DataCore Storage Server, identify it's corresponding SR UUID with the command

#xe sr-list params=uuid name-label=ControlSR_\ <DataCore Storage Server name>

Substitute the SR UUID identified in the command line above, and for each Control SR issue

#xe vdi-create sr-uuid=<SR_UUID> type=user name-label=datacoreMGT\ virtual-size=64

Important: The command above will return with the UUID for the created VDI. Substitute the UUID from the command above for VDI_UUID in the command line below: The command above will return with a VDI UUID for each Control SR. 2. Now on every host in the pool, execute the following command:

static-vdis add <VDI_UUID> DataCore

3. Reboot each XenServer Host in the pool in turn starting with the Master. Note: you can use the 'Maintenance Mode' feature in XenCenter to migrate existing virtual machines onto other nodes in order to maintain continued operation while each physical host is rebooted.

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Discover and Add New Storage Repository for Fibre Channel or Hardware iSCSI configurations only
All management operations go through the Master server. In order to discover mapped DataCore virtual volumes, all XenServer pool members must have the virtual volume mapped to them before using the same LUN. Therefore, if a pool member has the virtual volumes mapped but the Master Server does not, then the XenCenter GUI will never see the new LUN as the probe actually happens from the point of view of the Master Server.

Important: Before executing this section of steps, assure that multipath has been enabled for the host. (See the previous section Configure XenServer Host for Multipath) if planning to use multipathing. Wait for any Third Party AP virtual volume to be healthy before mapping to the XenServer Host(s). To use a virtual volume as a shared Storage Repository (SR) map it to all XenServer Hosts that will use the SR.

1.

Run the following command from any XenServer Host console in the Server Pool

xe host-list
This will list every XenServer Host in the Pool and their corresponding UUID 2. Run the following command for every XenServer Host UUID in the Pool (determined in the previous step)

xe sr-probe type=lvmohba
3. 4. 5. 6. 7.

host-uuid=<UUID>

Note: It is safe to ignore Error code: SR_BACKEND_FAILURE_107 Select the XenServer Host >Storage >New Storage Repository Select the Hardware HBA radio button, and Click Next Select the appropriate LUN on which to create the SR and Click Finish Click Yes to format the disk Check that the SR was created in the Resource pane (see the screenshot below).

Verify multi-pathing status

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Discover and Add New Storage Repository for Software iSCSI configurations only
All management operations go through the Xen Master server. In order to discover any mapped DataCore virtual volumes, all XenServer pool members must have the virtual volume mapped to them before being able to using the same DataCore LUN. If a pool member has the virtual volumes mapped but the Master Server does not, then the XenCenter GUI will not see the new LUN because the probe happens from the point of view of the Xen Master Server. Important: Before executing this section of steps, assure that multipath has been enabled for the host. (See the previous section Configure XenServer Host for Multipath) if planning to use multipathing. Wait for any Third Party AP virtual volume to be healthy before mapping to the XenServer Host(s). To use a virtual volume as a shared Storage Repository (SR) map it to all XenServer Hosts that will use the SR. Wait for any Third Party AP virtual volume to be healthy before mapping to the XenServer Host(s). To use a virtual volume as a shared Storage Repository (SR) map it to all XenServer Hosts that will use the SR. A Citrix-provided shell script will need to be downloaded to prevent the need for Xen Member Servers to be rebooted for multipathing to work.

1. 2.

Download DC-srprobe.sh from Citrix here: ftp://support.datacore.com/psp/TBs/XenScript/ Create a directory on the XenMaster Server

mkdir -p /opt/DataCore/bin
3. 4. On the XenMaster Server, upload the script into the directory created above. Make the script executable:

chmod +x /opt/DataCore/bin/DC-sprobe.sh
5. Run the script:

./opt/DataCore/bin/DC-srprobe.sh
6. 7. 8. Select the XenServer Host >Storage >New Storage Repository Choose iSCSI and Click Next. Target Host: Enter IP address of the Primary iSCSI target channel

9.

Click Discover IQNs (i.e. the IQN of the iSCSI target channel on the Storage Server)

10. Click Discover LUNs (the DataCore virtual volumes should appear in the box). Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

Note: If this step does not reveal the newly mapped virtual volume, repeat step 5. 11. Click Finish to create the SR and check the XenCenter GUI to verify the SR was created. 12. Click Yes to format the disk 13. The XenServer Hosts should see the multi-pathed SR down both paths. Verify this under the Multipathing status.

Verify multipathing status

Technical Bulletin 15b: Configuring Citrix XenServer Application Servers with DataCore Storage Servers

10