VMWARE Vi3!35!25 U2 Iscsi San CFG

iSCSI SAN Configuration Guide
Update 2 and later for ESX Server 3.5, ESX Server 3i version 3.5, VirtualCenter 2.5
iSCSI SAN Configuration Guide Revision: 20090313 Item: EN-000035-01
You can find the most up-to-date technical documentation on our Web site at: http://www.vmware.com/support/ The VMware Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: docfeedback@vmware.com
20072009 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware, the VMware boxes logo and design, Virtual SMP and VMotion are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.
VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

2 VMware, Inc.
Contents
AboutThisBook
1 UsingESXServerwithaStorageAreaNetwork 11
UnderstandingVirtualization 12 NetworkVirtualization 12 StorageVirtualization 12 StorageAreaNetworkConcepts 15 Ports 16 MultipathingandPathFailover 17 StorageSystemTypes 17 TargetComparedtoLUNRepresentations 17 iSCSINamingConventions 19 OverviewofUsingESXServerwithaSAN 20 BenefitsofUsingESXServerwithaSAN 20 UseCases 21 FindingSANConfigurationInformation 21 BasicsofUsingSANStorageSystemswithan ESXServer 22 NetworkConfigurationandAuthentication 22 SharingaVMFSAcrossESXServers 22 MetadataUpdates 24 VolumeDisplayandRescan 24 LevelsofIndirection 24 DataAccess:VMFSorRDM 25 ThirdPartyManagementApplications 26 Discovery,Authentication,andAccessControl 26 ErrorCorrection 27 UnderstandingVMFSandSANStorageChoices 27 ChoosingLargerorSmallerLUNs 27 MakingLUNDecisions 28 TipsforMakingLUNDecisions 29
VMware, Inc.
UnderstandingDataAccess 30 AccessingDataonaSAN 30 HowVirtualMachinesAccessData 30 HowVirtualMachinesAccessDataonaSAN 31 PathManagementandFailover 32 SCSIStorageStackFailover 32 NICTeamingandFailover 33 ArrayBasedFailover 33 ChoosingVirtualMachineLocations 35 DesigningforServerFailure 36 UsingVMwareHA 36 ServerFailoverandStorageConsiderations 36
2 InstallationofiSCSIInitiatorsandStorage 37
PreparingforiSCSIStorageSetup 37 ESXServerSANRequirements 38 Restrictions 38 Recommendations 39 SettingLUNAllocations 39 SettingUpHardwareiSCSIInitiatorsandStorage 40 InstallingandViewingHardwareiSCSIInitiators 40 ConfiguringHardwareiSCSIInitiators 41 AddingHardwareInitiatediSCSIStorage 47 SettingAdditionalParameters 50 SettingUpSoftwareiSCSIInitiatorsandStorage 50 NetworkingConfigurationforSoftwareiSCSIStorage ConfiguringSoftwareiSCSIInitiators 56 ViewingSoftwareiSCSIInitiators 60 AddingSoftwareInitiatediSCSIStorage 62
50
3 ModifyingSANStorageSystemswithESXServer 65
SetupOverview 66 GeneralConsiderations 66 EMCCLARiiONStorageSystems 67 EMCCLARiiONAX100iandAX150iandRDM 68 PushingHostConfigurationChangestotheStorageSystem 68 EMCSymmetrixStorageSystems 68 HPStorageWorksStorageSystems 69 HPStorageWorksMSA 69 HPStorageWorksEVA 71
VMware, Inc.
Contents
NetworkApplianceStorageSystems 71 Multipathing 71 SettingLUNTypeandInitiatorGroupType 72 ProvisioningStorage 72 EqualLogicStorageSystems 74 LeftHandNetworksSAN/iQStorageSystems 75 BasicConfiguration 75 AutomaticVolumeResignaturing 75
4 BootingfromaSANwithESXServerSystems 77
BootingfromaSANOverview 77 BenefitsofBootingfromaSAN 78 DecidingtoBootFromaSAN 78 EnablingBootingfromaSAN 79 PreparingtheSAN 79 ConfiguringiSCSIHBAstoBootfromaSAN 81
5 ManagingESXServerSystemsThatUseSANStorage 85
IssuesandSolutions 86 GuidelinesforAvoidingSANProblems 86 GettingInformation 87 ViewingHBAInformation 87 ViewingDatastoreInformation 88 ResolvingDisplayIssues 89 UnderstandingLUNNamingintheDisplay 89 ResolvingIssueswithLUNsThatAreNotVisible 89 UsingRescan 90 RemovingDatastores 91 AdvancedLUNDisplayConfiguration 92 ChangingtheNumberofLUNsScannedbyUsingDisk.MaxLUN 92 MaskingLUNsbyUsingDisk.MaskLUNs 93 ChangingSparseLUNSupportbyUsingDiskSupportSparseLUN 94 Multipathing 94 ViewingtheCurrentMultipathingState 94 ActivePaths 97 SettingaLUNMultipathingPolicy 97 DisablingandEnablingPaths 98 SettingthePreferredPath(FixedPathPolicyOnly) 99 PathManagementandManualLoadBalancing 100 PathFailover 101
VMware, Inc.
VMkernelConfiguration 102 SharingDiagnosticPartitions 102 AvoidingandResolvingSANProblems 103 OptimizingSANStoragePerformance 103 StorageSystemPerformance 104 ServerPerformance 104 NetworkPerformance 105 ResolvingPerformanceIssues 108 MonitoringPerformance 108 CheckingEthernetSwitchStatistics 109 ResolvingPathThrashing 109 UnderstandingPathThrashing 110 EqualizingDiskAccessBetweenVirtualMachines 111 RemovingVMFS2Drivers 112 ReducingSCSIReservations 112 SettingMaximumQueueDepthforSoftwareiSCSI 112 SANStorageBackupConsiderations 113 SnapshotSoftware 114 UsingaThirdPartyBackupPackage 114 ChoosingYourBackupSolution 115 LayeredApplications 115 ArrayBased(ThirdParty)Solution 116 FileBased(VMFS)Solution 116 VMFSVolumeResignaturing 117 MountingOriginal,Snapshot,orReplicaVMFSVolumes 117 UnderstandingResignaturingOptions 118
A MultipathingChecklist 121 B Utilities 123

esxtopUtility 123 storageMonitorUtility 124 Options 124 Examples 125 esxcfgswiscsiUtility 125 esxcfghwiscsiUtility 126 vmkpingUtility 126
Index 127
VMware, Inc.
About This Book
Thisbook,theiSCSISANConfigurationGuide,explainshowtouseanESXServer systemwithastorageareanetwork(SAN).Themanualdiscussesconceptual background,installationrequirements,andmanagementinformationinthesetopics:
UsingESXServerwithaSANDiscussesrequirements,differencesinSANsetup ifESXServerisused,andhowtomanagethetwosystemstogether. EnablingyourESXServersystemtobootfromaLUNonaSANDiscusses requirements,limitations,andmanagementofbootingfromaSAN.
NOTEForinformationonFibreChannel(FC)orNFSstoragedevices,seetheFibre ChannelSANConfigurationGuideandtheESXServerConfigurationGuide. TheiSCSISANConfigurationGuidecoversbothESXServer3.5andESXServer3i version3.5.Foreaseofdiscussion,thisbookusesthefollowingproductnaming conventions:

FortopicsspecifictoESXServer3.5,thisbookusesthetermESXServer3. FortopicsspecifictoESXServer3iversion3.5,thisbookusestheterm ESXServer3i. Fortopicscommontobothproducts,thisbookusesthetermESXServer. Whentheidentificationofaspecificreleaseisimportanttoadiscussion,thisbook referstotheproductbyitsfull,versionedname.
WhenadiscussionappliestoallversionsofESXServerforVMwareInfrastructure3, thisbookusesthetermESXServer3.x.
VMware, Inc.
Intended Audience
TheinformationpresentedinthismanualiswrittenforexperiencedWindowsorLinux systemadministratorswhoarefamiliarwithvirtualmachinetechnologydatacenter operations.
Document Feedback
VMwarewelcomesyoursuggestionsforimprovingourdocumentation.Ifyouhave comments,sendyourfeedbackto: docfeedback@vmware.com
VMware Infrastructure Documentation

TheVMwareInfrastructuredocumentationconsistsofthecombinedVMware VirtualCenterandESXServerdocumentationset.
Abbreviations Used in Figures

ThefiguresinthisbookusetheabbreviationslistedinTable 1. Table 1. Abbreviations
Abbreviation database datastore dsk# hostn SAN tmplt user# VC VM# Description VirtualCenterdatabase Storageforthemanagedhost Storagediskforthemanagedhost VirtualCentermanagedhosts Storageareanetworktypedatastoresharedbetweenmanagedhosts Template Userwithaccesspermissions VirtualCenter Virtualmachinesonamanagedhost
Technical Support and Education Resources

Thefollowingsectionsdescribethetechnicalsupportresourcesavailabletoyou. To accessthecurrentversionsofthisbookandotherbooks,goto: http://www.vmware.com/support/pubs
8 VMware, Inc.
About This Book
Online and Telephone Support

Useonlinesupporttosubmittechnicalsupportrequests,viewyourproductand contractinformation,andregisteryourproducts.Goto: http://www.vmware.com/support Customerswithappropriatesupportcontractsshouldusetelephonesupportforthe fastestresponseonpriority1issues.Goto: http://www.vmware.com/support/phone_support.html
Support Offerings
FindouthowVMwaresupportofferingscanhelpmeetyourbusinessneeds.Goto: http://www.vmware.com/support/services
VMware Education Services

VMwarecoursesofferextensivehandsonlabs,casestudyexamples,andcourse materialsdesignedtobeusedasonthejobreferencetools.Formoreinformationabout VMwareEducationServices,goto: http://mylearn1.vmware.com/mgrreg/index.cfm
VMware, Inc.
10
VMware, Inc.
Using ESX Server with a Storage Area Network
YoucanuseESXServerinconjunctionwithastorageareanetwork(SAN),aspecialized highspeednetworkthatconnectscomputersystemstohighperformancestorage subsystems.UsingESXServertogetherwithaSANprovidesextrastoragefor consolidation,improvesreliability,andhelpswithdisasterrecovery. TouseESXServereffectivelywithaSAN,youmusthaveaworkingknowledgeof ESX ServersystemsandSANconcepts.Also,whenyousetupESXServerhoststouse iSCSISANstoragesystems,specialconsiderationsarenecessary.Formoreinformation aboutusingESXServer,seetheESXServerConfigurationGuide. Thischapterdiscussesthefollowingtopics:

UnderstandingVirtualizationonpage 12 StorageAreaNetworkConceptsonpage 15 OverviewofUsingESXServerwithaSANonpage 20 BasicsofUsingSANStorageSystemswithanESXServeronpage 22 UnderstandingVMFSandSANStorageChoicesonpage 27 UnderstandingDataAccessonpage 30 PathManagementandFailoveronpage 32 ChoosingVirtualMachineLocationsonpage 35 DesigningforServerFailureonpage 36
VMware, Inc.
11
Understanding Virtualization
TheVMwarevirtualizationlayeriscommonacrossVMwaredesktopproducts(suchas VMwareWorkstation)andserverproducts(suchasVMwareESXServer).Thislayer providesaconsistentplatformfordevelopment,testing,delivery,andsupportof applicationworkloadsandisorganizedasfollows:
Eachvirtualmachinerunsitsownoperatingsystem(theguestoperatingsystem) andapplications. Thevirtualizationlayerallowsvirtualdevicestomaptosharesofspecificphysical devices.ThesedevicesincludevirtualizedCPU,memory,I/Obuses,network interfaces,storageadaptersanddevices,humaninterfacedevices,andBIOS.
Network Virtualization
Thevirtualizationlayerguaranteesthateachvirtualmachineisisolatedfromother virtualmachines.Virtualmachinescantalktoeachotheronlythroughnetworking mechanismssimilartothoseusedtoconnectseparatephysicalmachines. Theisolationallowsadministratorstobuildinternalfirewallsorothernetwork isolationenvironments,allowingsomevirtualmachinestoconnecttotheoutsidewhile othersareconnectedonlythroughvirtualnetworkstoothervirtualmachines. Formoreinformationonnetworkingconceptsandvirtualswitches,seetheESXServer ConfigurationGuide.
Storage Virtualization
InanESXServerenvironment,eachvirtualmachineincludesfromonetofourvirtual SCSIinitiators.ThesevirtualadaptersappearaseitherBuslogicorLSILogicSCSI controllers.TheyaretheonlytypesofSCSIcontrollersthatavirtualmachinecan access. EachvirtualdiskthatavirtualmachinecanaccessthroughoneofthevirtualSCSI initiatorsresidesintheVMwareVirtualMachineFileSystem(VMFS)oronarawdisk.
12
VMware, Inc.
Chapter 1 Using ESX Server with a Storage Area Network
Figure 11givesanoverviewofstoragevirtualization.ItillustratesstorageusingVMFS andstorageusingrawdevicemapping.ItalsoshowshowiSCSIstorageisaccessed througheitheriSCSIHBAsorbyusingageneralpurposeNICthatusesiSCSIinitiator software. Figure 1-1. iSCSI SAN Storage Virtualization
ESX Server virtual machine SCSI controller (Buslogic or LSI Logic) virtual disk virtual machine SCSI controller (Buslogic or LSI Logic) virtual disk
VMware virtualization layer software iSCSI initiator ethernet NIC LAN
hardware iSCSI initiator (HBA) LAN VMFS LUN1 .dsk
LUN2
LUN5
iSCSI Initiators
Toaccessremotetargets,yourESXServerhostusesiSCSIinitiators.Initiatorstransport SCSIrequestsandresponsesbetweentheESXServersystemandthetargetstorage deviceontheIPnetwork. ESXServersupportshardwarebasedandsoftwarebasediSCSIinitiators: Hardware iSCSI initiator. Athirdpartyhostbusadapter(HBA)withiSCSIover TCP/IPcapability.ThisspecializediSCSIadapterisresponsibleforalliSCSIprocessing andmanagement.
VMware, Inc.
13
Software iSCSI initiator. CodebuiltintotheVMkernelthatallowsanESXServerto connecttotheiSCSIstoragedevicethroughstandardnetworkadapters.Thesoftware initiatorhandlesiSCSIprocessingwhilecommunicatingwiththenetworkadapter throughthenetworkstack.Withthesoftwareinitiator,youcanuseiSCSItechnology withoutpurchasingspecializedhardware. NOTEGuestoperatingsystemsinvirtualmachinescannotseeiSCSIstoragedirectly. Totheguestoperatingsystems,iSCSIstorageattachedtotheESXServersystem appearstobeavailablethroughaSCSIHBA. ThediagraminFigure 11depictstwovirtualmachinesthatusedifferenttypesofiSCSI initiators. InthefirstexampleofiSCSIstorageconfiguration,theESXServersystemusesthe hardwareiSCSIinitiator.ThisspecializediSCSIinitiatorsendsiSCSIpacketstoadisk overaLAN. Inthesecondexample,theESXServersystemisconfiguredwiththesoftwareiSCSI initiator.Usingthesoftwareinitiator,theESXServersystemconnectstoaLANthrough anexistingNIC.
Disk Configuration Options

YoucanconfigurevirtualmachineswithmultiplevirtualSCSIdrives.Forsupported drivers,seetheStorage/SANCompatibilityGuideat www.vmware.com/support/pubs/vi_pubs.html.Theguestoperatingsystemcanplace limitationsonthetotalnumberofSCSIdrives. AlthoughallSCSIdevicesarepresentedasSCSItargets,physicalimplementation alternativesareavailable:
Virtualmachine.vmdkfilestoredonaVMFSvolume.SeeVirtualMachineFile Systemonpage 15. DevicemappingtoaSANlogicalunit(LUN).SeeRawDeviceMappingon page 15.
Fromthestandpointofthevirtualmachine,eachvirtualdiskappearsasifitwerea SCSIdriveconnectedtoaSCSIinitiator.Whethertheactualphysicaldiskdeviceis beingaccessedthroughSCSI,iSCSI,orFCcontrollersistransparenttotheguest operatingsystemandtoapplicationsrunningonthevirtualmachine.
14
VMware, Inc.
Virtual Machine File System

Inasimpleconfiguration,thevirtualmachinesdisksarestoredasfileswithinavirtual machinefilesystem(VMFS).WhenguestoperatingsystemsissueSCSIcommandsto theirvirtualdisks,thevirtualizationlayertranslatesthesecommandstoVMFSfile operations. ESXServersystemsuseVMFStostorevirtualmachinefiles.TominimizediskI/O overhead,VMFSisoptimizedtorunmultiplevirtualmachinesasoneworkload.VMFS alsoprovidesdistributedlockingforyourvirtualmachinefiles,sothatyourvirtual machinescanoperatesafelyinaSANenvironmentwheremultipleESXServerhosts shareasetofLUNs. VMFSisfirstconfiguredaspartoftheESXServerinstallation.Whenyoucreateanew VMFS3volume,itmustbe1200MBorlarger.SeetheInstallationGuide.Itcanthenbe customized,asdiscussedintheESXServerConfigurationGuide. AVMFSvolumecanbeextendedover32physicalstorageextents,includingSAN LUNsandlocalstorage.Thisallowspoolingofstorageandflexibilityincreatingthe storagevolumenecessaryforyourvirtualmachine.WiththenewESX3LogicalVolume Manager(LVM),youcanextendavolumewhilevirtualmachinesarerunningonthe volume.ThisletsyouaddnewspacetoyourVMFSvolumesasyourvirtualmachine needsit.
Raw Device Mapping

Arawdevicemapping(RDM)isaspecialfileinaVMFSvolumethatactsasaproxyfor arawdevice.TheRDMprovidessomeoftheadvantagesofavirtualdiskintheVMFS filesystem,whilekeepingsomeadvantagesofdirectaccesstophysicaldevices. RDMmightberequiredifyourunSANSnapshotorotherlayeredapplicationsinthe virtualmachine.RDMsbetterenablesystemstousethehardwarefeaturesinherentto SANstoragesystems.SeeUsingRawDeviceMappingintheESXServer ConfigurationGuide.
Storage Area Network Concepts

IfyouareanESXServeradministratorplanningtosetupESXServerhoststoworkwith SANs,youmusthaveaworkingknowledgeofSANconcepts.Youcanfindinformation aboutSANinprintandontheInternet.Twowebbasedresourcesare:

www.searchstorage.com www.snia.org
VMware, Inc.
15
IfyouarenewtoSANtechnology,readthefollowingsectiontofamiliarizeyourself withthebasicterminologythisdocumentuses.TolearnaboutbasicSANconcepts,see theSANConceptualandDesignBasicswhitepaperat http://www.vmware.com/support/pubs. ThisconfigurationguidediscussesiSCSISANs,whichuseEthernetconnections betweencomputersystems,orhostservers,andhighperformancestoragesubsystems. TheSANcomponentsincludehostbusadapters(HBAs)orNetworkInterface Cards (NICs)inthehostservers,switchesandroutersthattransportthestoragetraffic,cables, storageprocessors(SPs),andstoragedisksystems. Totransfertrafficfromhostserverstosharedstorage,theSANusestheiSCSIprotocol thatpackagesSCSIcommandsintoiSCSIpacketsandtransmitstheseonanEthernet network.
Ports
Inthecontextofthisdocument,aportistheconnectionfromadeviceintotheSAN. EachnodeintheSAN,ahost,storagedevice,andEthernetswitchhasoneormoreports thatconnectittotheSAN.Portsareidentifiedinanumberofways: IP address EachiSCSIporthasanIPaddressassociatedwithitsothatroutingand switchingequipmentonyournetworkcanestablishtheconnectionbetweentheserver andstorage.ThisisjustliketheIPaddressyouassigntoyourcomputertogetaccessto yourcompanysnetworkortheInternet. iSCSI Name Auniquenameforidentifyingtheport.TheiSCSInamestartswith eitheriqn.(foriSCSIqualifiedname)oreui.(forextendeduniqueidentifier).Multiple iSCSIdevicescanbepresent,withmultipleiSCSInames,andcanbeconnectedthrough asinglephysicalEthernetport.iSCSInamesareusuallysettoauniquedefaultvalue, anddonotneedtobesetbytheuser.AnexampleofaVMwaregeneratediSCSIname isiqn.199801.com.vmware:iscsitestox68158ef2. iSCSI alias AmoremanageablenameforaniSCSIdeviceorport,usedinsteadofthe iSCSIname.iSCSIaliasesarenotunique,andareintendedtobejustafriendlyname toassociatewithaport.OnanESXServersystem,thedefaultiSCSIaliasisthenameof thesystem.
16
VMware, Inc.
Multipathing and Path Failover

Whentransferringdatabetweenthehostserverandstorage,theSANusesa multipathingtechnique.Multipathingallowsyoutohavemorethanonephysicalpath fromtheESXServerhosttoaLUNonastoragesystem. IfapathoranycomponentalongthepathHBAorNIC,cable,switchorswitchport, orstorageprocessorfails,theserverselectsanotheroftheavailablepaths.Theprocess ofdetectingafailedpathandswitchingtoanotheriscalledpathfailover.
Storage System Types

Storagedisksystemscanbeofthefollowingtypes:
Anactiveactivestoragesystem,whichallowsaccesstotheLUNssimultaneously throughallthestorageportsthatareavailablewithoutsignificantperformance degradation.Allthepathsareactiveatalltimes(unlessapathfails). Anactivepassivestoragesystem,inwhichoneportoranSPisactivelyproviding accesstoagivenLUN.TheotherportsorSPsactasbackupfortheLUNandcan beactivelyprovidingaccesstootherLUNI/O.I/Ocanbesentonlytoanactive port.Ifaccessthroughtheprimarystorageportfails,oneofthesecondaryportsor storageprocessorsbecomesactive,eitherautomaticallyorthroughadministrator intervention. Avirtualportstoragesystem,whichallowsaccesstoallavailablevolumesthrough asinglevirtualport.Theseareactiveactivestoragedevices,buthidetheirmultiple connectionsthoughasingleport.TheESXServermultipathinghasnoknowledge ofthemultipleconnectionstothestorage.Thesestoragesystemshandleport failoverandconnectionbalancingtransparently.Thisisoftenreferredtoas transparentfailover.
Target Compared to LUN Representations

DifferentiSCSIstoragevendorspresentstoragetoserversindifferentways.Some vendorspresentmultipleLUNsonasingletarget,whileotherspresentmultipletargets withoneLUNeach(seeFigure 12).WhilethewaythestorageisusedbyanESXServer issimilar,thewaytheinformationispresentedthroughadministrativetoolsis different.
VMware, Inc.
17
Figure 1-2. Target compared to LUN Representations

target LUN LUN LUN target LUN target LUN target LUN
storage array
storage array
Threevolumes(orLUNs,theSCSItermforlogicalunit,whichmeansaportionof storage)areavailableineachoftheseconfigurations.Inthefirstcase,ESXServersees onetarget(representedbyanIQNname)butthattargethasthreeLUNsthatcanbe used.EachoftheLUNsrepresentindividualstoragevolumes.Inthesecondcase,the ESXServerseesthreedifferenttargets,representedbythreeseparateIQNnames. Each ofthesetargetshasoneLUN. Targetnumbering(thoughnotIQNtargetnaming)isdecidedbytheESXServer,so targetsthataresharedbydifferentESXServersystemsmightnothavethesametarget number(vmhba1:2:3:4,wherethe2showsthetargetnumberposition).Also,multiple pathstothesamestoragearerepresentedbydifferenttargetnumbers. LUNnumberingisdecidedbythestoragesystem,soLUNnumbersacrosspathsand multipleESXServersystemsalwaysstaythesame(vmhba1:2:3:4,wherethe3shows theLUNposition).Onstoragesystemswithmultipletargets,likethesecondexample above,theLUNnumberisalwayszero(0). ESXServerbasediSCSIinitiatorsestablishonlyoneconnectiontoeachtarget. This meansstoragesystemswithasingletargetcontainingmultipleLUNshaveall LUNtrafficonthatoneconnection.WithasystemthathasthreetargetswithoneLUN each,threeconnectionsexistbetweenanESXServerandthethreevolumesavailable. Thisinformationisusefulwhenyouaretryingtoaggregatestoragetrafficonmultiple connectionsfromanESXServerwithmultipleiSCSIHBAs,wheretrafficforonetarget canbesettoaparticularHBA,whiletrafficforanothertargetcanuseadifferentHBA. ThisdocumentusesthetermLUNtomeanavolumeavailablefromastoragesystem, evenifthatLUNistheonlyoneavailableonaniSCSItarget.Thismeaningistoavoid confusionwithotherusesofthewordvolumeinthisdocument.Althoughstorage systemvendorsmightnotcalltheirvolumesLUNs,thisisultimatelyhowtheyare representedtotheESXServerstoragesystem.
18
VMware, Inc.
iSCSI Naming Conventions

iSCSIusesanametouniquelyidentifyaniSCSIdevice,eithertargetorinitiator. This nameissimilartotheWorldWideNameassociatedwithFibreChanneldevices, usedasawaytouniversallyidentifythedevice. iSCSInamesareformattedtwodifferentways.ThefirstisbyaniSCSIqualifiedname, commonlyreferredtoasanIQNname.Thesecond,muchlesscommonmethod,is throughanenterpriseuniqueidentifier,alsoreferredtoasanEUIname.
iSCSI Qualified Names

iSCSIqualifiednamestaketheformiqn.yyyy-mm.naming-authority:unique name yyyy-mm istheyearandmonthwhenthenamingauthoritywasestablished. naming-authority isusuallyreversesyntaxoftheInternetdomainnameofthe namingauthority.Forexample,theiscsi.vmware.comnamingauthoritycouldhavethe iSCSIqualifiednameformofiqn.199801.com.vmware.iscsi.Thevmware.comdomain namewasregisteredinJanuaryof1998,andiscsiisasubdomain,maintainedby vmware.com. unique name theiscsi.vmware.comnamingauthorityneedstoensurethatanynames assignedfollowingthecolonareunique,suchas:

iqn.199801.com.vmware.iscsi:name1 iqn.199801.com.vmware.iscsi:name2 iqn.199801.com.vmware.iscsi:name999
Enterprise Unique Identifiers

Enterpriseuniqueidentifierstaketheformeui.<16 hex digits>. Forexample,eui.0123456789ABCDEF. The16hexadecimaldigitsaretextrepresentationsofa64bitnumberofanIEEEEUI (extendeduniqueidentifier)format.Thetop24bitsareacompanyIDthatIEEE registerswithaparticularcompany.Thelower40bitsareassignedbytheentity holdingthatcompanyID,andmustbeunique. Inmanycases,theIQNformatischosenovertheEUIformatforreadabilityandasa moreuserfriendlymethodofassigningnames.
VMware, Inc.
19
Overview of Using ESX Server with a SAN

SupportforQLogiciSCSIHBAsandsoftwarebasediSCSIimplementationsallowan ESXServersystemtobeconnectedtoiSCSIstorage.YoucanthenuseiSCSIstorage volumestostorevirtualmachineconfigurationinformationandapplicationdata. UsingESXServerwithaSANimprovesflexibility,efficiency,andreliability.Italso supportscentralizedmanagementaswellasfailoverandloadbalancingtechnologies.
Benefits of Using ESX Server with a SAN

UsingaSANwithESXServerallowsyoutoimproveyourenvironmentsfailure resilience:
YoucanstoredataredundantlyandconfiguremultipleEthernetpathstoyour iSCSIstorage,eliminatingasinglepointoffailure.Yourenterpriseisnotcrippled whenonedatacenterbecomesunavailable. ESXServersystemsprovidemultipathingbydefaultandautomaticallysupportit foreveryvirtualmachine.SeePathManagementandFailoveronpage 32. UsingaSANwithESXServersystemsextendsfailureresistancetotheserver. WhenyouuseSANstorage,allapplicationscaninstantlyberestartedafterhost failure.SeeDesigningforServerFailureonpage 36.
UsingESXServerwithaSANmakeshighavailabilityandautomaticloadbalancing affordableformoreapplicationsthanifdedicatedhardwareisusedtoprovidestandby services:
Ifvirtualmachinesareusedasstandbysystemsforexistingphysicalservers, sharedstorageisessentialandaSANisthebestsolution. UsetheVMwareVMotioncapabilitiestomigratevirtualmachinesseamlessly fromonehosttoanother. UseVMwareHighAvailability(HA)inconjunctionwithaSANforacoldstandby solutionthatguaranteesanimmediate,automaticresponse. UseVMwareDistributedResourceScheduler(DRS)tomigratevirtualmachines fromonehosttoanotherforloadbalancing.BecausestorageisonaSANstorage system,applicationscontinuerunningseamlessly. IfyouuseVMwareDRSclusters,putanESXServerhostintomaintenancemode tohavethesystemmigrateallrunningvirtualmachinestootherESXServerhosts. Youcanthenperformupgradesorothermaintenanceoperations.
20
VMware, Inc.
ThetransportabilityandencapsulationofVMwarevirtualmachinescomplementsthe sharednatureofiSCSIstorage.WhenvirtualmachinesarelocatedonSANbased storage,youcanshutdownavirtualmachineononeserverandpowerituponanother serverortosuspenditononeserverandresumeoperationonanotherserveronthe samenetworkinamatterofminutes.Thisallowsyoutomigratecomputingresources whilemaintainingconsistentsharedaccess.
Use Cases
UsingESXServersystemsinconjunctionwithSANiseffectiveforthefollowingtasks: Maintenance with zero downtime. Whenyouperformmaintenance,useVMware DRSorVMotiontomigratevirtualmachinestootherservers.Ifsharedstorageisonthe SAN,youcanperformmaintenancewithoutinterruptionstotheuser. Load balancing. UseVMotionexplicitlyoruseVMwareDRStomigratevirtual machinestootherhostsforloadbalancing.IfsharedstorageisonaSAN,youcan performloadbalancingwithoutinterruptiontotheuser. Storage consolidation and simplification of storage layout. Ifyouareworking withmultiplehosts,andeachhostisrunningmultiplevirtualmachines,thehosts storageisnolongersufficientandexternalstorageisneeded.ChoosingaSANfor externalstorageresultsinasimplersystemarchitecturewhilegivingyoutheother benefitslistedinthissection.Youcanstartbyreservingalargevolumeandthen allocateportionstovirtualmachinesasneeded.Volumeallocationandcreationfrom thestoragedeviceneedstohappenonlyonce. Disaster recovery. HavingalldatastoredonaSANcangreatlyfacilitateremote storageofdatabackups.Inaddition,youcanrestartvirtualmachinesonremote ESX Serverhostsforrecoveryifonesiteiscompromised.
Finding SAN Configuration Information

Inadditiontothisdocument,anumberofotherresourcescanhelpyouconfigureyour ESXServersysteminconjunctionwithaSAN:
Useyourstoragevendorsdocumentationformostsetupquestions.Yourstorage vendormightalsoofferdocumentationonusingthestoragesysteminan ESX Serverenvironment. TheFibreChannelSANConfigurationGuideDiscussestheuseofESXServerwith FibreChannelstorageareanetworks. TheVMwareI/OCompatibilityGuideListsthecurrentlyapprovedHBAs,HBA drivers,anddriverversions.
VMware, Inc.
21
TheVMwareStorage/SANCompatibilityGuideListscurrentlyapprovedstorage systems. TheVMwareReleaseNotesGiveinformationaboutknownissuesand workarounds. TheVMwareKnowledgeBaseHasinformationoncommonissuesand workarounds.
Formoreinformation,seetheVMwareDocumentationWebsiteat http://www.vmware.com/support/pubs.
Basics of Using SAN Storage Systems with an ESX Server

UsingaSANinconjunctionwithanESXServerhostdiffersfromtraditionalSANusage inavarietyofways,discussedinthissection.
Network Configuration and Authentication

BeforevolumesusingsoftwareiSCSIcanbediscovered,thestoragenetworkmustbe configuredandauthenticationmighthavetobesetup.
ForsoftwareiSCSI,networkingfortheVMkernelneedstobeconfigured.Youcan verifythenetworkconfigurationbyusingthevmkpingutility(seeAppendix B, Utilities,onpage 123).ForhardwareiSCSI,networkparameters,suchas IP address,subnetmask,anddefaultgatewaymustbeconfiguredontheHBA. Checkandchangethedefaultinitiatornameifnecessary. Thediscoveryaddressofthestoragesystemmustbesetandshouldbepingable usingvmkping. ForCHAPauthentication,enableitontheinitiatorandthestoragesystemside. Afterauthenticationisenabled,itappliesforallofthetargetsthatarenotyet discovered,butdoesnotapplytotargetsthatarealreadydiscovered.Afterthe discoveryaddressisset,thenewvolumesdiscoveredareexposedandcanbeused atthatpoint.
Sharing a VMFS Across ESX Servers

ESXServerVMFSisdesignedforconcurrentaccessfrommultiplephysicalmachines andenforcestheappropriateaccesscontrolsonvirtualmachinefiles.Forbackground informationonVMFS,seeVirtualMachineFileSystemonpage 15.Foradditional information,seetheESXServerConfigurationGuide.
22
VMware, Inc.
VMFScan:
Coordinateaccesstovirtualdiskfiles.ESXServerusesfilelevellocks,whichthe VMFSdistributedlockmanagermanages. CoordinateaccesstoVMFSinternalfilesysteminformation(metadata).
NOTESCSIreservationsareheldduringmetadataupdatestotheVMFSvolume. ESX ServerusesshortlivedSCSIreservationsaspartofitsdistributedlockingprotocol. BecausevirtualmachinesshareacommonVMFS,itmightbedifficulttocharacterize peakaccessperiodsortooptimizeperformance.Youneedtoplanvirtualmachine storageaccessforpeakperiods,butdifferentapplicationsmighthavedifferent peakaccessperiods.ThemorevirtualmachinesaresharingaVMFS,thegreaterthe potentialforperformancedegradationbecauseofI/Ocontention. NOTEVMwarerecommendsthatyouloadbalancevirtualmachinesoverservers, CPU,andstorage.Runamixofvirtualmachinesoneachserversothatnotall experiencehighdemandinthesameareaatthesametime. Figure 13showsseveralESXServersystemssharingthesameVMFSvolume. Figure 1-3. Accessing Virtual Disk Files
ESX Server A ESX Server B ESX Server C
VM1
VM2
VM3
VMFS volume disk1 disk2 disk3
virtual disk files
VMware, Inc.
23
Metadata Updates
AVMFSholdsfiles,directories,symboliclinks,RDMs,andsoon,alongwith correspondingmetadatafortheseobjects.Metadataisaccessedeachtimetheattributes ofafileareaccessedormodified.Theseoperationsinclude,butarenotlimitedto:

Creating,growing,orlockingafile. Changingafilesattributes. Poweringavirtualmachineonoroff.
Volume Display and Rescan

ASANisdynamic,andwhichvolumesareavailabletoacertainhostcanchangebased onanumberoffactorsincluding:

NewvolumescreatedontheiSCSIstorage Changestovolumeaccesscontrol Changesinnetworkconnectivity
TheVMkerneldiscoversvolumeswhenitbootsandthosevolumesarethenvisiblein VIClient.Ifchangesaremadetothevolumes,youmustrescantoseethosechanges. CAUTIONAfteryoucreateanewVMFSvolumeorextendanexistingVMFSvolume, youmustrescantheSANstoragefromallESXServerhoststhatcouldseethatparticular volume(LUN).Ifthisisnotdone,thesharedvolumemightbecomeinvisibletosome ofthosehosts.
Levels of Indirection
IfyoureusedtoworkingwithtraditionalSANs,thelevelsofindirectioncaninitially beconfusing.
Youcannotdirectlyaccessthevirtualmachineoperatingsystemthatusesthe storage.Withtraditionaltools,youcanmonitoronlytheVMwareESXServer operatingsystem(butnotthevirtualmachineoperatingsystem).Youusethe VI Clienttomonitorvirtualmachines. Eachvirtualmachineis,bydefault,configuredwithonevirtualharddiskandone virtualSCSIcontrollerduringinstallation.YoucanmodifytheSCSIcontrollertype andSCSIbussharingcharacteristicsbyusingtheVIClienttoeditthevirtual machinesettings,asshowninFigure 14.Youcanalsoaddharddiskstoyour virtualmachine.SeetheESXServerConfigurationGuide.
24
VMware, Inc.
Figure 1-4. Setting the SCSI Controller Type
TheHBAvisibletotheSANadministrationtoolsispartoftheESXServersystem, notthevirtualmachine. YourESXServersystemperformsmultipathingforyou.Multipathingsoftware (suchasPowerPath)inthevirtualmachineisnotsupported(andnotrequired).
Data Access: VMFS or RDM

Bydefault,avirtualdiskiscreatedinaVMFSvolumeduringvirtualmachinecreation. WhenguestoperatingsystemsissueSCSIcommandstotheirvirtualdisks,the virtualizationlayertranslatesthesecommandstoVMFSfileoperations.SeeVirtual MachineFileSystemonpage 15. AnalternativetoVMFSisusingRDMs.RDMsarespecialfilesinaVMFSvolumethat actasaproxyforarawdevice.TheRDMgivessomeoftheadvantagesofavirtualdisk intheVMFSwhilekeepingsomeadvantagesofdirectaccesstoaphysicaldevice. See RawDeviceMappingonpage 15.
VMware, Inc.
25
Third-Party Management Applications

MostiSCSIstoragehardwareispackagedwithstoragemanagementsoftware.Inmany cases,thisisawebapplicationthatcanbeusedwithanywebbrowserconnectedto yournetwork.Inothercases,thissoftwaretypicallyrunsonthestoragesystemorona singleserver,independentoftheserversthatusetheSANforstorage. Youcanusethisthirdpartymanagementsoftwareforanumberoftasks:
StoragesystemmanagementincludingLUNcreation,storagesystemcache management,LUNmapping,andLUNsecurity. Setupofreplication,checkpointing,snapshotting,ormirroring.
IfyoudecidetoruntheSANmanagementsoftwareonavirtualmachine,youcanrun anapplicationonavirtualmachine(failoverusingVMotion,failoverusingVMware HA,andsoon).Becauseoftheadditionallevelofindirection,however,the managementsoftwaremightnotbeabletoseetheSAN.Thiscanberesolvedbyusing anRDM.SeeLayeredApplicationsonpage 115. NOTEWhetheravirtualmachinecanrunmanagementsoftwaresuccessfullydepends onthestoragesysteminquestion.
Discovery, Authentication, and Access Control

SeveralmechanismscanbeusedtolimitwhichvolumesanESXServerhostcanaccess onaniSCSIstoragesystem.YoumustconfiguretheESXServerandtheiSCSIstorage systemtosupportyourstorageaccesscontrolpolicy. Discovery. AdiscoverysessionispartoftheiSCSIprotocol,anditreturnsthesetof targetsyoucanaccessonaniSCSIstoragesystem.Thetwotypesofdiscoveryavailable onESXServeraredynamicandstatic.Dynamicdiscoveryobtainsalistofaccessible targetsfromtheiSCSIstoragesystem,whilestaticdiscoverycanonlytrytoaccessone particulartargetbytargetname. Authentication. iSCSIstoragesystemsauthenticateaninitiatorbyanameandkey pair.ESXServersupportstheCHAPprotocol,whichVMwarerecommendsforyour SANimplementation.TheESXServerhostandtheiSCSIstoragesystemneedtohave CHAPenabled,andtohavecommoncredentials.IntheiSCSIloginphrase,theiSCSI storagesystemexchangesandchecksthesecredentials.
26
VMware, Inc.
Access Control. ApolicysetupontheiSCSIstoragesystem.Mostimplementations supportoneormoreofthreetypesofaccesscontrol:

Byinitiatorname ByIPaddress BytheCHAPprotocol
OnlyinitiatorsthatmeetallrulesattachedtotheiSCSIvolumecanaccessit.
Error Correction
ToprotecttheintegrityofiSCSIheadersanddata,theiSCSIprotocoldefineserror correctionmethodsknownasheaderdigestsanddatadigests.Thesearedisabledby default,buttheusercanenablethem.Thesedigestspertainto,respectively,theheader andSCSIdatabeingtransferredbetweeniSCSIinitiatorsandtargets,inbothdirections. Headeranddatadigestschecktheendtoend,noncryptographicdataintegrity beyondtheintegritychecksthatothernetworkinglayersprovide,suchasTCPand Ethernet.Theychecktheentirecommunicationpath,includingallelementsthatcan changethenetworkleveltraffic,suchasrouters,switches,andproxies. TheexistenceandtypeofthedigestsarenegotiatedwhenaniSCSIconnectionis established.Whentheinitiatorandtargetagreeonadigestconfiguration,thisdigest mustbeusedforalltrafficbetweenthem. Enablingheaderanddatadigestsdoesrequireadditionalprocessingforboththe initiatorandthetargetandcanaffectthroughputandCPUuseperformance.
Understanding VMFS and SAN Storage Choices

ThissectiondiscussestheavailableVMFSandSANstoragechoicesandgivesadviceon howtomakethesechoices.
Choosing Larger or Smaller LUNs

DuringESXServerinstallation,youarepromptedtocreatepartitionsforyoursystem. PlanhowtosetupstorageforyourESXServersystemsbeforeyouperforminstallation. Chooseoneoftheseapproaches:

ManyLUNswithoneVMFSvolumeoneachLUN OnelargeLUNormanyLUNswithasingleVMFSvolumespanningallLUNs
YoucanhaveatmostoneVMFSvolumeperLUN.Youcan,however,decidetouseone largeLUNormultiplesmallLUNs.
VMware, Inc.
27
Youmightwantfewer,largerLUNsforthefollowingreasons:
MoreflexibilitytocreatevirtualmachineswithoutaskingtheSANadministrator formorespace. Moreflexibilityforresizingvirtualdisks,doingsnapshots,andsoon FewerLUNstoidentifyandmanage
Youmightwantmore,smallerLUNsforthefollowingreasons:

LesscontentiononeachVMFSbecauseoflockingandSCSIreservationissues. DifferentapplicationsmightneeddifferentRAIDcharacteristics. Moreflexibility(themultipathingpolicyanddisksharesaresetperLUN).
NOTEYoucandivideyourdatacenterintoserversthatarebestconfiguredwithfewer, largerLUNsandotherserversthatusemore,smallerLUNs.
Making LUN Decisions

Whenthestoragecharacterizationforavirtualmachineisnotavailable,thereisoften nosimpleanswerwhenyouneedtodecideontheLUNsizeandnumberofLUNsto use.Youcanuseoneofthefollowingapproaches:

Predictivescheme Adaptivescheme
Predictive Scheme
Inthepredictivescheme,you:

CreateseveralLUNswithdifferentstoragecharacteristics. BuildaVMFSvolumeineachLUN(andlabeleachvolumeaccordingtoits characteristics). LocateeachapplicationintheappropriateRAIDforitsrequirements. Usedisksharestodistinguishhighpriorityfromlowpriorityvirtualmachines. DisksharesarerelevantonlywithinagivenESXServerhost.Thesharesassigned tovirtualmachinesononeESXServerhosthavenoeffectonvirtualmachineson otherESXServerhosts.
28
VMware, Inc.
Adaptive Scheme
Intheadaptivescheme,you:

CreatealargeLUN(RAID1+0orRAID5),withwritecachingenabled. BuildaVMFSinthatLUN. PlacefourorfivevirtualdisksontheVMFS. Runtheapplicationsandseewhetherdiskperformanceisacceptable. Ifperformanceisacceptable,youcanplaceadditionalvirtualdisksontheVMFS. Ifitisnot,youcreateanew,largerLUN,possiblywithadifferentRAIDlevel,and repeattheprocess.Youcanusecoldmigrationsoyoudonotlosevirtualmachines whenyourecreatetheLUN.
Tips for Making LUN Decisions

WhenyoumakeyourLUNdecision,keepinmindthefollowingconsiderations:
EachLUNshouldhavethecorrectRAIDlevelandstoragecharacteristicfor applicationsinvirtualmachinesthatuseit. OneLUNmustcontainonlyoneVMFSvolume. IfmultiplevirtualmachinesaccessthesameVMFS(andthereforethesameLUN), usedisksharestoprioritizevirtualmachines.
To use disk shares to prioritize virtual machines 1 2 3 4 StartaVIClientandconnecttoaVirtualCenterServer. Selectthevirtualmachineintheinventorypanel,rightclick,andchooseEdit Settingsfromthemenu. ClicktheResourcestabandclickDisk. RightclicktheSharescolumnforthedisktomodifyandselecttherequiredvalue fromthedropdownmenu.
VMware, Inc.
29
Understanding Data Access

Thissectiondiscusses:

AccessingDataonaSANonpage 30 HowVirtualMachinesAccessDataonpage 30 HowVirtualMachinesAccessDataonaSANonpage 31
Accessing Data on a SAN

Ifyouarenotfamiliarwithhowaphysical(nonvirtual)machineaccessesdataona SANstoragesystem,seetheSANConceptualandDesignBasicswhitepaperonthe VMwareDocumentationWebsiteatwww.vmware.com/support/pubs/.
How Virtual Machines Access Data

Virtualmachinesuseoneofthefollowingmethodstoaccessdata:
VMFSInasimpleconfiguration,thevirtualmachinesdisksarestoredas.vmdk fileswithinanESXServerVMFS.WhenguestoperatingsystemsissueSCSI commandstotheirvirtualdisks,thevirtualizationlayertranslatesthese commandstoVMFSfileoperations. Inadefaultsetup,thevirtualmachinealwaysgoesthroughVMFSwhenitaccesses afile,whetherthefileisonaSANorahostslocalharddrives.SeeVirtual MachineFileSystemonpage 15.
Rawdevicemapping(RDM)AnRDMisamappingfileinsidetheVMFSthat actsasaproxyforarawdevice.TheRDMgivestheguestoperatingsystemaccess totherawdevice. VMwarerecommendsRDMwhenavirtualmachinemustinteractwitharealdisk ontheSAN.Thisisthecase,forexample,whenyoumakestoragesystem snapshotsor,morerarely,ifyouhavealargeamountofdatathatyoudonotwant tomoveontoavirtualdisk.
FormoreinformationonVMFSandRDMs,seetheESXServerConfigurationGuide.
30
VMware, Inc.
How Virtual Machines Access Data on a SAN

WhenavirtualmachineinteractswithaSAN,thefollowingprocesstakesplace: 1 2 Whentheguestoperatingsysteminavirtualmachineneedstoreadorwritetoa SCSIdisk,itissuesSCSIcommandstothevirtualdisk. Devicedriversinthevirtualmachinesoperatingsystemcommunicatewiththe virtualSCSIcontrollers.VMwareESXServersupportstwotypesofvirtualSCSI controllers:BusLogicandLSILogic. ThevirtualSCSIControllerforwardsthecommandtotheVMkernel. TheVMkernel:
3 4
LocatesthefileintheVMFSvolumethatcorrespondstotheguestvirtual machinedisk. Mapstherequestsfortheblocksonthevirtualdisktoblocksonthe appropriatephysicaldevice. SendsthemodifiedI/OrequestfromthedevicedriverintheVMkerneltothe iSCSIinitiator(hardwareorsoftware).
IftheiSCSIinitiatorisahardwareiSCSIinitiator(iSCSIHBA),theHBAdoesthe following:

EncapsulatesI/OrequestsintoiSCSIProtocolDataUnits(PDUs). EncapsulatesiSCSIPDUsintoTCP/IPpackets. SendsIPpacketsoverEthernettotheiSCSIstoragesystem.
IftheiSCSIinitiatorisasoftwareiSCSIinitiator,it:

EncapsulatesI/OrequestsintoiSCSIPDUs. SendsiSCSIPDUsthroughTPC/IPconnections. TheVMkernelTCP/IPstackrelaysTCP/IPpacketstoaphysicalNIC. ThephysicalNICsendsIPpacketsoverEthernettotheiSCSIstorage system.
DependingonwhichporttheiSCSIinitiatorusestoconnecttothenetwork, Ethernetswitchesandrouterscarrytherequesttothestoragedevicethatthehost wantstoaccess. Fromthehostsperspective,thisstoragedeviceappearstobeaspecificdisk,butit mightbealogicaldevicethatcorrespondstoaphysicaldeviceontheSAN.
VMware, Inc.
31
Path Management and Failover

YoucanusemultiplewaystomanagepathsandfailoverinaniSCSISAN:

SCSIstoragestackfailover NICteamingandfailover Arraybasedfailover
SCSI Storage Stack Failover

ESXServersupportsmultipathingtomaintainaconstantconnectionbetweenthe servermachineandthestoragedeviceincaseofthefailureofanHBAorswitch. Multipathingsupportdoesnotrequirespecificfailoverdrivers. Tosupportpathswitching,theservertypicallyhastwoormoreHBAsavailablefrom whichthestoragesystemcanbereachedusingoneormoreswitches.Alternatively,the setupmightincludeoneHBAandtwostorageprocessorssothattheHBAcanusea differentpathtoreachthestoragesystem. Figure 1-5. Multipathing and Failover
ESX Server ESX Server
HBA2
HBA1
software initiator
IP network
SP
SP
iSCSI storage
InFigure 15,multiplepathsconnecteachserverwiththestoragedevice.Forexample, ifHBA1orthelinkbetweenHBA1andthenetworkfails,HBA2takesoverand providestheconnectionbetweentheserverandthenetwork.TheprocessofoneHBA takingoverforanotheriscalledHBAfailover.

32 VMware, Inc.
Similarly,ifSP1failsorthelinksbetweenSP1andtheswitchesbreaks,SP2takesover andprovidestheconnectionbetweentheswitchandthestoragedevice.Thisprocessis calledSPfailover.ESXServersupportsbothHBAandSPfailoverwithitsmultipathing capability. Youcanchooseamultipathingpolicyforyoursystem,eitherFixedorMostRecently Used.IfthepolicyisFixed,youcanspecifyapreferredpath.EachLUN(disk)thatis visibletotheESXServerhostcanhaveitsownpathpolicy.Forinformationonviewing thecurrentmultipathingstateandonsettingthemultipathingpolicy,see Multipathingonpage 94.
NIC Teaming and Failover

WithsoftwareiSCSI,youcanconnectasinglevirtualVMKerneliSCSInetworkswitch tomultiplephysicalEthernetadaptersbyusingtheVMwareInfrastructurefeature calledNICteaming.NICteamingprovidesnetworkredundancyandsomeload balancingcapabilitiesforiSCSIconnectionsbetweenESXServerandstoragesystems. SimilartotheSCSImultipathcapabilities,NICteamingprovidesfailoverifconnections orportsontheESXServersystemfail. NICteamingpathsdonotshowupasmultiplepathstostorageinESXServer configurations.NICteamingishandledentirelybythenetworklayerandmustbe configuredandmonitoredseparatelyfromtheESXServerSCSIstoragemultipath configuration.
Array-Based Failover
SomeiSCSIstoragesystemsmanagepathuseoftheirportsautomatically (transparentlytoESXServer).Whenusingoneofthesestoragesystems,ESXServer doesnotseemultipleportsonthestorageandcannotchoosethestorageportit connectsto.ThesesystemshaveasinglevirtualportaddressthatESXServerusesto initiallycommunicate.Duringthisinitialcommunication,thestoragesystemcan redirectESXServertocommunicatewithanotherportonthestoragesystem.TheiSCSI initiatorsinESXServerobeythisreconnectionrequestandconnectwithadifferentport onthesystem.Thestoragesystemusesthistechniquetospreadtheloadacross availableports. IfESXServerlosesconnectiontooneoftheseports,itautomaticallyattemptsto reconnectwiththevirtualportofthestoragesystem,andshouldberedirectedtoan active,usableport.Thisreconnectionandredirectionhappensquicklyandgenerally doesnotdisruptrunningvirtualmachines.Thesestoragesystemscanalsorequestthat iSCSIinitiatorsreconnecttothesystem,tochangewhichstorageporttheyare connectedto.Thisallowsthemosteffectiveuseofthemultipleports.
VMware, Inc.
33
Figure 16showsanexampleofportredirection.ESXServerattemptstoconnecttothe 10.0.0.1virtualport.Thestoragesystemredirectsthisrequestto10.0.0.2.ESXServer connectswith10.0.0.2andusesthisportforI/Ocommunication. NOTEThestoragesystemdoesnotalwaysredirectconnections.Theportat10.0.0.1 couldbeusedfortraffic,also. Figure 1-6. Port Redirection
ESX Server Connect to storage at 10.0.0.1 10.0.0.1 Reconnect to 10.0.0.2 10.0.0.2 storage
ESX Server
iSCSI storage traffic 10.0.0.1
10.0.0.2 storage
Iftheportonthestoragesystemthatisactingasthevirtualportbecomesunavailable, thestoragesystemreassignstheaddressofthevirtualporttoanotherportonthe system.Figure 17showsanexampleofthistypeofportreassignment.Inthiscase,the virtualport10.0.0.1becomesunavailableandthestoragesystemreassignsthevirtual portIPaddresstoadifferentport.Thesecondportrespondstobothaddresses. Figure 1-7. Port Reassignment
10.0.0.1
10.0.0.2 storage
10.0.0.1 10.0.0.1 10.0.0.2 storage
NOTEVirtualmachineI/Ocanbedelayedforuptosixtysecondswhilefailovertakes place,particularlyonanactivepassivearray.ThisdelayallowstheSANtostabilizeits configurationaftertopologychanges.Withactivepassivearrayswithpathpolicy Fixed,paththrashingcanbeaproblem.SeeResolvingPathThrashingonpage 109.
34
VMware, Inc.
Choosing Virtual Machine Locations

Whenyoureworkingonoptimizingperformanceforyourvirtualmachines,storage locationisanimportantfactor.Atradeoffalwaysexistsbetweenexpensivestoragethat offershighperformanceandhighavailabilityandstoragewithlowercostandlower performance.Storagecanbedividedintodifferenttiersdependingonanumberof factors: High Tier. Offershighperformanceandhighavailability.Mightofferbuiltin snapshotstofacilitatebackupsandpointintime(PiT)restorations.Supports replication,fullSPredundancy,andSASdrives.Useshighcostspindles. Mid Tier. Offersmidrangeperformance,loweravailability,someSPredundancy,and SCSIorSASdrives.Mayoffersnapshots.Usesmediumcostspindles. Lower Tier. Offerslowperformance,littleinternalstorageredundancy.Useslowend SCSIdrivesorSATA(seriallowcostspindles). Notallapplicationsneedtobeonthehighestperformance,mostavailablestorageat leastnotthroughouttheirentirelifecycle. NOTEIfyouneedsomeofthefunctionalityofthehightier,suchassnapshots,butdo notwanttopayforit,youmightbeabletoachievesomeofthehighperformance characteristicsinsoftware.Forexample,youcancreatesnapshotsinsoftware. Whenyoudecidewheretoplaceavirtualmachine,askyourselfthesequestions:

Howcriticalisthevirtualmachine? Whatareitsperformanceandavailabilityrequirements? WhatareitsPiTrestorationrequirements? Whatareitsbackuprequirements? Whatareitsreplicationrequirements?
Avirtualmachinemightchangetiersthroughoutitslifecyclebecauseofchangesin criticalityorchangesintechnologythatpushhighertierfeaturestoalowertier. Criticalityisrelativeandmightchangeforavarietyofreasons,includingchangesinthe organization,operationalprocesses,regulatoryrequirements,disasterplanning,and so on.
VMware, Inc.
35
Designing for Server Failure

TheRAIDarchitectureofSANstorageinherentlyprotectsyoufromfailureatthe physicaldisklevel.ASANprovidesmultiplepathsbetweenserversandstorage,which protectsagainstnetworkorportfailures.Thefinalstepinmakingyourwhole environmentfailureresistantistoprotectagainstserverfailure.ESXServersystems failoveroptionsarediscussedinthefollowingsections.
Using VMware HA
VMwareHAallowsyoutoorganizevirtualmachinesintofailovergroups.Whenahost fails,allitsvirtualmachinesareimmediatelystartedondifferenthosts.Whenavirtual machineisrestoredonadifferenthost,itlosesitsmemorystatebutitsdiskstateis exactlyasitwaswhenthehostfailed(crashconsistentfailover).Sharedstorage(such asaSAN)isrequiredforHA.SeetheResourceManagementGuide. NOTEYoumustbelicensedtouseVMwareHA.
Server Failover and Storage Considerations

Foreachtypeofserverfailover,youmustconsiderstorageissues:
Approachestoserverfailoverworkonlyifeachserverhasaccesstothesame storage.Becausemultipleserversrequirealotofdiskspace,andbecausefailover forthestoragesystemcomplementsfailoverfortheserver,SANsareusually employedinconjunctionwithserverfailover. WhenyoudesignaSANtoworkinconjunctionwithserverfailover,allvolumes theclusteredvirtualmachinesusemustbeseenbyallESXServerhosts. Althoughavolumeisaccessibletoahost,allvirtualmachinesonthathostdonot necessarilyhaveaccesstoalldataonthatvolume.Avirtualmachinecanaccess onlythevirtualdisksforwhichitwasconfigured.Incaseofaconfigurationerror, virtualdisksarelockedwhenthevirtualmachinebootssonocorruptionoccurs.
NOTEAsarule,whenyourebootingfromaSAN,eachbootvolumeshouldbeseen onlybytheESXServersystemthatisbootingfromthatvolume.Anexceptioniswhen youretryingtorecoverfromafailurebypointingasecondESXServersystemtothe samevolume.Inthiscase,theSANvolumeinquestionisnotreallyforbootingfroma SAN.NoESXServersystemisbootingfromitbecauseitiscorrupted.TheSANvolume isaregularnonbootvolumethatismadevisibletoanESXServersystem.
36
VMware, Inc.
Installation of iSCSI Initiators and Storage
BeforeESXServercanworkwithaSAN,youmustsetupyouriSCSIinitiatorsand storage.Todothis,youmustfirstobservecertainbasicrequirements.Thischapter discussestheserequirements,providesrecommendations,andthendetailshowto provideaccesstotheSANbyinstallingandsettingupyourhardwareorsoftwareiSCSI initiators. Thischapterdiscussesthefollowingtopics:

PreparingforiSCSIStorageSetuponpage 37 SettingUpHardwareiSCSIInitiatorsandStorageonpage 40 SettingUpSoftwareiSCSIInitiatorsandStorageonpage 50
Afterinitialinstallationstepsareperformed,youmightneedtomodifyyourstorage system.ThisisdiscussedinChapter 3,ModifyingSANStorageSystemswithESX Server,onpage 65.
Preparing for iSCSI Storage Setup

ToprepareforsettingupyourESXServersystemtouseSANstorage,reviewthe followingrequirements,restrictions,recommendations,andLUNallocationtips.
VMware, Inc.
37
ESX Server SAN Requirements

ThefollowingrequirementsmustbemetbeforeyourESXServerenvironmentcan workproperlywithaSAN:
Hardwareandfirmware.OnlyalimitednumberofSANstoragehardwareand firmwarecombinationsaresupportedinconjunctionwithESXServersystems. For anuptodatelist,seetheStorage/SANCompatibilityGuide. OneVMFSvolumeperLUN.ConfigureyoursystemtohaveonlyoneVMFS volumeforeachLUN.(InVMFS3,youdonotneedtosetaccessibility.) Unlessyoureusingdisklessservers(bootingfromaSAN),donotsetupthe diagnosticpartitiononaSANLUN.Inthecaseofdisklessserversthatbootfrom aSAN,ashareddiagnosticpartitionisappropriate. VMwarerecommendsthatyouuseRDMsforaccesstoanyrawdisk.Formore informationonRDMs,seetheESXServerConfigurationGuide. Multipathing.Formultipathingtoworkproperly,eachLUNmustpresentthe sameLUNnumbertoallESXServerhosts. Queuesize.SettheBusLogicorLSILogicdriverintheguestoperatingsystemto specifyalargeenoughqueue.YoucansetthequeuedepthforthephysicalHBA duringsystemsetup.Forsupporteddrivers,seetheStorage/SANCompatibility Guide. SCSITimeout.OnvirtualmachinesrunningMicrosoftWindows,increasethe valueoftheSCSITimeoutValueparametertoallowWindowstobettertolerate delayedI/Oresultingfrompathfailover.
Restrictions
ThefollowingrestrictionsapplywhenyouuseESXServerwithaSAN:

ESXServerdoesnotsupportiSCSIconnectedtapedevices. Youcannotusevirtualmachine,multipathingsoftwaretoperformI/Oload balancingtoasinglephysicalLUN.
38
VMware, Inc.
Chapter 2 Installation of iSCSI Initiators and Storage
Recommendations
Considerthefollowingrecommendationswhenyousetupyourenvironmentwith ESX ServerhostsandaSAN:
Userawdevicemappingforavirtualdiskofavirtualmachinetousesomeofthe hardwaresnapshottingfunctionsofthestoragesystem,ortoaccessadiskfrom bothavirtualmachineandaphysicalmachineinacoldstandbyhost configurationfordataLUNs. TouseVMotiontomoveavirtualmachinetoadifferenthost,theLUNsthathold thevirtualdisksofthevirtualmachinesmustbevisiblefromallofthehosts.
Setting LUN Allocations

WhenyousetLUNallocations,notethefollowingpoints:
StorageProvisioning.ToensurethattheESXServersystemrecognizestheLUNs atstartuptime,provisionallLUNStotheappropriateHBAsbeforeconnectingthe SANtotheESXServersystem. NOTEVMwarerecommendsthatyouprovisionallLUNstoallESXServerHBAs atthesametime.HBAfailoverworksonlyifallHBAsseethesameLUNs.
VMotionandVMwareDRS.WhenyouuseVirtualCenterandVMotionorDRS, makesurethattheLUNsforthevirtualmachinesareprovisionedtoallESXServer hosts.Thisconfigurationprovidesthegreatestfreedominmovingvirtual machines. Activeactiveversusactivepassivearrays.WhenyouuseVMotionorDRSwith anactivepassiveSANstoragedevice,makesurethatallESXServersystemshave consistentpathstoallstorageprocessors.Notdoingsocancausepaththrashing whenaVMotionmigrationoccurs.SeeResolvingPathThrashingonpage 109. ForactivepassivestoragearraysnotlistedintheStorage/SANCompatibilityGuide, VMwaredoesnotsupportstorageportfailover.Youmustconnecttheservertothe activeportonthestoragesystem.ThisconfigurationensuresthattheLUNsare presentedtotheESXServerhost.
VMware, Inc.
39
Setting Up Hardware iSCSI Initiators and Storage

WithhardwarebasediSCSIstorage,youuseaspecializedthirdpartyadaptercapable ofaccessingiSCSIstorageoverTCP/IP.ThisiSCSIinitiatorhandlesalliSCSIprocessing andmanagementforyourESXServersystem. HardwareiSCSIinitiatorsrequireconfigurationtoworkproperly,soyoumustinstall andconfigurethehardwareiSCSIinitiatorsasdetailedinthefollowingsectionsbefore settingupthedatastorethatresidesonaniSCSIstoragedevice. NOTEYoucanconfiguresomeESXServersystemstoloadbalancetrafficacross multipleHBAstomultipleLUNswithcertainactiveactivearrays.Todothis,assign preferredpathstoyourLUNssothatyourHBAsarebeingusedevenly.
Installing and Viewing Hardware iSCSI Initiators

Forinformationonwhichinitiatorsaresupported,seetheI/OCompatibilityGuideonthe VMwareWebsiteatwww.vmware.com. BeforeyoubeginconfiguringthehardwareiSCSIinitiator,makesurethattheiSCSI HBAissuccessfullyinstalledandappearsonthelistofinitiatorsavailablefor configuration.Iftheinitiatorisinstalled,youcanviewitsproperties. To view the hardware iSCSI initiator properties 1 2 LogintotheVIClientandselectaserverfromtheinventorypanel. ClicktheConfigurationtabandclickStorageAdaptersintheHardwaregroup. Thelistofavailablestorageadapters(initiators)appears.TheiSCSIinitiator appearsinthelistofstorageadapters.
UnderHBA,choosetheinitiatortoconfigure. Thedetailsfortheinitiatorappear,includingthemodel,IPaddress,iSCSIname, discoverymethods,iSCSIalias,andanydiscoveredtargets.
40
VMware, Inc.
ClickProperties. TheiSCSIInitiatorPropertiesdialogboxappears.TheGeneraltabdisplays additionalcharacteristicsoftheinitiator.
Youcannowconfigureyourhardwareinitiatororchangeitsdefaultcharacteristics.
Configuring Hardware iSCSI Initiators

WhenyouconfigurethehardwareiSCSIinitiator,setupyourinitiatorsiSCSIname, IP address,anddiscoveryaddresses.VMwarerecommendsthatyousetupCHAP parametersalso. AfteryouconfigureyourhardwareiSCSIinitiator,performarescansothatallLUNs thattheinitiatorhasaccesstoappearonthelistofstoragedevicesavailabletoyour ESX Serverhost.
Setting up Naming Parameters

WhenyouconfigureyourhardwareiSCSIinitiators,makesurethattheirnamesand IP addressesareformattedproperly.SeeiSCSINamingConventionsonpage 19.
VMware, Inc.
41
To set up the iSCSI name, alias, and IP address for the hardware initiator 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedin ToviewthehardwareiSCSIinitiatorpropertiesonpage 40. ClickConfigure. TheGeneralPropertiesdialogboxappears.
TochangethedefaultiSCSInameforyourinitiator,enterthenewname. Formatthenameyouenterproperly;otherwise,somestoragedevicesmightnot recognizethehardwareiSCSIinitiator.Youcanusethedefaultnamesuppliedby thevendor.Itdoesnothavetobechanged.
EntertheiSCSIalias. ThealiasisanamethatyouusetoidentifythehardwareiSCSIinitiator.
IfyouselectedUsethefollowingIPsettings,entervaluesforthefollowing:

IPAddress SubnetMask DefaultGateway
ClickOKtosaveyourchanges,thenreboottheserverforthechangestotakeeffect.
42
VMware, Inc.
Setting Up Discovery Addresses for the Hardware Initiator

Setuptargetdiscoveryaddressessothatthehardwareinitiatorcandeterminewhich storageresourceonthenetworkisavailableforaccess.Youcandothiswitheither dynamicdiscovery,wherealltargetsassociatedwithanIPaddressarediscovered,or withstaticdiscovery,whereyoumustspecifytheIPaddressandtheiSCSInameofthe targettobeseen. To set up target discovery addresses using Dynamic Discovery 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedin ToviewthehardwareiSCSIinitiatorpropertiesonpage 40. IntheiSCSIInitiatorPropertiesdialogbox,clicktheDynamicDiscoverytab.
VMware, Inc.
43
ToaddanewiSCSItargetthatyourESXServerhostcanuseforaSendTargets session,clickAdd. TheAddSendTargetsServerdialogboxappears.
EntertheIPaddressofastoragesystemandclickOK. AfteryourESXServerhostestablishestheSendTargetssessionwiththistarget device,anynewlydiscoveredtargetsappearintheStaticDiscoverylist.
TochangeordeleteaspecificIPaddress,selectitandclickEditorRemove.
To set up target discovery addresses by using Static Discovery 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedinTo viewthehardwareiSCSIinitiatorpropertiesonpage 40. IntheiSCSIInitiatorPropertiesdialogbox,clicktheStaticDiscoverytab.
44
VMware, Inc.
Thetabdisplaysalldynamicallydiscoveredtargetsandanystatictargetsalready entered.
ToaddatargetaccessibletoyourESXServerhost,clickAddandenterthetargets IPaddressandfullyqualifieddomainname.
Tochangeordeleteaspecificdynamicallydiscoveredtarget,selectthetargetand clickEditorRemove. NOTEIfyouremoveadynamicallydiscoveredstatictarget,thetargetcanbe returnedtothelistthenexttimearescanhappens,theHBAisreset,orthesystem isrebooted.
VMware, Inc.
45
Setting up CHAP Parameters

WhenyouconfigureyourhardwareiSCSIinitiator,ensurethatCHAPconfiguration matchesyouriSCSIstorage.IfCHAPisenabledonthestorage,itmustbeenabledon theinitiator.Iftheyareenabled,youmustsetuptheCHAPauthenticationcredentials tomatchyouriSCSIstorage. NOTEESXServerhostsonlysupportonesetofCHAPcredentialsperinitiator.Youcan notassigndifferentCHAPcredentialsfordifferenttargetsthroughaVIClient. To set up CHAP parameters for the hardware initiator 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedin ToviewthehardwareiSCSIinitiatorpropertiesonpage 40. ClicktheCHAPAuthenticationtab. ThetabdisplaysthedefaultCHAPparameters.
46
VMware, Inc.
TomakeanychangestotheexistingCHAPparameters,clickConfigure. TheCHAPAuthenticationdialogboxopens.
4 5 6
TokeepCHAPenabled,selectUsethefollowingCHAPcredentials. EitherenteranewCHAPnameorselectUseinitiatorname. Ifneeded,specifytheCHAPSecret. AllnewtargetswillusetheCHAPsecrettoauthenticatetheinitiator.
ClickOKtosavechanges. NOTEIfyoudisableCHAP,existingsessionsremainuntilarebootorthestorage systemforcesalogout,thenyoucannotconnecttotargetsthatrequireCHAP.
Adding Hardware-Initiated iSCSI Storage

WhenyoucreateadatastoreonahardwareinitiatediSCSIstoragedevice,theAdd Storagewizardguidesyouthroughtheconfiguration. To create a datastore on a hardware-initiated iSCSI device 1 2 3 LogintotheVIClientandselectaserverfromtheinventorypanel. ClicktheConfigurationtabandclickStorage. ClickAddStorage.
VMware, Inc.
47
TheSelectStorageTypepageappears.
SelectDisk/LUN,andclickNext. TheSelectDisk/LUNpageappears.Thiscantakeafewsecondsdependingonthe numberoftargetsthatyouhave.
48
VMware, Inc.
SelecttheiSCSIdevicetouseforyourdatastoreandclickNext. TheCurrentDiskLayoutpageappears.
LookoverthecurrentdisklayoutandclickNext. TheDisk/LUNPropertiespageappears.
Enteradatastorename. ThedatastorenameappearsintheVIClient,andthelabelmustbeuniquewithin thecurrentVirtualInfrastructureinstance.
ClickNext. TheDisk/LUNFormattingpageappears.
9 10
Ifneeded,adjustthefilesystemvaluesandcapacityyouuseforthedatastore. By default,theentirefreespaceavailableonthestoragedeviceisofferedtoyou. ClickNext. TheSummarypageappears.
11
ReviewthedatastoreinformationandclickFinish. ThiscreatesthedatastoreonthehardwareinitiatediSCSIdevice.
VMware, Inc.
49
Setting Additional Parameters

Youcanusetheesxcfg-hwiscsiutilitytoconfigureadditionalparametersforyour hardwareiSCSIHBA.Forexample,someiSCSIstoragesystemsrequireARP redirectiontomoveiSCSItrafficdynamicallyfromoneporttoanother.Youmustallow ARPredirectiononyourhardwareiSCSIHBA. SeeAppendix B,Utilities,onpage 123.
Setting Up Software iSCSI Initiators and Storage

WiththesoftwarebasediSCSIimplementation,youcanuseastandardnetwork adaptertoconnectyourESXServersystemtoaremoteiSCSItargetontheIPnetwork. TheESXServersoftwareiSCSIinitiatorbuiltintoVMkernelfacilitatesthisconnection communicatingwiththenetworkadapterthroughthenetworkstack. BeforeyouconfiguredatastoresthatuseasoftwareinitiatediSCSIconnectiontoaccess theiSCSIstorage,youmustenablenetworkconnectivityandtheninstallandconfigure thesoftwareiSCSIinitiator.
Networking Configuration for Software iSCSI Storage

BeforeyoucanconfigureiSCSIstorage,youmustcreateaVMkernelporttohandle iSCSInetworkingand(forESXServer3only)aserviceconsoleconnectiontotheiSCSI network. CAUTIONIfyouareconfiguringasoftwareiSCSIimplementationwithESXServer3, youmustsetupboththeVMkernelportandtheserviceconsoleconnectioninyour networkingconfiguration. To create a VMkernel port for software iSCSI 1 LogintotheVI Clientandselecttheserverfromtheinventorypanel. Thehardwareconfigurationpageforthisserverappears. 2 ClicktheConfigurationtabandclickNetworking.
50
VMware, Inc.
ClickAddNetworking. TheAddNetworkWizardappears.
WithESXServer3i,theoptionforServiceConsoledoesnotappearinthiswizard screen. 4 SelectVMkernelandclickNext. ThisletsyouconnecttheVMkernel,whichrunsservicesforiSCSIstorage,tothe physicalnetwork. TheNetworkAccesspageappears. 5 SelectthevSwitchtouseortheCreateavirtualswitchradiobutton.
VMware, Inc.
51
SelectthecheckboxesforthenetworkadaptersyourvSwitchwilluse.
YourchoicesappearinthePreviewpane. SelectadaptersforeachvSwitchsothatvirtualmachinesorotherservicesthat connectthroughtheadaptercanreachthecorrectEthernetsegment.Ifnoadapters appearunderCreateavirtualswitch,existingvSwitchesareusingallofthe networkadaptersinthesystem. NOTEDonotuseiSCSIon100MBNICs. 7 ClickNext. TheConnectionSettingspageappears. 8 UnderPortGroupProperties,selectorenteranetworklabeland,optionally,a VLANID.YoucanalsoenterorchangetheIPAddressandSubnetMaskunder IPSettings. NetworkLabelAnamethatidentifiestheportgroupthatyouarecreating.Thisis thelabelthatyouspecifywhenyouconfigureavirtualadaptertobeattachedto thisportgroup,whenyouconfigureiSCSIstorage.
52
VMware, Inc.
VLANIDIdentifiestheVLANthattheportgroupsnetworktrafficwilluse. VLANIDsarenotrequired.Ifyouarenotsurewhetheryouneedthem,askyour networkadministrator.
Youreceiveawarningthatnodefaultgatewayisset.Agatewayisneededfor connectivitytomachinesnotonthesameIPsubnetastheserviceconsole (ESX Server3only)orVMkernel.Toconnecttomultiplesubnets,clickYesto configureone.
VMware, Inc.
53
10
OntheRoutingtabintheDNSandRoutingConfigurationdialogbox,theservice consoleandtheVMkerneleachneedtheirowngatewayinformation.
NOTESetadefaultgatewayfortheportthatyoucreated.Youmustuseavalid staticIPaddresstoconfiguretheVMkernelstack. 11 12 13 14 ClickOKtosaveyourchangesandclosetheDNSandRoutingConfiguration dialogbox. ClickNext. UsetheBackbuttontomakeanychanges. ReviewyourchangesontheReadytoCompletepageandclickFinish.
AfteryoucreateaVMkernelportforiSCSI,youmustcreateaserviceconsole connectiononthesamevSwitchastheVMkernelport.IfyouareusingESXServer3i, thisisnotnecessary. To configure a service console connection for software iSCSI (ESX Server 3 only) 1 LogintotheVI Clientandselecttheserverfromtheinventorypanel. Thehardwareconfigurationpageforthisserverappears. 2 3 ClicktheConfigurationtabandclickNetworking. Ontherightsideofthescreen,clickPropertiesforthevSwitchassociatedwiththe VMkernelportyoujustcreated.
54
VMware, Inc.
OnthePortstab,clickAdd. TheAddNetworkWizardappears.
Asaconnectiontype,selectServiceConsoleandclickNext. TheConnectionSettingspageappears.
InthePortGroupPropertiesarea,enteranetworklabelthatidentifiestheport groupthatyouarecreating.
NewerportsandportgroupsappearatthetopofthevSwitchdiagram.
VMware, Inc.
55
EntertheIPAddressandSubnetMask,orselecttheObtainIPsetting automaticallyDHCPoptionfortheIPaddressandsubnetmask.Thismustbea differentIPaddressthantheonechosenfortheVMkernel. ClickEdittosettheServiceConsoleDefaultGateway. ClickNext. TheReadytoCompletepageappears.
8 9
10
AfteryouhavedeterminedthatthevSwitchisconfiguredcorrectly,clickFinish. AfteryoucreateaVMkernelportandserviceconsoleconnection,youcanenable andconfiguresoftwareiSCSIstorage.
Configuring Software iSCSI Initiators

ToconfigurethesoftwareiSCSIinitiator,youenableitandsetupitstargetaddresses. VMwarerecommendsthatyoualsosetupitsCHAPparameters. AfteryouconfigureyoursoftwareiSCSIinitiator,performarescan,sothatallLUNs thattheinitiatorhasaccesstoappearonthelistofstoragedevicesavailabletoyour ESX Serversystem.
56
VMware, Inc.
Enabling Software iSCSI Initiators

EnableyoursoftwareiSCSIinitiatorsothatESXServercanuseit. To enable the software iSCSI initiator 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedin ToviewthesoftwareiSCSIinitiatorpropertiesonpage 60. ClickConfigure. TheGeneralPropertiesdialogboxopens,displayingtheinitiatorsstatus,default name,andalias. 3 Toenabletheinitiator,selectEnabled.
TochangethedefaultiSCSInameforyourinitiator,enterthenewname. You do notneedtochangethedefaultname. Formatthenameyouenterproperly;otherwise,somestoragedevicesmightnot recognizethesoftwareiSCSIinitiator.
ClickOKtosaveyourchanges.
VMware, Inc.
57
Setting up Discovery Addresses

Setuptargetdiscoveryaddressessothatthesoftwareinitiatorcandeterminewhich storageresourceonthenetworkisavailableforaccess. To set up target discovery addresses for the software initiator 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedin ToviewthesoftwareiSCSIinitiatorpropertiesonpage 60. ClicktheDynamicDiscoverytab.
ToaddanewiSCSItargetyourESXServerhostcanuseforaSendTargetssession, clickAdd. TheAddSendTargetsServerdialogboxappears.
4 5
EntertheSendTargetsserverIPaddressandclickOK. TochangeordeleteaSendTargetsserver,selecttheserverandclickEditor Remove.

VMware, Inc.
58
Setting up CHAP Parameters

WhenyouconfigureyoursoftwareiSCSIinitiator,ensurethatCHAPconfiguration matchesyouriSCSIstorage.IfCHAPisenabledonthestorage,itmustbeenabledon theinitiator.Iftheyareenabled,youmustsetuptheCHAPauthenticationcredentials tomatchyouriSCSIstorage. To set up CHAP parameters for the software initiator 1 2 OpentheiSCSIInitiatorPropertiesdialogboxbyperformingthestepslistedin ToviewthesoftwareiSCSIinitiatorpropertiesonpage 60. ClicktheCHAPAuthenticationtab. ThetabdisplaysthedefaultCHAPparameters.
TomakeanychangestotheexistingCHAPparameters,clickConfigure.
VMware, Inc.
59
TheCHAPAuthenticationdialogboxopens.
4 5 6
TokeepCHAPenabled,selectUsethefollowingCHAPcredentials. EitherenteranewCHAPnameorselectUseinitiatorname. Ifneeded,specifytheCHAPSecret. AllnewtargetswillusetheCHAPsecrettoauthenticatetheinitiator. Any establishedsessionsarenotaffected.
ClickOKtosavechanges. NOTEIfyoudisableCHAP,existingsessionsremainuntilarebootorthestorage systemforcesalogout,youcannotconnecttotargetsthatrequireCHAP.
Viewing Software iSCSI Initiators

ThesoftwareiSCSIinitiatorthatyourESXServersystemusestoaccessa softwareinitiatediSCSIstoragedeviceappearsonthelistofavailableadapters. After configuringyoursoftwareinitiator,youcanusetheVIClienttoreviewits properties. To view the software iSCSI initiator properties 1 2 LogintotheVIClientandselectaserverfromtheinventorypanel. ClicktheConfigurationtabandclickStorageAdaptersunderhardware. Thelistofavailablestorageadaptersappears. 3 UnderiSCSISoftwareAdapter,choosetheavailablesoftwareinitiator.
60
VMware, Inc.
Thedetailsfortheinitiatorappear,includingthemodel,IPaddress,iSCSIname, discoverymethods,iSCSIalias,andanydiscoveredtargets.
ClickProperties. TheiSCSIInitiatorPropertiesdialogboxopens.TheGeneraltabdisplays additionalcharacteristicsofthesoftwareinitiator.
VMware, Inc.
61
Adding Software-Initiated iSCSI Storage

WhenyoucreateadatastoreonasoftwareinitiatediSCSIstoragedevice,theAdd Storagewizardguidesyouthroughtheconfiguration. To create a datastore on a software-initiated iSCSI device 1 2 3 LogintotheVIClientandselectaserverfromtheinventorypanel. ClicktheConfigurationtabandclickStorage. ClickAddStorage. TheSelectStorageTypepageappears.
SelecttheDisk/LUNstoragetypeandclickNext.
62
VMware, Inc.
TheSelectDisk/LUNpageappears.Thiscantakeafewsecondsdependingonthe numberoftargetsthatyouhave.
SelecttheiSCSIdevicetouseforyourdatastoreandclickNext. TheCurrentDiskLayoutpageappears.
LookoverthecurrentdisklayoutandclickNext. TheDisk/LUNPropertiespageappears.
Enteradatastorename. ThedatastorenameappearsintheVIClient,andthelabelmustbeuniquewithin thecurrentVirtualInfrastructureinstance.
ClickNext.
VMware, Inc.
63
TheDisk/LUNFormattingpageappears.
Ifneeded,adjustthefilesystemvaluesandcapacityyouuseforthedatastore. Bydefault,theentirefreespaceavailableonthestoragedeviceisofferedtoyou.
10
ClickNext. TheReadytoCompletepageappears.
11
ReviewthedatastoreconfigurationinformationandclickFinish. ThiscreatesthedatastoreonthesoftwareinitiatediSCSIstoragedevice.
64
VMware, Inc.
Modifying SAN Storage Systems with ESX Server
AfteryouinstallyouriSCSIinitiatorsandstorage,youmightneedtomodifyyour storagesystemtoensurethatitworksproperlywithyourESXServerimplementation. ThischapterdiscussesmanyoftheiSCSIstoragesystemssupportedinconjunction withVMwareESXServer.Foreachdevice,itlistsmajorknownpotentialissues,points tovendorspecificinformation(ifavailable),orincludesinformationfromVMware knowledgebasearticles. NOTEInformationinthisdocumentisupdatedonlywitheachrelease.New informationmightalreadybeavailable.Also,otheriSCSIstoragesystemsare supportedbutarenotcoveredinthischapter.ConsultthemostrecentStorage/SAN CompatibilityGuide,checkwithyourstoragevendor,andexploretheVMware knowledgebasearticles. Thischapterdiscussesthefollowingtopics:

SetupOverviewonpage 66 GeneralConsiderationsonpage 66 EMCCLARiiONStorageSystemsonpage 67 EMCSymmetrixStorageSystemsonpage 68 HPStorageWorksStorageSystemsonpage 69 NetworkApplianceStorageSystemsonpage 71 EqualLogicStorageSystemsonpage 74 LeftHandNetworksSAN/iQStorageSystemsonpage 75
VMware, Inc.
65
Setup Overview
VMwareESXServersupportsavarietyofSANstoragesystemsindifferent configurations.Notallstoragedevicesarecertifiedforallfeaturesandcapabilitiesof ESXServer,andvendorsmighthavespecificpositionsofsupportwithregardto ESX Server.Forthelatestinformationregardingsupportedstoragesystems,seethe Storage/SANCompatibilityGuide. VMwaretestsESXServerwithstoragesystemsinthefollowingconfigurations: Basic Connectivity. TestswhetherESXServercanrecognizeandoperatewiththe storagesystem.Thisconfigurationdoesnotallowformultipathingoranytypeof failover. HBA Failover. TheserverisequippedwithmultipleHBAsconnectingtooneormore SANswitches.TheserverisrobusttoHBAandswitchfailureonly. Storage Port Failover. Theserverisattachedtomultiplestorageportsandisrobust tostorageportfailuresandswitchfailures. Booting from a SAN. TheESXServerhostbootsfromaLUNconfiguredontheSAN ratherthanfromtheserveritself.
General Considerations
Forallstoragesystems,makesurethatthefollowingrequirementsaremet:
LUNsmustbepresentedtoeachHBAofeachhostwiththesameLUNIDnumber. Ifdifferentnumbersareused,theESXServerhostsdonotrecognizedifferent pathstothesameLUN.BecauseinstructionsonhowtoconfigureidenticalSAN LUNIDsarevendorspecific,consultyourstoragedocumentationformore information. Unlessspecifiedforindividualstoragesystemsdiscussedinthischapter,setthe hosttypeforLUNspresentedtoESXServertoLinuxorLinux Cluster,if applicabletoyourstoragesystem.ThemethodESXServerusestoaccessthe storagesystemismostcompatiblewithLinuxaccess,however,thiscanvary dependingonthestoragesystemyouareusing.
66
VMware, Inc.
Chapter 3 Modifying SAN Storage Systems with ESX Server
IfyouareusingVMotion,DRS,orHA,makesurethatsourceandtargethostsfor virtualmachinescanseethesameLUNswithidenticalLUNIDs. SAN administratorsmightfinditcounterintuitivetohavemultiplehostsseethe sameLUNsbecausetheymightbeconcernedaboutdatacorruption.However, VMFSpreventsmultiplevirtualmachinesfromwritingtothesamefileatthesame time,soprovisioningtheLUNstoallrequiredESXServersystemisappropriate. IfyoudonothaveCHAPauthenticationsetupontheLUNsthatarebeing accessed,youmustalsodisableCHAPontheESXServerhost.Otherwise, authenticationofthestoragesystemfails,althoughtheLUNshavenoCHAP requirement.
EMC CLARiiON Storage Systems

EMCCLARiiONstoragesystemsworkwithESXServermachinesinSAN configurations.Basicconfigurationstepsinclude: 1 2 3 4 5 Installingandconfiguringthestoragedevice. CreatingRAIDgroups. CreatingandbindingLUNs. RegisteringtheserversconnectedtotheSAN. CreatingstoragegroupsthatcontaintheserversandLUNs.
UsetheEMCsoftwaretoperformconfiguration.Formoreinformation,seetheEMC documentation. NOTEThisisanactivepassivediskarray,sorelatedissuesdescribedelsewhereinthis documentapply. Toavoidthepossibilityofpaththrashing,thedefaultmultipathingpolicyisMost RecentlyUsed,notFixed.TheESXServersystemsetsthedefaultpolicywhenit identifiesthestoragesystem.SeeResolvingPathThrashingonpage 109. AutomaticvolumeresignaturingisnotsupportedforAX100iandAX150istorage devices.Forinformationonresignaturing,seeVMFSVolumeResignaturingon page 117. NOTETobootfromaSAN,choosetheactivestorageprocessorforthebootLUNs targetintheHBABIOS.
VMware, Inc.
67
EMC CLARiiON AX100i and AX150i and RDM

OnEMCCLARiiONAX100iandAX150isystems,RDMsaresupportedonlyifyouuse theNavisphereManagementSuiteforSANadministration.NavisphereExpressisnot guaranteedtoconfigurethemproperly. TouseRDMssuccessfully,agivenLUNmustbepresentedwiththesameLUNIDto everyESXServerhostinthecluster.TheAX100iandAX150idonotdothisbydefault.
Pushing Host Configuration Changes to the Storage System

WhenyouuseanAX100iorAX150istoragesystem,nohostagentperiodicallychecks thehostconfigurationandpusheschangestothestoragesystem.The axnaviserverutil cliutilityisusedtoupdatethechanges.Thisisamanual operationthatyoushouldperformasneeded.
EMC Symmetrix Storage Systems

ThefollowingsettingsarerequiredforESXServeroperationsontheSymmetrix networkedstoragesystem:

Commonserialnumber(C) Autonegotiation(EAN)enabled SCSI3(SC3)set(enabled) Uniqueworldwidename(UWN) SPC2(Decal)(SPC2)SPC2flagisrequired
YouuseEMCsoftwaretoconfigurethestoragesystem.Forinformation,seeyourEMC documentation. NOTETheESXServerhostconsidersanyLUNsfromaSymmetrixstoragesystemthat haveacapacityof50MBorlessasmanagementLUNs.TheseLUNsarealsoknownas pseudoorgatekeeperLUNs.TheseLUNsappearintheEMCSymmetrixManagement Interfaceandshouldnotbeusedtoholddata.
68
VMware, Inc.
HP StorageWorks Storage Systems

ThissectionincludesconfigurationinformationforHPStorageWorksstoragesystems. Foradditionalinformation,seethesectiononVMwareESXServeratthe HP ActiveAnswersWebsite.
HP StorageWorks MSA
Thissectiondescribesthesetupandconfigurationstepsneededtoallowan HP StorageWorksMSA1510istoragesystemtocommunicatewithinanESX Server environment. To enable MSA1510i storage systems to communicate with ESX Server 1 2 Install,connect,andpowerupthenetworkdevicesasdetailedinthevendor installationdocument. ObtaintheIPaddressassignedtotheMSA1510icontrollermanagementport. a b 3 4 ScrollthroughthemessagesontheLCDpaneluntilthefollowingmessage appears:603PortMA0IP<address> RecordthemanagementportIPaddressthatappearsinBasicMSA1510i information.
FromtheserveroraworkstationontheMSA1510iLANsegment,openaWeb browserandentertheaddressobtainedinStep 2. Whenprompted,enterthefollowingdefaultaccesspermissions: Username:root Password:root
5 6
Whenprompted,setauniqueusernameandpassword. Usingthewizard,completethefollowingactions: a Storageconfiguration i ii b SettheFaultTolerantmode(RAIDmode). AssignasparediskforappropriateRAIDlevel.
iSCSIconfiguration(configureaniSCSIportal) i ii Selectadataport. AssignanIPaddresstothedataport.
VMware, Inc.
69
iii
VLANsaresetupontheswitchandareusedasonemethodof controllingaccesstothestorage.IfyouareusingVLANs,entertheVLAN IDtouse(0=notused).
iv ThewizardsuggestsadefaultiSCSITargetNameandiSCSITargetAlias. Acceptthedefaultorenteruserdefinedvalues.
NOTEToconfiguretheremainingdataports,completetheInitialSystem ConfigurationWizardprocess,andthenusetasksavailableonthe Configuretab. c d 7 LoginSettings ManagementSettings
ClickFinishtoapplytheconfigurationsettings.
NOTEWizardsareavailableforbasicconfigurationtasksonly.UsetheManageand Configuretabstoviewandchangeyourconfiguration. Afterinitialsetup,performthefollowingtaskstocompletetheconfiguration:

Createanarray. Createalogicaldrive. Createatarget. Createaportalgroup. Associateorassigntheportalscreatedusingthewizardwiththeportalgroup created. Maplogicaldrivestothetarget. Addinitiators(initiatorIQNnameandalias). UpdatetheACLsofthelogicaldrivestoprovideaccesstoinitiators(selectthelist ofinitiatorstoaccessthelogicaldrive).
SeetheMSA1510iConfigurationGuide.
70
VMware, Inc.
HP StorageWorks EVA
ThetwotypesofHPStorageWorksEVAsystemsareEVA_GL,anactivepassive system,andEVA_XL,anactiveactivesystem. SettheconnectiontypetoCustomwhenyoupresentaLUNtoanESXServerhost. The valueisoneofthefollowing:
ForHPEVAgl3000/5000(activepassive),usethe000000002200282E hostmode type. ForHPEVAglfirmware4.001(activeactivefirmwareforGLseries)andabove,use theVMware hostmodetype. ForEVA4000/6000/8000activeactivearrayswithfirmwareearlierthan5.031,use the000000202200083E hostmodetype. ForEVA4000/6000/8000activeactivearrayswithfirmware5.031andlater,usethe VMware hostmodetype.
Otherwise,EVAsystemsdonotrequirespecialconfigurationchangestoworkwithan ESXServersystem. Formoredetails,checkVMwareInfrastructure3,HPStorageWorksBestPracticesatthe HPWebsite.
Network Appliance Storage Systems

ThissectiondescribestheissuesandstepsassociatedwithallowingaNetwork AppliancestoragesystemtocommunicatewithinanESXServerenvironment.
Multipathing
WhenyousetupmultipathingbetweentwoQLogicHBAsandmultipleportsona NetworkAppliancestoragesystem,givethetwoQLogicHBAsdifferentdynamicor staticdiscoveryaddressestoconnecttothestorage. TheNetworkAppliancestoragesystemonlypermitsoneconnectionforeachtarget andeachinitiator.Attemptstomakeadditionalconnectionscausethefirstconnection todrop.Therefore,singleQLogicHBAsshouldnotattempttoconnecttomultiple IP addressesassociatedwiththesameNetworkAppliancetarget.
VMware, Inc.
71
Setting LUN Type and Initiator Group Type

SettheappropriateLUNtypeandinitiatorgrouptypeforthestoragesystem:

LUNtypeVMware(ifVMwaretypeisnotavailable,useLinux). InitiatorgrouptypeVMware(ifVMwaretypeisnotavailable,useLinux).
Provisioning Storage
Youmustprovisionstorage,usingeitherFilerVieworCLI. To provision storage by using FilerView storage management 1 2 LogintoNetworkAppliancestoragesystemmanagement(FilerView). Createavolume. a b c d e 3 SelectVolumesandclickAdd.ClickNext. SelectFlexibility(Default)orTraditional,thenclickNext. EnteraVolumeName,selectaLanguage,andclickNext. EntervaluesforContainingAggregate,TotalVolumeSize,andSpace GuaranteeandclickNext. ClickCommittocreatethevolume.
CreateLUNs. a b SelectLUNsandclickAdd. Enterthefollowing: i ii iii iv Path:Enterapath,forexample,/vol/vol1/lun1. LUNProtocolType:VMware. Description:Abriefdescription. SizeandUnit:Enterasize,forexample,10GBandselectSpaceReserved.
Createaninitiatorgroup. a b SelectLUNs>InitiatorGroupandclickAdd. Enterthefollowing: i ii GroupName:Enteragroupname Type:ChooseiSCSI.
72
VMware, Inc.
iii iv c 5
OperatingSystem:EnterVMware Initiators:Enterfullyqualifiedinitiatornames.Ifthereismorethanone initiator,eachinitiatorhastobeseparatedwithareturncarriage.
ClickAdd.
MaptheLUNtotheinitiatorgroup. a b c d e SelectLUNsandclickManage.ALUNslistappears. Fromthislist,clickthelabelontheMapsrowforthespecificLUNs. ClickAddGroupstoMap. SelecttheinitiatorgroupandclickAdd. Whenprompted,entertheLUNID(anynumberfrom0to255)andclick Apply.
YoucanalsoprovisionthestorageusingaCLI. To provision storage by using a CLI 1 UseaCLItocreateanaggregateifrequired.

aggr create <vmware-aggr> <number of disks>
Createaflexiblevolume.
vol create <aggregate name> <volume size>
CreateaQtreetostoreeachLUN.
qtree create <path>
CreateaLUN.
lun create -s <size> -t vmware <path>
Createaninitiatorgroup.
igroup create -f -t vmware <igroup name>
MaptheLUNtotheinitiatorgroupyoucreated.
lun map (<path>) <igroup name> <LUN ID>
VMware, Inc.
73
ForadditionalinformationonusingNetworkApplianceStoragewithVMware technology,seethefollowingNetworkAppliancedocuments:
NetworkAppliance&VMwareESXServer:InstantaneousBackup&Recovery withNetAppSnapshotTechnologyathttp://www.netapp.com/library/tr/3428.pdf. TechnicalCaseStudy:UsingaNetworkApplianceSANwithVMwaretoFacilitate StorageandServerConsolidationathttp://www.netapp.com/library/tr/3401.pdf.
EqualLogic Storage Systems

TosetupyourEqualLogicstoragesystemstoworkinanESXServerimplementation, youmustaddressthefollowingissues: Multipathing. NospecialsetupisneededbecauseEqualLogicstoragesystemssupport storageproccesorfailoverthatistransparenttoiSCSI.MultipleiSCSIHBAsor NICscanconnecttothesametargetorLUNonthestorageside.However,normal restrictionsonESXServerconfigurationapply,forexampleNICteamingmustbe correctlysetup. Creating iSCSI LUNs. FromtheEqualLogicwebportal,rightclickVolumes,and thenselectCreateVolume. Enable ARP redirection on hardware iSCSI HBAs. Formoreinformationon enablingARPredirection,seeesxcfghwiscsiUtilityonpage 126. FormoreinformationaboutconfiguringandusingEqualLogicstoragesystems,seethe vendorsdocumentation. NOTEYoumustaddtheserviceconsoleandVMkernelIPaddressesbeingusedfor iSCSIaccesstotheIPaddressACLonyourEqualLogicstoragesystems.Ifyouare usingCHAPauthenticationorinitiatornamebasedACLs,however,thisisnot necessary.
74
VMware, Inc.
LeftHand Networks SAN/iQ Storage Systems

SAN/iQSANssupportESXServeriSCSIconnectionsfromasoftwareinitiatorand HBAs.
Basic Configuration
Basicconfigurationstepsinclude: 1 2 3 4 5 InstallSAN/iQstoragenodes. CreateSAN/iQmanagementgroupsandclusters. Createvolumes. Assignvolumestoauthenticationgroupsandvolumelists. EnableARPredirectiononhardwareiSCSIHBAs. SeeesxcfghwiscsiUtilityonpage 126.
Asabestpractice,configurevirtualIPloadbalancinginSAN/iQforallESXServer authenticationgroups.
Automatic Volume Resignaturing

EnableautomaticvolumeresignaturingforSAN/iQstoragedevicestoallowaccessto SAN/iQsnapshotsandremotecopies. FormoreinformationonconfiguringLeftHandNetworksSANsforVMware Infrastructure3,seetheVMwareFieldGuideforSAN/iQPoweredSANs.
VMware, Inc.
75
76
VMware, Inc.
Booting from a SAN with ESX Server Systems
Thischapterdiscussesthefollowingtopics:

BootingfromaSANOverviewonpage 77 EnablingBootingfromaSANonpage 79
Booting from a SAN Overview

YoucanbootfromaSANonlywithESXServer3andwithhardwareiSCSI.Whenyou setupyoursystemtobootfromaSAN,thebootimageisnotstoredontheESXServer systemslocaldisk,butinsteadisstoredonaSANLUN,asFigure 48shows. Figure 4-8. How Booting from a SAN Works
ESX Server service console VMkernel
hardware iSCSI initiator (HBA) LAN storage array
boot disk
VMware, Inc.
77
Benefits of Booting from a SAN

WhenbootingfromaSAN,theoperatingsystemisinstalledononeormoreLUNsin theSANstoragesystem.Theserversareinformedaboutthebootimagelocation.When theserversstart,theybootfromtheLUNsontheSANstoragesystem. NOTEWhenyoubootfromaSANinconjunctionwithaVMwareESXServersystem, eachservermusthaveitsownbootLUN. BootingfromaSANprovidesnumerousbenefits,including:
CheaperserversServerscanbemoredenseandruncoolerwithoutinternal storage. EasierserverreplacementYoucanreplaceserversandhavethenewserverpoint totheoldbootlocation. Lesswastedspace. EasierbackupprocessesThesystembootimagesintheSANcanbebackedup aspartoftheoverallSANbackupprocedures. ImprovedmanagementCreatingandmanagingtheoperatingsystemimageis easierandmoreefficient.
Deciding to Boot From a SAN

BeforeyouconsiderhowtosetupyoursystemforbootingfromaSAN,decidewhether itmakessenseforyourenvironment. BootfromaSAN:

Ifyoudonotwanttohandlemaintenanceoflocalstorage. Ifyouneedeasycloningofserviceconsoles. Indisklesshardwareconfigurations,suchasonsomebladesystems.
DonotbootfromaSANifyouriskI/Ocontentionbetweentheserviceconsoleand VMkernel.
78
VMware, Inc.
Chapter 4 Booting from a SAN with ESX Server Systems
Enabling Booting from a SAN

EnablingyourESXServerhosttobootfromaSANrequiresthecompletionofanumber oftasks. To enable booting from a SAN 1 2 Reviewanyvendorconfigurationrecommendationsthatapplytothestorage systemortheserverbootingfromSAN. Configurethehardwareelementsofyourstoragenetwork,including:

SANseePreparingtheSANonpage 79. HBAsseeConfiguringiSCSIHBAstoBootfromaSANonpage 81
ConfigureACLsonyourstoragesystem. ProperaccesscontrolonthestoragesystemsisimportantwhenanESXServerhost isbootingfromiSCSI.
BootLUNsshouldonlybevisibletotheserverusingthatLUNtoboot.No otherserverorsystemontheSANshouldbepermittedtoseethatbootLUN. MultipleESXServerhostscanshareadiagnosticpartition.ACLsonthe storagesystemscanallowyoutodothis.
Choosethelocationforthediagnosticpartition. DiagnosticpartitionscanbeputonthesameLUNasthebootpartition.Core dumpsarestoredindiagnosticpartitions.
SetupyourESXServertobootfromCDROMfirstbecausetheVMware installationCDisintheCDROMdrive. Toachievethis,changethesystembootsequenceinyoursystemBIOSsetup.
Preparing the SAN

BeforeyouconfiguretheiSCSIHBAstobootfromaSAN,firstprepareyourstorage areanetworkbycheckingthecablingandswitchwiringandconfiguringthestorage system.
VMware, Inc.
79
To prepare the SAN 1 2 Connectnetworkcables,referringtoanycablingguidethatappliestoyoursetup. EnsureIPconnectivitybetweenyourstoragesystemandserver. Thisincludesproperconfigurationofanyroutersorswitchesonyourstorage network.StoragesystemsmustbeabletopingtheiSCSIHBAsinyourESXServer hosts. Configurethestoragesystem: a b Createavolume(orLUN)onthestoragesystemforESXServertobootfrom. ConfigurethestoragesystemsothattheESXServersystemhasaccesstothe assignedLUN.ThiscouldinvolveupdatingACLswiththeIPaddresses,iSCSI IQNnames,andtheCHAPauthenticationparameteryouuseonthe ESX Serversystem.Onsomestoragesystems,inadditiontoprovidingaccess informationfortheESXServerhost,youmustalsoexplicitlyassociatethe assignedLUNwiththehost. EnsurethattheLUNispresentedtotheESXServersystemasLUN0.(On storagesystemsthatpresentvolumesasmultipletargetsratherthanmultiple LUNS,thevolumesarealwayspresentedasLUN0). EnsurethatnoothersystemhasaccesstotheconfiguredLUN. RecordtheiSCSI(IQN)nameandIPaddressesofthetargetsassignedtothe ESXServerhost.YouneedthisinformationtoconfigureyouriSCSIHBA. CAUTIONIfyouusescriptedinstallationtoinstallESXServerwhenbooting fromaSAN,youneedtotakespecialstepstoavoidunintendeddataloss.See VMwareknowledgebasearticle1540at http://www.vmware.com/support/kb/enduser/std_adp.php?p_faqid=1540.
d e
80
VMware, Inc.
Configuring iSCSI HBAs to Boot from a SAN

ThissectiondiscusseshowtoconfigureaQLogiciSCSIHBAforbootingfromaSAN. OnasystemsetuptobootfromaSAN:

ThesystemBIOSmustdesignatetheiSCSIcardasthebootcontroller. TheBIOSmustbeenabledontheiSCSIHBAtolocatethetargetbootLUN.
To enable the QLogic iSCSI HBA to Boot from a SAN 1 2 DuringserverPOST,pressCrtl+qtoentertheQLogiciSCSIHBAconfiguration menu. SelecttheI/Oporttoconfigure. Bydefault,theAdapterBootmodeissettoDisable.
ConfiguretheHBA. a b FromtheFast!UTILOptionsmenu,selectConfigurationSettings>Host AdapterSettings. Configurethefollowingsettingsforyourhostadapter:initiatorIP address, subnetmask,gateway,initiatoriSCSIname,andCHAP(ifrequired).
VMware, Inc.
81
ConfigureiSCSIBootSettings. a b FromtheFast!UTILOptionsmenu,selectConfigurationSettings>iSCSI BootSettings. BeforeyoucansetSendTargets,setAdapterBootmodetoManual.
SelectPrimaryBootDeviceSettings. i ii EnterthediscoveryTargetIPandTargetPort. YoucanleavetheBootLUNandiSCSINamefieldsblankifonlyone iSCSItargetandoneLUNareatthespecifiedaddresstobootfrom. Otherwise,youmustspecifythesefieldstoensurethatyoudonotboot fromavolumeforsomeothersystem.Afterthetargetstoragesystemis reached,thesefieldswillbepopulatedafterarescan. Savechanges.
iii
82
VMware, Inc.
d e
FromtheiSCSIBootSettingsmenu,selecttheprimarybootdevice.Anauto rescanoftheHBAismadetofindnewtargetLUNS. SelecttheiSCSItarget.
NOTEIfmorethenoneLUNexistswithinthetarget,youcanchooseaspecific LUNIDbypressingEnterafteryoulocatetheiSCSIdevice.
VMware, Inc.
83
ReturntothePrimaryBootDeviceSettingmenu.Aftertherescan,theBoot LUNandiSCSINamefieldsarepopulated.ChangethevalueofBootLUNto thedesiredLUNID.
Saveyourchangesandrestartthesystem.
FormoreinformationandmoreuptodatedetailsaboutQLogichostadapter configurationsettings,seetheQLogichostadapterreadmefileattheQLogicwebsite.
84
VMware, Inc.
5\
Managing ESX Server Systems That Use SAN Storage
ThischaptercanhelpyoumanageyourESXServersystem,useSANstorageeffectively, andperformtroubleshooting. Thischapterdiscussesthefollowingtopics:

IssuesandSolutionsonpage 86 GettingInformationonpage 87 ResolvingDisplayIssuesonpage 89 AdvancedLUNDisplayConfigurationonpage 92 Multipathingonpage 94 PathFailoveronpage 101 VMkernelConfigurationonpage 102 AvoidingandResolvingSANProblemsonpage 103 OptimizingSANStoragePerformanceonpage 103 ResolvingPerformanceIssuesonpage 108 SANStorageBackupConsiderationsonpage 113 LayeredApplicationsonpage 115 VMFSVolumeResignaturingonpage 117
VMware, Inc.
85
Issues and Solutions

Table 51liststheissuesthataremostfrequentlyencounteredandeitherexplainshow toresolvethemorpointstothelocationwheretheissueisdiscussed. Table 5-1. Issues and Solutions
Issue ALUNisnotvisibleintheVIClient. Understandhowpathfailoverisperformedor changehowpathfailoverisperformed. Vieworchangethecurrentmultipathing policyorpreferredpath,ordisableorenablea path. IncreasetheWindowsdisktimeouttoavoid disruptionduringfailover. TheservercannotaccessaLUN,oraccessis slow. YouaddedanewLUNoranewpathto storageandwanttoseeitintheVIClient. Solution SeeResolvingDisplayIssuesonpage 89. TheVIClientallowsyoutoperformthese actions.SeeMultipathingonpage 94. TheVIClientallowsyoutoperformthese actions.SeeMultipathingonpage 94. SeeSettingGuestOperatingSystem Timeoutonpage 101. Paththrashingmightbetheproblem.See ResolvingPathThrashingonpage 109. Rescan.SeeUsingRescanonpage 90.
Guidelines for Avoiding SAN Problems

FollowtheseguidelinestoavoidpotentialproblemswithyourSANconfiguration:
PlaceonlyoneVMFSvolumeoneachLUN.MultipleVMFSvolumesononeLUN isnotrecommended. Donotchangethepathpolicythesystemsetsforyouunlessyouunderstandthe implicationsofmakingsuchachange.Inparticular,workingwithan activepassivearrayandsettingthepathpolicytoFixedcanleadtopaththrashing.
86
VMware, Inc.
Chapter 5 Managing ESX Server Systems That Use SAN Storage
Getting Information
ThissectionexplainshowtofindinformationaboutHBAs,status,multipathing,andso on.Ifyouexperienceproblemswhenperformingthesetasks,seeResolvingDisplay Issuesonpage 89.
Viewing HBA Information

UsetheVIClienttodisplayallavailablestorageadaptersandtheirinformation. To see a list of HBA types 1 SelectthehosttoseetheHBAsofandclicktheConfigurationtab. YoucanviewalistofallstoragedevicesfromtheSummarytab.However,you cannotseedetailsormanagethedevicefromthere. 2 IntheHardwarepanel,chooseStorageAdapters. Thelistofstorageadaptersappears.Youcanselecteachadapterforadditional information.
VMware, Inc.
87
Viewing Datastore Information

UsetheVIClienttodisplayallformatteddatastoresandreviewdetailsaboutaspecific datastore. To view all storage devices and details about them 1 SelectthehosttoseethestoragedevicesofandclicktheConfigurationtab. NOTETheServiceConsoleResourceslinkundertheSoftwareheadingappears forESXServer3only.ESXServer3idoesnotprovideaserviceconsole. 2 IntheHardwarepanel,chooseStorage. Thelistofdatastores(volumes)appearsintheStoragepanel.Thedisplayshows thewholeVMFSfortheselectedhost.OnlystoragethatwasformattedwithVMFS isincludedinthedisplay.
Toviewdetailsaboutanydatastore,selectit. TheDetailspaneldisplaysadditionalinformation.Thisincludesthelocationand capacity,numberofpaths,pathpolicy,andproperties.Italsoincludesextent information.AnextentisaVMFSformattedpartition(apieceofaLUN).For example,vmhba0:6:0isaLUN,andvmhba0:6:0:1isapartition.OneVMFS volumecanhavemultipleextents. NOTETheabbreviationvmhbareferstotheHBAontheESXServersystem,not totheSCSIcontrollerthatthevirtualmachinesuse.
4
88
ClickPropertiestoviewandchangeproperties.
VMware, Inc.
Resolving Display Issues

Thissectiondiscusseshowtotroubleshootcommonstatusandvisibilityissues. IfyouareusinganAX100iorAX150istoragesystem,inactiveconnectionscancause displayproblems.
Understanding LUN Naming in the Display

IntheVIClient,aLUNappearsasasequenceofthreeorfournumbers,separatedby colons:
<SCSI HBA>:<SCSI target>:<SCSI LUN>:<disk partition>
Ifthelastnumberis0ornotdisplayed,thenamereferstotheentireLUN. ThefirstthreenumbersinanESXdevicenamemightchange,butstillrefertothesame physicaldevice.Forexample,vmhba1:2:3representsSCSILUN3,attachedtoSCSI target2,onSCSIHBA1.WhentheESXServersystemisrebooted,thedevicenamefor LUN3mightchangetovmhba1:1:3.Thenumbershavethefollowingmeaning:
Thefirstnumber,theSCSIHBA,changesifaniSCSInetworkoutageoccursatthe timethesystemisbootedorrescannedandESXisrequiredtoaccessthephysical deviceoveradifferentSCSIHBA. Thesecondnumber,theSCSItarget,changesifachangeoccursinthemappingsin theiSCSItargetsvisibletotheESXServerhost. Thethirdnumber,theSCSILUN,neverchanges.
Resolving Issues with LUNs That Are Not Visible

YoucanusetheVIClienttoviewLUNs. Ifthedisplay(oroutput)differsfromwhatyouexpect,checkthefollowing: Cable connectivity. Ifyoudonotseeaport,theproblemcouldbecableconnectivity orrouting.Checkthecablesfirst.Ensurethatcablesareconnectedtotheportsanda linklightindicatesthattheconnectionisgood.Ifeachendofthecabledoesnotshowa goodlinklight,replacethecable. Routing. ControlsconnectivitybetweendifferentsubnetsonyourEthernet configuration.IfyourESXServersystemandiSCSIstoragearenotonthesamesubnet, ensurethatappropriateroutingexistsbetweenthesubnets.Also,ensurethatthesubnet maskandgatewayaddressaresetcorrectlyontheiSCSIstorageandtheiSCSIinitiator intheESXServerhost.
VMware, Inc.
89
Access Control. IftheexpectedLUNsdonotappearafterrescan,accesscontrol mightnotbeconfiguredcorrectlyonthestoragesystemside:
IfCHAPisconfigured,ensurethatitisenabledontheESXServerhostand matchesthestoragesystemsetup. IfIPbasedfilteringisused,ensurethattheiSCSIHBAortheVMkernelport groupIPaddressandserviceconsoleIPaddressareallowed. Ifyouareusinginitiatornamebasedfiltering,ensurethatthenameisa qualifiediSCSInameandmatchesthestoragesystemsetup.
ForbootingfromaSAN,ensurethateachESXServerhostseesonlyrequired LUNs.DonotallowanyESXServerhosttoseeanybootLUNotherthanitsown. UsestoragesystemsoftwaretomakesurethattheESXServerhostcanseeonlythe LUNsthatitissupposedtosee. EnsurethattheDisk.MaxLUNandDisk.MaskLUNssettingsallowyoutoview theLUNyouexpecttosee.SeeChangingtheNumberofLUNsScannedbyUsing Disk.MaxLUNonpage 92andMaskingLUNsbyUsingDisk.MaskLUNson page 93. Storage processor. Ifastoragesystemhasmorethanonestorageprocessor,make surethattheSANswitchhasaconnectiontotheSPthatownstheLUNsyouwantto access.Onsomestoragesystems,onlyoneSPisactiveandtheotherSPispassiveuntil afailureoccurs.IfyouareconnectedtothewrongSP(theonewiththepassivepath) youmightnotseetheexpectedLUNs,oryoumightseetheLUNsbutgeterrorswhen tryingtoaccessthem. Software iSCSI Network Configuration. ThesoftwareiSCSIinitiatorinESXServer requiresthataserviceconsolenetworkportandaVMkernelnetworkporthaveaccess totheiSCSIstorage.ThesoftwareinitiatorusestheserviceconsoleforiSCSIdiscovery anderrorhandling.ItusestheVMkernelfordatatransferbetweentheESXServer systemandtheiSCSIstorage.SeeNetworkingConfigurationforSoftwareiSCSI Storageonpage 50.
Using Rescan
Performarescaneachtimeyou:

CreatenewLUNsonaSAN. ChangetheLUNmaskingonanESXServerhoststoragesystem. Reconnectacable.
90
VMware, Inc.
Makeachangetoahostinacluster. ChangeCHAPsettingsoraddnewdiscoveryaddresses.
NOTEDonotrescanwhenapathisunavailable.Ifonepathfails,theothertakesover andyoursystemcontinuestobefullyfunctional.If,however,yourescanatatime whenapathisnotavailable,theESXServerhostmightremovethepathfromitslistof pathstothedevice.ThepathcannotbeusedbytheESXServerhostuntilthenexttime arescanisperformedwhilethepathisactive. To perform a rescan 1 2 IntheVIClient,selectahostandclicktheConfigurationtab. IntheHardwarepanel,chooseStorageAdaptersandclickRescanabovethe StorageAdapterspanel. YoucanalsorightclickanindividualadapterandclickRescantorescanjustthat adapter.
Removing Datastores
UsingtheVIClient,youcanremoveadatastorefrombeingusedasstorageforvirtual machines.YoucannotremovetargetsinsoftwareiSCSI,exceptbyrebooting. To remove a datastore 1 2 3 4 Turnoffallvirtualmachinesthatusethedatastorebeingremoved. Selectandremoveeachvirtualmachinefromtheinventorybyrightclickingthe virtualmachineandclickingRemovefromInventory. ClicktheConfigurationtabandclickStoragetodisplayallstoragedevices. SelectthedatastoretoremoveandclickRemove.
VMware, Inc.
91
Atthispointthedatastoreisremovedandtheinventoryshouldberefreshed automatically. NOTEAfteryouremoveadatastorefromanESXServerhost,maskorremovetheLUN fromthestoragesystemandrescanwiththeVIClienttopreventESXServerfrom discoveringtheLUN.
Advanced LUN Display Configuration

Thissectiondiscussesanumberofadvancedconfigurationoptions.
Changing the Number of LUNs Scanned by Using Disk.MaxLUN

Bydefault,theVMkernelscansforLUN0toLUN255foreverytarget(atotalof256 LUNs).YoucanchangetheDisk.MaxLunparametertochangethisnumber.This changemightimproveLUNdiscoveryspeed. NOTEYoucannotdiscoverLUNswithaLUNIDnumberthatisgreaterthan255. Reducingthevaluecanshortenrescantimeandboottime.ThetimetorescanLUNs dependsonseveralfactors,includingthetypeofstoragesystemandwhethersparse LUNsupportisenabled.SeeChangingSparseLUNSupportbyUsing DiskSupportSparseLUNonpage 94. To change the value of Disk.MaxLUN 1 2 IntheVIClientinventorypanel,selectthehost,clicktheConfigurationtaband clickAdvancedSettings. SelectDisk.
92
VMware, Inc.
ScrolldowntoDisk.MaxLUN,changetheexistingvaluetothevalueofyour choice,andclickOK.
Masking LUNs by Using Disk.MaskLUNs

TheDisk.MaskLUNsparameterallowsyoutomaskspecificLUNsonspecificHBAs. MaskedLUNsarenottouchedoraccessiblebytheVMkernel,evenduringinitial scanning. UsethisoptiontopreventtheESXServersystemfromaccessingsomeiSCSILUNs withoutthestoragesystemsmaskingmechanisms. To change the value of Disk.MaskLUNs 1 2 3 IntheVIClientinventorypanel,selectthehost,clicktheConfigurationtaband clickAdvancedSettings. SelectDisk. ScrolldowntoDisk.MaskLUNs,changetheexistingvaluetothevalueofyour choice,andclickOK.
CAUTIONIfatarget,LUN,orvmhbanumberchangesbecauseofaserverorSAN reconfiguration,theincorrectLUNmightbemaskedorexposed.
VMware, Inc.
93
Changing Sparse LUN Support by Using DiskSupportSparseLUN

Bydefault,theVMkernelisconfiguredtosupportsparseLUNs,thatis,acasewhere someLUNsintherange0toN1arenotpresent,butLUNNispresent.Forexample, ifastoragesystempresentsLUNsnumbering0,6,and23,butnoLUNswithanyofthe numbersbetweenthese. IfallLUNsaresequential,youcanchangetheDisk.SupportSparseLUNparameter. ThischangedecreasesthetimeneededtoscanforLUNs. To change the value of Disk.SupportSparseLUN 1 2 3 IntheVIClientinventorypanel,selectthehost,clicktheConfigurationtab,and clickAdvancedSettings. IntheAdvancedSettingsdialogbox,selectDisk. ScrolldowntoDisk.SupportSparseLUN,changethevalueto0,andclickOK.
Multipathing
SANimplementationswithahighnumberofLUNsandpathstothoseLUNscancause ESXServertorunoutofresourcesbeforeallofthepathsareenumerated.Thisscenario preventsESXServerfromseeingallofthepathstothestorage.Toavoidthissituation, reducethepathcounttotheLUNs. Foranintroductiontomultipathingconcepts,seePathManagementandFailoveron page 32.
Viewing the Current Multipathing State

YoucanusetheVIClienttoviewthecurrentmultipathingstate. To view the current multipathing state 1 2 IntheVIClientinventorypanel,selectahostandclicktheConfigurationtab. IntheStoragepanel,selectoneofthedatastores.
94
VMware, Inc.
InformationaboutthatdatastoreappearsintheDetailspanel.
3 4
Toviewadditionalinformation,ortochangethemultipathingpolicy,select PropertiesabovetheDetailspanel. IntheExtentspanel,selecttheextenttovieworchangeinformation. TheExtentDevicepaneldisplaysinformationabouttheextent,thepathselection algorithm,theavailablepaths,andtheactivepath.
Thedisplayincludesinformationonthestatusofeachpathtothedeviceextent. Thefollowingpathinformationappears:
ActiveThepathisworkingandisthecurrentpathbeingusedtotransfer data. DisabledThepathisdisabledandnodatacanbetransferred.

95
VMware, Inc.
StandbyThepathisworkingbutisnotcurrentlyusedfordatatransfer. DeadThesoftwarecannotconnecttothediskthroughthispath.
IfyouareusingpathpolicyFixedandwanttoseewhichpathisthePreferredpath, clickManagePaths. Thepreferredpathismarkedwithanasterisk(*)inthefourthcolumn.
Table 52summarizeshowthebehaviorofanESXServersystemchanges,depending onthetypeofstoragesystemandthefailoverpolicy. Table 5-2. Path Policy Effects

Policy/Controller MostRecentlyUsed Fixed Active-Active Administratoractionisrequired tofailbackafterpathfailure. VMkernelresumesusingthe preferredpathwhenconnectivity isrestored. Active-Passive Administratoractionisrequired tofailbackafterpathfailure. VMkernelattemptstoresume usingthepreferredpath.Thiscan causepaththrashingorfailure becauseanotherSPnowownsthe LUN.SeeResolvingPath Thrashingonpage 109.
NOTEiSCSIstoragesystemsthatprovidetransparentfailoverbehaveasactiveactive.
96
VMware, Inc.
Active Paths
ESXServerdoesnottypicallyperformI/OloadbalancingacrosspathsforagivenLUN. Atanyonetime,onlyasinglepathisusedtoissueI/OtoagivenLUN.Thispathis knownastheactivepath.
IfthepathpolicyofaLUNissettoFixed,ESXServerselectsthepathmarkedas Preferredastheactivepath. Ifthepreferredpathisdisabledorunavailable,theESXServersystemusesan alternativeworkingpathastheactivepath.
IfthepathpolicyofaLUNissettoMostRecentlyUsed,theESXServerhostselects anactivepathtotheLUNthatpreventspaththrashing.Thepreferredpath designationisnotconsidered.
NOTEInsomeSANterminology,thetermactivemeansanypaththatisavailablefor issuingI/OtoaLUN.FromtheESXServerhostspointofview,thetermactivemeans theoneandonlypaththattheESXServerhostisusingtoissueI/OtoaLUN.
Setting a LUN Multipathing Policy

Thefollowingmultipathingpoliciesarecurrentlysupported:
FixedTheESXServerhostalwaysusesthepreferredpathtothediskwhenthat pathisavailable.Ifitcannotaccessthediskthroughthepreferredpath,ittriesthe alternativepaths.Fixedisthedefaultpolicyforactiveactivestoragedevices. MostRecentlyUsedTheESXServerhostusesthemostrecentpathtothedisk untilthispathbecomesunavailable.Thatis,theESXServerhostdoesnot automaticallyrevertbacktothepreferredpath.MostRecentlyUsedisthedefault policyforactivepassivestoragedevicesandisrequiredforthosedevices. RoundRobinTheESXServerhostusesanautomaticpathselectionrotating throughallavailablepaths.Inadditiontopathfailover,roundrobinsupportsload balancingacrossthepaths. NOTERoundrobinloadbalancingisexperimentalandnotsupportedfor productionuse.Formoreinformation,seetheRoundRobinLoadBalancingwhite paper.
VMware, Inc.
97
TheESXServerhostsetsthemultipathingpolicyaccordingtothemakeandmodelof thestoragesystemitdetects.Ifthedetectedstoragesystemisnotsupported,itis treatedasactiveactive.Foralistofsupportedstoragesystems,seetheStorage/SAN CompatibilityGuide. NOTEVMwarerecommendsthatyounotchangeMostRecentlyUsedtoFixed. The systemsetsthispolicyforthestoragesystemsthatrequireit. To set the multipathing policy by using a VI Client 1 2 3 4 5 6 IntheVIClientinventorypanel,selectthehostandclicktheConfigurationtab. IntheHardwarepanel,selectStorage. Selectthedatastoretochangethemultipathingpolicyfor,andclickPropertiesin theDetailspanel. IntheExtentpanel,selectthedevicetomakethechangefor,andclickManage PathsintheExtentDevicepanelontheright. IntheManagePathsdialogbox,clickChange. SelectthemultipathingpolicyinthedialogboxthatappearsandclickDone.
NOTEForactivepassivestoragedevices,VMwarerecommendsMostRecentlyUsed.
Disabling and Enabling Paths

Youcantemporarilydisablepathsformaintenanceorotherreasons.Youcandoso usingtheVIClient. To disable or enable a path 1 2 3 IntheVIClientinventorypanel,selectthehostandclicktheConfigurationtab. IntheHardwarepanel,selectStorage. Selectthedevicetodisableapathfor,andclickPropertiesintheDetailspanel.
98
VMware, Inc.
4 5
IntheExtentpanel,selectthedevicetomakethechangefor,andclickManage PathsintheExtentDevicepanelontheright. ClickChangeinthePathspaneloftheManagePathsdialogboxandclick DisabledtodisablethepathorEnabledtoenablethepath.
Setting the Preferred Path (Fixed Path Policy Only)

IfyouareusingaFixedpathpolicy,theserveralwaysusesthepreferredpathwhen available. To set the preferred path 1 2 3 4 5 6 IntheVIClientinventorypane,selectthehostandclicktheConfigurationtab. IntheHardwarepanel,selectStorage. SelectthedevicetosetapreferredpathforandclickPropertiesintheDetails panel. IntheExtentpanel,selectthedevicetomakethechangeto,andclickManage PathsintheExtentDevicepanelontheright. SelectthepathtomakethepreferredpathforandclickChange. InthePreferencepane,clickPreferred. IfPreferredisnotanoption,makesurethatthepathpolicyisFixed. 7 ClickOKandclickOKagaintoexitthedialogboxes.
VMware, Inc.
99
Path Management and Manual Load Balancing

Balancingloadsamongavailablepathsimprovesperformance.Youcanconfigureyour systemtousedifferentpathstodifferentLUNsbychangingthepreferredpathforthe HBAs.YoucandothisonlyforactiveactiveSPs,andrequiresthatyouhavethepath policysettoFixed. Ifapathfails,thesurvivingpathscarryallthetraffic.Pathfailovermighttakeaminute ormore,becausethenetworkmightconvergewithanewtopologytotrytorestore service.ThisdelayisnecessarytoallowtheSANtostabilizeitsconfigurationafter topologychangesorothernetworkevents. Thefollowingexampledemonstrateshowmanualloadbalancingisperformed. Whenyouuseanactiveactivearray,youcansetupyoursystemforloadbalancing. Assumethefollowingsetup,showninFigure 59:

ActiveActiveSPs AnESXServersystem TwoiSCSIHBAs
Figure 5-9. Manual Load Balancing

ESX Server HBA1 HBA2
IP network
SP1
SP2
storage array
Forloadbalancing,setthepreferredpathsasfollows.

LUN1:vmhba1:1:1 LUN2:vmhba2:1:2 LUN3:vmhba1:2:3 LUN4:vmhba2:2:4
100
VMware, Inc.
Path Failover
PathfailoverreferstosituationswhentheactivepathtoaLUNischangedfromone pathtoanother,usuallybecauseofsomeSANcomponentfailurealongthecurrent path.AserverusuallyhasoneortwoHBAsandeachHBAseesoneortwostorage processorsonagivenSANstoragesystem.Youcandeterminetheactivepaththe paththattheservercurrentlyusesbylookingattheLUNsproperties. Figure 5-10. Active and Standby Paths
Active and standby paths
Whenanetworkcableispulled,I/Omightpausefor30to60secondsuntiltheiSCSI driverdeterminesthatthelinkisunavailableanduntilfailoveroccurs.Asaresult,the virtualmachines(withtheirvirtualdisksinstalledonSANstorage)canappear unresponsive.Ifyouattempttodisplaythehost,itsstoragedevices,oritsadapter,the operationmightappeartostall.Afterfailoveriscomplete,I/Oresumesnormally. Incaseofmultiplebreakages,allconnectionstoSANstoragedevicesmightbelost. If noneoftheconnectionstothestoragedeviceisworking,somevirtualmachines mightencounterI/OerrorsontheirvirtualSCSIdisks. Setting Guest Operating System Timeout Youmightwanttoincreasethestandard disktimeoutvaluesothataWindowsguestoperatingsystemisnotextensively disruptedduringfailover.
VMware, Inc.
101
ForWindows2000andWindowsServer2003guestoperatingsystems,youcanset operatingsystemtimeoutbyfusingtheregistry. To set operating system timeout for Windows servers 1 2 3 4 BackupyourWindowsregistry. SelectStart>Run,typeregedit.exeandclickOK. Intheleftpanelhierarchyview,doubleclickHKEY_LOCAL_MACHINE,then System,thenCurrentControlSet,thenServices,andthenDisk. SelecttheTimeOutValueandsetthedatavaluetox03c(hexadecimal)or 60 (decimal). Afteryoumakethischange,Windowswaitsatleast60secondsfordelayeddisk operationstocompletebeforeitgenerateserrors. 5 ClickOKtoexittheRegistryEditor.
VMkernel Configuration
WhenyouinstallyourESXServersystem,decidewheretoplacedifferentstorage elementssuchastheroot(/)and/bootpartitionsoftheserviceconsole(ESXServer3 only).
Sharing Diagnostic Partitions

WhenyouuseahardwareiSCSIinitiator,ifyourESXServerhosthasalocaldisk,that diskismostappropriatelyusedforthediagnosticpartition.Onereasonisthatifan issuewithremotestoragecausesacoredump,thecoredumpislostandresolvingthe issuebecomesmoredifficult. However,fordisklessserversthatbootfromaSAN,multipleESXServersystemscan shareonediagnosticpartitiononaSANLUN.IfmorethanoneESXServersystemis usingaLUNasadiagnosticpartition,thatLUNmustbeconfiguredsothatallthe serverscanaccessit. Eachserverneeds100MBofspace,sothesizeoftheLUNdetermineshowmanyservers canshareit.EachESXServersystemismappedtoadiagnosticslot.VMware recommendsatleast16slots(1600MB)ofdiskspaceifserversshareadiagnostic partition. Ifonlyonediagnosticslotisonthedevice,allESXServersystemssharingthatdevice maptothesameslot.Thiscaneasilycreateproblems.IftwoESXServersystems performacoredumpatthesametime,thecoredumpsareoverwrittenonthelastslot onthediagnosticpartition.
102 VMware, Inc.
Ifyouallocateenoughmemoryfor16slots,itisunlikelythatcoredumpsaremapped tothesamelocationonthediagnosticpartition,eveniftwoESXServersystems performacoredumpatthesametime.
Avoiding and Resolving SAN Problems

ThissectiongivessometipsforavoidingandresolvingproblemswithyourSAN configuration:
Documenteverything.Includeinformationaboutconfiguration,accesscontrol, storage,switch,serverandiSCSIHBAconfiguration,softwareandfirmware versions,andstoragecableplan. Planforfailure:
Makeseveralcopiesofyourtopologymaps.Foreachelement,considerwhat happenstoyourSANiftheelementfails. Crossoffdifferentlinks,switches,HBAsandotherelementstoensureyoudid notmissacriticalfailurepointinyourdesign.
CAUTIONTheinstallerletsyoueraseanyaccessibledisks,includingSANLUNs inusebyotherservers.
EnsurethattheiSCSIHBAsareinstalledinthecorrectslotsintheESXServerhost, basedonslotandbusspeed.BalancePCIbusloadamongtheavailablebussesin theserver. Becomefamiliarwiththevariousmonitorpointsinyourstoragenetwork,atall visibilitypoints,includingESXServerperformancecharts,Ethernetswitch statistics,andstorageperformancestatistics.
Optimizing SAN Storage Performance

ThemajorfactorsforoptimizingatypicaliSCSIenvironmentarestoragesystem performance,serverperformanceandnetworkperformance.Ifthenetwork environmentisproperlyconfigured,theiSCSIcomponentsshouldprovideadequate throughputandlowenoughlatencyforiSCSIinitiatorsandtargets.Ifthenetworkis congestedandlinks,switchesorroutersaresaturated,iSCSIperformancesuffersand mightnotbeadequateforESXServerenvironments.
VMware, Inc.
103
Storage System Performance

Ifissuesoccurwithstoragesystemperformance,consultyourstoragesystemvendors documentationforanyrelevantinformation. WhenyouassignLUNs,rememberthatyoucanaccesseachLUNthroughanumberof ESXServerhosts,andthatanumberofvirtualmachinescanrunoneachhost.One LUNusedbyanESXServerhostcanserviceI/Ofrommanydifferentapplications runningondifferentoperatingsystems.Becauseofthisdiverseworkload,theRAID groupthatcontainstheESXServerLUNsshouldnotincludeLUNsthatotherhostsuse thatarenotrunningESXServerforI/Ointensiveapplications. Enablereadcachingandwritecaching. LoadbalancingistheprocessofspreadingserverI/OrequestsacrossallavailableSPs andtheirassociatedhostserverpaths.Thegoalistooptimizeperformanceintermsof throughput(I/Opersecond,megabytespersecond,orresponsetimes). SANstoragesystemsrequirecontinualredesignandtuningtoensurethatI/Oisload balancedacrossallstoragesystempaths.Tomeetthisrequirement,distributethepaths totheLUNsamongalltheSPstoprovideoptimalloadbalancing.Closemonitoring indicateswhenitisnecessarytomanuallyrebalancetheLUNdistribution.SeePath ManagementandManualLoadBalancingonpage 100foranexample. Tuningstaticallybalancedstoragesystemsisamatterofmonitoringthespecific performancestatistics(suchasI/Ooperationspersecond,blockspersecond,and responsetime)anddistributingtheLUNworkloadtospreadtheworkloadacrossall theSPs. NOTEDynamicloadbalancingisnotcurrentlysupportedwithESXServer.
Server Performance
Ensuringoptimalserverperformancerequireslookingatanumberoffactors.Each serverapplicationmusthaveaccesstoitsdesignatedstoragewiththefollowingitems:

HighI/Orate(numberofI/Ooperationspersecond) Highthroughput(megabytespersecond) Minimallatency(responsetimes)
104
VMware, Inc.
Becauseeachapplicationhasdifferentrequirements,youcanmeetthesegoalsby choosinganappropriateRAIDgrouponthestoragesystem.Toachieveperformance goals,performthefollowingtasks:
PlaceeachLUNonaRAIDgroupthatprovidesthenecessaryperformancelevels. PayattentiontotheactivitiesandresourceuseofotherLUNSintheassignedRAID group.AhighperformanceRAIDgroupthathastoomanyapplicationsdoingI/O toitmightnotmeettheperformancegoalsthatanapplicationrunningonthe ESX Serverhostrequires. ProvideeachserverwithasufficientnumberofNICsorHBAstoallowmaximum throughputforalltheapplicationshostedontheserverforthepeakperiod.I/O spreadacrossmultipleportsprovideshigherthroughputandlesslatencyforeach application. ToprovideredundancyforsoftwareiSCSI,intheinitiatorbindmultipleNICsto thevswitchusedforiSCSIconnectivity. AllocateadequateperformancecapacitywhenyouallocateLUNsorRAIDgroups forESXServersystems,multipleoperatingsystemswilluseandsharethat resource.Asaresult,theperformancerequiredfromeachLUNinthestorage subsystemcanbemuchhigherifyouareworkingwithESXServersystemsthanif youareusingphysicalmachines.Forexample,ifyouexpecttorunfour I/Ointensiveapplications,allocatefourtimestheperformancecapacityforthe ESXServerLUNs. WhenyouusemultipleESXServersystemsinconjunctionwithaVirtualCenter Server,theperformanceneededfromthestoragesubsystemincreases correspondingly.
MatchthenumberofoutstandingI/OsthatapplicationsrunningonanESXServer systemwiththenumberofI/OstheSANcanhandle.
Network Performance
AtypicalSANconsistsofacollectionofcomputersconnectedtoacollectionofstorage systemsthroughanetworkofswitches.Often,numerouscomputersareaccessingthe samestorage.Figure 511showsseveralcomputersystemsconnectedtoastorage systemthroughanEthernetswitch.Inthisconfiguration,eachsystemisconnected throughasingleEthernetlinktotheswitch,whichisalsoconnectedtothestorage systemthroughasingleEthernetlink.Inmostconfigurations,withmodernswitches andtypicaltraffic,thisisnotaproblem.
VMware, Inc.
105
Figure 5-11. Single Ethernet Link Connection to Storage
Whensystemsreaddatafromstorage,themaximumresponsefromthestorageisto sendenoughdatatofillthelinkbetweenthestoragesystemsandtheEthernetswitch. Itisunlikelythatanysinglesystemorvirtualmachinewillgetfulluseofthenetwork speed,butthisistheexpectedsituationwhenmanysystemsshareonestoragedevice. Whenwritingdatatostorage,multiplesystemsorvirtualmachinesmightattemptto filltheirlinks.AsFigure 512shows,whenthishappens,theswitchbetweenthe systemsandthestoragesystemhastodropdata.Thishappensbecause,whileithasa singleconnectiontothestoragedevice,ithasmoretraffictosendtothestoragesystem thanasinglelinkcancarry.Inthiscase,theswitchdropsnetworkpacketsbecausethe amountofdataitcantransmitislimitedbythespeedofthelinkbetweenitandthe storagesystem. Figure 5-12. Dropped Packets
1 Gbit 1 Gbit
1 Gbit dropped packets
Recoveringfromdroppednetworkpacketsresultsinlargeperformancedegradation. Inadditiontotimespentdeterminingthatdatawasdropped,theretransmissionuses networkbandwidththatcouldotherwisebeusedforcurrenttransactions.
106
VMware, Inc.
iSCSItrafficiscarriedonthenetworkbytheTransmissionControlProtocol(TCP). TCP isareliabletransmissionprotocolthatensuresthatdroppedpacketsareretried andeventuallyreachtheirdestination.TCPisdesignedtorecoverfromdropped packetsandretransmitsthemquicklyandseamlessly.However,whentheswitch discardspacketswithanyregularity,networkthroughputsufferssignificantly.The networkbecomescongestedwithrequeststoresenddataandwiththeresentpackets, andlessdataisactuallytransferredthaninanetworkwithoutcongestion. MostEthernetswitchescanbuffer,orstore,dataandgiveeverydeviceattemptingto senddataanequalchancetogettothedestination.Thisabilitytobuffersome transmissions,combinedwithmanysystemslimitingthenumberofoutstanding commands,allowssmallburstsfromseveralsystemstobesenttoastoragesystemin turn. Ifthetransactionsarelargeandmultipleserversaretryingtosenddatathrougha singleswitchport,aswitchsabilitytobufferonerequestwhileanotheristransmitted canbeexceeded.Inthiscase,theswitchdropsthedataitcannotsend,andthestorage systemmustrequestretransmissionofthedroppedpacket.Forexample,ifanEthernet switchcanbuffer32KBonaninputport,buttheserverconnectedtoitthinksitcansend 256KBtothestoragedevice,someofthedataisdropped. Mostmanagedswitchesprovideinformationondroppedpackets,similartothe following:
*: interface is up IHQ: pkts in input hold queue OHQ: pkts in output hold queue RXBS: rx rate (bits/sec) TXBS: tx rate (bits/sec) TRTL: throttle count
Interface * GigabitEthernet0/1 IHQ 3 IQD 9922 OHQ 0
IQD: pkts dropped from input queue OQD: pkts dropped from output queue RXPS: rx rate (pkts/sec) TXPS: tx rate (pkts/sec)
OQD 0 RXBS 476303000 RXPS 62273 TXBS 477840000 TXPS 63677 TRTL 0
InthisexamplefromaCiscoswitch,thebandwidthusedis476303000bits/second, whichislessthanhalfofwirespeed.Inspiteofthis,theportisbufferingincoming packetsandhasdroppedquiteafewpackets.Thefinallineofthisinterfacesummary indicatesthatthisporthasalreadydroppedalmost10,000inboundpacketsintheIQD column. Configurationchangestoavoidthisprobleminvolvemakingsureseveralinput Ethernetlinksarenotfunneledintooneoutputlink,resultinginanoversubscribed link.Whenanumberoflinkstransmittingnearcapacityareswitchedtoasmaller numberoflinks,oversubscriptionisapossibility.
VMware, Inc.
107
Generally,applicationsorsystemsthatwritealotofdatatostorage,suchasdata acquisitionortransactionloggingsystems,shouldnotshareEthernetlinkstoastorage device.Thesetypesofapplicationsperformbestwithmultipleconnectionstostorage devices. Figure 513,showsmultipleconnectionsfromtheswitchtothestorage. Figure 5-13. Multiple Connections from Switch to Storage
1 Gbit 1 Gbit 1 Gbit 1 Gbit
UsingVLANsorVPNsdoesnotprovideasuitablesolutiontotheproblemoflink oversubscriptioninsharedconfigurations.VLANsandothervirtualpartitioningofa networkprovideawayoflogicallydesigninganetwork,butdonotchangethephysical capabilitiesoflinksandtrunksbetweenswitches.Whenstoragetrafficandother networktrafficendupsharingphysicalconnections,astheywouldwithaVPN,the possibilityforoversubscriptionandlostpacketsexists.ThesameistrueofVLANsthat shareinterswitchtrunks.PerformancedesignforaSANsmusttakeintoaccountthe physicallimitationsofthenetwork,notlogicalallocations.
Resolving Performance Issues

Thissectiondiscussesperformancemonitoringandpossiblewaysofresolving performanceissues.Forbestperformance,placeeachvirtualmachineonthe appropriatetierofstorage.SeeChoosingVirtualMachineLocationsonpage 35.
Monitoring Performance
TheVIClientoffersextensivefacilitiesforcollectingperformanceinformation. The informationisthengraphicallydisplayedintheVIClient.TheVIClientupdates itsdisplayperiodically.Forinformation,seetheVirtualInfrastructureUsersGuide.
108
VMware, Inc.
WithESXServer3,youcanalsousetheesxtoputility,availablefromtheservice console.Forinformationaboutesxtop,seetheResourceManagementGuide,orlookat themanpagefromtheserviceconsole.Youcanuseesxtoptomonitorperformancein realtime.IfyouareusingESXServer3i,theresxtoputilityprovidessimilar functionality.
Checking Ethernet Switch Statistics

ManyEthernetswitchesprovidemethodsformonitoringswitchhealth.Switchesthat haveportsoperatingnearmaximumthroughputmuchofthetimedonotprovide optimumperformance.IfyouhaveportsinyouriSCSISANrunningnearthe maximum,reducetheload.IftheportisconnectedtoanESXServersystemoriSCSI storage,youcanreducetheloadbyusingmanualloadbalancingasdescribedinPath ManagementandManualLoadBalancingonpage 100. Iftheportisconnectedbetweenmultipleswitchesorrouters,considerinstalling additionallinksbetweenthesecomponentstohandlemoreload.Ethernetswitchesalso commonlyprovideinformationabouttransmissionerrors,queuedpackets,and droppedEthernetpackets.Iftheswitchregularlyreportsanyoftheseconditionson portsbeingusedforiSCSItraffic,performanceoftheiSCSISANwillbepoor.For additionalinformationonproperlyconfiguringyouriSCSInetwork,seeNetwork Performanceonpage 105.
Resolving Path Thrashing

IfyourservercannotaccessaLUN,oraccessisveryslow,youmighthaveaproblem withpaththrashing(alsocalledLUNthrashing).Paththrashingmightoccurwhentwo hostsaccesstheLUNthroughdifferentSPsand,asaresult,theLUNisneveractually available. Usually,onlyspecificSANconfigurationsinconjunctionwiththefollowingconditions cancausethepaththrashing:
Youareworkingwithanactivepassivearray.Paththrashingonlyoccurson activepassivearrays.Foractiveactivearraysorarraysthatprovidetransparent failover,paththrashingdoesnotoccur. PathpolicyissettoFixed. TwohostsaccesstheLUNusingoppositepathorder.Forexample,HostAissetup toaccessthelowernumberedLUNthroughSPA.HostBissetuptoaccessthe lowernumberedLUNthroughSPB. PaththrashingcanalsooccurifHostAlostacertainpathandcanuseonlypaths toSPAwhilehostBlostotherpathsandcanuseonlypathstoSPB.
VMware, Inc.
109
Paththrashingcanalsooccuronadirectconnectstoragesystem(suchasAX100ior AX150i)withHBAfailoverononeormorenodes. Paththrashingisaproblemthatyoutypicallywillnotexperiencewithotheroperating systems:
NoothercommonoperatingsystemusessharedLUNsformorethantwoservers (thatsetupistypicallyreservedforclustering). Forclustering,onlyoneserverissuesI/Osatatimeandpaththrashingdoesnot becomeaproblem.
Incontrast,multipleESXServersystemsmightbeissuingI/OtothesameLUN concurrently. To resolve path thrashing
EnsurethatallhostssharingthesamesetofLUNsonthoseactivepassivearrays thataccessthesamestorageprocessorsimultaneously. CorrectanycablinginconsistenciesbetweendifferentESXServerhostsandSAN targetssothatallHBAsseethesametargetsinthesameorder. SetthepathpolicytoMostRecentlyUsed(thedefault).
Understanding Path Thrashing

Inallstoragesystems,theSPsarelikeindependentcomputersthathaveaccesstosome sharedstorage.Algorithmsdeterminehowconcurrentaccessishandled.
Foractivepassivearrays,onlyoneLUNatatimecanaccessallthesectorsonthe storagethatmakeupagivenLUN.Theownershipispassedbetweenthestorage processors.StoragesystemsusecachesandSPAmustnotwritesomethingtodisk thatinvalidatestheSPBcache.BecausetheSPhastoflushthecachewhenits finishedwithitsoperation,ittakesalittletimetomovetheownership.Duringthat time,neitherSPcanprocessI/OtotheLUN. Foractiveactivearrays,thealgorithmsallowmoredetailedaccesstothestorage andsynchronizecaches.AccesscanhappenconcurrentlythroughanySPwithout extratimerequired.
Considerhowpathselectionworks:

OnanactiveactivearraythesystemstartssendingI/Oonthenewpath. Foractivepassivearrays,theESXServersystemchecksallstandbypaths.TheSP attheendofthepaththatiscurrentlyunderconsiderationsendsinformationto thesystemonwhetheritcurrentlyownstheLUN.
110
VMware, Inc.
IftheESXServersystemfindsanSPthatownstheLUN,thatpathisselected andI/Oissentonthatpath. IftheESXServerhostcannotfindsuchapath,theESXServerhostpicksone ofthepathsandsendstheSP(attheotherendofthepath)acommandtomove theLUNownershiptothisSP.
Paththrashingcanoccurasaresultofthefollowingpathchoice:IfserverAcan reachaLUNonlythroughoneSP,andserverBcanreachthesameLUNonly throughadifferentSP,theybothcontinuallycausetheownershipoftheLUNto movebetweenthetwoSPs.Becausethesystemmovestheownershipquickly,the storagesystemcannotprocessanyI/O(orcanprocessonlyverylittle).Asaresult, anyserversthatdependontheLUNstarttimingoutI/O.
Equalizing Disk Access Between Virtual Machines

Youcanadjustthemaximumnumberofoutstandingdiskrequestswiththe Disk.SchedNumReqOutstandingparameterintheVIClient.Whentwoormore virtualmachinesareaccessingthesameLUN,thisparametercontrolsthenumberof outstandingrequeststhateachvirtualmachinecanissuetotheLUN.Adjustingthe limitcanhelpequalizediskaccessbetweenvirtualmachines. ThislimitisinapplicablewhenonlyonevirtualmachineisactiveonaLUN.Inthat case,thequeuedepthofthestorageadapterandthenetworkscapacitytopasstraffic limitsthebandwidth. To set the number of outstanding disk requests 1 2 3 4 5 IntheVIClient,selectthehostintheinventorypanel. ClicktheConfigurationtabandclickAdvancedSettings. ClickDiskintheleftpanelandscrolldowntoDisk.SchedNumReqOutstanding. ChangetheparametervalueandclickOK. Reboottheserver.
Thischangecanaffectdiskbandwidthscheduling,butmightalsoleadto improvementsfordiskintensiveworkloads. IfyouadjustthisparametervalueintheVMkernel,youmightalsowanttoadjustthe queuedepthinyourstorageadapter.SeeSettingMaximumQueueDepthforSoftware iSCSIonpage 112.
VMware, Inc.
111
Removing VMFS-2 Drivers

IfyouhavealotofLUNsandVMFSvolumes,andallofthemareVMFS3,youcan potentiallyimproveperformancebyunloadingtheVMFS2driver.Atacommandline prompt,type:
vmkload_mod -u vmfs2
Asignificantincreaseinthespeedofcertainmanagementoperationslikerefreshing datastoresandrescanningstorageadaptersshouldresult.
Reducing SCSI Reservations

OperationsthatrequiregettingafilelockorametadatalockinVMFSresultin shortlivedSCSIreservations.SCSIreservationslockanentireLUN.ExcessiveSCSI reservationsbyaservercancauseperformancedegradationonotherserversaccessing thesameVMFS. Examplesofoperationsthatrequiregettingfilelocksormetadatalocksinclude:

Virtualmachinepoweron VMotion Virtualmachinesrunningwithvirtualdisksnapshots Fileoperationsthatrequireopeningfilesordoingmetadataupdates(See MetadataUpdatesonpage 24.)
Performancedegradationcanoccurifsuchoperationsarehappeningfrequentlyon multipleserversaccessingthesameVMFS.Forinstance,VMwarerecommendsthat youdonotrunmanyvirtualmachinesfrommultipleserversthatareusingvirtualdisk snapshotsonthesameVMFS.LimitthenumberofVMFSfileoperationswhenmany virtualmachinesarerunningontheVMFS.
Setting Maximum Queue Depth for Software iSCSI

IfyounoticeunsatisfactoryperformanceforyoursoftwareiSCSILUNs,youcanchange theirmaximumqueuedepthbyusingtheesxcfg-modulecommandasfollows:
esxcfg-module
-s
iscsi_max_lun_queue=value
iscsi_mod
After you issue this command, reboot your system.
112
VMware, Inc.
Theiscsi_max_lun_queueparameterisusedtosetthemaximumoutstanding commands,orqueuedepth,foreachLUNaccessedthroughthesoftwareiSCSIadapter. Thedefaultis32,andthevalidrangeis1to255. CAUTIONSettingthequeuedepthhigherthanthedefaultcandecreasethetotal numberofLUNssupported.
SAN Storage Backup Considerations

IntheSANenvironment,backupshavetwogoals.Thefirstgoalistoarchiveonline datatoofflinemedia.Thisprocessisrepeatedperiodicallyforallonlinedataonatime schedule.Thesecondgoalistoprovideaccesstoofflinedataforrecoveryfroma problem.Forexample,databaserecoveryoftenrequiresretrievalofarchivedlogfiles thatarenotcurrentlyonline. Schedulingabackupdependsonanumberoffactors:
Identificationofcriticalapplicationsthatrequiremorefrequentbackupcycles withinagivenperiodoftime. Recoverypointandrecoverytimegoals.Considerhowpreciseyourrecoverypoint needstobe,andhowlongyouarewillingtowaitforit. Therateofchange(RoC)associatedwiththedata.Forexample,ifyouareusing synchronousasynchronousreplication,theRoCaffectstheamountofbandwidth requiredbetweentheprimaryandsecondarystoragedevices. OverallimpactonSANenvironment,storageperformance(whilebackingup), andotherapplications. IdentificationofpeaktrafficperiodsontheSAN(backupsscheduledduringthose peakperiodscanslowtheapplicationsandthebackupprocess). Timetoscheduleallbackupsinthedatacenter. Timeittakestobackupanindividualapplication. Resourceavailabilityforarchivingdata;usuallyofflinemediaaccess(tape).
Includearecoverytimeobjectiveforeachapplicationwhenyoudesignyourbackup strategy.Thatis,considerthetimeandresourcesnecessarytoreprovisionthedata.For example,ifascheduledbackupstoressomuchdatathatrecoveryrequiresa considerableamountoftime,examinethescheduledbackup.Performthebackupmore frequently,sothatlessdataisbackedupatatimeandtherecoverytimedecreases.
VMware, Inc.
113
Ifaparticularapplicationrequiresrecoverywithinacertaintimeframe,thebackup processneedstoprovideatimescheduleandspecificdataprocessingtomeetthis requirement.Fastrecoverycanrequiretheuseofrecoveryvolumesthatresideon onlinestoragetominimizeoreliminatetheneedtoaccessslowofflinemediafor missingdatacomponents.
Snapshot Software
Snapshotsoftwareallowsanadministratortomakeaninstantaneouscopyofanysingle virtualdiskdefinedwithinthedisksubsystem.Snapshotsoftwareisavailableat differentlevels:
ESXServerhostsallowyoutocreatesnapshotsofvirtualmachines.Thissoftware isincludedinthebasicESXServerpackage. Thirdpartybackupsoftwaremightallowformorecomprehensivebackup proceduresandmightcontainmoresophisticatedconfigurationoptions.
Administratorsmakesnapshotsforavarietyofreasons,including:

Backup Disasterrecovery Availabilityofmultipleconfigurations,versions,orboth Forensics(lookingatasnapshottofindthecauseofproblemswhileyoursystem isrunning) Datamining(lookingatacopyofyourdatatoreduceloadonproductionsystems)
Using a Third-Party Backup Package

Ifyouareusingthirdpartybackupsoftware,makesurethatthesoftwareissupported withESXServerhosts.SeetheBackupSoftwareCompatibilityGuide. Usingthirdpartysoftwarehastheadvantageofauniformenvironment.However,the additionalcostofthethirdpartysnapshottingsoftwarecanbecomehigherasyour SANgrows. Ifyouusesnapshotstobackupyourdata,considerthefollowingpoints:
SomevendorssupportsnapshotsforVMFSandRDMs.Ifbotharesupported,you canmakeeitherasnapshotofthewholeVMFSforahost,orsnapshotsforthe individualvirtualmachines(oneforeachdisk). SomevendorssupportsnapshotsonlyforasetupusingRDM.IfonlyRDMis supported,youcanmakesnapshotsofindividualvirtualmachines.
114
VMware, Inc.
Seeyourstoragevendorsdocumentation. NOTEESXServersystemsalsoincludeaConsolidatedBackupcomponent,whichis discussedindetailintheVirtualMachineBackupGuide.
Choosing Your Backup Solution

Whenyouchooseyourbackupsolution,considerthatabackupcanbeoneorallof these:

Crashconsistent Filesystemconsistent Applicationconsistent
VMwareoffersafilesystemconsistentbackup.Inmostcases,afilesystemconsistent backupallowsyoutocompletelyrecoverfromfailure.However,ifyourapplications requiresynchronizationacrossfilesystemsorwithadatabase,theVMwaresolution mightnotprovideenoughconsistency.Inthesecases,investigateathirdpartybackup solutiontoseewhetheritbettersuitsyourneeds.
Layered Applications
SANadministratorscustomarilyusespecializedarraybasedsoftwareforbackup, disasterrecovery,datamining,forensics,andconfigurationtesting. Storageproviderstypicallysupplytwotypesofadvancedservicesfortheir LUNssnapshottingandreplication.
SnapshottingcreatesspacewithefficientcopiesofLUNsthatsharecommon blocksofdata.Ingeneral,snapshottingisusedlocallyonthesamestoragesystem astheprimaryLUNforquickbackups,applicationtesting,forensics,ordata mining. ReplicationcreatesfullcopiesofLUNs.Replicasareusuallymadetoseparate storagesystems,possiblyseparatesitestoprotectagainstmajoroutagesthat incapacitateordestroyanentirestoragesystemorsite.
WhenyouuseanESXServersysteminconjunctionwithaSAN,decidewhether arraybasedorhostbasedtoolsaremoresuitableforyourparticularsituation.
VMware, Inc.
115
Array-Based (Third-Party) Solution

Whenyouconsideranarraybasedsolution,considerthefollowingpoints: NOTEESXServersystemsalsoincludeaconsolidatedbackupcomponent,whichis discussedindetailintheVirtualMachineBackupGuide.
Arraybasedsolutionsusuallyresultinmorecomprehensivestatistics.WithRDM, dataalwaystakesthesamepath,whichresultsineasierperformance management. SecurityismoretransparenttothestorageadministratorwhenyouuseRDMand anarraybasedsolutionbecausewithRDM,virtualmachinesmoreclosely resemblephysicalmachines. Ifyouuseanarraybasedsolution,physicalcompatibilityRDMsareoftenusedfor thestorageofvirtualmachines.IfyoudonotintendtouseRDM,checkthestorage vendordocumentationtoseeifoperationsonLUNswithVMFSvolumesare supported.Furthermore,ifyouusearrayoperationsonVMFSLUNs,carefully readthesectiononresignaturing.
File-Based (VMFS) Solution

WhenyouconsiderafilebasedsolutionthatusesVMwaretoolsandVMFS(insteadof thearraytools),beawareofthefollowingpoints:
UsingVMwaretoolsandVMFSisbetterforprovisioning:onelargeLUNis allocatedandmultipleVMDK filescanbeplacedonthatLUN.WithRDM,anew LUNisrequiredforeachvirtualmachine. SnapshottingisincludedwithyourESXServerhostatnoextracost.Thefilebased solutionisthereforemorecosteffectivethanthearraybasedsolution. ForESXServeradministrators,usingVMFSiseasier. ESXServeradministratorswhousethefilebasedsolutionaremoreindependent fromtheSANadministrator.
116
VMware, Inc.
VMFS Volume Resignaturing

ESXserversneedtobeabletodifferentiatebetweentheirVMFSvolumesandusea volumesignaturetodoso.WhenaVMFSvolumeisreplicatedorasnapshotistaken, theresultingLUNcopyhasthesamesignatureasthesource.WhenanESXServersees twoLUNswiththesamesignature,theESXServermusthandletheconditionto preventdowntimecausedbyconfusionoverwhichLUNitshouldbeusingtoaccess theregisteredvirtualmachines.ResignaturingisafeatureintroducedinESXServer3.0 tosolvethisproblem. NOTEWhenaLUNneedstoberesignatured,analertappearsinthevmkernellog. If youencountersuchanalert,setyourresignaturingoptionsappropriately,as describedinthefollowingsections.
Mounting Original, Snapshot, or Replica VMFS Volumes

Youcanmountoriginal,snapshot,orreplicaVMFSvolumesonthesameESXServer host. To mount original, snapshot, or replica VMFS volumes 1 Performtherequiredstoragetasks: a b 2 3 4 5 Makethestoragesystemsnapshotorreplica. ConfigureaccesscontroltoallowESXServertoaccessthesnapshotorreplica.
IntheVIClient,selectthehostintheinventorypanel. ClicktheConfigurationtabandclickAdvancedSettings. SelectLVMintheleftpanel,thensettheLVM.EnableResignatureoptionto1. RescanforanynewLUNsorVMFSvolumes. Aftertherescan,thecopiedVMFSvolumeappearsas /vmfs/volumes/snap-<DIGIT>-<old-label>. IftheVMX fileforanyofthevirtualmachinesortheVMSDfileforvirtualmachine snapshotscontains/vmfs/volumes/<label or UUID>/paths,youmustchange theseitemstoreflecttheresignaturedvolumepath.
SettheLVM.EnableResignatureoptionto0afterresignaturingiscomplete.
NOTEAnyvirtualmachinesonthisnewsnapshotvolumearenotautodiscovered. Youmustmanuallyregisterthevirtualmachines.
VMware, Inc.
117
Understanding Resignaturing Options

ThissectiondiscusseshowtheEnableResignatureandDisallowSnapshotLUNoptions interactandexplainsthethreestatesthatresultfromchangingtheseoptions:
State1:EnableResignature=0,DisallowSnapshotLUN=1(theESXServer3.x default) State2:EnableResignature=1(DisallowSnapshotLUNisnotrelevant) State3:EnableResignature=0,DisallowSnapshotLUN=0(ESXServer2.xbehavior)
State 1: EnableResignature=0, DisallowSnapshotLUN=1 (default)

Inthisstate:
YoucannotbringsnapshotsorreplicasofVMFSvolumesmadebythestorage systemintotheESXServerhostregardlessofwhethertheESXServerhasaccessto theoriginalLUN. LUNsformattedwithVMFSmusthavethesameIDforeachESXServerhost.
State1isthesafeststatebut:
IfyouuseClariionAX100iorAX150iwithNavisphereExpress,youcannot configurethesameLUNIDacrossstoragegroups.Youmustinsteaduseaversion ofNavispheresoftwarethathasmorecomprehensivemanagementcapabilities. ForIBMTotalStorage8000,ifyouhaveLUNsthatarenotconfiguredtopresentthe sameLUNIDtoallservers,youneedtoeitherusethesettingsinstate3orrecreate theLUNsfromscratch.
State 2: EnableResignature=1, (DisallowSnapshotLUN is not relevant)

Inthisstate:
YoucansafelybringsnapshotsorreplicasofVMFSvolumesintothesameservers astheoriginalandtheyareautomaticallyresignatured. VMFSvolumescontainingLUNsfromAX100iorAX150ithatarenotpresented withthesameLUNnumberstoallserverseffectivelylosetheabilitytousethe virtualmachinesstoredonthatVMFSvolume.Avoidthissituation.
118
VMware, Inc.
State 3: EnableResignature=0, DisallowSnapshotLUN=0

ThisissimilartoESXServer2.xbehavior.Inthisstate,theESXServerassumesthatit seesonlyonereplicaorsnapshotofagivenLUNandnevertriestoresignature.Thisis idealinaDRscenariowhereyouarebringingareplicaofaLUNtoanewclusterofESX Servers,possiblyonanothersitethatdoesnothaveaccesstothesourceLUN.Insucha case,theESXServerusesthereplicaasifitistheoriginal. IfyouhaveanAX100iorAX150ithatcannotbeconfiguredtopresentthesameLUN numberstoallserversforsomereason,youneedthissettingtoallowallESXServer systemstousethesameLUNsforfeatureslikeVMotion,VMwareDRS,and VMware HA. DonotusethissettingifyouarebringingsnapshotsorreplicasofaLUNintoaserver withaccesstotheoriginalLUN.Thiscanhavedestructiveresultsincluding:
IfyoucreatesnapshotsofaVMFSvolumeoneormoretimesanddynamically bringoneormoreofthosesnapshotsintoanESXServer,onlythefirstcopyis usable.Theusablecopyismostlikelytheprimarycopy.Afterreboot,itis impossibletodeterminewhichvolume(thesourceoroneofthesnapshots)is usable.Thisnondeterministicbehaviorshouldbeavoided. IfyoucreateasnapshotofaspannedVMFSvolume,anESXServerhostmight reassemblethevolumefromfragmentsthatbelongtodifferentsnapshots.Thiscan corruptyourfilesystem.
VMware, Inc.
119
120
VMware, Inc.
Multipathing Checklist
Thisappendixprovidesachecklistofmultipathingsetuprequirementsfordifferent storagesystems. Table A-1. Multipathing Setup Requirements

Component Allstoragesystems Topology EMCSymmetrix EMCClariion Comments Writecachemustbedisabledifnotbatterybacked. NosinglefailureshouldcauseHBAandSPfailover,especiallywith activepassivestoragearrays. EnabletheSPC2andSC3settings.ContactEMCforthelatestsettings. SettheAdvancedSettingfortheESXServerhost: AllInitiatorrecordsmusthave:

FailoverMode=1 InitiatorType=ClariionOpen ArrayCommPath=Enabledor1
HPMSA HPEVA
Nospecificrequirements ForEVA3000/5000firmware4.001andlater,andEVA4000/6000/8000firmware 5.031andlater,setthehosttypetoVMware. Otherwise,setthehostmodetypetoCustom.Thevalueis:

EVA3000/5000firmware3.x:000000002200282E EVA4000/6000/8000:000000202200083E
NetApp EqualLogic
Nospecificrequirements Nospecificrequirements
VMware, Inc.
121
Table A-1. Multipathing Setup Requirements (Continued)

Component LeftHand ESXServer Configuration Comments Nospecificrequirements SetthefollowingAdvancedSettingsfortheESXServerhost:

SetDisk.UseLunResetto1 SetDisk.UseDeviceResetto0
AmultipathingpolicyofMostRecentlyUsedmustbesetforallLUNshosting clustereddisksforactivepassivearrays.AmultipathingpolicyofMost RecentlyUsedorFixedmaybesetforLUNsonactiveactivearrays. AllowARPredirectionifthestoragesystemsupportstransparentfailover.See esxcfghwiscsiUtilityonpage 126.
122
VMware, Inc.
Utilities
Inmostcases,theVIClientiswellsuitedformonitoringanESXServerhostconnected toSANstorage.Advancedusersmight,attimes,wanttousesomecommandline utilitiesforadditionaldetails. Thisappendixprovidesinformationonthefollowingutilities:

esxtopUtilityonpage 123 storageMonitorUtilityonpage 124 esxcfgswiscsiUtilityonpage 125 esxcfghwiscsiUtilityonpage 126 vmkpingUtilityonpage 126
esxtop Utility
TheesxtopcommandlineutilityprovidesadetailedlookatESXServer3resourceuse inrealtime.ItrunsontheESXServerhostsserviceconsole.Fordetailedinformation aboutesxtop,seetheResourceManagementGuideortype man esxtop atthe commandlineprompt. NOTEIfyouareusingESXServer3i,theresxtoputilityprovidessimilarfunctionality. SeeRemoteCommandLineInterfaceInstallationandReferenceGuide.
VMware, Inc.
123
storageMonitor Utility
ThestorageMonitorutilitymonitorsSCSIsenseerrorsthatstoragedevicesattachedto VMwareESXServerexperience.Theutilitygatherssenseerrorinformationby periodicallypollingthestorageMonitorrunninginsidetheVMkernel,andsendserror informationtoastandardoutputfile,afile,orthesystemlog.Itformatserror informationbeforesendingittooutput.Forexample,itconvertssenseerrorcodesto correspondingtextaccordingtoSCSI3specification. Ifnoconfigurationfileisspecified,storageMonitorparsesthe /etc/vmware/storageMonitor.conf defaultconfigurationfiletofiltercertainerrorsand allowothererrorstobedisplayed.Youcanusethe-doptiontorunstorageMonitorin interactivemodeordaemonmode.
Options
YoucanstartstorageMonitorfromtheESXServercommandlineusingoneofthe followingoptions. Table B-1. storageMonitor Command-Line Options
Option <config-file> Description Allowsyoutospecifyaconfigurationfile.Ifthisoptionisleft unspecified,thedefaultisused. TheconfigurationfilespecifieswhichtypeoferrorsstorageMonitor shouldallowandwhichonesitshouldfilterbeforedisplayingthem. Thedefaultconfigurationfileillustratestheformatoftheentries. -d SpecifiesthatstorageMonitorshouldberunindaemonmode.Whenthis optionisspecified,alloutputgoeseithertosyslogortoalogfilethat theuserspecifies.Ifthe-soptionisalsospecified,outputiswrittento astandardoutputfileaswell. Displayshelpinformation. Whenthisoptionisspecified,outputfromtheprogramiswrittento <log_file>.Thisoptionisvalidonlyifthe-doptionisalsospecified. Allowsyoutospecifytheinterval(inseconds)usedforpollingkernel residentstorageandforretrievingthestatusorerrorsofthestorage devices.Ifthisoptionisnotspecified,thedefaultpollingintervalof 10 secondsisused. SpecifiesthatstorageMonitorshouldsendoutputtoastandardoutput file.ThisoptionisonlyvalidifyoustartstorageMonitorindaemon mode(-doptionisspecified).
-h -l <log_file> -p <poll_interval>
-s
124
VMware, Inc.
Appendix B Utilities
Examples
storageMonitor -p 60
Setsthepollingintervalto60seconds.Sendsoutputtoastandardoutputfile(because storageMonitorisnotrunningindaemonmode).Usesthefiltersspecifiedinthedefault configurationfilebeforesendingtheoutput.

storageMonitor -d -c myconf.conf
RunsstorageMonitorindaemonmodebyusingthemyconf.conf configurationfile. Writes outputtosyslog.Bydefault,syslogislocatedat/var/log/storageMonitor.

storageMonitor -d -l mylog.log -s
RunsstorageMonitorindaemonmodebyusingthedefaultconfigurationfile. Sends outputtomylog.loginsteadofsyslog.Alsowritesoutputtoastandardoutputfile becausethe-soptionisspecified.
esxcfg-swiscsi Utility
Theesxcfg-swiscsiutilityallowsyoutoenableordisablesoftwareiSCSIonESXhosts. Usageexample: esxcfgswiscsi[e][d][h][q][s]<vmkernelSCSIadaptername> Table B-2. esxcfg-swiscsi Command-Line Options
Option -e -d -q -s -h Description EnablessoftwareiSCSI. DisablessoftwareiSCSI.Usingthisoptioncanleadtoproblemsifyou areusingiSCSIvolumes. ChecksifsoftwareiSCSIisonoroff. ScansfordisksavailablethroughthesoftwareiSCSIinterface. Displayshelpinformation.
VMware, Inc.
125
esxcfg-hwiscsi Utility
Theesxcfg-hwiscsiutilityallowsyoutoconfiguresupportedparametersforhardware iSCSI. Usageexample: /sbin/esxcfg-hwiscsi [-l] [-a allow|deny] [-h] <vmkernel SCSI adapter name> Table B-3. esxcfg-hwiscsi Command-Line Options
Option -l -a -h Description Listscurrentconfiguration(overridessettingsoptions). AllowsordeniesARPredirectiononadapter. Displayshelpinformation.
vmkping Utility
ThevmkpingutilityallowsyoutoverifytheVMkernelnetworkingconfiguration. Usageexample: vmkping [options] [host|IP address] Table B-4. vmkping Command-Line Options
Option -D -c <count> -i <interval> -s <size> Description VMkernelTCPstackdebugmode. Setspacketcount. Setsinterval. Setssendsize.
126
VMware, Inc.
Index
A
access data 30 equalizing disk access 111 access control 26 active path status 95 active paths 97 active-active disk arrays 17, 39, 71, 96, 100, 109 active-passive disk arrays 17, 34, 39, 67, 71, 96, 109 path policy reset 86 path thrashing 110 adding iSCSI hardware-initiated storage 47 iSCSI software-initiated storage 62 allocations, LUN 39 applications, layered 115 array-based (third-party) solution 116 authentication 26, 46, 59, 80, 90 avoiding problems 86, 103
enabling Qlogic HBA for 81 overview 77 BusLogic queue depth 38 SCSI controller 12
C
cable connectivity issues 89 changing disk.supportSparseLun 94 CHAP authentication 22, 26, 46, 59, 80, 90 configuring hardware-initiated iSCSI storage 47 software-initiated iSCSI storage 62 current multipathing state 94
D
data access 30 RDM 25 VMFS 25 datastores creating on hardware-initiated iSCSI storage 47 creating on software-initiated iSCSI storage 62 removing 91 viewing information 88 dead paths 96 diagnostic partitions 38 sharing 102 disabled path status 95 disabling paths 98 DisallowSnapshotLUN 118
127
B
backups and disaster recovery 21 considerations 113 solution 115 third-party backup package 114 booting from a SAN benefits 78 enabling 79
VMware, Inc.
disaster recovery 21 discovery 26 address 43, 58 static 44 disk access, equalizing 111 disk arrays active-active 39, 96, 100 active-passive 39, 96, 110 disk shares 29 disk.maskLuns 93 disk.maxLun 92 Disk.SchedNumReqOutstanding 111 disk.supportSparseLun 94 disks, configuration options 14 distributed locking 15 drivers, VMFS-2 112 dump partitions 38 sharing 102
EVA (HP StorageWorks) 71 extents 15 definition 88 information about 95
F
failover 32, 36, 101 I/O delay 34 failure, server 36 finding information 21 Fixed path policy 34, 96, 97 path thrashing 110 preferred path 99
H
hardware iSCSI initiators installing 40 setting up CHAP parameters 46 setting up discovery addresses 43 setting up naming parameters 41 HBA enabling Qlogic HBA for booting from a SAN 81 list of types 87 queue depth 112 high-tier storage 35 host type 66 HP StorageWorks 69 EVA 71 MSA 69
E
EMC CLARiiON 67 EMC Symmetrix 68 pseudo LUNs 68 EnableResignature 118 enabling paths 98 enabling Qlogic HBA for booting from a SAN 81 equalizing disk access 111 EqualLogic storage systems 74 ESX Server basics of using with SAN 22 benefits 20 sharing VMFS 22 ESX Server 3i 88, 109, 123 esxcfg-hwiscsi utility 126 esxcfg-module 112 esxcfg-swiscsi utility 125 esxtop utility 109, 123
128
I
I/O delay 34, 38 indirection, levels of 24 iSCSI hardware-initiated storage, adding 47 iSCSI HBA alias 42
VMware, Inc.
Index
iSCSI initiators hardware 13 software 14 iSCSI networking creating a VMkernel port 50 iSCSI software-initiated storage adding 62 iSCSI software-initiated storage, adding 62 iscsi_max_lun_queue 113 issues performance 108 visibility 89
L
layered applications 115 LeftHand Networks SAN/iQ storage systems 75 levels of indirection 24 Linux Cluster host type 66 Linux host type 66 list of HBA types 87 load balancing 21 manual 100 locations of virtual machines 35 locking 15 Logical Volume Manager (LVM) 15 lower-tier storage 35 LSI Logic SCSI controller 12 LSILogic queue depth 38 LUN discovery, VMkernel 24 LUN not visible cable connectivity 89 issues 89 SP visibility 90 LUNs allocations 39 changing number scanned 92 creating, and rescan 90
decisions 28 disk.maskLuns 93 display and rescan 24 display configuration 92 fewer, larger compared to smaller 27 masking changes and rescan 90 multipathing policy 97 number scanned 92 one VMFS volume per 38 removing 91 setting multipathing policy 97 sparse 94 LVM (Logical Volume Manager) 15 LVM.EnableResignature 117
M
maintenance, zero downtime 21 management applications 26 manual load balancing 100 mapping file 15 masking, using disk.maskLuns 93 maximum HBA queue depth 112 maxLun 92 metadata updates 24 mid-tier storage 35 monitoring performance 108 Most Recently Used path policy 96, 97 path thrashing 110 MRU path policy 96 MSA (HP StorageWorks) 69 multipathing 38, 94 viewing the current state of 94 multipathing policy 97 multipathing software 25 multipathing state 94 multiple extents 88
VMware, Inc.
129
N
Network Appliance storage provisioning storage 73 Network Appliance storage system 71 network performance 105 network virtualization 12 number of extents 15 number of outstanding disk requests 111
O
operating system timeout 101 outstanding disk requests 111
network 105 optimizing 103 removing VMFS-2 drivers 112 SCSI reservations 23 server 104 storage system 104 preferred path 96, 99 prioritizing virtual machines 29 problems avoiding 103 performance 108 visibility 89
P
passive disk arrays 39, 96 path thrashing 110 path failover 32 path failure rescan 91 path management 32, 100 path policies Fixed 34, 96, 97 Most Recently Used 97 MRU 96 Round Robin 97 path policy reset active-passive disk array 86 path status 95, 96 path thrashing 67, 109, 110 path, asterisk next to 96 paths active 97 disabling 98 enabling 98 preferred 96, 99 performance checking Ethernet switch statistics 109 issues 108 monitoring 108
130
Q
Qlogic HBA, enabling for booting from a SAN 81 queue depth 112
R
raw device mapping (RDM) 30, 68 data access 25 mapping file 15 removing datastores 91 removing LUNs 91 removing VMFS-2 drivers 112 rescan 90 LUN creation 90 LUN display 24 LUN masking 90 when path is unavailable 91 reservations reducing 112 resignaturing options 118 resolving problems 103 resxtop utility 109, 123 Round Robin path policy 97
VMware, Inc.
Index
S
SAN backup considerations 113 basics of using with ESX Server 22 benefits 20 concepts 15 optimizing performance of 103 server failover 36 scanning, changing number 92 SCSI HBA configuring 42 SCSI reservations 22 reducing 112 server failover 36 server failure 36 server performance 104 service console 88, 102, 109 sharing diagnostic partitions 102 sharing VMFS across servers 22 snapshot 117 snapshot software 114 software iSCSI networking 50 software iSCSI initiators enabling 56 setting up CHAP parameters 59 setting up discovery addresses 58 SP visibility, LUN not visible 90 sparse LUN support 94 standby path status 96 storage choices 27 storage consolidation 21 storage devices details 88 viewing 88 storage systems EMC CLARiiON 67 EMC Symmetrix 68
EqualLogic 74 HP StorageWorks 69 LeftHand Networks SAN/iQ 75 Network Appliance 71 performance 104 types 17 storage virtualization 12 storageMonitor utility 124
T
testing, storage systems 66 third-party backup package 114 third-party management applications 26 timeout 101 TimeoutValue parameter 38 troubleshooting 86, 89, 103
U
use cases disaster recovery 21 load balancing 21 maintenance 21 storage consolidation 21 utilities esxcfg-hwiscsi 126 esxcfg-swiscsi 125 esxtop 124 resxtop 123 storageMonitor 124 vmkping 126
V
virtual machines data access 30 data access on SAN 31 default configuration 24 equalizing disk access 111 I/O delay 34 locations 35
VMware, Inc.
131
prioritizing 29 SAN data access 31 visibility issues 89 vmdk file 14, 30, 116 VMFS 15, 25, 27 locking 15 number of extents 15 one volume per LUN 38 SCSI reservations 22 sharing across ESX Servers 22 volume resignaturing 117 VMFS-2 drivers 112 vmhba 88 VMkernel configuration 102 LUN discovery 24 vmkping utility 126 VMotion 20, 21, 26, 39, 67, 112, 119 VMware DRS 20, 21, 67, 119 using with VMotion 39 VMware HA 20, 26, 36, 67, 119 volume resignaturing 117
132
VMware, Inc.

VMWARE Vi3!35!25 U2 Iscsi San CFG

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

VMWARE Vi3!35!25 U2 Iscsi San CFG

Încărcat de

Drepturi de autor:

Formate disponibile

iSCSI SAN Configuration Guide

iSCSI SAN Configuration Guide

iSCSI SAN Configuration Guide Revision: 20090313 Item: EN-000035-01

VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

iSCSI SAN Configuration Guide

NetworkApplianceStorageSystems 71 Multipathing 71 SettingLUNTypeandInitiatorGroupType 72 ProvisioningStorage 72 EqualLogicStorageSystems 74 LeftHandNetworksSAN/iQStorageSystems 75 BasicConfiguration 75 AutomaticVolumeResignaturing 75

iSCSI SAN Configuration Guide

A MultipathingChecklist 121 B Utilities 123

About This Book

Thisbook,theiSCSISANConfigurationGuide,explainshowtouseanESXServer systemwithastorageareanetwork(SAN).Themanualdiscussesconceptual background,installationrequirements,andmanagementinformationinthesetopics:

UsingESXServerwithaSANDiscussesrequirements,differencesinSANsetup ifESXServerisused,andhowtomanagethetwosystemstogether. EnablingyourESXServersystemtobootfromaLUNonaSANDiscusses requirements,limitations,andmanagementofbootingfromaSAN.

NOTEForinformationonFibreChannel(FC)orNFSstoragedevices,seetheFibre ChannelSANConfigurationGuideandtheESXServerConfigurationGuide. TheiSCSISANConfigurationGuidecoversbothESXServer3.5andESXServer3i version3.5.Foreaseofdiscussion,thisbookusesthefollowingproductnaming conventions:

iSCSI SAN Configuration Guide

VMware Infrastructure Documentation

Abbreviations Used in Figures

Technical Support and Education Resources

About This Book

Online and Telephone Support

VMware Education Services

iSCSI SAN Configuration Guide

Using ESX Server with a Storage Area Network

iSCSI SAN Configuration Guide

Eachvirtualmachinerunsitsownoperatingsystem(theguestoperatingsystem) andapplications. Thevirtualizationlayerallowsvirtualdevicestomaptosharesofspecificphysical devices.ThesedevicesincludevirtualizedCPU,memory,I/Obuses,network interfaces,storageadaptersanddevices,humaninterfacedevices,andBIOS.

Chapter 1 Using ESX Server with a Storage Area Network

VMware virtualization layer software iSCSI initiator ethernet NIC LAN

hardware iSCSI initiator (HBA) LAN VMFS LUN1 .dsk

iSCSI SAN Configuration Guide

Disk Configuration Options

Virtualmachine.vmdkfilestoredonaVMFSvolume.SeeVirtualMachineFile Systemonpage 15. DevicemappingtoaSANlogicalunit(LUN).SeeRawDeviceMappingon page 15.

Fromthestandpointofthevirtualmachine,eachvirtualdiskappearsasifitwerea SCSIdriveconnectedtoaSCSIinitiator.Whethertheactualphysicaldiskdeviceis beingaccessedthroughSCSI,iSCSI,orFCcontrollersistransparenttotheguest operatingsystemandtoapplicationsrunningonthevirtualmachine.

Chapter 1 Using ESX Server with a Storage Area Network

Virtual Machine File System

Raw Device Mapping

Storage Area Network Concepts

iSCSI SAN Configuration Guide

Chapter 1 Using ESX Server with a Storage Area Network

Multipathing and Path Failover

Storage System Types

Target Compared to LUN Representations

iSCSI SAN Configuration Guide

Figure 1-2. Target compared to LUN Representations

Chapter 1 Using ESX Server with a Storage Area Network

iSCSI Naming Conventions

iSCSI Qualified Names

iqn.199801.com.vmware.iscsi:name1 iqn.199801.com.vmware.iscsi:name2 iqn.199801.com.vmware.iscsi:name999

Enterprise Unique Identifiers

iSCSI SAN Configuration Guide

Overview of Using ESX Server with a SAN

Benefits of Using ESX Server with a SAN

UsingESXServerwithaSANmakeshighavailabilityandautomaticloadbalancing affordableformoreapplicationsthanifdedicatedhardwareisusedtoprovidestandby services:

Chapter 1 Using ESX Server with a Storage Area Network

Finding SAN Configuration Information

iSCSI SAN Configuration Guide

TheVMwareStorage/SANCompatibilityGuideListscurrentlyapprovedstorage systems. TheVMwareReleaseNotesGiveinformationaboutknownissuesand workarounds. TheVMwareKnowledgeBaseHasinformationoncommonissuesand workarounds.

Basics of Using SAN Storage Systems with an ESX Server

Network Configuration and Authentication

Sharing a VMFS Across ESX Servers

Chapter 1 Using ESX Server with a Storage Area Network

Coordinateaccesstovirtualdiskfiles.ESXServerusesfilelevellocks,whichthe VMFSdistributedlockmanagermanages. CoordinateaccesstoVMFSinternalfilesysteminformation(metadata).

VMFS volume disk1 disk2 disk3

virtual disk files

iSCSI SAN Configuration Guide