ThisdocumentationisbroughttoyoubyopenQRMEnterprise [http://www.openqrm-enterprise.com/] DocumentVersion:24.02.2010 openQRMEnterpriseGmbH BerrenratherStrae188c 50937Kln/Germany Telefon:+49(0)221995589-10 Fax:+49(0)221995589-20 Mail:info@openqrm-enterprise.

com

openQRMisthenextgeneration,OpenSourceDataCenterManagementandCloudComputingPlatformdesigned to fully automate data centers and manage them in a scalable way. Among its many characteristics is a unique architecture thatunifiesphysicaland virtualmachine deploymentwithin a single managementconsole.openQRM integrates with all mainstream virtualization technologies and supports transparent P-to-V, V-to-P and also V-to-V migrations. openQRM's storage integration uses snapshots to clone servers for rapid deployment, backup/restore, server versioning,diskre-sizing and persistentcloud storage.openQRMalso providesN-to-1failoverallowing groupsof serverstouseasinglestandby.Italsohastheabilitytofailoverfromphysicaltovirtualmachines,andanopenAPI tointegratewithexistingbusinessprocessesandotherdatacenterrelatedtools. With its concept of plug-ability openQRM combines proven open-source products and commercial third-party components for data center administration, system- and service-monitoring, high-availability and automated provisioningwithinasinglemanagementconsole. HereascreenshotofopenQRM'sDatacenterDashboard:

The initial version of openQRM was developed by the Qlusters company founded in 2001. While Qlusters first concentrateson High-Performance-Computing (HPC)itchanged itsbusinessfocusto Data-CenterManagementin 2003.ThefirstopenQRMversionwasbasedontheJavaprogramminglanguageundfurtherdevelopeduntilversion 3.1.4whenthecompanycloseditsdoorsbeginningof2008.Amazingthat2005openQRMalreadyincludedafully automated provisioning system and the support for Virtual Machine deployment very much similar than todays conceptofCloudComputing,justitwasnamedUtilityComputing.Causedbyalongdevelopmenthistorythelast versionsofopenQRM3.xgrewverybig and complex.Because ofthatdevelopementprogressslowed down very muchinthelastyearsofQlusters.LuckilytheydecidedtochangetheinitialcommerciallicenseforopenQRMtoan open-sourceone(MPL)in2006.MatthiasRechenburgwasworkingforQlusterssincedayoneandfromthestartof theopen-sourcereleaseProjectManageroftheopenQRMproject.WhenQlustersshutdownin2008hedecidedto continueopenQRMasancommunity-drivenopen-sourceproject. At this point, starting with the 4.0 release version, openQRM got re-written from the scratch, its features and mechanismswereportedtothemuchlighterPHPlanguageanditslicensewasupdatedtoGPLVersion2.Within just2yearsofre-design,developementandQAtheopenQRMteamsucceededinoutperformingthefunctionalities oftheold,Java-basedopenQRMandgotthenewversionenterprise-ready.

Datacenterarealwayscustomandcomplexenvironments.Itistakesalotofeffortandhassletomaintainthem.The complexity originates from the number of involved subsystems and from the complexity of each subsystem. In a modernDatacentertherearealwaysquiteafewsubsystemsinvolvedlikephysicalservers,virtualmachines,different operating systems, network components, network configuration and network services like DNS and dhcpd plus a system-andservicemonitoring,backup/restore,out-of-bandmanagementandsoon.Preferredwouldbetohaveall those differentaspectmanaged centralized within a single administration console which isexactlythe goalofthe openQRMframework. Fittingthiskindofcomplexityintoasingleapplicationsoundsimpossible? openQRM'smain conceptisto breakDatacentersinto manageable subsystems.In openQRMeach subsystemis separately implemented via a plugin which provides the functionality to manage it. Then openQRM creates automated and generic interfaces between the different components via its completely plug-able software architecture.Actuallythe base serverofopenQRMisdesigned to justhave a single function,to manage plugins. That way new features like additional deployment-, storage- and virtualization types can be added to openQRM withoutchangingasinglelineofcodeinthebaseserver.Notonlythatthisconceptskeepsthebaseserveralways small,staticandrobustbutitalsoallowsseveraldevelopersworkingondifferentpluginsinparallelwithoutinterfering witheachotherschanges.

OneofthemainprincipalsofopenQRMisthatitgeneratesgenericinterfacesbetweendifferentsubsystemresulting in a standardized,flexible and dynamicscalable ITenvironmentwhile avoiding notnecessarydependencies.The specificsubsystemsareimplementedbyeitherOpen-Sourceorcommercialthird-partycomponentsbyfittingtheminto openQRM's Datacenter abstraction layer. To provide variety and specific sysadmin preferences openQRM always triestogiveseveraloptionsregardingtheimplementationofeachsubsystem.Anexampleforthataretheautomated monitoring solutionssupported and integrated in openQRMsuch asNagios3,Zabbixand Collectd plusopenQRM ownbasicmonitoringservice. Youshouldusewhatyoulikeandprefer! The differentVirtualization technologiesand approachesto unifythemalmostalltryto solve the same problemof

specific,sometimesevenclosedformatofthevirtualHarddisks.ThefactthateveryVirtualizationtechnologyisusing their own virtual Hard disk format makes migrating Systems, running in those Virtual Machines, to another Virtualizationtechnologyorevenbacktophysicalsystem. ToavoidanylockingtospecificVirtualizationVendorsopenQRMthereforeprovidesaunifiedvirtualizationlayerwhich sits on-top of each Virtualization technology and conforms them within the openQRM server. In openQRM the (server-)imagesaredirectlyconnectedfromthestorage,throughthenetwork,tothevirtualMachinesfromanytype. ViathisunifiedvirtualizationlayeropenQRMsupportsseamlessmigrationfromphysicalsystemstovirtualmachines (P2V),fromvirtualmachinesbacktophysicalsystems(V2P)andalsothemigrationfromVirtualizationtechnologyAto VirtualizationtechnologyB(V2V). openQRMalsoavoidsVendor-Locksonthestoragesubsystemsbyintegratingwithvariousstorageproviderssuchas NFS,iSCSI,AOE(Coraid),Equallogic,NetappandZFS. Sysadminsshouldbeabletochooseandselectthetechnologyfittingbestfortheirbusiness

The name openQRMmeansopen QlustersResource Managerand isbased on itscommercialhistory.To avoid confusionandtokeepthealreadybigopenQRMcommunitytheopenQRMteamdecidedtocontinuewiththisname afterthepreviousmainsponsorQlustersshutdowntheircompany.TheopenQRMProjectisnowafullycommunity driven,Open-SourceprojectbackedupagainbyitsnewmainSponsoropenQRMEnterprise.

AnimportantquestionaboutDatacentersis: ArewelookingatServicesorServers? Isitimportantforusthatspecificphysical(orvirtual)hardwarecontinuestoworkorisitmoreimportanttokeepthe servicesprovidedbythewholeDatacenterupandrunning? Sinceeverythingwhichmoves(likeCPUventilatorsandHarddisks)willbreakatsomepointitseemskindofserious toavoidbindingDatacenterservicestophysicalhardware.ButevenvirtualmachinesarerunningontheirHypervisors physicalhardware. But then, what is the best place to store our most important services ? The answer is : Modern Storage Systems Modern Storage appliances coming with out-of-the-box support for High-Availability, Scalability and Data-security through enhanced RAIDHard diskarraysallowing to hot-swap failed diskwithoutinterruption.Allseriousfilesand datawithinaDatacentershouldbestoredonthosekindofstoragesystemstoensuredata-availability,itsintegrity andtohaveasingleplaceforbackupandrestore.Attheenditwillalsohelptomakeyoursysadminsleepbetter openQRM'sbest-practicerecommendationistoavoidusinglocalHarddisksbecausetheyarethefirstthingwhich failsinphysicalservers. Andhowweusuallyinstallserver? Servergetsorderedandplacedinthedatacenter Itgetsconnectedtothenetwork

TheoperatingsystemisbeinginstalledonthelocalHarddisks TheOSgetsitsspecialconfiguration Applicationarebeinginstalledonit TheApplicationsgetstheirfinalconfigurationanddata ThesystemisbeingaddedtotheBackup Monitoringisbeingsetup

WhenknowingthattheHarddisksaretheweakestpartofourphysical(orvirtual)hardwarewhywestillbind themostimportantparts,theservices,applicationsandserversrunningontheOS,toit? With itsunique architecture openQRMprovidesa genericDatacenter-Abstraction layerwhich completelyseparates the services from physical servers or virtual machines by storing and using them directly from a robust and high-availablecentralizedstorage. Fromthe pointofviewthatthose Datacenterservicesare the mostimportantpartthe openQRMteamespecially lookedatLinuxserverandaskedthemelves: Whatisarunningserversystem? Arunning(Linux-)serverconsistofthefollowingcomponents: aLinuxKernel,whichisafile somekernelmodules,whicharefiles aninitialRamdisk(initrd),whichisafile aroot-filesystemcontainingtheapplicationsandservers,whicharealljustfiles WhenarunningLinuxsystem,providingalltheimportantDatacenterserviceswewouldliketokeepupandrunning allthetime,isjustabunchoffilesthenweshouldstarttreatingthemasfiles.Firstthingistostoretheminasave place, on a modern Storage systems (there are several more advantages of using modern Storage systems

explainedlater). Nextspecialobservationwasabouttheinitrd-stage,whenLinuxsystemsareusingasmall,initialramdisktopre-setup theinitprocedurebyrunning/sbin/init.TheonlypurposeandresponsibilitytheinitrdhasinLinuxistofindandmount the root-filesystem, normally located on a local Hard disk. openQRM uses this generic Linux mechanism in an enhancedwaybymakingtheprocessoffindingandmountingtheroot-filesystemcompletelyplug-able.InopenQRM the specific storage-plugin tells a booting system within the initrd stage from where and how it should mount its root-filesystem. Sincestorage-typesarealsoplug-ableinopenQRManykindofexternal,remotestorgedevicescanbeused! Onefunnyexampleisapossiblegmailfs-storagepluginwhichwouldsupportbootingupsystemsfromGoogle-Mail accountstorage..sure,justafunnyexample,wouldbeslowasdrybreadbutontheotherhandGooglewould careaboutyourbackups. PleasefindoutmoreaboutuniquebootconceptinthenextchapterTheopenQRMBootConcept

RegularbootfromHarddisk Systemisbeingturnedon Biosreadsboot-sectorfromfirstHarddisk Boot-managerisbeingexecutedfromtheDisk Boot-managerloadsOperatingSystemfromlocalHarddisk OperatingSystemisbeingexecutedandloadsitscomponentsfromthelocalHarddisk OperatingSystemstartsapplicationsandservices NowatsomepointtheHarddiskwillbreakandloadingtheboot-managerand/ortheoperatingsystemwillfail. HerehowopenQRMovercomesthissituationwithacentralized,networkboot-manager: TheopenQRMway Systemisbeingturnedon Biosisconfiguredtodoanetwork-boot(PXE) SystemsendsaPXErequestandasksforanautomatednetwork-configurationviadhcp openQRM'sdhcpd-serveranswerstherequestandprovidesanip-address Systemactivatesitsnetwork-configurationandreadsitsPXE-configurationfromtheopenQRMserver Systemdownloadsitsoperatingsystemkernelandinitialramdisk(initrd) SystemexecutestheOperatingSystemkernelandloadstheinitialramdisk Withintheramdisknetwork-hardwareisbeingautomaticallydetectedandinitialized Havingfullnetworkconnectionthesystemnowdownloadsitsfullsetofkernelmodules Additionalhardware-detectionwithallavailablekernelmodules SystemgetsconfigurationparametersfromtheopenQRMserver Atthisstageallavailablehardwareisdetectedandthenetworkisfullyinitialized.Herethesystemcancontinuein twodifferentways: 1. Asidle,free(meaningavailable)serverresource 2. Asanactive,assignedresourceactingasaserviceprovider Ifthesystemisnotassignedyet(1)itwillsimplystaywithinitsinitial,minimalramdiskwaitingforfurtheractionsfrom

openQRM.Incasethesystemisactiveassignedtoastartedappliance(2)herethefurtherstepsitwillexecute: Systemchecksthedeploymenttypeandmethodoftheserver-imageassignedtoitsappliance According to the deployment type the system downloads image-deployment hook provided by specific storage-plugin Systemexecutesmount_rootfunctionprovidedintheimage-deploymenthook Theimage-deploymenthookmountstheserver-image-locationfromtheremotestorageread-writeable Kernel-andkernel-modulesfilesarebeingtransferredtothemountedroot-filesystem TheopenQRM-clientisbeinginstalledontothemountedroot-filesystem Theimage-deploymenthookre-mountstheserver-image-locationread-only Systemcontinueswithregularinit(running/sbin/initontheroot-filesystem) DuringfutherinitprocedureoftheSystemtheopenQRM-clientisstarted AccordingtotheactivatedpluginsinopenQRMtheSystemnowstartsfurtherpluginservices

Andwhatifthenetworkfails? thentheprovidednetworkservicesoftheDatacenterwillnotworkanywayandoneshouldfixthenetworkasap.

For the past openQRM used a combination of hwsetup, kudzu-knoppix and hwdata to automatically detect

hardwareduringbootup.AdaptedfromKnoppixthisworkedgreatforalongtime.Whilethismethodmeanwhileis notverywellmaintainedanymoreitrecentlycreatestroublesespeciallyonCentOSKernelsasreported. Toovercomethissituationweresearchedanew,muchmoreefficientwayforopenQRM'shardwaredetectionusing pcimodules.Thepcimodulescommandisavailableasapatchforthepcituilspackageandsimplylistsallneeded kernelmodulesaccordingthepciidsofthedetectedhardware.UsingthistoolopenQRM'shardwaredetectionnow couldbereducedto3linesofcode:

for module in $(pcimodules) ; do modprobe -s -k "$module" done

WhiletestingthisnewpcimoduleshardwaredetectionmethodonphysicalandvirtualsystemsourQAreportedthat much more hardware was detected compared to the previous hwsetup utility. Even USB devices are detected flawlessly.

DefinitionofaResource
AresourceinopenQRMiseverythingwhichhasaCPUandsomememory.ResourcesinopenQRMhavedifferent typessuchasPhysicalSystems,VirtualizationHostorVirtualMachine.AccordingitstypeopenQRMinterfacesand communicateswiththespecificresource. SincetheopenQRMplatformisrecommending(notforcing)toavoidusinglocalHarddisksaresourceinopenQRMis justCPUandmemory.Theresource'slocalHarddiskisingeneralNOTpartoftheresourceitselfbutcanbeusedfor e.g.swapspaceorevenforlocal-deployment.IngeneralopenQRMmainconceptisbasedonrapid,image-based deploymentandtheserver-imagesofthecurrentrunningservicesarelocatedonremotestoragedevices. TheopenQRMteamrecommendsto use the LocalHard disksavailable in resources(e.g.physicalservers)asthe localswapdevicesforvirtualmachinesrunningonthissystem.Makessensetohavetheswapspacelocal. Otherthanthatthelocaldisksstillcanbeusedasstorageforapplicationsorcustomerdata.Onejustneedtokeep inmindthatthisagainbindstheservicecurrentlyrunningonthespecificresourcetoitslocaldisk.Inthiscaseitisof courserecommendedtohaveaseparatedbackupprocedureforthelocaldiskdata.

Kernel
Kernels in openQRM are Linux Operating System kernels which can be assigned to resources. This happens automatically through the appliance model through openQRM's integrated, centralized network-boot-manager PXELINUXfromtheSyslinuxproject[http://syslinux.zytor.com] .

Image
Thedefinitionofanimage(server-image)inopenQRMisthatitislocatedonanetwork-attachedstoragedevice(NAS or SAN) and contains a root-filesystem of an Operating system supported by openQRM. The image is completly self-containedand,viatheappliancemodelitandincombinationwithakernelcanbestartedonanyavailable(idle) resource. Theserver-imageroot-filesystemmaybeaminimalOperatingsysteminstallationwhichisthenfurtherleveraged(e.g. viathePuppetintegration)oritalsocanbeafullinstalledcompletelypre-configuredsetofapplications.Animage canevenbeasnapshotofanexistingserveroracloneorcopyofananexistingserver-image.

Appliance
Appliances in openQRM representing one (or more) of the actual services which should be provided by the Datacenter.Anapplianceisthecombinationofakernel,an(server-)image,aresourceandservicerequirementsplus service-level-agreements (SLA). With those informations openQRM then fully automates the management of the specificservicesrunningontheappliance'sserver-image. Appliances are justlike TVs,ittakes a single buttontostartorstopthemandthey willaways provide the desiredservice.

Please check the chapter about High-Availability to see how openQRM automatically keeps Datacenter services running.

Storage
Astorage componentin openQRMconsistsofan integrated resource containing some kind ofnetwork-attachable storage (NASorSAN).Storagesin openQRMare providing the image-locations,meaning the place where serverimagesarestoredanddirectlyattachedtoresourceasrequired.BycreatingastorageserveropenQRMthenexactly knows how to interface with its specific storage technology and further allows automated management of the availablestoragespaceandvolumes. openQRMsupportsawholebunchofdifferentstorageservertypessuchasNFS,iSCSI,AOE,Equallogic,Netapp andZFSbutitalsoprovidesowncustomstoragetypessuchasthelvm-storagepluginwhichisbasedonLogical

VolumeManagement(LVM).Asvirtualizationtypesstoragetypesarefullyplug-ableinopenQRMsothatnewcustom storagedevicescanbeintegratedeasily. Just like appliances the storage resources are also running the openQRM-client service which allows remotemanagementofthestorage-subsystembyopenQRM.

HereascreenshotofopenQRM'sPlugin-Manager:

ThePlug-inManager
SinceitsPlug-insareprovidingallthefeaturesoftheopenQRMServerthePlug-inManageristhecentralpointto enable(ordisable)additionalfunctionalitiesfortheopenQRMmanagedenvironment.Bydefaultitpresentsalistof allavailablePlug-insplustheirstate.Plug-insinopenQRMcanhave3differentstates: Disabled(Stopped) EnabledandStopped

10

EnabledandStarted ThePlug-instatecanbechangedeitherthroughtheactioniconsinthespecificPlug-inroworviathegroupactions buttonattheendofthelist. DetailedPlug-instatesdescription Disabled(Stopped) ThespecificPlug-infunctionalityanditsmenuentryaredisabled EnabledandStopped ThespecificPlug-infunctionalityisinitializedanditsmenuentryisenabled.EventualPlug-inservicesarenotstarted. EnabledandStarted ThespecificPlug-infunctionalityisinitialized,itsmenuentryisenabledandeventualPlug-inservicesarestarted. Enabling,Disabling,StartingorStoppingaPlug-ininopenQRMresultsin4differentPlug-inactions: Plug-inInit Plug-inStart Plug-inStop Plug-inUninstall ForeachactionopenQRMsubmitsaPlug-incommandtoitsinternalcommandqueuewhichthenrunsthecommands throughtheRemotecommandexecutionsystem(nextChapter). DetaileddescriptionofthePlug-incommands Plug-inInit ForthePlug-inInitactionopenQRMruns:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/PLUGIN_NAME/etc/init.d/openqrm-plugin-PLUGIN_NAME init ADMIN_USER ADMI

Plug-inStart ForthePlug-inStartactionopenQRMruns:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/PLUGIN_NAME/etc/init.d/openqrm-plugin-PLUGIN_NAME start

Plug-inStop ForthePlug-inStopactionopenQRMruns:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/PLUGIN_NAME/etc/init.d/openqrm-plugin-PLUGIN_NAME stop

Plug-inUninstall ForthePlug-inUninstallactionopenQRMruns:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/PLUGIN_NAME/etc/init.d/openqrm-plugin-PLUGIN_NAME uninstall ADMIN_USER

For the Init and Uninstall actions a valid openQRM administrator user and password are required. Those parametersareusedbythePlug-instocreate(ordrop)additionalPlug-intablesintheopenQRMdatabase.

11

ManualrunningPlug-inactions ByrunningtheabovelistedPlug-inactioncommandsPlug-insstatescanalsobechangedfromthecommand-line. PluginHooks openQRMoffersagreatselectionofPlug-inHooksallowingeachPlug-intoruncommandswhene.g.resourceor appliancestatechanged.ThismechanismisdescribedindetaildinthelaterPlug-inDevelopementsection.

RemoteCommandExecution
openQRM'sFrameworkfor(remote)commandexecution As the central management tool of complete IT environment openQRM needs to be able to run administrative commandsontheopenQRMserveritselfbutalsoonotherintegrated,remotesystemswithintheopenQRMmanaged network.ThereforeopenQRMprovidesaSSL-securedremotecommandexecutionsubsystemconsistingof3major components: AdropbearServerautomaticallydeployedandstartedoneveryresourcemanagedbyopenQRM Thepublic-sharedSSLkeyoftheopenQRMServeralsoautomaticallydeployedtoeveryresource ThedbclientutilexecutedbytheopenQRMcommandqueue Dropbear is a minimal SSH-Server and Client which openQRM uses to create a secure transmission channel betweentheopenQRMserveranditsmanagedresources.Viaapublic-sharedSSLkeymechanismopenQRMgains password-less,secureaccesstoallsystemswithinthemanagednetwork.Alsocommandswhicharebeingrunonthe openQRMserveritselfareusingthissafemechanism. FlowofanopenQRMcommand 1. PHPresourceclasssendsacommandthroughitssend_commandmethod. 2. Via the openqrm-executil(OPENQRM_SERVER_BASE_DIR/openqrm/sbin/openqrm-exec)thismethod puts the new command plus its parameters and unique token in the openQRM command queue (OPENQRM_SERVER_BASE_DIR/openqrm/var/spool/)usingthetokenforthecommandfilename. 3. From there the command is being picked up by the openqrm-cmd-queue (OPENQRM_SERVER_BASE_DIR/openqrm/sbin/openqrm-cmd-queue), a service started by the openQRM Serverinitscript. 4. The openQRM command queue sequentially runs the commands and executes them on the specified (remote)hoststhroughthedbclientutil. AllcommandsinopenQRMarefullyloggedtosyslog.FailedcommandsarerecognizedbytheopenQRMcommand queue. In case a command fails in the queue it is re-scheduled to run 3 times. After a command failed 3 times openQRMgeneratesanError-EventwhichshowsupintheCentralEventlog.Intheeventlogsysadmingetabrief overviewofthecommandwhichfailed,itserrormessageandtheopportunitytoRe-Runit.

openQRMisdesigned with a infinite linearscalable architecture providing flexible solutionsforcustomDatacenter. DependingontherequirementsoftheITenvironmentmanagedbyopenQRMthereforeitoffersnearlyinfiniteways tosetitup.Following3exampleUse-casesandtheirsystemrequirements: SimpleProof-of-ConceptSetup 1PhysicalSystemdedicatedfortheopenQRMServer

12

VTCPUExtension(fullVirtualizationSupport) Afreepartitiondedicatedfortheserver-imagestore(atleast20GB) 1GBMemory(themorethebetter) ItisnoproblematalltorunopenQRM,theStorage-andVirtualization-partonasinglesystemfore.g.aPOCsetup. BasicSetup 3PhysicalSystems(onefortheopenQRMServer,oneforStorageandoneasaVirtualizationHost) VTCPUExtension(fullVirtualizationSupportforthesystemdedicatedasVirtualizationHost) A free partition dedicated for the server-image store (at least 100 GB) on the system dedicated for the Storage The basic setup basically starts to distribute the Storage- and Virtualization components to external servers and thereforeprovidesabetterperformanceandscalability. ProductionSetup 4+NPhysicalSystems(twoforopenQRMServerHA,NStorageandNVirtualizationHosts) VTCPUExtension(fullVirtualizationSupportforthesystemsdedicatedasVirtualizationHosts) Foraproductionsetupitisrecommendedtotakecareofthe high-availabilityofopenQRM.Thisisarchivedbyan active-passive setup for the openQRM Server (e.g. using Linux-ha). For perfect scalability openQRM supports an unlimitednumberofStorage-andVirtualizationresources.Newresources(Storageand/orVirtualizationHosts)can beaddedtotheopenQRMmanagednetworkatanytime. Scalingup Since openQRMfullysupportsdynamicITenvironmentsitispossible and easyto scale froma simple setup to a basiconeoreventoaproductionsetupatanytime.

openQRMdistinguishesbetweendifferenttypesofresourcesinthemanagednetwork.ThereareStorageresources, VirtualizationHostsresources,VirtualMachineresources,SystemsdedicatedforrapiddeploymentandtheopenQRM Server itself. According to the resource type openQRM includes support for various different Operating Systems, Storage-andVirtualizationappliances. SupportedOperatingSystemsfortheopenQRMServer: Ubuntu Debian CentOS ItisrecommendedtoalwaysusethelatestversionoftheabovelistedLinuxDistributions. SupportedOperatingSystemsforrapiddeployment: Ubuntu Debian CentOS Fedora Suse/openSuse Windows(XP/Windows7)

13

Solaris/openSolaris AlsofortheSystemsdedicatedforrapiddeploymentitisrecommendedtoalwaysusethelatestversionoftheabove listedLinuxDistributions. SupportedStorageAppliances: DellEquallogic NetAppFiler Solaris/openSolarisZFSStore(e.g.Nexentastore) Additionalto those third-partyStorage providersopenQRMcomeswith a whole setofown,customstorage types which are turning a simple Linux Box with an LVM partition into a rapid cloning, robust server-image store for openQRM. SupportedVirtualizationVendors VMWareServer1 VMWareServer2 VMWareESX/VMWareVsphere CitrixXenServer Xen KVM

openQRM comes with a complete generic deployment abstraction layer and appliance model which conforms provisioningofphysicalsystemandvirtualmachinesfromanytype.Firstthingtodotodeployaserver-imagetoa physical server is to integrate its resource into the openQRM environment. Adding new physical resource to openQRMiswayeasy.Turningon network-boot(PXE)in theserversBIOSand powering themon isenoughfor openQRMtoauto-discoverandaddnewservers.AsexplainedindetailinthesectionopenQRMBootconceptthe startedserverwillautomaticallybootviathenetworkandappearasnew,idleresourceinopenQRM'sresourcelist. Please notice : To initialize and start openQRM's network-boot environment the dhcpd and tftpd plugin needstobeenabledandstarted! ThroughopenQRM'sVirtualizationPlug-insvirtualMachinesarecreatedandaddedtoopenQRMinthesameway. VirtualMachinesfromdifferenttypesaresimilarlycreatedviatheVirtualizationtypespecificVM-Managerwhichallows to set various VM parameters such as the VM name, amount of memory, number of cpus and its virtual network connection. The Virtual Machines boot sequence is then set to netboot (PXE) allowing them to get seamlessly addedtoopenQRMinthesamewayasphysicalsystems.TheywillappearinopenQRMresourcelistasnew,idle resourcefromthespecificVirtualMachinetype.

AllprovisioninganddeploymentiscontrolledthroughopenQRM'sBaseMenusectionusinganappliancemodel. The appliance model turns the complex workflows of deploying a new system according a bunch of various configuration parameters, requirements and SLAs into one, single mouse click. Following the basic rules of an applianceinopenQRM: 1)AppliancesareprovidingtheDatacenterservices 2)Anapplianceconsistof

14

AnOperatingSystemKernel AServer-Imagecontainingaroot-filesystem AResource,actingastheServicecontainer Additionalconfigurationparametersdefiningservicerequirementsandservicelevelagreements 3)Everycomponentofanappliancecanbetransparentlyexchanged/replaced 4)Eachcomponentcanbemanagedseparatelywithoutanyfurtherdependencies 4)Appliancesarestartedorstoppedbyasingleaction 5)Plug-inHooksinopenQRMtakingcaretosetupandconfigureallinvolvedappliancesubsystems This appliance model is a central concept in openQRM's framework and all integrated subsystems are unexceptionallyfollowingit.

Virtualization Hosts (Hypervisors) consisting of physical systems which are being parted into several autonomous virtualmachines.openQRMdeploysVirtualizationHostsviaitsgenericprovisionconceptusingtheappliancemodel. To enable managementofa Virtualization Hostthe Hosts-appliance resource-type needsto be setto the specific VirtualizationtechnologyHosttype.

VirtualMachinesinopenQRMarenetwork-booted,deployedandmanagedinthesamewayasallotherresources. Pleasenotice: ThereareseveralVirtualizationpluginsavailablewhichbypassthenetwork-bootandsupportlocal-deploymentand local-boot for Virtual Machines. Examples for those kind of Plug-ins are the kvm-storage and the xen-storage Plug-in which are emulating the idlestate ofitsvirtualresourcesand using openQRM'shookforvirtual-resourcecommands. The virtual-resource-command hook provides a method to map directly executed resource commands such asrebootto itsvirtualcomplementwhich isthen executed on the virtualresourceshost(more detailsin the developmentsection). ThosePlug-insaddingsupportfordeploymentandmanagementofOperatingSystemswhichdonotsupportrapid deploymentvianetwork-bootsuchasWindowsandSolaris/openSolariswithinVirtualMachines.

SinceopenQRM'srapiddeploymentmethodsarebasedoncentralizedStorageSystemstheyareaakeycomponent in the openQRM management network. The Storage layer in openQRM is providing the remote, (server-) image location.DependingontheStoragetypethisimage-locationcanbeaNFS-export,aniSCSILun,anAOEVolume, anykindofremoteblock-deviceoranythingelsewhichcontainsavalidroot-filesystemcontent. Similar to Appliances, Storages in openQRM consist of a resource which is already integrated and available in openQRM.ThereforethefirstthingtodotocreateanewStorageinopenQRMistoadditsresource.Thiscanbe donebydeployingastorageserver-imagetoanexistingidleresource.IncasesanalreadyexistingStorageServer shouldbeusedforopenQRMdeploymentitsresourceeasilycanbeintegratedthroughthelocal-serverPlug-in. SomeofthesupportedcommercialStorageVendorsofferingkindofStorageApplianceswhichareclosedsystemnot allowingtobedirectlyintegratedasmonitoredresourcewithinopenQRM.ExamplesforthatkindofStorageProviders

15

aree.g.NetAppandEquallogic.ForthosePlug-insthespecificStorageresourceshouldbemanuallycreatedtothe resourcenewWebform. Pleasenotice: AllresourcesmanualaddedthroughtheresourcenewWebformwillbeautomaticallyexcludedfromopenQRM basicmonitoring.Theywillalwaysappearinunknownstate(yellowicon)inopenQRMsinceitisnotpossible(ornot supported)torunadditionalthird-partycomponents,liketheopenQRM-Client,onthosestorageappliances.

To enable support for infinite different Storage Technologies in openQRM the Storage types are Plug-able. New StoragetypesareaddedtoopenQRMviaitsStoragePlug-inswhichalsoprovidethedefinedinterfaceformanaging thespecificStorageVendor. HerealistofdifferentStoragePlug-insinopenQRM: aoe-storage ThisPlug-in providesan AOE-Storage-and deploymenttype supporting too bootresourcesdirectlyfroman AOE Storage-Server.Italso providesthe Storage-Serverpartwhich turnsa simplyLinuxbox+AOE-and Vblade-Tools installedintoanAOEStorage-ServerfullymanagedbyopenQRM. equallogic-storage This Plug-in integrates Dell's Equallogic Storage Appliance into the openQRM framework. It supports seamless managementofEquallogicStorageSystemsandthecapabilitytobootresourcesdirectlyfromtheEquallogic'siSCSI Luns. netapp-storage The netapp-storage Plug-in interfaces with NetApp Filer Appliances allowing resources to directly boot from the NetAppiSCSILuns.SameasforallStoragePlug-insitincludesanembeddedmanagementfortheNetAppFiler. nfs-storage Thenfs-storagePlug-inisalmostthesimplestStorageTypesupportedbyopenQRM.Anexporteddirectoryincluding aserversroot-file-systemcontentisenoughforopenQRMtoenableresourcebootingdirectlyfromaNFSStorageServer.Server-imagesonNFS(NFSROOT)providingagenerictransferlayersotheirareaverygoodcandidatefor server-image-templateswhicharethenautomaticallytransferredtoe.g.iSCSILuns,AOEVolumesoreventolocal block-devices. local-storage The local-storage is an exceptional Storage Plug-in in openQRM. It consist of a NFS Storage-Server with an underlaying Logical Volume Management (LVM) taking care of the supports for rapid cloning. Its deployment mechanismprovidesagrab,deployandun-deploylife-cylcleforresources.Itsgrabstagetransfersthecontent ofallattachedHarddisksofaresourcetotheremoteNFSStoragelocationusingabinaryimageformat(dd).The deploystageofthelocal-storagePlug-inre-transmitsthosebinaryserver-imagesbacktothelocalHarddisksofthe sameorotherresources.Withitsun-deploystagewhichautomaticallyupdatestheserver-imageontheStoragein case the involved appliance is stopped it keeps a 1:1 relation between the specific server-image on the remote StorageandthelocalHarddiskofaspecificresourcedeployedviaanappliance. Pleasenotice: The local-storage Plug-ins life-cycle will re-set the involved boot-sequence after the deploy stage to local-boot insteadoftheregularnetboot.Afterun-deploymentitautomaticallysettheboot-sequencetonetbootagain.

16

iscsi-storage ThisPlug-inintegrateswiththeEnterpriseiSCSITarget,anOpen-Sourceimplementationwhichisfreelyavailablein many modern Linux Distributions. It automatically manages the iSCSI Targets configuration and exported blockdevicesplusitsupportstobootresourcesdirectlyfromthoseiSCSILuns. lvm-storage Thelvm-storagePlug-inisacombinationoftheaoe-storage,theiscsi-storageandthenfs-storagePlug-infeaturing anunderlayingLogicalVolumeManagement(LVM)tosupportrapidcloningandsnapshots.Itincludesanembedded managementconsolefortheLVMvolumeswithintheopenQRMUI. xen-storageandkvm-storage ThosePlug-insprovidingaVirtualization-(Xen,KVM)andStorage-part(LVM)withinasinglePlug-in.ItsVirtualization part is using the local (local to the Virtualization Host) LVM volumes to attach them as local (local to the Virtual Machine)Hard disk.ItsStorage partincludesfulladministration capabilitiesforthe Storage ServersLVMvolumes. ThePlug-insalsofollowingstrictlyusingopenQRM'sappliancemodelbut,otherthantheotherStoragetypesbased on direct network-boot deployment, they are booting their Virtual Machines from their local attached, virtual Hard disks. Independent from openQRM usual network-deployment those Plug-ins providing the support for Non-Linux OperatingSystemssuchasWindowsandSolaris/openSolarisinopenQRM. Pleasenotice: Thisresultsinadependencytolocal-diskdevicesontheStorageHost.ThatmeansthatVMsfromthistypemustbe locatedonthesameStoragehostwherethelogicalvolume(theVMsroot-disk)islocated tmpfs-storage ThetmpfsStoragePlug-inprovidesthepossibilitytodeploy(Linux-)Systemin-Memory.Dependingontheimageconfigurationitcreatesatmpfs-root-mountpointwhichisbeingpopulatedviainstall-from-nfs. sanboot-storage Sanboot-storageintegratesGPXE(etherboot.org [http://etherboot.org] )intoopenQRMasasecond,optionalnetworkbootloader.Itsupportsanimage-basedWindows-deployment.UsingSanbootWindowssystemsaredirectlyfroma iSCSITargetorAOE-Storage. zfs-storage TheZFSStoragePlug-inallowstobenefitfromSun'snewrobustZFSfile-systemprovidedbySolaris,openSolarisor FreeBSDsystems.TheZFSfile-systemisknownforitsawesomefeaturessuchashighstoragecapacities,integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checkingandautomaticrepair,RAID-ZandnativeNFSv4ACLs.TheZFSStoragePlug-ininopenQRMdependson the solx86 Plug-in which integrates Solaris, openSolaris and FreeBSD systems seamlessly into the openQRM framework.ItsupportsbootingresourcesdirectlyfromtheremoteZFSfile-systemthroughtheiSCSIprotocolandalso providesthestoragemanagementcapabilitiesembeddedintheopenQRMUI.

openQRM unifies and automates the administration of the different Storage Providers by its integrated Storage Management. Therefore, during the design phase of openQRM, the openQRM team especially analyzed the frequentandcommonstorageactionsrequiredintheopenQRMenvironment.Thoseare: CreateanewVolumewithagivennameandSize RemoveaVolumewithagivenname

17

CloneaVolumewithagivenname(createsa1:1copy) AuthenticateaVolumeagainstaresource IfsupportedbytheStorageTechnologytherearealsothefollowingStorageactionpossible: SnapshotaVolumewithagivensnapshotnameandsnapshotsize ResizeaVolumewithagivennameandanewsize AllthoseStorageactionsareimplementedbytheirspecificStoragePlug-inandexposedthroughaPlug-inspecific Volume manager. While the Create, Remove, Clone, Snapshot and Resize Storage actions are executed by the sysadmintheauthenticateactionishandledfullyautomaticallybyopenQRMviaanStorage-Auth-Hookasdescribed inthefollowingchapter. Storage actions are also integrated within the openQRM Cloud to automate the Storage management for rapid deploymentthroughtheCloudPortal.

openQRM's rapid deployment is all about centralized server-management through network-booting and directly attachedremoteroot-file-systemstorage.Tosecurethestorage-andmanagement-networkandtoensureonlythe resourcededicatedforaspecificapplianceisallowedtoaccessandmountaremote(server-)image-locationfroman externalStorageServeropenQRMautomaticallytakescaretoauthenticatetheresourceagainstitsimage-location. Thishappensthrough a Storage-Auth-Hookprovided bythe Storage Plug-ins.According to the appliance imagedeploymenttypethishookautomaticallygetsexecutedintheopenQRMServerenginewhenanapplianceisstarting orstopping.Thehookincludesallrequiredparametersandinformationsabouttheappliancewhichisthenusedby thespecificStoragePlugintoenableordisableaccesstoanimage-locationontheStorageServer. AdetailedfunctionsdescriptionoftheStorge-Auth-hookisavailableinthePlug-inDevelopmentsection.

In openQRM Virtualization Hosts are managed through the appliance model. The specific resource-type in the appliance configuration tellsopenQRMwhich Virtualization interface type to use.Forthisreason the Virtualization Host needs to be integrated and available in openQRM. This can be done by deploying a Virtualization Host server-image to an existing idle resource. In cases an already existing Virtualization Host should be used its resourceeasilycanbeintegratedthroughthelocal-serverPlug-in.

Notonlythestoragetypesbutalsothevirtualizationtypesarefullyplug-ableinopenQRM.ViaitsopenpluginAPI openQRMintegratswithVMware-ESX,Vmware-Server2,Vmware-server(1),Xen,KVMandCitrixXenServer.Adding supportforfurthervirtualization technologieslike Virtualbox,openVZand othersison the future road map.To be able to seamlessly handle all those different kinds of virtual machines openQRM puts a layer on top of the virtualization methods to unify their management. In openQRM virtual machines are simply net-booting into the openQRMmanagementenvironmentinthesamewayasphysicalsystems.

Continuing with the full separation between hard- and software, meaning on one side physical- and also virtual machines(becausetheVmsarerunningontheHypervisorwhichisrunningonthebaremetal)andontheotherside thesoftwarelayer,the server-imageslocated on a safe storagedevice,in an openQRMenvironmenta Hypervisor

18

becomesjustaresourceprovider,justbeingresponsibletohostthevirtualcomputeresourceoftheuserschoice. ThatwaytheappliancerunningonavirtualmachinealsogetsfullyindependentfromitsHypervisorHostandcanbe transparently(live-)migrated to anotherHypervisorsofthe same ordifferentvirtualization technologyoreven from physicalsystemstovirtualmachinesandtheotherwayaround.OpenQRMsupportsP2V,V2P,V2V,P2Pmigration withoutanychangesontheactualserver-imagesitself. OpenQRMcannotonlymanagedifferenttypesofHypervisortechnologiesbutitcanalsodeploythemviatheregular genericdeploymentmechanismofitsframework.ThatoffersscalabilityforthecompleteITinfrastructurebecausethe data-centercangrow(andshrink)asdemandedbyjustadding(orremoving)moreHypervisors.

SinceeachVirtualizationTechnologyhasdifferentadvantages(andeventualdisadvantages)itshouldbeselected dependingonwhichapplicationservicesshouldbevirtualized.Thatmeansfordifferentpurposesoneshouldchoose the Virtualization type fitting bestto the appliacations.Italso meansthatin a perfectworldsystemadministrator endingupwithmanaginglotsofdifferentVirtualMachineTypesandTechnologiesplusthephysicalsystemactingas Virtual Machine hosts. By abstracting physical and virtual machines as Data center resources openQRM is the perfecttoolfortheperfectworldscenariosinceitseparatestheactualservice(OS+applications)fromthephysical orvirtualmachine (the resource)itisrunning on and therefore allowsthe administratorto easilyand dynamically adapttheresourcetype(theVirtualizationtype)ofservicesaccordingtotheservicedemands. ThisuniqueabstractionofthedifferentVirtualizationTechnologiesinopenQRMalsohasanotherhugeadvantage: openQRMavoidsandpreventsVendorlocking! With openQRMswitching fromone Technologyto anotherisabsolute transparentand doesnoteffectthe actual service(theserver-image)atall.

MostlyallrapiddeploymentmethodsinopenQRMtakingadvantagesofmodernStorageServerandfocusingonan image-baseddeployment.ThatmeansthatsystemsarenotinstalledbyopenQRM,whichofcoursecanbedone withopenQRMinanfullyautomatedwayusingtheLinuxCOEPlug-inintegration,butsystemsarejustconnected throughthenetworktoready-to-runserver-imagesonremoteStorageserver.Hereafewofthegreatadvantagesof theimage-baseddeploymentcomparedtoregularprovisioningvia(manual)local-installation. Server-imagesarebasedonwell-known-to-workserver-templates,workingandpre-configuredout-of-the-box Server-imagessavestheinstallationstepandtime.DeploymenttimeisequalBoottime. Server-imagescanbeefficientlysnapshotted.Snapshotsjuststorethechangesbetweentheoriginimage Server-imagescanbecloned-on-demand.Evensnapshotscanbedeployedseamlessly Server-imagesguaranteeareproducibleIT-environment Server-imagescaneasilybeingbacked-upandrestored Server-imagesprovidesasingle-placeforupdates Server-imagescanbere-visioned.Thinkofitlikesubversionforyourservers Server-imagesarejustfiles.Treatthemlikefiles. andmuchmore SinceopenQRMsupportsawholebunchofdifferentStoragetypesitprovidesalogicalabstractionwhichmakesit reallyeasytousethedifferentserver-imagestypes. An image in openQRM consist of different configuration parameters such as its Storage Server, its root-device identifierandseveralotherdeploymentparameters.ThetwomostimportantparametersareonwhichStorageServer theimageislocatedplusastorage-typedependentroot-deviceidentifierprovidingtheexactinformationwherethe

19

imageislocatedontheStorage. On NFS-based Storage Server this root-device identifier is the exported path to the image root-file-system, on iSCSI-basedStoragesitistheLunIDandonAOE/CoraidbasedStoragedevicesitistheshelfandslotnumberof the exposed volume. Since the root-device identifer it is also a plug-able component provided by the Storage Plug-ins. WhencreatinganewimageopenQRMfirstaskonwhichStorageserveritsimage-locationislocated.Afterselecting the Storage Server openQRM contacts the Storage and triggers it to send its list of current available root-device identifiers.Thislististhenusedtopopulatetheroot-identifierselect-boxintheimagenewform. Pleasefindadetaileddescriptiononhowtheroot-identifierhooksisusedinthedevelopmentsection. RapiddeploymentinopenQRMisbasedonnetwork-bootandresourcesarealwayspassingtheinitrdstagetaking care to mount the server-image's root-file-system content from the (remote) Storage Server. Within this stage the resource enjoysfullnetworkconnection plusthe capabilityto download additionaltoolsprovided byPlug-ins.The initrdstagealsoprovidessomeunique,greatwaystotransferroot-file-systemcontentfromoneStorageproviderto another. One thing which alltransfermethodshave in common isthattheyrequire a NFS-based Storage systemeitheras source ordestination.Therefore itisan advantage to have atleastone NFS-based Storage Serveracting asan storeforgoldenimages(server-imagetemplates). Pleasenotice: All of the below listed server-image transfer methods are configured as single shot meaning the transferconfigurationisbeingresetedafterthefirstdeploy. Followingadescriptionoftheavailabletransfermethods.

TheinstallfromNFStransfermethodisconfiguredthroughtheimageneworimageeditformviatheinstallfrom-nfsselectbox.ThisboxprovidesalistofallNFS-basedserver-imagesavailableinopenQRM. AtypicalUse-caseispopulatinganemptyiSCSILunwhichworksasexplainedbythefollowingstep-by-steplist: Createanew(empty)iSCSILunonanavailableStorageServer Createanewimageusingthe(stillempty)iSCSILunasitsroot-deviceidentifier InthenewImageconfigurationselectanavailableNFS-basedserver-imagefromtheinstall-from-nfsselect box Createanewapplianceusinganavailableresource,akernelandthejustcreatedimage Starttheappliance Thefollowinghappenswithintheinitrdstageofthestartingappliance: TheresourcediscoverstheiSCSITarget(theStorageServeroftheimage) Theresourceconnectsthe(stillempty)iSCSILunsasitsroot-file-system Theresourcemountstheinstall-from-nfsimage-locationinatemporarylocation Then the resource copies the root-file-system content from the temporary mount point (the NFS-based server-image template selected by the install-from-nfs select box) to its root-device (the iSCSI Lun) using rsync. Whenthetransfercompletettheresourceumountsthetemporarylocationandcontinuesregularboot-upand initfromthe,nowpopulatediSCSILun.

20

The transfer to NFS transfer method is configured through the image new or image edit form via the transfer-to-nfsselectbox.ThisboxprovidesalistofallNFS-basedserver-imagesavailableinopenQRM. A typicalUse-case iscreating a newserver-image template froma existing server-image located on an Lun.This worksasexplainedbythefollowingstep-by-steplist: Createanew(empty)NFSexportonanavailableNFS-basedStorageServer Createanewimageusingthe(stillempty)NFSexportasitsroot-deviceidentifier In the new Image configuration of an existign iSCSI based image select the just created (still empty) NFS-basedserver-imagefromthetransfer-to-nfsselectbox Createanewapplianceusinganavailableresource,akernelandtheiSCSIbasedimage Starttheappliance Thefollowinghappenswithintheinitrdstageofthestartingappliance: TheresourcediscoverstheiSCSITarget(theStorageServeroftheimage) TheresourceconnectstheiSCSILunsasitsroot-file-system Theresourcemountsthetransfer-to-nfsimage-locationinatemporarylocation Thentheresourcecopiestheroot-file-systemcontentfromthemountediSCSILUNtothetemporarymount point(theNFS-basedserver-imagetemplateselectedbythetransfer-to-nfsselectbox)byrsync. Whenthetransfercompletettheresourceumountsthetemporarylocationandcontinuesregularboot-upand initfromtheiSCSILun. Result is a new, fully populated NFS-based server-image containing the root-file-system content of the existingiSCSIbasedimage.

TheinstallfromLocaltransfermethodisconfiguredthroughtheimageneworimageeditformviatheinstallfrom-localselectbox.Thisboxprovidesalistofcommonlocalroot-devicesonresources. AtypicalUse-caseispopulatinganemptyiSCSILunwhichworksasexplainedbythefollowingstep-by-steplist: Createanew(empty)iSCSILunonanavailableStorageServer Createanewimageusingthe(stillempty)iSCSILunasitsroot-deviceidentifier InthenewImageconfigurationselectaroot-devicefromtheinstall-from-nfsselectbox Create a new appliance using an available resource (one with an existing root-file-system content on the root-deviceasconfiguredbytheinstall-from-localselectbox),akernelandthejustcreatedimage Starttheappliance Thefollowinghappenswithintheinitrdstageofthestartingappliance: TheresourcediscoverstheiSCSITarget(theStorageServeroftheimage) Theresourceconnectsthe(stillempty)iSCSILunsasitsroot-file-system Theresourcemountsitslocalroot-device(definedbytheinstall-from-localdefinition)inatemporarylocation Thentheresourcecopiestheroot-file-systemcontentfromthetemporarymountpoint(thelocaldevice)toits root-device(theiSCSILun)usingrsync. Whenthetransfercompletettheresourceumountsthetemporarylocationandcontinuesregularboot-upand initfromthe,nowpopulatediSCSILun.

21

The transfer to Local transfer method is configured through the image new or image edit form via the transfer-to-nfsselectbox.Thisboxprovidesalistofcommonlocalroot-devices. AtypicalUse-case isdeploying an network-booted appliance with itsroot-file-systemlocated on a partition on the localHarddisk.Thisworksasexplainedbythefollowingstep-by-steplist: Editanexisting(fullypopulated)NFS-basedserver-imageandsetitstransfer-to-localselectboxtoalocal root-device Deploythisimageonaresourceviaanappliance Thefollowinghappensduringtheinitrdstage: Theresourcemountsitroot-file-systembyNFS Theresourcemountsthelocaldeviceconfiguredbythetransfer-to-localimageconfigurationinatemporary location Then the resource copies the root-file-system content (the NFS root-file-system content) to the temporary mountpoint(thelocaldeviceselectedbythetransfer-to-localselectbox)usingrsync. Whenthetransfercompletettheresourceumountsthetemporarylocationandcontinuesregularboot-upand initfromitsNFSROOTroot-file-system Tostartthiskindofserver-imagefromitsresourcelocalHarddiskpartitionfollowthestepsbelow: CreateaStoragefromthetypelocalinstalledserverusingtheopenQRMserverastheresource Createanewimage,selectthelocalinstalledserverStorageasitsstorageprovider Fillinthecustomroot-devicefromwhicharesourceshouldboot(theonepopulatedbytransfer-to-local) Create a new appliance using an available resource, a kernel and the just created local installed server image Starttheappliance The resource will reboot, network boot kernel and initrd and startup init on its local root-device as defined in the imagedefinition.

AllrapiddeploymentinopenQRMisdonevianetwork-boot(PXE)usingaspecialopenQRMinitrd.Asdescribedin detailinthesectionaboutopenQRMbootsequencetheopenQRMinitrdsareplug-ableandallowingadeployment type (created bya Storage Plug-in)to tellthe booting systemfromwhereand howitshould mountit'sroot-filesystemfrom. For all remote deployment the usage of local Hard disks of the resources is avoided expect for Swap-Space. Swap-SpaceavailableonlocalattachedHarddisksisautomaticallydetectedandusedbytheresources.

Localdeploymentin openQRMisalso donevia network-boot(PXE)using a specialopenQRMinitrd and comesin threeflavours: Local-StorageDeployment Local-ServerDeployment

22

AutomaticInstallation Local-StorageDeployment (describedintheStoragesection) Local-ServerDeployment openQRMalsosupportsalocaldeploymentmethodwhichallowstocentralized,network-boottheresourceandletit mountandusealocalHarddiskpartitionforfurtherboot-up.Basicallyitallowstodefinethelocalboot-deviceviaa network-bootloader. HINT:ThelocalHarddiskpartitiondefinedforboot-upintheimage-configurationcanbeemptyifusedincombination withinstall-from-nfstoautomaticallypopulatethepartitionwithroot-file-systemcontent. AutomaticInstallation The third flavour of local deployment in openQRM is automatic profile-based installation via the integration with LinuxCOE [http://linuxcoe.sourceforge.net/] . This mechanism automatically installs systems on their local disk. Those installationthencanbeusedforeitherdirectdeploymentthroughlocal-serverdeploymentorasserver-templatesfor server-imagecreation.

Since resources are decoupled from their root-files-systems migration appliances from physical server to virtual machinesiscompletetransparentandeasy.Followthestepsbelowtoexchangeanappliancephysicalresourcewith avirtualone: stoptheappliance edittheapplinace changetheresource-typefromPhysicalSystemtoaVirtulization-typeVM selectanewresourcefromthetypeVirtulization-type savetheappliance starttheappliance Theappliancewillnowstartusingthenew,virtualresourceasdefinedintheapplianceconfiguration.

SimilartotheP2Vmigrationherethestepstomigrateanappliancefromavirtualmachinetoaphysicalsystem: stoptheappliance edittheappliance changetheresource-typetoPhysicalSystem selectanewresourcefromthetypePhysicalSystem savetheappliance starttheappliance Theappliancenowrunsonaphysicalsystem.

23

UsingthesamemethodappliancescanbemovedfromoneVirtualizationtypetoanother.Herethestepsnecessary forthemigration: stoptheappliance(VirtualizationtypeA) edittheappliance changetheapplianceresourcetypefromVirtualization-VM(typeA)toVirtualization-VM(typeB) selectanewresourcefromVirtualization-VM(typeB) savetheapliance starttheappliance The appliance will now start using the new, virtual resource (Virtualization type B) as defined in the appliance configuration.

WhentalkingaboutGreenITthecurrentapproachistouseVirtualizationtoconsolidatemanyphysicalserversto runvirtualizedonone(ormore)HypervisorHosts.Whilethismethodisgoodtosavetheoverallpowerconsumption itisoften forgotten thatthe in the new,virtualized situation in the case the oneHypervisorHostsbreaksthose manyserversrunning in virtualmachineson thisHostswillalso getunavailable.Thismeansthatin the modern, virtualizedWorldsweneedtoespeciallytakecareofthehighavailability. TheusualmethodtokeepsystemHighavailable(10customservers): Getadditional10serverspreferredofthesamemanufactureconsistingofthesameparts Configuresyncingofthedisksbetweenthe10pairsofserver ImplementaFail-oversolutionfortheservicerunningonthe10clusters Asaresultthismethodrequires20physicalsystemstokeep10servershighavailable. HAinopenQRM: Deploythe10customserversviaopenQRM Add1serverasHot-Standby Inthecaseoneofthe10customserversbreakopenQRMwillusetheoneavailablesystemtorestartitviaitsrapid deployment methods. As the result 10 (or more) servers can made high available with just a single Hot-Standby system (since resource types are not linked to the actual server-image in openQRM physical servers even could fail-overtovirtualmachines). Thissavesthepowerconsumptionof9servers!

Application HA can be archived in openQRM by adding custom Nagios checks to monitor the service state of a specificapplication.Incasethischeckfailsitcandovariousthingstore-activatetheserviceagain: Restartingtheservice Forcingareboot Forcingafail-overtoanpassiveapplication-standby

24

HAonApplicationlevelisnotyetautomatedinopenQRMbutmustbesetupmanual.

Toavoidasinglepointoffailure(SPOF)andtokeeptheopenQRMServerhigh-availableitisrecommendedfora production setups to run openQRM in an active/passive HA-configuration. openQRM's architecture of keeping everythingwithinasingle-base-directorymakesitreallyeasytoinstallopenQRMinHigh-availablemode. TheopenQRMTeamrecommendstouseLinux-HA [http://www.linux-ha.org/] fortheopenQRMHAsetup. RequirementsforopenQRMHA 2ormoresystemsfortheactiveandpassiveopenQRMClusternodes Shared-StorageprovidingtheopenQRMServerfile-data External(remote)Database HINT:openQRMdoesnotmatterifinstalledonphysicalsystemoronavirtualmachine. StepstoinstallopenQRMinHA-mode InstallOSonthesystemsdedicatedfortheopenQRMClusternodes MountthesharedstorageinopenQRM'sbase-dir(normally/usr/share/openqrm)onallClusternodes SetupLinux-HAwithaClusterip-address InstallopenQRMonONEsystemonly! During setup use the External (remote) Database and the Network-interface with the Cluster ip-address Ontherestofthesystem link/usr/share/openqrm/web/toDOCUMENT_ROOT/openqrm link/usr/share/openqrm/etc/init.d/openqrmto/etc/init.d/openqrm Wrap/etc/init.d/openqrmintoaLinux-HAinitscript AfterthatLinux-HAwilltakecaretoalwayskeeponesystemsrunningastheactiveClusternodeusingtheglobal Clusterip-address.

aoe-storage TheAoe-storagepluginintegratesAoe/CoraidStorageintoopenQRM.Itaddsanewstorage-type'aoe-storage'and anewdeployment-type'aoe-root'totheopenQRM-serverduringinitialization. Aoe-storage type : A linux-box (resource) with 'vblade' installed should be used to create a new Storage-server through the openQRM-GUI. The Aoe-storage system can be either deployed via openQRM or integrated into openQRMwiththe'local-server'plugin.openQRMthenautomaticallymanagesthevbladedisksontheAoe-storage server. Aoe-deploymenttype :The Aoe-deploymenttype supportsto bootservers/resourcesfromthe Aoe-stoage server. Serverimagescreatedwiththe'aoe-root'deploymenttypearestoredonStorage-serverfromthestorage-servertype 'aoe-storage'. During startup of an appliance they are directly attached to the resource as its rootfs via the aoe-protokol. Howtouse:

25

CreateanAoe-storageserverviathe'Storage-Admin'(Storagemenu) CreateaDisk-shelfontheAoe-storageusingthe'Shelfs'link(Aoe-pluginmenu) Createan(Aoe-)Image('AddImage'intheImage-overview).ThenselecttheAoe-storageserverandselect anAoe-devicenameastheimageroot-device. CreateanApplianceusingoneoftheavailablekernelandtheAoe-Imagecreatedintheprevioussteps. StarttheAppliance

aws Theaws-pluginprovidesanintegrationwiththeAmazonWebService(AWS)andprovidesaseamlessmigration-path fromandtoAWS. Configure AWS Account Create a new AWS Account configuration using the AWS Accounts menu item. The followinginformationsarerequired: AWSAccountName JavaHomeDir EC2HomeDir AWSPrivatekeyfile AWSCertfile SSHkeyfileusedfortheAMI AWSRegion ImportServersfromAWSToimportanAWSServer(theAMIofanactiveEC2Instance)followthestepsbelow: SelectanAWSAccounttousefortheimport SelectanactiveAWSEC2InstancerunningtheAMItoimport Selectan(empty)openQRMServerimage(fromtypeNFS-orLVM-NFS) ThiswillautomaticallyimporttheAMIfromtheselctedAWSEC2Instanceintothe(previouslycreated)emptyServer ImageinopenQRM. TheimportedAMInowcanbeusedwithallexistingresource-typesinopenQRMsoe.g.itcannowalsorunona physicalsystemoronanyothervirtulizationtype.

citrix TheCitrixpluginistestedwithCitrixXenServer5.5.0 Howtouse: installCitrix-XenServeronaserversystem alsoinstallthesecondcdcontainingthesupportforLinuxvms logintotheCitrixXenServerviasshandscpthe/usr/sbin/xeutiltotheopenQRMservers/usr/sbindir enabletheopenQRMCitrixpluginviathepluginmanager manuallyaddaresourceviaBaseResourcesNew,providetheCitrixserversmac-andip-address create astorage typeLocal-installed servervia Base Storage New,selectthe Citrixserverresource andprovideaname

26

createanimageviaBaseImageNew,provideaname createanapplianceviaBaseAppliancesNew,selecttheCitrixserversresource,thedefaultkerneland thepreviouslycreatedimage settheapplianceResourcetypetoCitrixHostandsave gotoPluginsVirtualizationCitrixCitrixVMManager,selecttheCitrixHostappliance nowclickonauthandprovidetheauthenticationdetailstologintotheCitrixHost createanewvmviathe+VMbutton thenewvmwillboot-upviathenetworkandinashorttimeappearintheresourceoverviewasanewidle resourceinthedata-center

cloud The openQRM cloud-plugin provides a fully automated request and provisioning deployment-cycle. External data-centeruserscansubmittheirCloudrequestsforsystemsviaasecondweb-portalontheopenQRM-server.After eithermanuallyorautomaticapprovaloftheCloudrequestsopenQRMhandlestheprovisioninganddeploymentfully automatically. Howtouse: To setup automaticdeploymentwith the cloud-plugin firstthe openQRMenvironmentneedsto be populated with available resources, kernels and server-images. The combination of those objects will be the base of the cloudrequestslater. Startsomeresources(phyiscaland/orvirtual) Createone(ormore)storage-server Createone(ormore)server-imageonthestorage-servers Cloud-Userscanbecreatedin2differentways: User can go to http://openqrm-server-ip/cloud-portal [http://openqrm-server-ip/cloud-portal] and register themselves AdministratorsofopenQRMcancreateUserswithintheCloud-pluginUI Cloud-RequestscanbesubmittedtotheopenQRMCloudeitherviatheexternalCloud-portalbyaloggedinuseror onbehalfofanexistinguserintheCloud-RequestmanagerintheopenQRMUI. starttime-Whentherequestedsystemsshouldbeavailable stoptime-Whentherequestedsystemsarenotneededanymore Kernel-Selectsthekernelfortherequestedsystem Image-Selectstheserver-imagefortherequestedsystem ResourceType-Whatkindofsystemshouldbedeployed(physicalorvirtual) Memory-Howmuchmemorytherequestedsystemshouldhave CPUs-HowmanyCPUstherequestedsystemshouldhave Disk-IncaseofClone-on-deployhowmuchdiskspaceshouldbereservedfortheuser NetworkCards-Howmanynetwork-cards(andip-addresses)shouldbeavailable Highavailable-Setsiftherequestedsystemshouldbehigh-available Clone-on-deploy-IfselectedopenQRMcreatesacloneoftheselectedserver-imagebeforedeployment CloudConfiguration

27

Explanationoftheconfigurationparamaters: 1. cloud_admin_email-TheemailaddressoftheCloud-Administrator 2. auto_provision-Canbesettotrueorfalse.Ifsettofalserequestsneedsmanualapproval. 3. external_portal_url-CanbesettotheexternalUrloftheCloud-portal 4. request_physical_systems-IftheCloudshouldprovidealsophysicalsystemtotheusers 5. default_clone_on_deploy-Settotrue(default)theClouddeploysclonesoftheselectedserver-image 6. max_resources_per_cr-Global-Cloud-Limit,setsthemax.numberofresourcesperrequest 7. auto_create_vms-IftheCloudshouldautomaticallycreatenewvirtualmachines

28

8. max_disk_size-Global-Cloud-Limit,setsthemax.diskusageontheCloud 9. max_network_interfaces-Global-Cloud-Limit,setsthemax.numberofnetwork-interfaces 10. show_ha_checkbox-IftoshowtheHigh-availabilityoptionstotheusers 11. show_puppet_groups-IftoshowthePuppetapplication-classestotheusers 12. auto_give_ccus-NumberofCCUsautomaticallygiventonewusers 13. max_apps_per_user-Global-Cloud-Limit,setsthemax.numberactiveappliancesperuser 14. public_register_enabled-IfthepublicregistrationtotheCloudisenabledordisabled. 15. cloud_enabled-GlobalswitchtoenableordisabletheCloud.SetstheCloudinmaintainanceifsettofalse 16. cloud_billing_enabled-SwitchtoenableordisabletheautomaticCloudbilling 17. show_sshterm_login-Iftoshowthessh-termloginoptiontotheusers 18. cloud_nat-IftheCloudisusingNaturalAddressTranslation(NAT) 19. show_collectd_graphs-IftoshowtheCollectdSystemgraphstotheusers 20. show_disk_resize-IftoshowtheDisk-resizeoptiontotheusers 21. show_private_image-EnablesordisablesthePrivate-Imagefeature 22. cloud_selector-EnablesordisablestheCloud-Selectorfeature 23. cloud_currency-ThecurrencyusedbytheCloud(e.g.Euro,US,etc.) 24. cloud_1000_ccus-DefinestheMappingbetweentherealcurrency(Euro,US,etc.)andvirtualone(CCU) CloudIpGroups TheopenQRMcloud-pluginprovidesautomaticallynetwork-configurationfortheexternalinterfacesofthedeployed systems.TocreateandpopulateaCloudIpGrouppleasefollowthestepsbelow: SelecttheCloudIpGrouplinkfromthecloud-pluginmenu Clickon'CreatenewCloudIpGroup'linkandfilloutthenetworkparametersforthenewIpGroup IntheIpGroupoverviewnowselectthenewcreatedIpGroupandclickonthe'load-ips'button Nowputablockofip-addressesforthisIpGroupintothetextareaandsubmit. CloudAdminSOAP-WebService Toeasilyintegratewiththird-partyprovsionenvironmentstheopenQRMCloudprovidesaSOAP-WebServiceforthe CloudAdministratorandtheCloudUsers.TheAPIdocumentationoftheopenQRMCloudSOAPWebServicecanbe foundat: http://www.openqrm-ng.net/downloads/plugins/cloud/openqrm-soap-api/ [http://www.openqrm-ng.net/downloads/plugins
/cloud/openqrm-soap-api/]

CloudLockfile TheCloudcreatesalockfileat/usr/share/openqrm/web/action/cloud-conf/cloud-monitor.locktoensuretransactions.

collectd The Collectd plugin provides automated monitoring and system graphs for appliances in openQRM. It seamlessly integratesCollectdwithinopenQRMandprovideshourly,daily,weeklyandmonthlysystemgraphscreatedfromthe collecteddataviarrdtool.Byenablingtheplugincollectdispre-configuredandinitialyzedautomatically.Thesystem graphsareupdatedsequentiallyviaacron-job. Itmaytakesomeminutesafterthestartofanappliancetocollectenoughdatatocreatethegraphs.

29

dhcpd The dhcpd-plugin automaticallymanagesyourip-addressassignmentand network-bootenvironemntforthe rapiddeployment features of openQRM. Since the dynamic deployment methods in openQRM are based on networkbooting (PXE)a dhcpd-serverisa fundamentalservice to assign ip-addressesto booting resources.An automatic configuredDhcpd-serverisprovidedbythisplugin. Howtouse: No manual configuration is needed for the dhcpd-plugin. It automatically configures a dhcpd.conf file during initialization.Tomanualaddresourcesforstaticip-assignmentpleasefindthedhcpd.confusedbythepluginat:
/usr/share/openqrm/plugins/dhcpd/etc/dhcpd.conf

dns The dns-plugin automatically manages ip-address to hostname resolving via bind/named for the entire openQRM network.Itconfiguresthehostname/ipentriesforthednsdatabaseandreloadsthename-severduringstart/stopof an appliance. The hostnames are automatically set to the appliance name with the ip address of the applianceresource. Howtouse: No manual configuration is needed for the dns-plugin. It automatically configures the dns-name server during initializationwiththedomainnameconfiguredin:
/usr/share/openqrm/plugins/dns/etc/openqrm-plugin-dns.conf

equallogic-storage The Equallogic-storage plugin integrates Equallogic iSCSI Storage hardware into openQRM. It adds a new storage-type 'equallogic-storage' and a new deployment-type 'equallogic-root' to the openQRM-server during initialization. Equallogic-storage type : An Equallogic-Server, added manually as a new resource with its ip-address set to the groupIP,shouldbeusedtocreateanewStorage-serverthroughtheopenQRM-GUI.openQRMthenautomatically managestheEquallogic-disks(Luns)ontheEquallogic-storageserver. Equallogic-deployment type : The Equallogic-deployment type supports booting servers/resources from the Equallogic-storageserver.Serverimagescreatedwiththe'equallogic-root'deploymenttypearestoredonStorageserverfromthestorage-servertype'equallogic-storage'.Duringstartupofanappliance,theyaredirectlyattachedto theresourceasitsrootfsviatheiSCSI-protocol. Usagephilosophy:VolumesontheEquallogichardwareareusedasablockdeviceoveriSCSI,onwhichOpenQRM will create a filesystem (no partitions). This filesystem is mounted through the rootfs deployment scripts. It is recommendedtouseaseparatenetworkorVLANpurelyforiSCSI,towhichtheEquallogicnetworkinterfacesare connected. When booting directly from the Equallogic volumes, the current implementation also requires a DHCP serveronthisnetworkforIPallocationonthesecond(storage)networkinterfaceof(cloud)appliances;duringboot, therootfshookwilluseudhcpdtosetanIPonthesecondaryinterface.

30

Currentlimitations: Snapshottingisnotimplemented;e.g.nosupportforusingavolumeasmasterimageandusingsnapshots ofthatasrootfsvolumes,aswithlvm-iscsi-storage. Clone-on-deployinthecloudisnotactuallycloningvolumes;instead,newvolumesaremadeonthestorage whichwillbeformattedwhenusinginstall-from-nfsdeployment.Privateimagesdoesusecloning. The clone function in the Equallogicstorage managerisexactlythat;itrunsthe clonecommand on the hardware. Onlyalphanumericcharacters,colon,dotanddashareallowedinvolumenames. Howtouse: EnableSSHaccessonyourEquallogicstoragegroup CreateanEquallogic-storageserverviathe'Storage-Admin'(Storagemenu) CreateavolumeontheEquallogic-storageusingthe'VolumeAdmin'link(Equallogic-pluginmenu) Createan(Equallogic-)Image('AddImage'intheImage-overview).ThenselecttheEquallogic-storageserver andselectanEquallogic-devicenameastheimagesroot-device. Create an Appliance using one of the available kernel and the Equallogic-Image created in the previous steps. StarttheAppliance Equallogicemulator:Forpre-productionandwithout-hardwaretesting,awrapperscriptcalledeqemu-scsthasbeen written.ThisscriptcanbeusedtoturnanylinuxserverwiththeSCST+SCST-iSCSIinitiatorstacktoemulatethe behaviourofanEquallogicgroup.Itwillneedtobeadaptedtoyourenvironmentandisonlyincludedfortestingand developmentpurposes.CanbefoundintheOpenQRMsourceatplugins/equallogic-storage/bin/eqemu-scst

highavailability Thehighavailability-pluginautomaticallyprovideshigh-availabilityfortheappliancesmanagedbyopenQRM. Howtouse: SimplyusetheHA-Managertoselecttheapplianceswhichshouldbehigh-available.IncaseofanerroropenQRM willtrytofindanewresourcefittingtotheapplianceprofileandre-start/re-deploytheappliance.

image-shelf The image-shelf-plugin provides ready-made Server-Images templates for various purposes. Those Server-Image templatesare transparenltytranferred to 'empty'Imageslocated on Storage-Serversmanaged byopenQRM.After thattheycandirectlyusedforrapid-deployment.Thisistheeasistmethodtogetstarted. Please notice that the Image-Shelfs are providing NFS-deployment Server-Image templates which then can be tranferredtoe.g.Iscsi-orAoe-deploymentImagesviatheINSTALL_FROMdeploymentparameters. Howtouse: Enablethe'nfs-storage'plugin CreateaStorage-serverfromthetype'NFS-Storage'or'LvmStorageServer(Nfs)' (YoucanusetheopenQRM-serveritselfasresource)

31

CreateannewexportontheNFS-Storageserverviathe'nfs-storage'or'lvm-storage'plugin CreateanewImageontheNFS-Storageserverandselectthepreviouslycreated NFSexport(thestorage-locationfortheimage)astheImage's'root-device' NowClickontheImage-shelf SelectanImage-Shelffromthelist SelectanServer-Templatefromthelist Selectthejustcreated(empty)NFS-Image ChecktheEvent-listfortheprogressoftheImagecreation

iscsi-storage TheIscsi-storagepluginintegratesIscsi-TargetStorageintoopenQRM.Itaddsanewstorage-type'iscsi-storage'and anewdeployment-type'iscsi-root'totheopenQRM-serverduringinitialization. Iscsi-storagetype: A linux-box (resource) with the Enterprise Iscsi-target installed should be used to create a new Storage-server through the openQRM-GUI. The Iscsi-storage system can be either deployed via openQRM or integrated into openQRM with the 'local-server' plugin. openQRM then automatically manages the Iscsi-disks (Luns) on the Iscsistorageserver. Iscsi-deploymenttype: The Iscsi-deploymenttype supportsto bootservers/resourcesfromthe Iscsi-stoage server.Serverimagescreated withthe'iscsi-root'deploymenttypearestoredonStorage-serverfromthestorage-servertype'iscsi-storage'.During startupofanappliancetheyaredirectlyattachedtotheresourceasitsrootfsviatheiscsi-protokol. Howtouse: CreateanIscsi-storageserverviathe'Storage-Admin'(Storagemenu) CreateaDisk-shelfontheIscsi-storageusingthe'Luns'link(Iscsi-pluginmenu) Createan(Iscsi-)Image('AddImage'intheImage-overview).ThenselecttheIscsi-storageserverandselect anIscsi-devicenameastheimagesroot-device. CreateanApplianceusingoneoftheavailablekernelandtheIscsi-Imagecreatedintheprevioussteps. StarttheAppliance

kvm ThispluginistestedwithKVMkvm-62TobenefitfromKVM's'virtio'featureatleastkvm-84isneeded TheKVMpluginaddssupportforKVM-VirtualizationtoopenQRM.Applianceswiththeresource-type'KVMHost'are listedintheKVM-ManagerandcanbemanagedviatheopenQRMGUI.Additionaltotheregularpartitioncommands like create/start/stop/remove the KVM-plugin provides a configuration form per vm to re-configure the partition as needed(e.g.addingavirtualnetworkcardorharddisks). Hint:TheopenQRM-serveritselfcanbeusedasaresourceforanKVM-Hostappliance.Inthiscasenetwork-bridging should be setup on openQRM-server system before installing openQRM. At least an internal bridge for the openQRM management network is needed. The name for this bridge can be configured in the KVM plugin-

32

configurationfileviatheparameterOPENQRM_PLUGIN_KVM_INTERNAL_BRIDGE. Additional an external bridge (e.g. pointing to the internet) can be setup and configured via the OPENQRM_PLUGIN_KVM_EXTERNAL_BRIDGEparameterintheKVMplugin-configurationfile.openQRMthenwill createeveryfirst(virtual)network-cardfortheKVMvmsontheinternalbridgeandeveryotherontheexternalone. Withthis2-bridgesetupeveryvmwillthenhaveitsfirstnicpointingtotheopenQRMmanagementnetwork(doingthe pxe-boot)andeveryothernicwillpointe.g.totheinternet. Afterhaving a network-bridge configured openQRMshould be installed on the internalbridge-interface (bydefault br0).ThiscanbedonebysettingtheopenQRMmanagementnetwork-interfacein/usr/share/openqrm/etc/openqrmserver.conftobr0beforeinitalyzingopenQRM. Howtouse: Createanapplianceandsetitsresource-typeto'KVMHost' Usethe'VMManager'intheKvm-pluginmenutocreateanewKvm-servervirtual-machinesontheHost ThecreatedKvm-servervmisthenbootingintoopenQRMasregularresources

kvm-storage Thekvm-storageplug-inisfurtherdescribedinthesectionaboutWindowsdeployment

linuxcoe Thelinuxcoe-pluginprovidesautomaticinstallationforthesystemsmanagedbyopenQRMviaLinuxCOE.Itembeds the LinuxCOE web-interface and supports assignment of custom installation profiles to systems managed by openQRM.Theautomaticinstalledsystemthencanbeusedtocreateserver-templates. MoreaboutLinuxCOEat:http://linuxcoe.sourceforge.net[http://linuxcoe.sourceforge.net/]

local-server The local-server-plugin provides an integration for already existing, local-installed systems in openQRM. After integratinganexisting,local-installedserveritcanbeused'grab'thesystemsroot-fsandtransformittoanopenQRM server-image.Italsoallowstodynamicallydeploynetwork-bootedserverimageswhilestillbeingabletorestore/restart theexistingserver-systemlocatedonthelocal-harddisk. Howtouse: Copy(scp)the'openqrm-local-server'utiltoanexisting,local-installedserverinyournetwork
scp /usr/share/openqrm/plugins/local-server/bin/openqrm-local-server [ip-address-of-existing-server]:/tmp/

Executethe'openqrm-local-server'utilontheremotesystemviasshe.g.:
ssh [ip-address-of-existing-server] /tmp/openqrm-local-server integrate -u openqrm -p openqrm -q [openqrm-s

ThesystemnowappearsintheopenQRM-serverasnewresource

33

Itshouldbenowsetto'network-boot'initsbiostoallowdynamicassign-anddeployment
The resource can now be used to e.g. create a new 'storage-server' within openQRM * After setting the system to 'network-boot' in its bios it also can be used to deploy server-images from dif

To remove a systemfromopenQRMintegrated via the local-serverplugin run the 'openqrm-local-server'util again.e.g.:

ssh [ip-address-of-existing-server] /tmp/openqrm-local-server remove -u openqrm -p openqrm -q [openqrm-serv

local-storage The Local-storage plugin addssupportto deployserver-imagesto local-harddiskon the resourcesto openQRM.It providesmechanismto'grab'server-imagesfromlocalharddisksinexisting,local-installedsystemsinthedata-center. Those'local-storage'server-imagesthencanbedynamicallydeployedtoanyavailableresourcesinopenQRM.The deploymentfunctionthen'dumps'theserver-image'grabbed'intheprevioussteptotheharddiskofaresourceand starts it from the local-disk. The 'local-storage' server-images are stored on a storage-server from the type 'localstorage'whichexportstheimagesviaNFS. Local-storage storage type : A linux-box (resource) which has NFS-server installed and a logical volume group available (lvm2) should be used to create a new Storage-server through the openQRM-GUI. The Local-storage systemcanbeeitherdeployedviaopenQRMorintegratedintoopenQRMwiththe'local-server'plugin. Local-storagedeploymenttype:TheLocal-deploymenttypesupportstocreateserver-imagesfromexistingsystems anddeploythoseimagestootheravailableservers/resources. Howtouse: Grabbingaserver-imagefromanexistingsystem CreateanLocal-storageserverviathe'Storage-Admin'(Storagemenu) Createa'local-storage'storagelocation(nfs-exportonalvol)usingthe Local-storagepluginslocal-storagemanager Createa'local-storage'image('AddImage'intheImage-overview) Settheroot-deviceandroot-device-typetothestoragelocationcreatedintheprevisousstep Goto'local-storage''Grab' Inthefirststepselectanidleresourcetograb Inthesecondstepselectthe'local-storage'imagecreatedinthepreviousstep Thecontentoftheidleresourcesharddiskarenowtransferred(grabbed)tothe 'local-storage'imagelocationonthestorageserver. Deployinga'local'server-imagetoanavailableresource Createa'local'image('AddImage'intheImage-overview) Selectastorage-serverfromthetype'local-storage' Settheroot-deviceandroot-device-typeaccordingtothe'local-storage'imagetodeploy CreateannewApplianceusingakernelandthejustcreated'local'server-image Selectanavailableresource

34

StarttheAppliance Thiswillrebootthe resource and 'dump'the server-image fromthe 'local-storage'storage serverthe the resource local-harddiskandstartsitafterthedeploymentfinished. Whenstoppingtheappliancethediskcontentoftheresourcewillbe'grabbed'againtoupdatetheserver-imageon the'local-storage'server.

lvm-storage The 'lvm-storage' plugin transforms a standard Linux-box into a rapid-fast-cloning storage-server supporting snap-shottingforNFS-,Aoe-,andIscsi-filesystem-images.Thesnapshots(clonesfroma'goldenserverimage')are immediatlyavailablefordeploymentandsavingspaceonthestorage-subsystembecausejustthedeltaoftheserver imageisbeingstored.Itaddsanewstorage-type'lvm-storage'andthreenewdeployment-types'lvm-nfs','lvm-aoe' and'lvm-issci'totheopenQRM-serverduringinitializationandbasicallycombinesthefunctionalityofthe'nfs-storage', the'aoe-storage'andthe'iscsi-storage'plugins. Lvm-storagetype:Alinux-box(resource)withtheEnterpriseIscsi-target,NFS-serverandvblade(aoetools)installed shouldbeusedtocreateanewStorage-serverthroughtheopenQRM-GUI.TheLvm-storagesystemcanbeeither deployed via openQRM or integrated into openQRM with the 'local-server' plugin. openQRM then automatically managestheAoe/Iscsi-disksandNFS-exportsontheLvm-storageserver. Lvm-deployment type : The three Lvm-deployment types ('lvm-nfs', 'lvm-aoe' and 'lvm-issci') supporting to boot servers/resourcesfromthe Lvm-storage servervia NFS,Iscsiorthe Aoe-protokol.Serverimagescreated with the 'lvm-nfs/iscsi/aoe'deploymenttypearestoredonStorage-serverfromthestorage-servertypes'lvm-storage'.During startupofanappliancetheyaredirectlyattachedtotheresourceasitsrootfs. How to use : The Lvm-storage server supports three diffrent storage technologies, NFS, Aoe and Iscsi. The functionalityandusageisconformtothecorresponding'nfs-storage','aoe-storage'and'iscsi-storage'pluginswiththe greatbenefitofthe underlaying logicalvolume manager.Thisaddsrapid-cloning capabilitiesthrough snapshotting andsupportstocreatenewserver-imagesfrom'golden-images'(server-templates)withinseconds. Pleasecheckthe'nfs/aoe/iscsi-storage'pluginfordetailedusageinformation.

nagios3 Thenagios3-pluginautomaticallymonitorsthesystemsandservicesmanagedbytheopenQRM-server. Howtouse:Togenerateand/orupdatetheNagiosconfigurationfortheopenQRM-networkandmanagedservers use the 'Config' link in the Nagios-plugin menu. The nagios-configuration is then created fully automatically by scanningthenetworkviathe'nmap'utility.Theoutputofthenmaprunthenisusedby'nmap2nagios-ng'togenerate theNagios-configuration.

netapp-storage The NetApp-storage plugin integrates NetApp-Filer Storage systems into openQRM. It adds a new storage-type 'netapp-storage'andanewdeployment-type'netapp-iscsi'totheopenQRM-serverduringinitialization. NetApp-storage type : A NetApp-Filer Storage system can be easily integrated into openQRM by adding a new

35

resourcewiththemac-andip-addressoftheNetAppserver.openQRMthenmanagestheVolumesandIscsi-Luns ontheNetApp-Filerautomatically. NetApp-deploymenttype:TheNetApp-deploymenttypesupportstobootservers/resourcesdirectlyfromtheNetAppstoage servervia the Iscsi-protokol.Serverimagescreated with the 'netapp-iscsi'deploymenttypesare stored on Storage-server from the storage-server type 'netapp-storage'. During startup of an appliance they are directly attachedtotheresourceasitsrootfseitherthroughiscsi. Howtouse: Createanewresourcewiththeip-andmac-addressoftheNetApp-storageserver(Resourcemenu) CreateanNetApp-storageserverviathe'Storage-Admin'(Storagemenu) Create an (NetApp-) Image ('Add Image' in the Image-overview). Then select the NetApp-storage server deployment-type('NetAppIscsi-root').SelectaNetAppstoragedeviceastheimageroot-device. CreateanApplianceusingoneoftheavailablekernelandtheNetApp-Imagecreatedintheprevioussteps. StarttheAppliance

nfs-storage TheNfs-storagepluginintegratesNfsStorage-serversintoopenQRM.Itaddsanewstorage-type'nfs-storage'anda newdeployment-type'nfs-root'totheopenQRM-serverduringinitialization. Nfs-storage type :A linux-box(resource)with 'nfs-server'installed should be used to create a newStorage-server through the openQRM-GUI. The Nfs-storage system can be either deployed via openQRM or integrated into openQRMwiththe'local-server'plugin.openQRMthenautomaticallymanagestheexportsontheNfs-storageserver. Nfs-deployment type : The Nfs-deployment type supports to boot servers/resources from the Nfs-stoage server. Serverimagescreatedwiththe'nfs-root'deploymenttypearestoredonStorage-serverfromthestorage-servertype 'nfs-storage'. During startup of an appliance they are directly attached to the resource as its rootfs via the nfs-protocol. Howtouse: CreateanNfs-storageserverviathe'Storage-Admin'(Storagemenu) Createannewnfs-exportontheNfs-storageusingthe'Exports'link(Nfs-pluginmenu) Createan(Nfs-)Image('AddImage'intheImage-overview).ThenselecttheNfs-storageserverandchoose oneoftheNfs-storage-devicesasthe'root-device'. CreateanApplianceusingoneoftheavailablekernelandtheNfs-Imagecreatedintheprevioussteps. StarttheAppliance

puppet The Puppet plugin provides automated configuration-management for appliances in openQRM. It seamlessly integratesPuppetwithintheopenQRMGUIandassiststoputspecificappliancesintopre-madeorcustomPuppetclasses. By enabling the plugin the puppet-environment (server and client) is pre-configured and initialyzed automaticallyaccordingtobest-practiceexperiencese.g.bykeepingthepuppet-configurationwithinasvn-repsitory. Thispuppet-configurationrepositoryisalsoavailableforexternalsvnclients.Tocheckoutthepuppet-repoplease run:
svn co svn+ssh://[user]@[openqrm-server-ip]/usr/lib/openqrm/plugins/puppet/etc/puppet/ .

36

Commitsthe thisrepositorywillautomaticallythe puppetconfiguration at/etc/puppet/*The puppet-configuration is organizedin'classes','goups'and'appliances'.Owncustomclassesshouldbeaddedtotheclass-directory.Classes shouldbethencombinedin'groups'whichwillbeautomaticallydisplayedinthepuppet-manager.The'appliances' sectionisfullymanagedviathepuppet-manageruser-interface. The default puppet-plugin configuration comes with a set of pre-made puppet-classes and groups. The available groupsare: basic-server webserver database-server lamp Thosepre-definedgroupscanofcoursebeadaptedandenhancedviathepuppetsvnrepository. Pleasenoticethatthepuppet-plugin dependson the dns-plugin !Make sure to have the dns-plugin enabledand startedbeforethepuppet-plugin. Howtouse: Gotothe'puppet-manager'inthepuppet-pluginmenu Selectanappliancetoautomaticconfigureviapuppet Selectthepuppet-groupstheapplianceshouldbelongto Withinshorttimethepuppet-serverwilldistributethenewconfigurationtotheapplianceautomatically.

sanboot-storage Thesanboot-storageplug-inisfurtherdescribedinthesectionaboutWindowsdeployment

solx86 The solx86-plugin provides an integration for already existing, local-installed openSolaris/Solaris X86 systems in openQRM.Afterintegratinganexisting,local-installedserveritcanbeusedasaZFS-Storageserver. Howtouse: To integrate a Solaris/openSolarisX86 systemplease login to the Solaris/openSolarissystemasrootand run the followingcommands:
wget https://[openqrm-server-ip-address]/openqrm/boot-service/openqrm-solx86 chmod +x openqrm-solx86 ./openqrm-solx86 integrate -u [openqrm-admin] -p [openqrm-admin-password] -q [openqrm-server-ip-address]

ToremovetheopenQRMintegrationfromyourSolaris/openSolarissystempleaserun:
./openqrm-solx86 remove -u [openqrm-admin] -p [openqrm-admin-password] -q [openqrm-server-ip-address]

37

sshterm Thesshterm-pluginintegrates'ajaxterm'intoopenQRMandprovidesasecurelogintotheopenQRM-serverandthe managedresourcesthroughtheWeb-interface. Nomanualconfigurationisneededforthesshterm-plugin.

tftpd Thetftpd-pluginautomaticallymanagestouploadkernelstotheresources(servers)managedbyopenQRM.Since thedynamicdeploymentmethodsinopenQRMarebasedonnetwork-booting(PXE)atftpd-serverisafundamental servicetoservertheoperation-systemfilesviathenetwork.AnautomaticconfiguredTftpd-serverisprovidedbythis plugin. Howtouse: Nomanualconfigurationisneededforthetftpd-plugin.Itautomaticallystartsupthetftpd-serviceduringstart-upof theplugin.

tmpfs-storage The tmpfs-storage plugin supports an In-Memory deployment method. It works in combination with the installfrom-nfshookandallowstoruncompleteserverwithinRAM. Apartfromthenoteableperformanceboostthisnon-permanentdeploymentmethodalsotakesadvantagesofbeing completely independent from local hard-disks since the whole system content is stored in memory. This makes it perfectforkiosk-modescenariossuch asinternetcafes,ITtraining roomsbutalso forlean Hypervisorprovisioning andHigh-PerformanceComputing(HPC).Anotherbenefitforthosesetupsisthatsincenolocaldisksareusedthere isnotreallymuchleftwhichcanbreakinthephysicalhardware;eventhenthehardwareisreplace-ableatanytime andnoserviceistiedtoit. For the administrators of those kiosk-mode scenarios this mean less maintenance, a complete centralized managementandthesecuritythatifusersbreaktheirsystems(thesoftwareinstallation)itwillbefixedautomatically byarebootwhichcausesafreshdeploymentofastaticserver-templatedirectlyintoRAM.

vmware-server Thevmware-serverpluginistestedwithVMware-server1 VMwareServerversion1isknowntobeagreatchoiceforapplicationswhichrequireafull-virtualizationtechnology. VMware-serverVirtualization hostscan be easilyprovisioned viaopenQRMbyenabling thisplugin.Italso enables the administrator to create, start, stop and deploy the 'vms' seamlessly through the web-interface. The virtual VMware-server-resources(vms)arethentransparentlymanagedbyopenQRMinthesamewayasphysicalsystems. Howtouse: Enableandstartthelocal-serverplugin

38

IntegrateaVMwareServersystemvialocal-server(pleasecheckthelocal-serveraboutpage) Settheappliance(automaticallycreatedbylocal-serverintegration)resource-typeto'VMware-ServerHost' Usethe'VMwareServerManager'intheVMwareServerpluginmenutocreateanewVMwarevirtual-machine ontheHost ThecreatedVMwarevmsarethenbootingintoopenQRMasnewresources

vmware-server2 Thevmware-server2pluginistestedwithVMware-server-2.0.1-156745.i386 VMwareServer2isknowntobeagreatchoiceforapplicationswhichrequireafull-virtualizationtechnology.VMwareserver Virtualization hosts can be easily provisioned via openQRM by enabling this plugin. It also enables the administratorto create,start,stop and deploythe 'vms'seamlesslythrough the web-interface.The virtualVMwareserver-resources(vms)arethentransparentlymanagedbyopenQRMinthesamewayasphysicalsystems. Howtouse: Enableandstartthelocal-serverplugin IntegratetheVMwareServer2systemvialocal-server(pleasecheckthelocal-serveraboutpage) Settheappliance(automaticallycreatedbylocal-serverintegration)resource-typeto'VMware-Server2Host' Use the 'VMware Server2 Manager'in the VMware Server2 plugin menu to create a newVMware virtualmachineontheHost ThecreatedVMwarevmsarethenbootingintoopenQRMasnewresources

vmware-esx Thevmware-esxpluginistestedwithVMwareESX3.5-ESXi4.0 Thisplugin integratesVMware ESXserverasanothervirtualresource providerforopenQRM.Since VMWare ESX doesnotprovideanAPIforthelinuxoperation-systemyettheintegrationiscurrentlydonevia'password-lessssh'to theESXserver(fromtheopenQRM-server). Howtogetsshenabledand'password-less'logintotheESXserverrunningiswelldocumentedintheinternet. PleasenoticethatthismodeisunsupportedbyVMware!stillwewouldliketobeabletomanageESX. Requirements:-Anexisting'DataStore'(Storage)ontheESXserver.DataStoresinVMwareESXarethelocation wherethevirtualmachinefilesarebeingsaved.FortheopenQRMVMware-ESXpluginthedefaultdatastorecanbe configuredinthe pluginsconfiguration file.BydefaultopenQRMwilltryto gatherthe firstavailable datastore(e.g. datastore1onafreshinstalledESXi4.0)anduseitforstoringthevirtualmachines. -password-lesssshaccess(asuserroot)fromtheopenQRMservertotheESXserver(asmentionedbefore).Hint: makesuretoset/.ssh/authorized_keystomode0600ontheESXhost(dirandfile) Howtouse: HowtointegrateaVMwareESXserverintoopenQRM: Firstmakesuretoenabled'password-lesssshlogin'ontheESXserver TocheckyoucanrunasrootontheopenQRM-server:

39

ssh [ip-address-of-the-esx-server] ls This should give you a directory listing. * Now integrate the ESX server by running the following command : /usr/share/openqrm/plugins/vmware-esx/bin/openqrm-vmware-esx init -i [ip-address-of-the-esx-server] This procedure will ask for a valid openQRM username and password. * The above procedure will integrate the ESX server within openQRM fully automatically. It will create the following components : - a resource (the ESX server) - a local storage placeholder for the ESX server resource - a local image placeholder for the ESX server resource - a local kernel placeholder for the ESX server resource - and a local appliance (the ESX server appliance) * Go to the 'ESX-Manager' within the VMware-ESX plugin menu. Select the ESX-appliance. * In the next screen you can now create/start/stop/remove/delete virtual machines on the ESX server. Created virtual machines will automatically start into openQRM and appear as new idle resources, ready for

windows Thewindowsplug-inisfurtherdescribedinthesectionaboutWindowsdeployment

xen ThexenpluginistestedwithXen3.2andhigherandrequiresfullvirtualization(HVMviaVT) Xen Virtualization hosts can be easily provisioned via openQRM by enabling this plugin. It also enables the administrator to create, start, stop and deploy the 'vms' seamlessly through the web-interface. The virtual Xen-resources(vms)arethentransparentlymanagedbyopenQRMinthesamewayasphysicalsystems. Hint:TheopenQRM-serveritselfcanbeusedasaresourceforanXEN-Hostappliance.Inthiscasenetwork-bridging should be setup on openQRM-server system before installing openQRM. At least an internal bridge for the openQRM management network is needed. The name for this bridge can be configured in the XEN pluginconfigurationfileviatheparameterOPENQRM_PLUGIN_XEN_INTERNAL_BRIDGE. Additional an external bridge (e.g. pointing to the internet) can be setup and configured via the OPENQRM_PLUGIN_XEN_EXTERNAL_BRIDGEparameterintheXENplugin-configurationfile.openQRMthenwill createeveryfirst(virtual)network-cardfortheXENvmsontheinternalbridgeandeveryotherontheexternalone. Withthis2-bridgesetupeveryvmwillthenhaveitsfirstnicpointingtotheopenQRMmanagementnetwork(doingthe pxe-boot)andeveryothernicwillpointe.g.totheinternet. Afterhaving a network-bridge configured openQRMshould be installed on the internalbridge-interface (bydefault eth0).Thiscan be done bysetting the openQRMmanagementnetwork-interface in /usr/lib/openqrm/etc/openqrmserver.conftobr0beforeinitalyzingopenQRM. Howtouse: Createanapplianceandsetitsresource-typeto'XenHost' Usethe'XenManager'intheXenmenutocreateanewXenvirtual-machinesontheHost ThecreatedXenvmisthenbootingintoopenQRMasregularresources

40

xen-storage Thexen-storageplug-inisfurtherdescribedinthesectionaboutWindowsdeployment

zabbix Thezabbix-pluginautomaticallymonitorsthesystemsandservicesmanagedbytheopenQRM-server. Howtouse: Afterenabling and starting the Zabbixplugin you can login to Zabbixas'Admin'with an emptypassword.Please makesuretosetpasswordforthe'Admin'accountatfirstlogin! AllmanagedsystemsbyopenQRMwillbeautomaticallydiscoveredandmonitoredbyZabbix.Youcannowindetail configurethesystemandservicechecksviatheintuitiveZabbixUI.

zfs-storage TheZFS-storagepluginintegratesZFS-TargetStorageintoopenQRM.Itaddsanewstorage-type'zfs-storage'anda newdeployment-type'iscsi-root'totheopenQRM-serverduringinitialization. ZFS-storagetype:Alinux-box(resource)withtheZFS-targetinstalledshouldbeusedtocreateanewStorage-server through the openQRM-GUI. The ZFS-storage system can be either deployed via openQRM or integrated into openQRM with the 'local-server' plugin. openQRM then automatically manages the ZFS-disks (Luns) on the ZFS-storageserver. ZFS-deploymenttype :The ZFS-deploymenttype supportsto bootservers/resourcesfromthe ZFS-stoage server. Serverimagescreatedwiththe'iscsi-root'deploymenttypearestoredonStorage-serverfromthestorage-servertype 'zfs-storage'. During startup of an appliance they are directly attached to the resource as its rootfs via the iscsiprotokol. Howtouse: CreateanZFS-storageserverviathe'Storage-Admin'(Storagemenu) CreateaDisk-shelfontheZFS-storageusingthe'Luns'link(ZFS-pluginmenu) Createan(ZFS-)Image('AddImage'intheImage-overview).ThenselecttheZFS-storageserverandselect anZFS-devicenameastheimagesroot-device. CreateanApplianceusingoneoftheavailablekernelandtheZFS-Imagecreatedintheprevioussteps. StarttheAppliance

To take the most advantages of openQRM's management and deployment functionality it is recommended to separate the network into a dedicated management- and storage-network and a public-network. The

41

management-network is completely managed by openQRM and separates the network-traffic from resources to storages from the public network-traffic. It is recommended to use a high-speed network especially for the management-andstorage-network. For the Virtualization Hosts in the openQRM environment it is recommended to adapt their bridging setup to the separated management-and plublic-network.The Virtualization plug-insin openQRMsupporting a custombridgedefinition in their specific virtualization plug-in's configuration file. Here e.g. the xen-plugin configuration options regardingthebridge-setupoftheHypervisorHosts:
# required parameter to set the internal Xen bridge # by default openQRM will use the internal bridge eth0 for the virtual # network-interface routed to the management/openQRM network on the Xen vms. OPENQRM_PLUGIN_XEN_INTERNAL_BRIDGE=eth0 # required parameter to set the external Xen bridge # by default openQRM will use external bridge eth0 for the virtual # network-interface(s) routed to the public network (internet) on the Xen vms OPENQRM_PLUGIN_XEN_EXTERNAL_BRIDGE=eth0

ThefirstparameterOPENQRM_PLUGIN_XEN_INTERNAL_BRIDGEdefinesthebridgetouseforthemanagementandstorage-network.ThesecondparameterOPENQRM_PLUGIN_XEN_EXTERNAL_BRIDGEdefinesthebridgeto thepublic-network.BothparameterneedstobeconfiguredontheintegratedHypervisorhost(afteritdownloaded thexen-plugin-boot-service)orontheopenQRMServeritself(byunpackingthexen-pluginboot-service,adaptingthe configurationandpackingitagain).Othervirtualizationplug-insinopenQRMareworkingcorresponding. Bydefaultthefirstvirtualnetwork-interfaceofVirtualMachinesisautomaticallyconnectedtothemanagement-and storage-networkwhileallfurthervirtualNICsareconnectedtothepublic-network.

Setting up the Operating System for the openQRM Server system is easy. Preferred Linux Distributions for the openQRM Server are latest Ubuntu or latest Debian. Just installing the base Operating System is enough since openQRM automatically fetches further package dependencies during setup. This is true for the installation by pre-buildopenQRMpackagesorviatheinstalltionfromthesources.

FromPackages
Pre-built openQRM packages for Debian, Ubuntu and CentOS are available on the openQRM Project sites files sectionathttp://sourceforge.net/projects/openqrm/[http://sourceforge.net/projects/openqrm/] . Debian/Ubuntu DebianandUbuntupackagescanbeinstalledviathedpkgutil:
dpkg -i openqrm-[version]-debian1_[architecture].deb

TofetchalldependenciesandinitializeopenQRMpleaserun:
apt-get install -f

AfterthatpleasecontinueintheWeb-basedconfigurationfortheopenQRMServer.

42

Another option is to use the package repositories of the openQRM Project at http://packages.openqrm.com/ [http://packages.openqrm.com/] .Tousethepackagerepositoriespleaseaddthefollowinglineto/etc/apt/sources.list
deb http://packages.openqrm.com/lenny/ ./

Thenrun:
apt-get update apt-get install openqrm

NowpleasecontinueintheWeb-basedconfigurationfortheopenQRMServer. CentOS OnCentOS(orotherRedhat-basedLinuxdistributions)openQRMcanbeinstalledusingtherpmtool:

rpm -iHv openqrm-[version]-centos[version]_[architecture].rpm

TousethepackagerepositoryoftheopenQRMProjectpleasecreateafile/etc/yum.repos.d/openQRM.repowiththe followingcontent:
[openqrm] name = Red Hat Enterprise $releasever - openQRM baseurl = http://packages.linuxfellaz.net/centos5 enabled = 1 protect = 0 gpgcheck=0

Afterthatpleaserun:
yum install openqrm

ThenpleasecontinueintheWeb-basedconfigurationfortheopenQRMServer.

FromSources
SinceopenQRM'sadvancedbuild-system(detailsintheDevelopmentsection)itisreallyeasytobuildopenQRMfrom the sources. It just requires to have make and svn (subversion client) installed. The following commands automaticallycompile,installandstartopenQRM:
svn co https://openqrm.svn.sourceforge.net/svnroot/openqrm openqrm cd openqrm/trunk/src make make install make start

Furthermake-targetsareexplainedintheDevelopmentsection.

FromPackages
In case the package repositoriesofthe openQRMProjectare used the update-managerofthe Linuxdistribution openQRMisinstalledonwillnotifywhennewpre-builtpackagesareavailable.

43

Itisrecommendedtocreateanewstate-backupoftheopenQRMServerstatebeforeapplyingtheupdate! Itisalsosavetouninstallandre-installtheopenQRMpackages.Theuninstallphaseautomaticallycreatesastatebackup.Duringre-installationthenpleasechecktherestorefromlastbackupcheckboxintheWeb-configurator.

FromSources
openQRM also supports direct updates from the sources. Since every base- and plug-in directory contains a separated makefile even partly updates are easy to apply. If updated source code is available change to the base-directorycontainingtheupdatesandrun:
make && make install

ThiswillapplytheupdatestotheinstalledopenQRMServer.Alsoplug-inscanbeupdatedthisway.Dependingon theplug-initmaybeneededtodisable,re-enableandre-starttheplug-inviatheplugin-manager.

Thisbasic-setup section describesa single-systemopenQRMsetupwhich isperfectto startwith openQRM.This setup can be scaled up with additional physical systems, virtual machines, external storage servers and more functionalityviaadditionalplug-insatanytimelater.

Whenusingasingle-systemopenQRMsetupthestorage-component,aHypervisorandopenQRMServeritselfare installed on a single physical system. In this setup the openQRM system acts as the storage server and virtual machineprovider.Thereforeheresomespecialconfigurationrecommendationsandrequirementsespeciallyforthe single-systemsetup: 1physicalsystem VTCPUextension(fullVirtualizationsupport)

44

1GBMemory(ormore) 10GBfreediskspace 1dedicatedstoragepartition,50GB(ormore) Pre-setuptheHypervisor: InstallandconfiguretheHypervisor(e.g.KVMorXen)onthesystem.PleasemakesuretoconfiguretheHypervisors bridge-setupaccordingthemanagement-andpublic-network(refertoPreparingtheNetwork). Pre-setupLVM For the basic-setup it is recommended to use the lvm-storage plug-in and its provided deployment types. To pre-configurethesystemtobeusedasanlvm-storageservermakesuretohavethefollowingpackagesinstalled: lvm2(logicalVolumemanagement) nfs-kernel-server(NFSServer) iscsi-target(EnterpriseiSCSITarget) Theninitializethe dedicated storagepartition to be used withlvm(inthisexample/dev/sdb1 isthe dedicatedfree partition):
fdisk /dev/sdb

Nowsetthefirstpartion(/dev/sdb1inthisexample)tothepartitiontype8e(LinuxLVM)andsave.Thepartitionis nowreadyto be used forLVM.Please run the following commandsto initialize the LVMpartition and to create a volumegroup(namedvolinthisexmaple)onit:
pvcreate /dev/sdb1 vgcreate vol /dev/sdb1

ThestoragepartisnowconfiguredtobemanagedbyopenQRM.

ActivateDHCP,TFTP,StoragePlugins
AftersettinguptheopenQRMServersystemandpre-configuringtheHypervisor-andstorage-componentstheinitial openQRMenvironmentnowcanbestartedbyenablingandstartingthefollowingplug-ins: dhcpd-automaticallyprovidesandmanagestheip-addressesinthemanagement-andstorage-network tftpd-providesthenetwork-boot-managerforacentralizedkernel-andboot-management lvm-storage-provides3differentstorage-anddeployment-types(lvm-nfs,lvm-iscsiandlvm-aoe) AlsopleaseenabletheVirtualizationplug-infortheselectedHypervisorTechnology(e.g.xenorkvmforthisexample setup) xen-integrateswithXen-Hypervisorsandprovides2newresourcetypes:XenHostandXenVM kvm-integrateswithKVM(kernelvirtualmachine)andprovides2newresourcetypes:KVMHostandKVM VM

CreatingImages
ImagesinopenQRMarelogicalobjectscontainingparameterswhicharedefininge.g.theimagestorageserver,the exactimagelocationonthestorageserver,theimagename,versionandmore. PleasenoticethatImagesinopenQRMarealwayslocatedonastorageserver!

45

Thatmeansbeforeactuallycreatinga(server-)imageinopenQRMastorageserverobjectneedstobecreated.In thisbasic-setupexampleweuseoneofthemostsimpleststoragetypelvm-nfs.Tocreateanewlvm-nfsstorage serverinopenQRMgoto:

Base -> Components -> Create -> Storage

SelecttheLvmStorageServer(Nfs)storagetypeandchoosetheopenQRMServerresource(resource0)asthe storageresource.Provideanameforthestorageserverobjectandsave.Thenewstorageserverisnowavailablein openQRMat:

Base -> Components -> Storage

Tocreateanewimagelocation(alogicalvolumeexportedviaNFS)clickontheMgmtbutton.

ImageLocations
AnImagelocationinopenQRMisthephysicalstoragelocationofa(server-)imagesrootfile-systemcontent.Itis theplaceonthestorageserverwheretheactualroot-file-systemcontentisstored.Theimagelocationisdefinedby thestorageplug-insroot-deviceidentifierhook(pleasechecktheDevelopmentsectionformoredetails). AfterclickingtheMgmtbuttoninthestorageoverviewthenextpagepresentsalistofallavailablevolumegroups on the system. In our example it will show a single line for the vol volume group created in the previous step Pre-setupLVM.Selectitandclickonselect. Thenextformdisplaysalistofexistinglogicalvolumeswithintheselectedvolumegroup(volinourexample).Use theAddnewlogicalvolumetoVolumegroupvolsectiontoaddanewlogicalvolumetothelvm-nfsstorageserver. Forthisbasic-setupexamplenamethenewlogicalvolumemyserver.Afterclickingaddthenewcreatedvolumeis listedinthevolumeoverview. Whatwehavejustcreatednowisanewimagelocationbasedonalogicalvolumeofthegivensize.openQRMalso createdafile-systemonthislogicalvolume,mounteditandexporteditviaNFS(thisisspecificforthelvm-nfsstorage type).Theimagelocationmountpointis:
/[volume-group-name]/[logical-volume-name]/

Inourexampleitis:
/vol/myserver

Ofcoursethisnewcreatedvolumeiscompletelyemptyfornow.HowtocreateanopenQRMimageoutofthisimage locationandhowtopopulateitwitharootfiles-systemcontentinthenextsection.

LogicalImages
Wehavecreatedastorageserveranda(yetempty)imagelocationonthestorageserverintheprevioussteps.Now wecreateanew,logicalopenQRMimageobjectfromthosecomponents.Tocreateanewimagegoto:
Base -> Components -> Create -> Image

Selectthestorageserveronwhichtheimageislocated(inourexamplethelvm-nfsstorageserver).Inthenextform pleaseprovideanamefortheimage(wewillusemyimageasimagenameforthisexample). The mostimportantandrequiredimage configurationoptionis the Root-deviceselectbox.Itpresents the image locations available on the selected storage server. In our example the select box has one entry namedmyserver.

46

Optionalfilloutotherimageparametersuchasversion,root-passwordetc.Thensavetheimage. Whatwehavedoneuntilnow: WehavenowcreatedanewopenQRMimage(object).Itislocatedonthe(lvm-nfs)storageserverandconsistofthe imagelocationcreatedintheprevioussteps. Buttheimagelocationisstillempty!? Howtoautomaticallypopulatethisemptyimagelocationwithrootfile-systemcontentisexplainedinthenextsection.

Usingapre-madeImagefromImageShelves
openQRMprovidestheimage-shelfplug-inwhichautomaticallypopulatesNFS-basedimagelocationsfromimage shelfs.Thisimageshelfscanbeeitherlocaldirectories,ftp-and/orhttplocationsprovidingzippedserver-templates. The openQRMProjectoffersa publicavailable image-shelfwhich isautomaticallypre-configured in the openQRM setup. Topopulatethestillemptyimagelocationinourexamplebasic-setuppleasegoto:
Plugins -> Deployment -> Image-shelf -> Import

Thepagepresentsalistofavailableimageshelfs.ForopenQRMversion4.6(andhigher)pleaseusethe1.image shelfprovided byopenqrm-enterprise.ForolderopenQRMsetups(4.5 and lower)please selectthe image shelf fromwww.openqrm-ng.net[http://www.openqrm-ng.net] . Onthefollowingpagealistofavailableserver-templatesonthisimageshelfisdisplayed.Forthisexamplewepick theUbuntu-x86_64-9.10server-template.Clickonget. NowopenQRMpresentsalistofNFS-basedserver-imagesavailabletoputthisserver-templateto.Pleaseselectthe previouscreatedmyimageimage. openQRMnowmountsthe myimageimage location,downloadsand unpacksthe rootfile-systemcontenton the mounted directory and finally creates a new kernel from the just populated image location fitting to the selected server-template(inourexamplea64bitUbuntukernel). Pleasechecktheeventlistforprogressonthedownloadandimagepopulationphase. Whatwehavesofar: Wenowhaveaready-to-run,fullypopulatedserver-imagewhichcanbedeployedviaanappliance.

AddingResources
Resources are added to openQRM in a fully automated way by simply net-booting them via PXE. openQRM auto-discoversnewsystemandbootsthemintotheidlemode,meaningfreeandavailableforprovisioning.Thisis true forphysicalsystemsand forvirtualmachines.Virtualresourcesare created through the specificvirtualization pluginsVM-Managerandalsonet-bootingintoopenQRMtransparently. The idle mode for resources is a quite important layer especially for resource-planning, dynamic scalability and high-availability. The pool of idle resources allows a detailed capacity planning and provides an exact overview aboutavailablesystemsforfail-oversituations(hot-standbys).Intheidlephaseresourcesarereadytobepickedup by an appliance which specifically contains the deployment informations e.g. which kernel and image to use for boot-up. Afterassignmentofaresourcetoa(started)appliancetheresourcerebootsfromidletoassignedstateandstarts

47

thedefinedserver-imageincludingitsservices.Whenstoppinganapplianceitsresourcearefreedandreleasedto theidleresourcepool.

AddingKernels
The initialization phase of openQRM creates a default kernel from the kernel running on the openQRM server systemitself.Thisdefaultkernelisusedtoautomaticallydiscoverandbootresourcesintotheidlestate. NewkernelsshouldbeaddedontheopenQRMserverwiththefollowingcommand:
OPENQRM_SERVER_BASE_DIR/openqrm/bin/openqrm kernel add -n name -v version -u username -p password [-l location

Namecanbeanyidentifieraslongasithasnospacesorotherspecialcharacters;itisusedaspartofthe filename. Version should be the version for the kernel you want to install. If the filenames are called vmlinuz2.6.26-2-amd64then2.6.26-2-amd64istheversionofthiskernel. UsernameandpasswordarethecredentialstoopenQRMitself. Locationisthe rootdirectoryforthe kernelyou wantto install.The filesthatare used are ${location}/boot /vmlinuz-${version},${location}/boot/initrd.img-${version}and${location}/lib/modules/${version}/* initramfs/ext2 should specify the type of initrd image you want to generate. Most people want to use initramfshere. path-to-initrd-template-fileshouldpointtoanopenqrminitrdtemplate.ThesecanbefoundintheopenQRM basedirunderetc/templates. Example:
OPENQRM_SERVER_BASE_DIR/openqrm/bin/openqrm kernel add -n openqrm-kernel-1 -v 2.6.29 -u openqrm -p openqrm -i

Tospecifyanewversionofthedefaultkernelgoto:
Base -> Components -> Kernels

Thenselectthenewkernelwhichshouldreplacethepreviousdefaultkernelandclickonset-default.

CreatingAppliances
AppliancesinopenQRMactastheservice-containerandcontainingthedetailedinformationsandparameterhowto deploy(andun-deploy)theserviceintheDatacenter.AsexplainedintheAppliancemodelofopenQRMappliances consist of the abstracted Data center components plus Service Level Agreements (SLA) defining e.g. CPU- and Memoryrequirements,machinetype,High-availabilityandmore.Astheabstractionoftheservice-layerinopenQRM appliances simplify the mostly very storage-, resource- and virtualization-type dependent deployment mechanisms sinceforstartingandstoppinganapplianceknowledgeaboutthecustomunderlayingsubsystemsisnotneeded. AppliancesinopenQRMarecreatedfromthefollowingcomponents: Aresource(physicalsystemorvirtualmachine) Akernel(theOperatingSystemkerneltoboot) Aserver-image(logicalobjectholdingthestorage-serverandimage-locationdetails) AdditionalServiceLevelAgreements(SLA) Appliances in openQRM are also used to define their special purpose as Virtualization-Host. Virtualization managementforappliancesisenabledbysettingtheresource-typetooneoftheVirtualization-Hostsystemtypes.

48

openQRM is to be known as the swiss-army-knife when it comes to Datacenter Management by integrating all involvedDatacentersubsystemswithinasinglemanagementconsole.Herealistofitsmostcommonusecases: ISPcustomerprovisioning Private-andPublicCloudComputing DeveloperTeams QATeams Serverconsolidation/Virtualmachinemanagement Thebasic-setupdescribedintheprevioussectionisagoodstartingpointforascalableopenQRMenvironment.From thesingle-systemopenQRMsetupthenextstepistodistributethestorage-andHypervisor-componentstoremote systems(physicalresources)andtoletopenQRMjustmanagethedistributedenvironmentbysendingcommandsto thespecificresources.

49

ForproductionsetupsitisrecommendedtouseadistributedopenQRMenvironmentandtomakesuretheopenQRM Server itself is installed in a High-availability mode. This section explains how to configure HA for the openQRM Server. RequirementsfortheHAsetup: 4(ormore)physicalsystems 1(ormore)system(s)dedicatedtobeVirtualizationHosts 1(ormore)system(s)dedicatedtobetheStorageservers 2(ormore)systemsdedicatedforopenQRMServer(systemA)plusHot-Standbysystems(systemB) forfail-over Theexplanationfocusonthe2systemsdedicatedfortheopenQRMServerplusitsHot-Standbysystem. Installthe2systemswithaLinuxDistributionsupportedbyopenQRM(AandB) Connectbothsystemstothenetworkandconfiguretheirip-addresses(AandB) Installand setup openQRMServeron systemA,use a remote and High-available Database and an alias network-interface(onlyA) Createasharedstorageon(oneof)theStorageserver StopopenQRMServeronsystmA(onlyA) MovetheopenQRMServerbase-directorytoatemporarydirectory(onlyA) MountthesharedstoragelocationfromtheStorageserverattheopenQRMServerbase-directory(AandB) MovetheopenQRMServercontentfromthetemporarydirectoryintothenow(mounted)base-directory(only A) InstallandsetupLinux-HAonbothsystems Createawrapper-initscriptresponsibleforstartingopenQRMServerontheactiveClusternode Addthewrapper-initscriptasaLinux-HAserviceusingaglobalClusterip-address Now Linux-HA automatically takes care to activate the openQRM Server only on its active Cluster node and to fail-overtheserviceplusitsip-addressincaseofasystemfailure.

ThemigrationofanexistingITinfrastructuretoanopenQRMmanagedenvironmentisseamlessandcanbedonein small,selfdefined and conceivable steps.Basicallyitisallabouttransferring existing server(theirrootfile-system content)toastoragemanagedbyopenQRM.TomigrateanexistingserversystemintoopenQRMpleasefollowthe stepsbelow: CreateanemptyimagelocationononeofthestorageserverinopenQRM Syncthecompleterootfile-systemcontentoftheexistingservertotheemptyimagelocationonthestorage e.g.viarsync This step can be done during run-time of the existing system. If it is a virtual machine it is recommendedtostopitandstartthesyncingprocedurewhenitfullyshutdown. Make sure to exclude /proc and /sys from the syncing procedure and create them in the image locationasemptydirectories. Shutdowntheexistingserver,setitsBIOStonetwork-boot(PXE)andrebootit. ItwillbeautomaticallyaddedtoopenQRMasnew,idleresource Createanimageusingthepreviouscreatedimagelocationwhichisnowpopulatedwiththerootfile-system contentoftheexistingserver

50

Createanapplianceusingtheexistingsystemsresource,thenewcreatedimageandakernel. Starttheappliance TheexistingserverisnowfullymigratedandmagedbyopenQRM.Itsroot-deviceislocatedonthestorageserver (theimagelocation)anditsresourceisnowfullyflexibleandcanbereplacedatanytime.Theappliancecannow alsomovetoavirtualmachineortoadifferentphysicalsystematanytimesinceitsrootfile-systemcontentisnottied toitslocalhard-diskanymore.

kvm-storage
The'kvm-storage'pluginisacombinationofthelvm-storageandthekvmplugin.Itprovidesanewstoragetype kvm-storagebasedonlvmwhichisusedalocal-diskdeviceforKVMvirtualmachinesonthesamesystem. Please notice : This plugin provides support to manage KVM virtual machines in the common way. That means that the KVM vms are using local-disks which are logical volumes on the KVM-Storage Host. This results ina dependency tolocal-diskdevices onthe KVM-Storage Host.the KVMVMs depends onthe logical volumes provides by KVM-Storage Host VMs must be on the same KVM-Storage host where the logicalvolume(theVMsroot-disk)islocated Requirements: AserverfortheKVM-StorageHost (thiscanbearemotesystemintegratedintoopenQRMe.g.viathelocal-serverpluginortheopenQRMserveritself) TheserverneedsVT(VirtualizationTechnology)SupportinitsCPU(requirementforKVM) lvm2toolsinstalled One(ormore)lvmvolumegroup(s)withfreespacededicatedfortheKVMVMstorage KVMinstalled OneormorebridgesenabledfortheKVMvirtualmachines 1.KVMStorageManagement: Createanewstoragefromtypekvm-storage Createanewlogicalvolumeonthisstorage Usethelocal-storageplugintopopulatethenewlogicalvolume Nowusethelinuxcoe-plugintoautomaticallyinstallaLinuxdistributiononit. AnotheroptionistoconnecttotheVMsVNCconsoleandinstallanOSintheregularway. CreateanImageusingthenewcreatedlogicalvolumeasroot-device ResultisanopenQRMImage(server-template)whichcanbedeployedtoaKVM-StorageVM(onthesamesystem) viaanAppliance. 2.KVM(Storage)VMManagement: CreateanewapplianceandsetitsresourcetypetoKVM-StorageHost CreateandmanageKVMvirtualmachinesviatheKVM-StorageVMManager Thisresultsin new(idle)resourcesin openQRMwhich can be deployed with KVM-Storage volumes(on the same system)viaanAppliance.

51

3.KVMStorageDeployment: Createanewappliance SelectanidleresourcewiththetypeKVM-StorageVM SelectanKVM-StorageImage(onthesamesytemastheidleresource) SettheresourcetypeoftheappliancetoKVM-StorageVM Starttheappliance This step will assign the logical volume on the KVM-Storage Host as the local-disk and boot device to the KVM-StorageVM(onthesamesystem).TheVMnowbootsuplocallyfromthelogicalvolumespecifiedbytheImage.

xen-storage
The'xen-storage'pluginisacombinationofthelvm-storageandthexenplugin.Itprovidesanewstoragetype xen-storagebasedonlvmwhichisusedalocal-diskdeviceforXenvirtualmachinesonthesamesystem. Please notice : This plugin provides support to manage Xen virtual machines in the common way. That meansthattheXenvmsareusinglocal-diskswhicharelogicalvolumesontheXen-StorageHost.Thisresults in a dependency to local-disk devices on the Xen-Storage Host. the Xen VMs depends on the logical volumes provides by Xen-Storage Host VMs must be on the same Xen-Storage host where the logical volume(theVMsroot-disk)islocated Requirements: AserverfortheXen-StorageHost (thiscanbearemotesystemintegratedintoopenQRMe.g.viathelocal-serverpluginortheopenQRMserveritself) TheserverneedsVT(VirtualizationTechnology)SupportinitsCPU(requirementforXen) lvm2toolsinstalled One(ormore)lvmvolumegroup(s)withfreespacededicatedfortheXenVMstorage Xeninstalled OneormorebridgesenabledfortheXenvirtualmachines 1.XenStorageManagement: Createanewstoragefromtypexen-storage Createanewlogicalvolumeonthisstorage Usethelocal-storageplugintopopulatethenewlogicalvolume Nowusethelinuxcoe-plugintoautomaticallyinstallaLinuxdistributiononit. AnotheroptionistoconnecttotheVMsVNCconsoleandinstallanOSintheregularway. CreateanImageusingthenewcreatedlogicalvolumeasroot-device ResultisanopenQRMImage(server-template)whichcanbedeployedtoaXen-StorageVM(onthesamesystem) viaanAppliance. 2.Xen(Storage)VMManagement: CreateanewapplianceandsetitsresourcetypetoXen-StorageHost CreateandmanageXenvirtualmachinesviatheXen-StorageVMManager Thisresultsin new(idle)resourcesin openQRMwhich can be deployed with Xen-Storage volumes(on the same system)viaanAppliance. 3.XenStorageDeployment:

52

Createanewappliance SelectanidleresourcewiththetypeXen-StorageVM SelectanXen-StorageImage(onthesamesytemastheidleresource) SettheresourcetypeoftheappliancetoXen-StorageVM Starttheappliance This step will assign the logical volume on the Xen-Storage Host as the local-disk and boot device to the Xen-StorageVM(onthesamesystem).TheVMnowbootsuplocallyfromthelogicalvolumespecifiedbytheImage.

sanboot-storage
The sanboot-storage plugin provides a rapid, image-based deployment method especially for the Windows Operatingsystem.ItintegrateswithGPXE [http://etherboot.org/] andsupportstoseamlesslyconnectiSCSILunsand AOE/Coraid Storage devicesdirectlyin the bootloaderstage,before the actualOperating Systemisloaded.This makes it extremely useful specifically for Windows systems because Windows will still think it is using a local hard-disk. Requirementsforasanboot-storageserver: oneormorededicatedLVMvolumegroup(s)withfreespaceavailable EnterpriseiSCSITargetinstalled SoftwareAOEShelfvbladeinstalled HowtodeployWindowssystemsbooteddirectlyfromSAN Enableandstartthedhcpd,tftpd,sanboot-storageandwindowsplugin Startaphysicalsystemwiththeboot-sequencesetto:firstnetwork-boot,secondCDROM ThesystemnowbootsfromthenetworkandisautomaticallyaddedtoopenQRMasnew,idleresource Createanewstoragefromtypesanboot-storage Createanewvolumeonthesanbootstorage Createanewimageoutofthejustcreatedvolume Createanewapplianceusingtheidleresource,thedefaultkernelandthepreviouscreatedimage. Starttheappliance ThesystemwillnowrebootassignedandconfiguredtobootdirectlyfromtheiSCSILun(orAOEShelf).Sincethe Lunisstillemptyitwillfailafterthenetwork-bootconnectedtheiSCSILun(orAOEShelf)aslocal-diskandthenjump tothenextcofniguredboot-device,theCDROM.AtthispointWindows(7)canbeinstallednormally. PleasenoticethatWindowsXPneedstobefirstinstalledonalocalhard-diskandthenthehard-diskimage istransferredtotheiSCSILunorAOEShelf. ThisstepisnotneededforWindows7whichcanbedirectlyinstalledontheiSCSILunorAOEShelf. ForthedetaileddocumentationhowtotransferWindowsXPinstallationstoaSANpleasereferto: http://etherboot.org/wiki/sanboot [http://etherboot.org/wiki/sanboot] and http://etherboot.org/wiki/sanboot/iscsi_install [http://etherboot.org/wiki/sanboot/iscsi_install] InstallingtheWindowsopenQRM-Client After successful installation the Windows system should be integrated into openQRM by installing the Windows openQRM-Clientonit.TheWindowsPluginaddssupportfortheWindowsOperatingSystemstoopenQRM.Itconsist of a basic monitoring agent and a remote-execution subsystem which run on the Windows systems as Windows services after integrating them with a simple setup program. The openQRM-Client setup program is available for downloadontheAboutpageofthewindowsplugin.

53

Pleasenotice:BeforeyourunthesetupprogramfortheWindowsopenQRM-Clientpleasecreateanewuser rootonthewindowssystem! Please notice :Afterrunningthe Windows openQRM-Clientinstallerplease make sure tohave TCP port22 (ssh)enabledintheWindowsfirewall! HintforWindowsXP:Pleaserungpedit.mscandaddthePermissiontoremoteshutdownthesystemtouser root openQRMisnowabletomanageandmonitortheWindowssystem(-image).CommandsontheWindowssystems are executed via the virtual resource command hook which transparently translates e.g. a reboot action into shutdown.exe. TheWindowsserver-imageisnowreadyandfullyfunctionalusingtheopenQRMappliancemodel. Itis recommendedtocreate snapshots fromthe originalWindows server-image afteritis finally setupand thenjusttodeploythesnapshotsoftheorigin! Thatwayone can keep a differentWindowssystemrevisionswithoutloosing the origin state.Justre-creating the snapshotwillcauseafreshWindowsinstallation.

Image-baseddeploymentbasicallycomesin2flavours: Ready-setupandpre-configuredServerimages MinimalbaseServerimageswhicharethenleveragedduringdeploymenttime This section describes automated methods to leverage server images according users requirements during deploymenttime.

Plug-inboot-servicescanbeusedtoe.g.: installadditionalpackagedependencies pre-setupapplications startadditionalservices ThebasicopenQRMClientboot-serviceprovidesahandyshellfunctionopenqrm_install_os_dependency()toinstall additionalpackagedependencies.Thefunctionislocatedin

OPENQRM_SERVER_BASE_DIR/openqrm/include/openqrm-package-functions

Sourcingthisfunctionfilefromaplug-inboot-serviceinitscriptmakesitavailable.Itrequiresthepackagenameto installasfirstargumentandcanbeusedlike:
openqrm_install_os_dependency [package-name]

DependingontheLinuxdistributionitautomaticallydecidesontherightpackagemanager(dpkgorrpm)andtriggers and installation ofthe named package.The function can be used in forcemode byexporting the following shell parameterbeforerunningit:

54

FORCE_INSTALL="true"

Additionaltoautomatedpackageinstallationaplug-inboot-serviceinitscriptcanofcoursebeusedtopre-configure severalcustomapplicationconfigurationsandtostartadditionalserviceontheresourceitisrunningon.

ThePuppetintegrationinopenQRMallowsfurthertofullyautomatethedeploymentprocessoftheapplicationstack ontherequestedCloudAppliances. PuppetwasselectedbytheopenQRMteamasthenumberonechoicewhenitcomestoautomatedconfiguration managementbecausewithitallowssuchafinegraineddevelopmentoflargeITinfrastructures.Comparedtoother tools Puppet allows to code the data-center environment via the Puppet language. This programmable configurationlayerallowssystem-administrators,QAanddeveloperstodeveloptheirITinfra-structureviathewell definedpuppet-language.Thedata-centerthenbecomesre-makeable. ThePuppetfeaturesinopenQRMarealsointegratedandavailableintheopenQRMCloud.

Thefirststeptomakesureallsystemsandservicesinadatacentersarerunningwellistomonitorthem.openQRM comeswithseveraldifferentDatacentermonitoringoptionsfurtherdescribedinthissection.

openQRMcomeswithaClient,automaticallyinstalledonallmanagedsystemsintheopenQRMnetworkthroughthe basicopenQRMClientboot-service,which includesa basicmonitoring utility.Thisopenqrm-monitordconsistofa shellscriptsendingstatisticstotheopenQRMServerviathehttpsprotocol.Thestatisticdataincludesuptime,load, cpumodel+count,network-interfacecount,memory,swapetc.openQRMgathersthosestatisticsandcollectsthem initsdatabase. In case a resource does not send statistics for 4 minutes openQRM sets its state to error and runs the highavailabilityhook(furtherdescribedintheDevelopmentsection). TheoepnQRMClientisalsoavailableforthefollowingnon-LinuxOperatingSystems: Windows-describedintheWindowsdeploymentsection Solaris/openSolaris-describedinthesolx85plug-insection

Awell-known,provenandwidelyusedmonitoringtoolisNagioswhichisavailableforopenQRMintheflavorofan additionalplugin.Thesecond,alsoessential,stepistheautomatichandlingoferrors,whatopenQRMisfamousfor. The combination ofthe enhanced monitoring utilityNagiosand the automated error-handling,high-availabilityand fail-overfeaturesoftheopenQRMdatacentermanagementplatformcreatesapowerfulanddynamicenvironment whichreducesdown-timeofsystemsandservicesinamoderndatacentertotheminimum. Nagios is a widely adopted open-source host, service and network monitoring program based on a client/server concept.TheNagios-servergetsmonitoringinformationsbyactive-orpassive-checks.Thatmeansiteitheractively teststhe availabilityofa systemorservice fromthe Nagios-serveritselforitpassivelyreceivesinformationsabout tests running on the remote system. The passive checks are initiated by the Nagios-client (nrpe) running on the systemsmonitoredbytheNagios-server.TheclientpartofNagiosisdesignedinaplug-ableway.Newservicechecks

55

can be added easily by creating new Nagios-plugins interfacing with the plugin-architecture of the base Nagiosserver. TheNagios-serverrunswithin an Apache web-serverand consistsofperl-and shellscriptsmixed with binarytools executedviatheCGI-interface.InconfigurableintervalsitchecksvariousserviceslikeSMTP,POP3,HTTP,NNTP, PING, etc. and provides the gathered data center informations in a nice web-interface. It also monitors system resourceslikeCPU-load,memoryanddiskusage,runningprocesses,logs,etc.andenvironmentalfactorssuchas CPU-temperature.

collectdisadaemonwhichcollectssystemperformancestatisticsperiodicallyandprovidesmechanismstostorethe valuesinRRDfiles.TheintegrationofcollectdasanopenQRMplug-inprovidesanautomatedsetupofthecollectd serverandclientonallmanagedapplianceswithintheopenQRMnetwork.ThecollectdClientssendtheirstatistics informationstothemaincollectdserverrunningontheopenQRMServerasaplug-inservice.Perappliancestatistics are stored in RRD Database files which are used as the data source to frequently create well-arranged system statistics graphs out of them. The system graphs are embedded into the openQRM UI and also available for openQRMCloudusers.

Zabbix is a quite new System- and Service monitoring tool which is known for its great scalability. It is an enterprise-classopensourcedistributedmonitoringsolutionalsoavailableinopenQRMasadditionalplug-in. TheZabbixplug-inprovidesanautomatedZabbixServerandClientsetupfortheappliancesmanagedbyopenQRM. ZabbixClientsare automaticallydiscovered bythe ZabbixServerand through the embedded UIcustomsystem-, service-andnetwork-checkscanbedefined.

56

HereascreenshotofopenQRM'sVisualCloudDesigner:

Approximatelysincebeginningbeginningof2008thetermCloudComputingcameupasnewhypeintheIT-World directly after Virtualization. Since then this term is used for different new and also well-known technologies for managing large data-centersand ITinfrastructures.Some use itasa synonymforSoftware asa service(SAAS) comingfromtheWeb2.0evolution,someotherconnectthetermCloudComputingwithAutomatedProvisioningof alargenumberofvirtualmachinesandServersondemand.IngeneralthedefinitionofthetermCloudComputing leavesitopenbutdefinesinmoregenericway: CloudcomputingisInternet(cloud)baseddevelopmentanduseofcomputertechnology(computing)[1][2][3].Itis abusinessinformationmanagementstyleofcomputinginwhichtypicallyreal-timescalable[4]resourcesareprovided

57

as a service[5] over the Internet[6] to users who need not have knowledge of, expertise in, or control over the technologyinfrastructure(inthecloud)thatsupportsthem[7].(WikipediaCloudComputing) Even more, both aspects of Cloud Computing, SAAS and Automated Provisioning, are co-existing in a kind of symbioses relationship. SAAS means to provide a Service via a network (Inter- or Intranet) to its customers on demand. Most common for this case is that the applications and services are deployed to a separated virtual machine.ThisisolationoftheapplicationinitsownVMresultsinbetterflexibilityandsecurityfortheSAASprovider. TherequirementtodeployahugenumberofVMs(oneperapplicationinstance)needsautomatismwhichissolved throughAutomatedProvisioning. Cloud Computing is not a new technology but a new name for a combination of two or more already known technologieslikeHPC-andHAClustering,Grid-Computing,Utility-Computing,Distributed-Computingetc.Theyareall theretomanagelargeITinfrastructuresusedfordifferentpurposesinanautomatedway. Theroleoftheopen-sourcecommunityespeciallyforCloudComputing ModernCloudsmostlyconsistsofalreadyexisting,well-knownandoftenopen-sourcecomponents.Foreveryaspect of a Cloud environment a different set of utilities is used e.g. Puppet for automated configuration management, Nagiosforsystem-andservicemonitoring,oneormorevirtualizationtechnologies,Linux-HAforhigh-availabilityetc. ThoseutilitiesarealreadyusedinlargeproductionandacceptedbysystemadministratorsandtheIT-management. Itwouldmakenosensetore-writeallthosetoolsfromthescratchjustbecauseofanewname(CloudComputing)for knownmethodsofdeployingandmanaginglargeserverenvironments.ForthatreasonsCloudsaremainlyasetof looselyconnectedtools.ThechallengeformodernCloudComputingistheintegrationofthoselooselyconnected tools into a single management User Interface. It can only be archived via a plug-ability which combines all the separated utilities to benefit from their cooperation. This is exactly the goal of the openQRM data-center managementplatform.

TypesofCloudSolutions
PublicClouds[http://en.wikipedia.org/wiki/Cloud_computing#Public_cloud] PrivateClouds[http://en.wikipedia.org/wiki/Cloud_computing#Private_cloud] HybridClouds[http://en.wikipedia.org/wiki/Cloud_computing#Hybrid_cloud] CurrentlymostCloudsarespecializedtoonlyautomatedVirtualMachinemanagementbutnottomanagecomplete ITinfrastructuresusingtheconceptofCloudComputingandresourcesondemand.

Publicvs.PrivateCloudComputing
Clouds can be divided into Public Clouds and Private Clouds. While within a private cooperated data-center environmentsecurityaspectsmaydependonthesystemadministratorandontheiractualusage.ForainternallyQA network,protectedbythecompaniesfirewall,securitymaynotbethatcriticalcomparedtotheloadbalancercluster ofthemaindatabaseapplication.InanywayforPublicCloudComputinganexhaustivesecurityconceptisneeded to ensure the integrity of the users data and provide continuously reliability for the Cloud service. OpenQRM's thoroughlydesignedsecurityconceptconsistofautomatedauthenticationoftheserver-images,theseparationofthe hard-andsoftwarelayertoensurethedata-integrityandenableservicehighavailabilityviarapidre-deploymentplus theseparation between the actualCloud managementsystemand the end usersCloud portal.To gain additional securityontopoftheCloud-securityvirtualprivatenetworking(VPN)andfilesystemencryptioncan(andshould)be additionallyusedintheapplicationlayer.

The Cloud Comparison Matrix [http://www.openqrm-enterprise.com/fileadmin/DATA/Whitepapers/cloud_computing_compare-

58

20091215.pdf]

CreatedbyparticipantsofDevopsdays09inGhent

On top ofthisstraightand genericopenQRMframeworksitsthe Cloud plugin asa simple UIforthe end user.It provides a fully automated provisioning cycle from physical systems or virtual machines deployment through automatedapplicationandconfigurationmanagementaccordingtheusersrequestsuptoautomatedde-provisioning tofreeuptheuserscomputeresources.ViaanadditionalexternalCloudweb-portalusersregisteredtotheCloud can login to manage their own Cloud environment by requesting new resources, de-provisioning existing ones or managingtheiractiveCloudrequests. ForthesystemadministratortheCloudprovidesaninternalinterfacepluggedintotheopenQRMserver.Itprovidesa finegrainedoverviewaboutthecurrentCloudactivitiesandseveralconfigurationoptionstotunetheCloudbehavior e.g.theCloudcan besetto automaticallyormanualapprove newCloud request,automaticallycreate newvirtual machinesifnotenoughexistingoneareavailable,enableordisabletheclone-on-deployfeaturesetc.Italsoconsist ofa Cloud Ip-managerwhich isused to automaticallyconfigure the externalnetwork-interfacesofthe provisioned machines. The default behavior of the openQRM Cloud is to use the clone-on-deploy mechanism to provision resources requestedbytheusers.ThismeansthatforeveryCloudrequesttheintegratedstoragemanagementwillexecutea clone command on the storage server hosting the golden-image (server-image template) and then use the snapshottedserver-imagetodeployitfortheuser.Thismethodhasthehugeadvantagethatthesnapshotsonthe storageserver(e.g.viatheLVMsnapshotstoragefeature)normallydonotconsumeanyspacebecausetheyare onlyread-onlycopiesfromtheoriginallogicalvolume(server-image).Thismeansonlythechangestheusersmakes toitsserveraresavedtothestorageandnoadditionalspacefortheactualrootfilesystemplusitsapplications. ForautomatedleveragingtheapplicationlayeroftheprovisionedsystemsaccordingtotheusersrequestopenQRM integrated with the Puppet configuration management system. Puppet takes care to setup and pre-configure the usersmachinesviapre-madeandknown-to-workrecipes.TheintegrationofPuppetasanadditionalpluginandthe cooperationofthePuppet-pluginandtheCloudadd-onprovidesaselectionofout-of-the-boxapplicationservers andgivesaddedvaluebyautomatismfortheusersandadministrators. TheopenQRMCloudcomeswithanintegratedbillingsystemforthecomputeresourcesconsumedbytheendusers. Based on Cloud Computing Units(CCU's),the virtualcurrencyin the openQRMCloud,systemadministratorsand userscanplanandkeeptrackoftheirusedcomputepowerandcosts.OpenQRMsimplycalculatestheamountof CCU'sperrequestandsubtractitfromtheUsersaccount.ThatwaytheCloudmanageriscompletelyopenhowto sellthecomputingpowertothemarketendusers(e.g.viaEbay). MoredetailsabouttheCloudconfigurationinthesectionPlugindescription.

openQRMplug-able architecture isdesigned to fullysupportcontinuousintegration.With itsstaticbase framework which basicallyjustmanagesthe differentplug-insopenQRMallowsseveraldeveloper(-teams)to workon several plug-insinparallelwithoutinterferingeachother.openQRMcomeswithseveralspecialfeaturesmakingthelifeofits developerseasiere.g.itsbuild-andpackagingsystemdescribedinthenextchapter.

ThesourcesofopenQRMareorganizedinthesamewayasinthelaterinstallation.Itsdirectorystructureisadapted to a standard unix filesystem layout plus the source (or base-dir) pre-fix. openQRM tools are located in src/bin,

59

configuration filesin src/etc,daemonsin src/sbin and pluginsin src/plugins.The main makefile runsa loop forall directoriesinsrc/.Ineachdirectoryitexecutesthecorrespondingmaketarget.Inthepluginsdiritrunsaseparated loopforallplug-indirectories. Themainmakefilecontainsthefollowingmaketargets: check-checksruntimerequirements configure-pre-configuresthesources(emptybydefault) compile-compileopenQRManditscomponenents install-installopenQRMfromthepreviouscompiledcomponents(needsroot) start-startstheopenQRMServer(needsroot) uninstall-uninstallstheopenQRMServer(needsroot) clean-cleansthesourcetree realclean-cleansthesourcetreeandbuildcache reinstall-stops,uninstalls,cleans,re-compiles,installsandstartsopenQRMServer(needsroot) rpm-buildsrpmpackages(needsroot) deb-buildsDebianpackages(needsroot) debsource-buildsDebiansourcepackage initrd-buildsaninitrd-template buildrequirements-checksthebuild-requirements update-updatessourcesfromthesvnrepo

build-cache
TheopenQRMbuild-systemusesitsowncachingfordownloadedcomponentsandalreadycompiledbinaries.This build-cache creates a directory src/../buildtmp (one level above src/) which keeps files once their are downloaded and/or compiled once. This method dramatically speeds up the development of openQRM since frequent re-compilations just takes a few seconds. Please notice that the first build will always take some time but all componentsarebeingcached,eachfurtherbuildjusttakesafewmoments.

build-functions
openQRMcomeswith2especiallyusefulbuildfunctionscontainedinsrc/include/openqrm-build-functions. openqrm_cache_or_download Makessureacomponentisavailableinthebuild-cache,ifnottriggerdownload openqrm_compile_from_source Compilesandsetsupcomponentsinthebuild-cache Thosefunctionstakesacomponent.confbuild-configurationasthefirstargument.Furtheractionisfullyautomated. Thecomponentsbuild-configurationfileacceptsthefollowingparameters:
OPENQRM_SOURCE_VERSION="[components-version]" OPENQRM_SOURCE_DOWNLOAD="http://[components-download-location]/[component]-$OPENQRM_SOURCE_VERSION.tar.gz" OPENQRM_SOURCE_BINARY_RESULT="[resulting-files]"

Additionalthefollowgingshellexportscontrollingtheexactcompilephase:
OPENQRM_MAKE_TARGET="[the-make-target-to-be-executed]" OPENQRM_PRE_MAKE="[commands-to-be-executed-before-the-make-phase]"

60

OPENQRM_POST_MAKE="[commands-to-be-executed-after-the-make-phase]"

Torunthosefunctionsdirectlyfromwithinmakefilesamake-assistantutilisavailableinthemainsourcedir.

automatedpackaging
The packaging-functions file (src/include/openqrm-package-functions) contains functions for fully automated packaging: openqrm_rpm_package - Builds RPM-packages of openQRM using rpmbuild and .spec file templates from src/rpm/(executedbymakerpm) openqrm_create_deb_source - Creates a Debian source-package of openQRM (executed by make debsource) openqrm_pbuilder - Builds Debian-packages via the Debian pbuilder packaging util (executed by make deb)

PluginTemplate
To make it easy starting to develop a new openQRM Plug-in an openqrm-create-plugin script is provided which automaticallycreatesanewPlug-inskeletonbasedonanexistingone.Herehowtouseit:
OPENQRM_SERVER_BASE_DIR/openqrm/bin/openqrm-create-plugin OLD_PLUGIN_NAME NEW_PLUGIN_NAME

BasicallythisscriptrenamesallOLD_PLUGIN_NAMEintoNEW_PLUGIN_NAME.

providedPluginsHooks
appliancehook(PHP) Duringadd/remove/start/stopapplianceactionsopenQRMchecksforeachpluginifitprovidesanappliance-hookfile at
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/openqrm-[plugin-name]-appliance-hook.php

Thishookshouldimplementthefollowingfunction:
openqrm_[plugin-name]_appliance($cmd, $appliance_fields)

openQRMthenrunsthisfunctionwiththefollowingparameters:
$cmd - add, remove, start, stop $appliance_fields - An array containing all appliance parameters

Example: The dns-plugin isusing the appliance hookto automaticallymanage DNSresolving ofthe appliance name during startandstopactions.

61

virtualresourcecommandhook(PHP) FortherebootandpoweroffresourceactionsopenQRMlooksfor
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/openqrm-[plugin-name]-resource-virtual-command-hook.

IfexistingopenQRMruns:
openqrm_kvm_storage_resource_virtual_command($cmd, $resource_fields)

Parametersinthishookfunctionare:
$cmd - reboot, halt $resource_fields - An array containing all resource parameters

Example: Thekvm-storageplug-inisusingthisvirtual-resource-commandhooktomape.g.therebootresourceactiontothe correspondingVMrestartcommand. kernelhook(Shell) WhenanewkerneliscreatedviatheOPENQRM_BASE_DIR/openqrm/bin/openqrmutilopenQRMchecksifplug-ins providingan

OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/include/openqrm-plugin-[plugin-name]-kernel-hook

shellfunctionfileimplementingthexen_kernel_hookfunctions.Thisfunctionisfeedwiththefollowingparameters:
KERNEL_NAME=$1 KERNEL_VERSION=$2 KERNEL_LOCATION=$3 KERNEL_TYPE=$4

Example: Thexenplug-inisusingthiskernelhooktoadditionallycopytheHypervisorfiletothetftpbootdirectory. assignhook(Shell) Duringassignment(start)ofanapplianceopenQRMchecksifplug-insprovidingan

OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/include/openqrm-plugin-[plugin-name]-assign-hook

shell function file implementing the [plugin]_assign_hook functions. This function is feed with the following parameters:
OPENQRM_RESOURCE_ID=$1 OPENQRM_KERNEL_NAME=$2 OPENQRM_RESOURCE_PXELINUXCFG_FILE=$3

Example: Thexenplug-inisusingthisassignhooktore-writetheresourcePXEconfiguration. resourcehook(PHP)

62

TheopenQRMresourcehookisexecutedifplug-insprovidingthefollowingresource-hookfile:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/openqrm-[plugin-name]-resource-hook.php

IfexistingopenQRMruns:
openqrm_[plugin-name]_resource($cmd, $resource_fields)

Parametersinthishookfunctionare:
$cmd - add, remove $resource_fields - An array containing all resource parameters

Example: Thedhcpdplug-inisusingthisresource-hookadd(andremove)resourceto(andfrom)itsdhcpd.confconfiguration file. image-authStoragehook(PHP) To automatically authenticate image-locations on storage servers openQRM checks if plug-ins providing the image-authhookfileat:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/openqrm-[deployment-type]-auth-hook.php

Thedifferentdeploymentstypesaredefinedbythespecificstorageplug-in.openQRMthenruns:
storage_auth_function($cmd, $appliance_id)

Parametersinthishookfunctionare:
$cmd - start, stop $appliance_id - the appliance id

Thisfunctionthenneedstomakesuretotriggeranauthenticationupdateoftheimagespecifiedintheapplianceon the storage server (where the image is located). For the start action the function makes sure to only allow the applinaceresourcetomounttheimagelocationfromthestorage.Duringstopitde-authenticatestheimage-location again. root-mounthook(Shell) The root-mounthookisa shellfile containsthe mount_rootfunction which isresponsible to mountthe rootfilessystemfromthe storage serverwithing the initrd-stage ofa booting resource.Thisfunction ispurelystorage type dependentanddefineshowtomounttheroot-fs.Thefilenameofthishookis:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/root-mount.[deployment-type]

ItshouldbelinkedtoopenQRM'sboot-servicedirectorywithintheopenQRMweb. Resourcesassignedtoanactiveappliancedownloadthisfileandrunthemount_rootfunctiontomounttheirroot file-systemduring boot-up.Thishookcan also define addtionaloptionalfunction to e.g.install-from-nfsortransferto-local. Example:

63

Theiscsiplug-inisusingtheroot-mounthooktodefinehowresourcesarebooteddirectlyfromtheiSCSILunsof thetarget. root-deviceidentifierhook(PHP) To abstractimagesand make themselectable e.g.in the appliance configuration formstorage plug-insneedsto provideanroot-deviceidentifierhookviathefile:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/image.[deployment-type].php

Thefileimplementsthefollowingfunction:
get_image_rootdevice_identifier($storage-id) $storage-id - The openQRM storage id

Thefunctionshouldreturnanarrayof[image-volume-identifier-on-the-storage],[image-name] Thehookfilecanalsoimplementget_image_default_rootfs()whichthenfillstheroot-devicetypeinputboxinthe image-newform. Plug-inmonitorhookforreiteratingcommands(PHP) For reiterating commands openQRM provides the plug-in monitor hook which is executed every 10 seconds. openQRMlooksforeachplug-inifitprovidesafilenamed:
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/openqrm-[plugin-name]-monitor-hook.php

IfsuchfileexistopenQRMrunsthefunction:
openqrm_[plugin-name]_monitor()

Example: Thecloudplug-inisusingtheopenQRMmonitorhooktofrequentlycheckforCloud-events. Cloud-Billinghook(PHP) ToallowcustombillingfortheopenQRMClouditprovidesabillinghooklocatedat:

OPENQRM_SERVER_BASE_DIR/openqrm/plugins/cloud/web/openqrm_custom_cloud_billing($cr_id, $cu_id, $cu_ccunits)

Insertcustombillingfunctionsthere. High-Availabilityhook(PHP) IncaseofaresourceerroropenQRMchecksifplug-insprovidingahigh-availabilityhook.Thehookfileisnamed:

OPENQRM_SERVER_BASE_DIR/openqrm/plugins/[plugin-name]/web/openqrm-[plugin-name]-ha-hook.php

Ifaplug-inprovidesthisHAhookopenQRMrunsthefollowingfunctionimplementedbythehook:
openqrm_highavailability_ha_hook($resource_id)

The function then isresponsible to re-actin the rightwayto make sure no service interruption occurs.Itisonly executedifaresourceisassignedtoanactiveappliance.

64

Example: Thehighavailabilityplug-inimplementsthishooktoreplacefailedresourcesinactiveappliances.

ImageHooks
Themount-pointconfigurationfile/etc/fstabonserver-imagesiscreatedautomaticallyaccordingtheimagesstoragelocationduringtheinitrd-stageoftheboot-upphase.Foraddingadditionalstaticmount-pointscreateafile/etc/fstabstaticontheserver-imagecontainingtheadditionalmount-points.Thisfileisautomaticallyaddedto/etc/fstabduring start-up.

Plug-inBoot-services
EachPlug-incanprovideaBoot-serviceforthemanagedresources.ThisBoot-serviceconsistofapackedPlug-in specific tools file which is being linked into the openQRM webspace at Plug-in Init. By the start phase of the openQRM-client this file is then being downloaded and unpacked by starting resources. In case this package includesaPlug-ininitscripttheopenQRM-clientwillalsostartthePlug-inontheresource.ThePlug-ininitscriptfor managedresourcesshouldbelocatedat
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/PLUGIN_NAME/etc/init.d/PLUGIN_NAME

withintheBoot-servicepackage.Itshouldsupportthestartandstopparameter.

Plug-inServices
Plug-in'salsohavethecapabilitytostart(andstop)additionalservicesontheopenQRMServeritself.TheopenQRM ServerinitscriptlooksforPlug-ininitscriptsat
OPENQRM_SERVER_BASE_DIR/openqrm/plugins/PLUGIN_NAME/etc/init.d/openqrm-plugin-PLUGIN_NAME

ontheopenQRMServer.IfthisfileisprovidedbythePlug-inopenQRMwillrunitwiththestart/stopparameterduring openQRMServerstart/stop.

EveryCloudComputingEnvironmentneedstobeintegratedintoacustomer-managementframework.Especiallythe externalCloud-PortalinopenQRMplusitsgenericbillingmechanismviaCloudComputingUnits(CCUs)makesthis integration easy. Still for a seamless integration with an existing customer-management application a WebService including a well defined API was missing. The openQRM Team understood this need and implemented a SOAP-WebServicefortheopenQRMCloud.ItsSOAP-Server,workinginWDSL-mode,isseparatedintoanCloudAdminandCloud-Userpart.Itisexposingtheautomaticprovisioning/de-provisioningmechanismstoSOAP-Clients andcanbeusedasafullremote-controlfortheopenQRMCloud. AdetaileddocumentationandhugeexamplesfortheCloud-AdminandCloud-UserSOAP-Clientsareincludedin theCloud-plugin.TheopenQRMCloudSOAP-WebServiceplusitsdocumentedAPImakesitveryeasytointegrate theopenQRMCloudwithanythird-partyapplication. The openQRM Cloud API-Documentation for the Cloud SOAP WebService can be easily generated using phpDocumentor [http://www.phpdoc.org/] . An online version of the openQRM Cloud API is available at : http://www.openqrm-ng.net/downloads/plugins/cloud/openqrm-soap-api/ [http://www.openqrm-ng.net/downloads/plugins
/cloud/openqrm-soap-api/]

65

HeretheCodingandDesignGuidelinesofopenQRM.Thegoalisittokeepthefollowingspecifications: KIS(keepitsimple) runeverywhere(supportforeverylinuxdistribution,especiallyDebian+Ubuntu) writeaslesscodeaspossible(lesscode=lessbugs) usingexistingcomponents(insteadofprovidingownbinaries/libs) supportmultipledatabases(db2,mysql,oracle,postgres) commentsourcecode!

openQRMenvironmentsare based on a separated management-and storage-networkplusone ormore publicnetwork. For production setups it is recommended to adapt a few security considerations to protect the management-andstorage-networkasdescribedinthechaptersbelow.

TheopenQRMServerconsistofseveralsecuredsubsystemstosupportmanagingmission-criticalDatacenters. SSLencryptionfortheUIaccess All http communication between a client browser and the openQRM Server UI are fully encrypted via SSL [http://en.wikipedia.org/wiki/Transport_Layer_Security] . Also the internal communication between the openQRM Client (e.g.forsendingstatistics)areusingtheSSLsecuredhttps. Hint:IfneededopenQRMcanstillbesetupusingthe(unsecure)httpprotocolbychangingthe
OPENQRM_WEB_PROTOCOL="https"

parameterinthemainopenQRMconfigurationfileat:
OPENQRM_SERVER_BASE_DRI/openqrm/etc/openqrm-server.conf

to:
OPENQRM_WEB_PROTOCOL="http"

SSLencryptedcommandexecutionlayer The openQRM remote command execution subsystem is based on Dropbear [http://matt.ucc.asn.au/dropbear /dropbear.html] , a tiny SSHD server automatically installed on behalf of the openQRM Client. It is using shared public-keymechanismtosecurelytransmitcommandstoremotesystems,encryptedbySSL. Storageauthentication ForeverydeploymentactionopenQRMautomaticallymakestotosetupauthenticationfortheaffectedvolume(s)on theinvolvedstoragesystems.Ittakescarethatonlytheinvolvedresourceisableandallowedtomounttheremote diskfromthe storage.Thismechanismpreventssituationswhere userAisable to mountusersBdiskimage in a robustandsecureway.

66

Definedcommandexecution IntheopenQRMCloudtheCloudusersdonotdirectlyinteractwithvirtual-orphysicalsystemsintheDatacenterbut allcommandsare pre-defined and executed on behalfofthe openQRMCloud.The openQRMCloud isthen just submittingthepre-definedcommandstotheSSL-securedcommandexecutionsubsystem.

Usingfirewallsbetweenthemanagement-andpublic-network To improve network-security package-filtering firewalls should be placed between the management- and publicnetwork.Viatheappliancestart-andstop-hookthefirewallrulescanbeadapteddynamicallyaccordingthenetworktrafficrequirementsfromtheapplianceresourcetothestorage. UsingVLANstophysicallyseparatemanagement-andpublic-network Tofurtherimprovenetwork-securityVLANtaggingcanbeusedtoprovideafullphysicalnetworkseparationforthe managed systems in openQRM. Same as for the dynamic adaption of the firewall rules the appliance start- and stop-hookaregoodcandidatestoconnectthemtothenetwork-switchconsoleforautomaticapplyingupdatedVLAN configurationsaccordingthenetwork-trafficrequirementsfromtheapplianceresourcetothestorage.

CustomfirewallingforAppliances To implementcustomfirewallrulesforthe managed appliancesitisrecommended to create Puppet-recipeswhich sets up specific firewall configurations according the appliances services. Those Puppet-recipes can be manually assignedtoappliancesbythePuppet-Manager(partofthePuppetopenQRMplug-in)orappliedautomaticallyusing theappliancestart-andstophooks. AutomatedServiceMonitoring To be bestinformed and up2date aboutthe servicesrunning in the DatacenteropenQRMoffersand automated servicemappingviatheNagios3integration.ThisspecialNagiosmodewilldisplayexactlywhichnetwork-servicesare available in the openQRM managed environment. For further, fine grained system- and service-monitoring it is recommendedtousetheCollectd-andZabbixplug-insavailableinopenQRM.

Endof2009MatthiasRechenburg,ProjectManagerofopenQRM,foundedtheopenQRMEnterpriseGmbHtogether withhisbusinesspartnersAndreWestbunkandChristophMoeller. AsastrongpartnerandthemainsponsoroftheopenQRMDataCenterManagementandCloudComputingPlatform openQRMEnterpriseprovidesprofessionalservicesandlong-termsupportwithfirsthandcompetence. Withahugepoolofexperienceinmanagingdatacentersinaflexible,consistentandtransparentway,openQRM EnterpriseenablesITorganizationstoefficientlyprovidetheirITservicesinarobust,standardized,performantand high-availablemode. openQRMEnterpriseconsolidatesthetechnicalcompentenceofthecoremembersanddevelopersoftheopenQRM Project, experienced with every detail of this open source solution, to supply expert knowledge for custom, sustainabledatacentersetupsinbestpractice.openQRMEnterprise'sfocusistolowertheTotalCostofOwnership (TCO)forITdepartmentsusingaprovenopensourceframework.

67

Copyright2010,MatthiasRechenburgmatt@openqrm-enterprise.com[mailto:matt@openqrm-enterprise.com] ThisdocumentationisbroughttoyoubyopenQRMEnterprise [http://www.openqrm-enterprise.com/]

openQRMEnterpriseGmbH BerrenratherStrae188c 50937Kln/Germany Telefon:+49(0)221995589-10 Fax:+49(0)221995589-20 Mail:info@openqrm-enterprise.com

68