Veritas Volume Manager

VM daemons Maintains system configuration in the kernel & disk (private region). If the daemon is stopped it does not disable any configuration state loaded into the kernel, it only affects the ability to make configuration changes until vxconfigd is restarted. vxconfigd It can be in three states: nabl ! "ormal mode #isable ! Most operations cannot be be used $ooted ! "ormal startup %hile using boot disk group Monitors for failure events and relocates failed subdisks &sed to backup configuration chnages, the files created can be used %ith vxmake to restored lost groups. #isplay vertias volume manager events used %ith the vxconfigd daemon Kernel Info )he kernel can be in three states: 'ernel (tates nabled * both private and public regions are accessible #isabled * no private or public regions are accessible #etached * only private regions are accessible

vxrelocd vxconfigbackupd vxnotify

VM utilities vxconfigd *k *m enable *x ,debug option+x+M debug *x log *x logfile ! ,name*x syslog *x timestamp *x tracefile!name log to .var.vxvm.vxconfigd.log log to filename log to syslog date and timestamp every entry log all possible tracing to file

)he vxiod utility starts, stops, or reports on + /I)0( +olume Manager (+x+M) I.1 daemons. 0n I.1 daemon provides a process context for performing I.1 in +x+M.Manage extended disk i.o & handles dirty regions, logging vxiod vxiod set ,number- ! set number of runnning viod daemon "ote: %hen run on its o%n it displays 2 of vxiod daemons that are running. vxdctl [option] )he vxdctl utility manages aspects of the state of the volume configuration daemon vxconfigd and also manages aspects of configuration for bootstrapping the rootdg disk group. mode ! %hat mode the vxconfigd is running in enable ! enable the vxconfigd daemon (reread the db) disable ! disable the vxconfigd daemon stop ! kill the vxconfigd daemon (&se 3vxconfigd *k *m disable3 to start again) license 4init5 ! print out license info or reread licenses

support ! display version and components list ! display entries in .etc.vx.volboot init 4dmp5 ! recreate .etc.vx.volboot "ote: %hen is disabled or stop mode no +6 commands %ill be able to run vxinstall Disk Regions )his is %ere veritas holds the meta data regarding the disk. 0 copy of the configuration database is copied to each private region %ithin the disk group. +eritas %ill try and keep 7 copies of the configuration database. "ormally configured as slice 8 )his is the area that %ill store the users data. Public "ormally configured as slice 9. Disk ayouts private region and public region slices are on seprate partitions (8 & 9), tis type of disk is not suitable for moving bet%een different 1.(3s but are suitable for boot partitions :an be converted to :#( "D! #"ross$ platform Data !%aring& !imple :an be converted to :#( 'one "one partitioning private and public regions are one slice (slice ;) , this type is suitable for moving bet%een different 1.(3s but not suitable for boot parttions. <rivate and public are the same partition but continuous (slice 8) Install volume manger (use .etc.vx.disk to exclude any disks or controllers)

Private

!liced Disk layout

VxVM "onfiguration Database #$ si=e #$ location (ile ocations vxinstall has not be run ?ost I#3s backup config files (vxconfigbackupd) delete or deported disk group .etc.vx.reconfig.d.state.d.install*db .etc.vx.volboot .etc.vx.cbr.bk .etc.vx.dgcfg.deport vxdg list ,group- > grep permlen * the si=e of the configuration db vxdisk list ,disk- > grep *i configs * db location

config files 0ll commands logs @icenses .var.adm.vx.veacmdlog .etc.vx.licenses.lic .var.vxvm.tempdb Imported disk groups info "ote: to clear the tempdb file: vxconfigd *k *x cleartempdir * clear the .var.vxvm.tempdb vxconfigd log file )ackup * Restore .usr.lib.vxvm.bin.vxconfigbackup *l .var.vxvm.backups $ackup *l ! location %here to store backup vxconfigrestore *p ,group2 either one of the belo% after the precommit vxconfigrestore *d ,groupvxconfigrestore *c ,group"ote: *p ! %hen you %ant to check that the restore is correct (use vxprint to check) *d ! abort the precommit *c ! commit the precommit Disks vxdisksetup *i cAtBdB privlen!;CD vxdisksetup *i ,device- format!sliced * initiali=ed a disk as a sliced disk Initiali=e disk "ote: format can be either sliced, simple, cdsdisk or none (see above * #isk /egions) &ninitiali=e disk #isk Information /esi=e a @&" 0dd a disk slice to volboot 0dd a disk slice 0dd a disk vxdiskunsetup *: cAtBdB vxdisk *g ,group- list ,diskvxdisk *s list vxdisk *g ,group- resi=e ,disk- length!DE vxdctl add disk ,device- type!simple vxdisk Ff ,device- type!simple vxdiskadd cGtBdB or cG (all disk on controller) vxdisksetup *i ,device.var.vxvm.vxconfigd.log

/estore (precommit.commit)

/emove a disk totally from vxdisk rm ,device+M /emove a disk from a volume vxdg *g ,group- rmdisk ,diskname-

/emove a disk slice from +M :lear any host I# flags /enaming a disk

vxdctl rm disk ,devicevxdisk clearimport ,disk namevxedit *g ,disk- rename ,old disk name- ,ne% disk name-

Move disk to different disk vxdg move ,source dg- ,target dg- ,diskgroup vxdisk offline ,dev name1ffline a disk "ote: disk must not be in a disk group vxdisk online ,dev name1nline a disk "ote: disk must have a private region other%ise you need to initialise the disk ?ot spare "o?ot&se )urn off failing flag ncapsulate a disk vxedit Fg ,group- set spare!on ,diskvxedit Fg ,group- set nohotuse!on ,diskvxedit *g ,group- set failing!off ,diskvxdisk define cBtBdBsB type!nopriv vxreattach 4*br > *c5 /eattach disk ((0") *b ! $ackground process *r ! /ecover volumes *c ! :hecks to see if reattach is possible vxdisk scandisks 4ne% > fabric 5 vxedit *g ,group- set comment!H.......H ,disk-

#iscover ne% disks #isk :omment Disk +roup :reate a disk group /emove a group 0dd a disk to a group /emove a disk from a group /eplace failed disk

vxdg init group ,disk-!,devicevxdg init ,group- ,disk-!,device- cds!off vxdg destroy ,groupvxdg Fg ,group- adddisk ,disk-!,devicevxdg Fg ,group- rmdisk ,diskvxdg *k *g ,group- adddisk ,disk-!,device-

* initiali=e a non*cds disk group

*k ! forces vxvm to take media name of the failed disk & assign it to the ne% disk vxdg import ,groupvxdg *n ,ne%*group*name- import ,old*group*namevxdg *: import ,groupvxdg import *: ,group*: * clears any exist host flags vxdg deport ,group-

Import a group

Import group (clear any flags) #epot a group

vxdg *n ,ne%*group*name- deport ,oldIgroupIname@ist no hot use on disk @ist spare space on disk #isplay free space vxdg Fg ,group- nohotuse ,diskvxdg Fg ,group- spare ,diskvxdg Fg ,group- free

$ackup disk group (vxvm vxconfigbackup 9.B) /estore disk group (vxvm vxconfigrestore 4*p>*d>*c5 9.B) #iskgroup +ersion &pgrade disk version vxdg list ,group- > grep *i version vxdg upgrade ,group* upgrade to current version vxdg *) 7B upgrade ,group* upgrade to version 7B vxdg *) 7B init ,group- ,disk-!,device- * creater ne% group J version 7B vxdg bootdg vxdg defaultdg vxdctl defaultdg ,group- * set defaultdg Volume 0dding mirror to root .etc.vx.bin.vxrootmir ,alternate- create rootvol, s%ap vol, vxassist make ,volume- ,si=e- ,diskvxassist *g ,group- make ,vol- ,si=e- Kctrl:cA * don3t use controller A

$oot.#efault #E

:reate a simple volume

vxassist make ,volume- ,si=e- ,disk- layout!4stripe*mirror>concat*mirror> mirror*concat>mirror*stripe5 stripe*mirror ! layered volume concat*mirror ! layered volume mirror*concat ! non*layered volume mirror*stripe ! non*layered volume

Mirror a simple volume

vxassist mirror ,volume- -diskvxassist *g ,group- remove mirror ,vol- KdiskBG vxassist make ,volume- ,si=e- layout!stripe

* remove the diskBG mirror

:reate a stripped volume

vxassist *g ,group- *o ordered make ,vol- ,si=e- layout!stripe ncol!8 ,diskG,diskA- ,disk8-

:reate mirrored volume vxassist make ,volume- ,si=e- layout!mirror, log nmirror!2 nlog!2 %ith log :reate a raid volume /emove a volume Initiali=ing a volume 1nline /elayout vxassist make ,volume- ,si=e- layout!raid7 vxedit Frf rm ,volumevxassist *g ,group- remove volume ,vol"ote: you must disable the volume first vxvol init state ,volume- 4plex5state!clean,enable,active vxassist *g ,group- relayout ,vol- layout!stripe ncol!A

vxassist *g ,group- relayout ,vol- layout!stripe ncol!LG vxassist *g ,group- relayout ,vol- layout!stripe ncol!*G vxassist *g ,group- relayout ,vol- layout!stripe stripe!8Ak ncol!7 vxassist *g ,group- relayout ,vol- layout!raid7 stripeunit!8Ak ncol8 vxassist *g ,group- convert ,vol- layout!stripe*mirror 2 #isplay the relayout operation vxrelayout *g ,group- 4status>reverse>start5 ,volvxtask list (tarting a volume (tart a disabled volume #isable a volume vacuate a volume Maintenance mode "ot clean "o kernel state xtending a volume si=e (hrinking a volume si=e add a #/@ log to a volume vxvol start ,volumevxrecover Fsb ,volume*s ! start volume after recovery *b ! background the recovery task vxvol stop ,volumevxevac *g ,group- ,from*disk- ,to*diskvxvol maint ,volumevxmend mirror clean ,plexvxplex att ,volIname- ,plexvxresi=e ,volume- ,ne% lengthvxresi=e *g ,group- ,volume- LGBBm vxresi=e ,volume- ,ne% lengthvxresi=e *g ,group- ,volume- *GBBm vxassist addlog ,volume* increase the volume by GBBMb * decrease the volume by GBBMb

remove a #/@ log from vxassist remove log ,volumea volume xtending log si=e #etering volume si=e vxvol set loglen ! Am ,volume.logvxassit *g ,group- maxsi=e layout!mirror * the maximum si=e you can create a mirror vxassist *g ,group- maxgro% ,volume- * the maximum si=e the volume can gro% too +xmend fix clean ,plexvxedit *g ,group- set o%ner!,user- group!,group- mode!,perms- ,vol-

/ecover a volume :hange volumes permissions Plexs :reating a plex /emove a plex Moving a plex

vxmake plex ,plex- sd !,sub disk namevxplex Fo rm dis ,plexvxplex *g ,group- dis ,plexvxedit *g ,group- *rf rm ,plexvxplex mv ,original plex- ,ne% plex-

:opying a plex 0ttaching a plex #etaching a plex 1fflining a plex

vxplex cp ,volume- ,ne% plexvxplex att ,volume- ,plexvxplex det ,plexvxmend off volBG*BA

!ub$disks :reating sub*disk /emoving sub*disk Moving sub*disk 0ssociating %ith a plex #issociating (plitting Noining relocating a sub disk vxmake sd ,sub*disk- ,disk-, offset, len vxedit rm ,sub*diskvxsd mv ,old sub*disk- ,ne% sub*diskvxmake plex ,plex- sd!,sub*disk-, M i.e vxmake plex home*G sd! diskBA*BG, diskBA*BB, diskBA*BA vxsd dis ,sub*diskvxsd Fs,si=e- split sd,ne% sub-,ne%subAvxsd Ooin ,sub*diskG-,subdiskA-,ne% subdiskvxassist *g ,disk group- move KdiskB7 diskBA

relocating a %hole disk vxprint *g rootdg *se 3sdIorigIdmname!HdiskBAH3 sub disks vxunreloc *g rootdg diskBA Volume Manager Information Disks #isplay all the physical disks vxdisk list vxdisk *o alldgs list vxstat Fg ,group-Fff Fd Disk +roup #isplay group properties vxdg list #isplay detailed group info vxdg list ,groupvxinfo *p *g ,groupVolume #isplay volume info #isplay volume properties #isplay unstartable volume :heck for volume vxprint F0ht ,volvxprint Fvl vxinfo *g ,group- ,volvxstat Fg ,group-Fff *v

#isplay detailed disk info vxdisk list ,disk:heck for disk failures

failures Plex #isplay plex properties :heck for plex failures #isplay sub*disk properties vxprint Fvp vxstat Fg ,group-Fff *p !ub$Disks vxprint Fst Veritas ,asks vxtask list vxtask monitor * continuously monitor #isplay tasks (tates: r ! running p ! pause a ! aborting !tatistics and ,racing Iostats vxstat *g ,group- *r Fd ,disks* reset all stats on disk vxstat *g ,group- *d * display stats vxstat *g ,group- *i G *d ,vol* display stats every G sec intervals for volume vxstat *g ,group- *i GB *c 7 *d * display 7 sets J GB secs intervals vxtrace *d ,filename- *o dev,disk ,volvxtrace *f ,filename- *o dev,disk ,vol- > more

)racing icensing 0dd +ie% <aths /eload ne% license V-. (tart.(top

vxlicinst vxlicense *c vxlicrep vxlicense *p

* versions greater than 8.7 * versions belo% 8.7 * versions greater than 8.7 * versions belo% 8.7

.etc.vx.licenses.lic * versions greater than 8.7 .etc.vx.elm * versions belo% 8.7 vxdctl license init

.etc.init.d.isisd 4start>stop>restart5 vxsvcctrl status vxsvc 4*m>*k>*v5

(tatus

*m ! status *k ! kill *v ! version .opt.+/)(ob.bin.vxsvc .opt.+/)(ob.bin.vxsvcctrl

#aemons

.opt.+/)(ob.bin.vea (tart + 0 E&I Pou must have 6*%indo%s running

ogging @ogging help in recovery and can speed it up dramatically, the main form of logging in veritas is the #/@ (dirty region log) %hich performs the follo%ing

log keeps track of changed regions if system fails only the changed regions of the volume are recovered vxassist *g ,group- addlog ,vol- logtype!drl vxassist *g ,group- addlog ,vol* used for raid logs (no type) vxassist *g ,group- remove log ,vol- 4nlog!n5 ,vol-

0dd /emove Volume Read Policy

<olicies can be used if you have slo%er disks %ithin a volume and you %ish to use the faster disks. /ound /obin <reffered <lex (elected <lex !torage -xpert #vxse& +eritas have created some scripots that can check the integrity of the vxvm setup i.e mirrored volumes, spares, etc. )he scripts are based on rules and there are a number of differents rules veritas has set, look in the rules directory to see all of them. #isplay #escription :heck rules @ist spare rules /un spare run "ote: you need to run H.etc.init.d.isisd startH to start the necessary daemons first V/!- Pat%s /ules #efault <arameters .opt.+/)(.vxse.vxvm .etc.default.vxse vxseIraid7logG info vxseIraid7logG *g ,group- check vxseIspares list vxseIspares run vxvol *g ,group- rdpol round ,volvxvol *g ,group- rdpol prefer ,vol- ,plexvxvol *g ,group- rdpol select ,vol-

VxDMP see +x#M< for more information Veritas links /ecovery features http:..prefetch.net.articles.veritasrecoveryfeatures.html

Veritas (ile !ystem

+eritas Qilesystem ( +xQ() is an extent based, intent logging file system. +xQ( is geared to%ard &nix environments that reRuire high performance and availability and deal %ith large amounts of data. +xQ( features: *

xtent based allocation xtent attributes Qast filesystem recovery 0ccess control lists (0:@) 1nline administration 1nline backup nchanced I.1 and mount options Improved synchronous %rites (upport for large filesystems (up to A terabytes) (upport for large files (up to G terabytes) +eritas Ruicklog

Version 0 disk ayout )he +xQ( version 9 disk layout divides the entire file system space into fixed si=ed allocation units. )he first allocation unit starts at block =ero and all allocation units 8A' blocks. 0ll structural information are contained in files. (o expanding the filesystem structures simply reRuires extending the appropriate structural files. 0ll version 9 structural files reside in the structural fileset. )he structural files in version 9 disk layout are:

Object location Table file

:ontains the obOect location table (1@)). )he 1@), %hich referenced from the superblock is used to locate the other structural files. ncapsulates the super*block and super*block replicas. 0lthough the primary location of the super*block is kno%n, the label file can be used to locate upper* block copies ifthere is structural damage to the filesystem.

Label File

Device File

/ecords device information such as volume length and volume label and contains pointers to other structural files. ?old information on a per* fileset basis. )his may include the inode of the filesetSs inode list file, the maximum number of inodes allo%ed, an indication of %hether the filesystem supports large files and the inode number of Ruotas file if the filesetsupports Ruotas. Then a filesystem is created, there are t%o filesets, the structural fileset, %hichdefines the file system structure and the primary fileset %hich contains user data.

Fileset Header File

Inode List File

$oth the structural fileset and the primary fileset have their o%n inode lists %hich are stored in inode list files. Increasing the number of inodes involves increasing the si=e of the file after expanding the inode allocation unit file. ?old the free inode map, extended operations map and a summary of the inode resources. Maps the block used by the filesystem intent log. Indicates the allocation state of each 0& by defining %hether each 0& is free allocated as a %hole (no bitmaps allocated) or expanded, in %hich case the bitmaps associated %ith each 0& determine %hich extents are allocated. :ontains the 0& summary for each allocation unit, %hich contains the number of free extents of each si=e. )he summary for an extent is created only %hen an allocation unit is expanded for use. :ontains the free extent maps for each of the allocation units. If the filesystem supports Ruotas, there is a Ruotas file, %hich is used to track the resources allocated to each user.

Inode Allocation Unit File Log File Extent Allocation Unit State File

Extent Allocation Unit Summar File Free Extent !a" File #uotas File

"reating a (ilesystem )he mkfs command creates veritas filesystems by %riting to a special character device. mkfs FQ vxfs Fo largefiles .dev.vx.rdsk.appdg.sambaIvol mkfs 4*Q vxfs5 4 genericIoption5 4*o specialIoption5 device 4si=e5 :reate genericIoption * 1ptions common to other filesystem types specialIoption * 1ptions specific to +xQ( filesystem device * disk device or +eritas volume si=e * the si=e in sectors

Mounting1 2nmounting a (ilesystem

Mount a +6Q( filesystem &mount a +6Q( filesystem Displaying a (ilesystem

mount FQ vxfs Fo largefiles .dev.vx.rdsk.appdg.sambaIvol .samba umount .samba

Information ?o% created

mount *v mount *p mkfs *Q vxfs *n " ,filesystemmkfs *Q vxfs *m ,character device-

Identifying (ilesystem types fstyp *v .dev.vx.dsk.appdg.sambaIvol Identify filesystem type fstyp *v .dev.vx.rdsk.appdg.sambaIvol (block device) (ra% device)

!%rink1gro3 a (ilesystem (hrink a filesystem Ero% a filesystem Reorganise a (ilesystem &se the fsadm ro reorganise (defrag) a filesystem: /eport on extent fragmentation: /eport on directory fragmentation: /eorganise extents: /eorganise directories: .usr.lib.fs.vxfs.fsadm * .samba .usr.lib.fs.vxfs.fsadm *# .samba .usr.lib.fs.vxfs.fsadm *e .samba .usr.lib.fs.vxfs.fsadm *d .samba .usr.lib.vxfs.fsadm *b GBA9BB .samba .usr.lib.vxfs.fsadm *b AB9DBB .samba

"reate and Mount a snaps%ot filesystem &se the mount command to create the snap shot filesystem then use vxdump (or tar, dd, cpio) to backup the filesystem to tape:

:reate the snapshot filesystem: $ackup the filesystem: /estore the filesystem (to .restore

mount *Q vxfs *o snapof!.dev.vx.dsk.appdg.sambaIvol snap,si=e!ABD9BB U .dev.vx.dsk.rootdg.snapshot.snapshot vxdump *cf .dev.rmt.Gcbn .snapshot vxrestore *vx .restore

filesystem): 2sing 4uotas )urn on a Ruota for a filesystem (make sure filesystem has been mounted %ith Ruota option): )o setup a users Ruota )o vie% a users Ruota: )o turn off Ruota on a filesystem: "ote: )he Ruota file is not removed. Vx(! consistancy Qull check fsck *Q vxfs *o full,nolog ,volumevxRuotaon ,mountIpoint"ote: a file %ill be created called Ruotas (it3s a text file) vxedRuota ,usernamevxRuota *v ,usernamevxRuotaoff ,mountIpoint-

)%o f.s in parallel fsck *Q vxfs *o p ,ra%*device-,ra%*device-

Intent og )he intent log is uded to speed up recovery :reate Mount fsadm *Q vxfs *o log!si=e,logdevice!,devicemount *Q vxfs *o log mount *Q vxfs *o delaylog (integrity) (performance)

Veritas (ile !ystem

+eritas Qilesystem ( +xQ() is an extent based, intent logging file system. +xQ( is geared to%ard &nix environments that reRuire high performance and availability and deal %ith large amounts of data. +xQ( features: *

xtent based allocation xtent attributes Qast filesystem recovery 0ccess control lists (0:@) 1nline administration 1nline backup nchanced I.1 and mount options Improved synchronous %rites

(upport for large filesystems (up to A terabytes) (upport for large files (up to G terabytes) +eritas Ruicklog

+ersion 9 disk @ayout

)he +xQ( version 9 disk layout divides the entire file system space into fixed si=ed allocation units. )he first allocation unit starts at block =ero and all allocation units 8A' blocks. 0ll structural information are contained in files. (o expanding the filesystem structures simply reRuires extending the appropriate structural files. 0ll version 9 structural files reside in the structural fileset. )he structural files in version 9 disk layout are:

Object location Table file

:ontains the obOect location table (1@)). )he 1@), %hich referenced from the superblock is used to locate the other structural files. ncapsulates the super*block and super*block replicas. 0lthough the primary location of the super*block is kno%n, the label file can be used to locate upper*block copies ifthere is structural damage to the filesystem. /ecords device information such as volume length and volume label and contains pointers to other structural files. ?old information on a per* fileset basis. )his may include the inode of the filesetSs inode list file, the maximum number of inodes allo%ed, an indication of %hether the filesystem supports large files and the inode number of Ruotas file if the filesetsupports Ruotas. Then a filesystem is created, there are t%o filesets, the structural fileset, %hichdefines the file system structure and the primary fileset %hich contains user data.

Label File

Device File

Fileset Header File

Inode List File

$oth the structural fileset and the primary fileset have their o%n inode lists %hich are stored in inode list files. Increasing the number of inodes involves increasing the si=e of the file after expanding the inode allocation unit file. ?old the free inode map, extended operations map and a summary of the inode resources. Maps the block used by the filesystem intent log. Indicates the allocation state of each 0& by defining %hether each 0& is free allocated as a %hole (no bitmaps allocated) or expanded, in %hich case the bitmaps associated %ith each 0& determine %hich extents are allocated. :ontains the 0& summary for each allocation unit, %hich contains the number of free extents of each si=e. )he summary for an extent is created only %hen an allocation unit is expanded for use.

Inode Allocation Unit File Log File Extent Allocation Unit State File

Extent Allocation Unit Summar File

Free Extent !a" File #uotas File

:ontains the free extent maps for each of the allocation units. If the filesystem supports Ruotas, there is a Ruotas file, %hich is used to track the resources allocated to each user.

"reating a (ilesystem
)he mkfs command creates veritas filesystems by %riting to a special character device. )o create a file system: mkfs 4*Q vxfs5 4 genericIoption5 4*o specialIoption5 device 4si=e5 genericIoption specialIoption device si=e example: mkfs FQ vxfs Fo largefiles .dev.vx.rdsk.appdg.sambaIvol output: version 9 layout AB9DBB sectors, GBA9BB blocks of si=e GBA9, log si=e GBA9 blocks unlimited inodes, largefiles supported GBA9BB data blocks, GBGADB free data blocks 9 allocation units of 8A;CD blocks, 8A;CD blocks last allocation unit has 9BVC data blocks 'ote: you do not get the super*block output list as in ne%fs 2 1ption largefiles set 1ptions common to other filesystem types 1ptions specific to +xQ( filesystem disk device or +eritas volume the si=e in sectors

Mounting1 2nmounting a (ilesystem

Mount a +6Q( filesystem &mount a +6Q( filesystem mount FQ vxfs Fo largefiles .dev.vx.rdsk.appdg.sambaIvol .samba umount .samba

Displaying a (ilesystem
)o see information on a filesystem: mount *v

.dev.dsk.cBtBdBsB on . type ufs read.%rite.setuid.largefiles on (un May G9 GV:9A:A7 ABBB .proc on .proc type proc read.%rite.setuid on (un May G9 GV:9A:A7 ABBB .fd on .dev.fd type fd read.%rite.setuid on (un May G9 GV:9A:A7 ABBB s%ap on .tmp type tmpfs read.%rite on (un May G9 GV:9A:A7 ABBB .dev.vx.dsk.appdg.sambaIvol on .samba type vxfs read.%rite.log.setuid.largefiles on (un May G9 GV:9A:A7 ABBB

Identifying (ilesystem types

&se the fstyp command to identify a filesystem type: fstyp *v .dev.vx.dsk.appdg.sambaIvol or .dev.vx.rdsk.appdg.sambaIvol vxfs magic a7BGfcf7 version 9 ctime (un May G9 GV:9A:A7 ABBB logstart B logend B bsi=e GBA9 si=e 7GABBB dsi=e 7GABBB ninode B nau B defiextsi=e B ilbsi=e B immedlen VC ndaddr GB aufirst B emap B imap B iextop B istart B bstart B femap B fimap B fiextop B fistart B fbstart B nindir AB9D aulen 8A;CD auimlen B auemlen D auilen B aupad B aublocks 8A;CD maxtier G7 inopb 9 inopau B ndiripau B iaddrlen D bshift GB inoshift A bmask fffffcBB boffmask 8ff checksum eGA8fa;; oltextG 88 oltextA GADA oltsi=e G checksumA CAb free A;9V9C ifree B efree 8B AD A8 A9 G7 G8 GC GA D D 7 8 C C 7 G

!%rink1gro3 a (ilesystem
&se the fsadm command to shrink.gro% a filesystem: (hrink a filesystem Ero% a filesystem .usr.lib.vxfs.fsadm *b GBA9BB .samba .usr.lib.vxfs.fsadm *b AB9DBB .samba

Reorganise a (ilesystem
&se the fsadm ro reorganise (defrag) a filesystem: /eport on extent fragmentation: /eport on directory fragmentation: /eorganise extents: /eorganise directories: .usr.lib.fs.vxfs.fsadm * .samba .usr.lib.fs.vxfs.fsadm *# .samba .usr.lib.fs.vxfs.fsadm *e .samba .usr.lib.fs.vxfs.fsadm *d .samba

"reate and Mount a snaps%ot filesystem

&se the mount command to create the snap shot filesystem then use vxdump (or tar, dd, cpio) to backup the filesystem to tape:

:reate the snapshot filesystem: $ackup the filesystem: /estore the filesystem (to .restore filesystem):

mount *Q vxfs *o snapof!.dev.vx.dsk.appdg.sambaIvol snap,si=e!ABD9BB U .dev.vx.dsk.rootdg.snapshot.snapshot vxdump *cf .dev.rmt.Gcbn .snapshot vxrestore *vx .restore

2sing 4uotas
)urn on a Ruota for a filesystem (make sure filesystem has been mounted %ith Ruota option): )o setup a users Ruota )o vie% a users Ruota: )o turn off Ruota on a filesystem: "ote: )he Ruota file is not removed. vxRuotaon ,mountIpoint"ote: a file %ill be created called Ruotas (it3s a text file) vxedRuota ,usernamevxRuota *v ,usernamevxRuotaoff ,mountIpoint-

/emove +eritas volume from root disk

In the example belo%, the server pegasus has t%o internal disks (cBtBdB and cBtGdB) under +eritas +olume Manager control. )he operating system is mirrored bet%een the t%o devices. G. Qrom the HokH prompt, boot from the cdrom device into single*user mode: A. <erform a filesystem check on the root filesystem: 8. &pdate the .a.etc.system file, removing references to the +x+M root device: 2 vi .a.etc.system )he system must not boot from the +eritas volumes. Pou can either delete the follo%ing t%o lines or change them from: rootdev:.pseudo.vxioJB:B set vxio:volIrootdevIisIvolume!G to: W rootdev:.pseudo.vxioJB:B W set vxio:volIrootdevIisIvolume!G 9. /estore the pre*+x+M version of the vfstab file. "ote that the vxinstall process makes a backup copy of the original vfstab file named vfstab.prevm. Te3ll make use of that backup no%: 7. If the install*db file exists, the +x+M daemons %ill not start on bootup: 2 touch .a.etc.vx.reconfig.d.state.d.install*db C. )he root*done file indicates that the root disk has been encapsulated. /emove it no%: 2 rm .a.etc.vx.reconfig.d.state.d.root*done ;. $oot from the root device

0t this point, the operating system is no longer under +eritas +olume Manager control. )he disk cBtGdB is unused, and the system %ould not survive the failure of cBtBdB if it %ere to die no%. "ote that even though the operating system is no longer under system control, in some cases you might %ish to continue running +eritas volume manager for non*os volumes. In this situation a fe% additional steps are reRuired: G. Manually start up volume manager processes: 2 rm .etc.vx.reconfig.d.state.d.install*db 2 vxiod set GB 2 vxconfigd *m disable 2 vxdctl enable A. :onfirm that all previous root volumes are accessible: 8. /emove the previous root volumes from +x+M: 2 vxedit *rf rm rootvol s%apvol var 9. /emove the root disk from +x+M: 2 vxdg rmdisk rootdisk 7. Qinally, restore the original vtoc to cBtBdB: 0t this point the root disk cBtBdB is completely free of +x+M, but any other +eritas volumes can be brought online and mounted if reRuired.

Replace a disk in .5666

G. A. 8. 9. 7. C. ;. D. run XvxdiskadmY select option 9 to Xremove a disk for replacementY run option GG to take disk offline run luxadm removeIdevice 4*f5 ,array-,,slot- (remember to do this both sides in a cluster) replace disk run luxadm insertIdevice ,array-,,slot(remember to do this both sides in a cluster) run vxdctl enable run XvxdiskadmY select option 7 to Xreplace a failed or removed diskY run Hvxtask listH to make sure volumes have been sync3d if mirrors are involved

Veritas Volume Manager Recovery Features Introduction

+eritas +olume Manager (+x+M) has become the standard @ogical +olume Manager (@+M) in many enterprises for its robust feature set, its ability to run on multiple operating systems (e.g., ?<*&6, @inux, (olaris, Tindo%s), and the numerous scalability, availability, and recoverability features that come %ith the base product. )he recoverability features help to ensure that data is protected %hen hard%are platforms fail and to ease the process reRuired to restore systems to an operational state.

)his article %ill provide an introduction to t%o important and often overlooked recovery features: failure notifications and configuration database backups. )he article %ill also provide t%o disaster*recovery case studies to sho% ho% these recovery features can be used to aide in recovering from disasters %hen they strike. 0 basic kno%ledge of +eritas +olume Manager %ill be assumed. If you are ne% to +eritas +olume Manager, consult the vxintro(Gm) man page for an introduction to terminology and basic usage.

Veritas Disk 7eaders

Then a device is initiali=ed or encapsulated %ith +eritas +olume Manager, a disk header is %ritten to the device3s private region. )he disk header contains a diskid to uniRuely identify the device, a disk group identifier to indicate %hich disk group the device is associated %ith, a set of flags to indicate the status and intended use (e.g., hot spare) of the device, and a hostid to indicate %hich host (if any) has the device imported. )he disk header can be printed by passing the +eritas device name to vxdisk(Gm)3s HlistH option: Z vxdisk list c8t8d6
Device: c1t1d0s2 devicetag: c1t1d0 type: auto hostid: pooh disk: name=c1t1d0 id=1123602295.10.pooh group: name=oradg id=1123603158.13.pooh in o: ormat=cdsdisk!privo set=256!pu"s#ice=2!privs#ice=2 #ags: on#ine ready private autocon ig autoimport imported pu"paths: "#ock=$dev$v%$dmp$c1t1d0s2 char=$dev$v%$rdmp$c1t1d0s2 version: 3.1 iosi&e: min=512 '"ytes( ma%=20)8 '"#ocks( pu"#ic: s#ice=2 o set=230) #en=35365968 disk*o set=0 private: s#ice=2 o set=256 #en=20)8 disk*o set=0 update: time=1123603160 se+no=0.6 ss": actua#*se+no=0.0 headers: 0 2)0 con igs: count=1 #en=1280 #ogs: count=1 #en=192 De ined regions: con ig priv 0000)8,000239-000192.: copy=01 o set=000000 ena"#ed con ig priv 000256,0013)3-001088.: copy=01 o set=000192 ena"#ed #og priv 0013)),001535-000192.: copy=01 o set=000000 ena"#ed #ockrgn priv 001536,0016/9-0001)).: part=00 o set=000000 0u#tipathing in ormation: numpaths: 1 c1t1d0s2 state=ena"#ed

#ue to the critical nature of the configuration information stored in the disk headers, it is important to periodically back up this information. 0s I %ill discuss in the section :onfiguration $ackups, +eritas introduced ne% features in +eritas +olume Manager 9.6 to ease the backup process.

Veritas "onfiguration Database

Then ne% obOects (e.g., subdisks, plexes, volumes) are created %ith the +eritas :@I or E&I, +eritas %ill %rite the configuration data to describe these obOects to a configuration database. )he configuration database is stored in the private region of several devices in each disk group for redundancy. )o list the devices in a disk group %ith a copy of the configuration database, the vxdg(Gm) utility can be invoked %ith the HlistH option and a disk group to Ruery: Z vxdg list oradg 9 egrep :config disk;<clean online:

con con con con con

ig ig ig ig ig

disk disk disk disk disk

c1t1d0s2 c1t2d0s2 c1t3d0s2 c1t)d0s2 c1t5d0s2

copy copy copy copy copy

1 1 1 1 1

#en=1280 #en=1280 #en=1280 #en=1280 #en=1280

state=c#ean state=c#ean state=c#ean state=c#ean state=c#ean

on#ine on#ine on#ine on#ine on#ine

)o print the contents of the configuration database, the vxprint(Gm) utility can be used. )he follo%ing example uses the vxprint(Gm) H*hH (list record hierarchies) and H*tH (use one*line format tailored for each record type) options to display the configuration database %ith an informational header and descriptive records: Z vxprint $%t
D1 ;> D0 :? :9 56 ?> ? =9 ;D ;? ;5 D5 ;= 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2562781 ;>3>4 D4?854 :982B*52> :?1 535C4?69 2?69<04 :?1$?;4>$56 ?69<04 =94D =94D =94D =3:42>?69 ;23=?69 de au#t c1t1d0s2 c1t2d0s2 c1t3d0s2 c1t)d0s2 c1t5d0s2 c1t6d0s2 2961 D0*52> >@=4 B;>3>4 B;>3>4 B;>3>4 B;>3>4 B;>3>4 B;>3>4 D8;B ?692304 535C4 961?69 D56 0826:; 1:6<=,8D ;=3:4*52> =:8?942 =<A942 ;>3>4 =:803:@ ;>3>4 :40*C6;> ;>3>4 ;>3>4 ;>3>4 9421>C ;>3>4 9421>C D8;B677; 9421>C 2?6993@: 9421>C D8;B677; 9421>C

3==?69*52> ;>3>4 D3>3?69; ;:9 :40*D1 :40*:92B :43D=69 =:47=94D <>@=4 93@6<> 2569$E8D 06D4 -569$.677 D4?854 06D4 -569$.677 30$20 06D4 -569$.677 D4?854 06D4

dg oradg dm dm dm dm dm dm v p# sv sv sv v p# sd p# sd v p# sd p# sd v p# sd p# sd c1t1d0 c1t2d0 c1t3d0 c1t)d0 c1t5d0 c1t6d0

de au#t auto auto auto auto auto auto

)6000 20)8 20)8 20)8 20)8 20)8 20)8

1123603158.13.pooh 35365968 35521)08 35521)08 35521)08 35365968 35521)08 , , , , , , oravo#01,03 3$128 :E 2$2 423 2$2 423 2$2 423 sgen

oravo#01 oravo#01,03 oravo#01,;01 oravo#01,;02 oravo#01,;03 oravo#01,901 oravo#01,=01 c1t1d0,02 oravo#01,=02 c1t)d0,02 oravo#01,902 oravo#01,=03 c1t2d0,02 oravo#01,=0) c1t5d0,02 oravo#01,903 oravo#01,=05 c1t3d0,02 oravo#01,=06 c1t6d0,02

, 423A94D 35>8?4 )19)30)0 ;4945> oravo#01 423A94D 35>8?4 )19)3168 ;>:8=4 oravo#01,03 oravo#01,901 1 13981056 0$0 oravo#01,03 oravo#01,902 1 13981056 1$0 oravo#01,03 oravo#01,903 1 13981056 2$0

, 423A94D 35>8?4 13981056 ;4945> , sgen oravo#01,901 423A94D 35>8?4 13981056 56253> , :E oravo#01,=01 c1t1d0 0 13981056 0 c1t1d0 423 oravo#01,901 423A94D 35>8?4 13981056 56253> , :E oravo#01,=02 c1t)d0 0 13981056 0 c1t)d0 423 , oravo#01,902 oravo#01,=03 oravo#01,902 oravo#01,=0) , oravo#01,903 oravo#01,=05 oravo#01,903 oravo#01,=06 423A94D 423A94D c1t2d0 423A94D c1t5d0 423A94D 423A94D c1t3d0 423A94D c1t6d0 35>8?4 35>8?4 0 35>8?4 0 35>8?4 35>8?4 0 35>8?4 0 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 ;4945> , sgen 56253> , :E 0 c1t2d0 423 56253> , :E 0 c1t5d0 423 ;4945> , sgen 56253> , :E 0 c1t3d0 423 56253> , :E 0 c1t6d0 423

)he obOect types (e.g., subdisks, disk groups, plexes, volumes, etc.), block offsets, obOect names, and locations can be useful for understanding the +eritas +olume Manager layout and reassembling the configuration layout during a disaster. #ue to the importance of this information, backing up the configuration database is imperativeK 0s you %ill see in the next section, +eritas +olume Manager makes backing up the configuration information a snap.

"onfiguration )ackups
)he vxconfigbackupd(Gm) daemon %as introduced in +eritas +olume Manager 9.6 to provide an automated %ay to back up disk headers and the configuration database. +xconfigbackupd(Gm) is started automatically at system boot time and actively monitors the configuration database and disk headers. Then vxconfigbackupd(Gm) detects a change to the active configuration, vxconfigbackupd(Gm) %ill %rite the configuration to a set of files in the .etc.vx.cbr.bk.. directory. (HH identifies the disk group, and HH identifies the disk group id.) )he vxconfigbackupd(Gm) man page provides the follo%ing descriptions for the files in the .etc.vx.cbr.bk.. directory:
$etc$v%$c"r$"k$.$.diskin o 9ocation o "ackup i#e $etc$v%$c"r$"k$.$."incon ig 9ocation o "ackup i#e $etc$v%$c"r$"k$.$.c grec 9ocation o "ackup i#e in v%print ,m ormat. or disk attri"utes. or "inary con iguration copy. records

or con iguration

In addition to the automated backups that are performed by vxconfigbackupd(Gm), manual backups can be performed %ith the vxconfigbackup(Gm) utility. )his is valuable for backing up specific versions of the configuration and archiving the configuration to a user*supplied destination. )he follo%ing example uses the vxconfigbackup(Gm) H*lH (location to store backup) to back up the current configuration to the .etc.dgbackups directory: Z 1usr1lib1vxvm1bin1vxconfigbackup $l 1etc1dgbackups
;tart "acking up diskgroup oradg to F $etc$dg"ackups$oradg.112/2)0283.19.Ginnie ... ?%?0 26>854 ?,5,2,3100 Aackup comp#ete or diskgroup: oradg

If you are using a version of +eritas +olume Manager prior to 9.6, you %ill be unable to use the vxconfigbackupd(Gm) and vxconfigbackup(Gm) utilities to manage backups. #on3t fear ** the sample shell script in @isting G can be used to archive the configuration to a text file, %hich can be passed to vxmake(Gm)3s H*dH option to re*create the configuration if disaster %ere to strike.

Restoring "onfiguration Data

Then configuration information is lost due to a system outage or a mistake at the keyboard, the configuration can often be recovered to a point in time (preferably to a point in time %hen a kno%n %orking backup %as taken %ith vxconfigbackup(Gm)) %ith the vxconfigrestore(Gm) utility. )he follo%ing example sho%s ho% to restore the contents on the oradg disk group using the configuration backup in the .etc.dgbackups directory: Z 1usr1lib1vxvm1bin1vxconfigrestore $l 1etc1dgbackups oradg

Diskgroup oradg con iguration restoration started ...... 8nsta##ing vo#ume manager disk header or c1t1d0s2 ... 8nsta##ing vo#ume manager disk header or c1t2d0s2 ... 8nsta##ing vo#ume manager disk header or c1t3d0s2 ... 8nsta##ing vo#ume manager disk header or c1t)d0s2 ... 8nsta##ing vo#ume manager disk header or c1t5d0s2 ... 8nsta##ing vo#ume manager disk header or c1t6d0s2 ... , oradgHs diskgroup con iguration is restored 'in a precommitted state(. Diskgroup can "e accessed in read on#y and can "e e%amined using v%print'1m( in this state. :un: v%con igrestore ,# $etc$dg"ackups$ ,c oradg ==I to commit the restoration. v%con igrestore ,# $etc$dg"ackups$ ,d oradg ==I to a"ort the restoration.

Then vxconfigrestore(Gm) is invoked, the configuration %ill be staged to a transient location, allo%ing the administrator to validate the configuration %ith the vxprint(Gm) utility. If the configuration looks sound, the configuration can be committed to the disk headers and the configuration database by invoking vxconfigrestore(Gm) %ith the H*cH (commit) option: Z 1usr1lib1vxvm1bin1vxconfigrestore $l 1etc1dgbackups1 $c oradg
5ommitting con iguration restoration or diskgroup oradg .... oradgHs diskgroup con iguration restoration is committed.

1nce the configuration is restored, the volumes can be started, and the file systems that reside on those volume can be mounted.

+etting 'otified =%en (ailures >ccur

Then +eritas volume manager is installed %ith the default options, the vxrelocd(Gm) failure event detection and subdisk relocation daemon is started at boot time. )his daemon is responsible for detecting device failures, relocating subdisks on failed devices to alternate devices, and notifying personnel that a device or +eritas obOect failed. )he notifications are sent to the root user by default and look similar to the follo%ing:
>o: rootJsystem ;u"Kect: ?o#ume 0anager 5ontent,9ength: 2)0 ai#ures on host Ginnie

7ai#ures have "een detected "y the ?4:8>3; ?o#ume 0anager: ai#ed disks: c1t)d0 ai#ed p#e%es: raid5vo#,=08

(ince email addressed to the root user is directed to the local mail spool by default, it is advantageous to for%ard all messages addressed to root to an actual email address. )his can be accomplished by enabling email for%arding and adding the follo%ing alias to .etc.aliases: root: adminJmycompany.com

If you %ould prefer to use a separate account for notifications, you can append the account name to the line that starts vxrelocd(Gm) in the vxvm*recover init script: Z grep vxrelocd 1etc1init;d1!?5vxvm$recover vxrelocd root storageproblems & :onfiguring the notification facility to route messages to a valid account is crucial and ensures timely notifications %hen failures are detected.

"ase !tudy @8A Recovering from a Po3er (ailure

Thile sitting at my desk on a Qriday afternoon (around Cpm), my email client chimed to alert me to the follo%ing message:
>o: rootJsystem ;u"Kect: ?o#ume 0anager 5ontent,9ength: 2)0 ai#ures on host Ginnie

7ai#ures have "een detected "y the ?4:8>3; ?o#ume 0anager: ai#ed vo#umes: oravo#01

I immediately logged into %innie and received the follo%ing error %hen trying to list the directory that %as on the failed volume oravolBG: Z ls $la 1u68 .: I.1 error 0 Ruick check of the system logs revealed numerous (:(I error messages: Z tail 1var1adm1messages
Lu# 26 $pciJ1 Lu# 26 Lu# 26 $pciJ1 Lu# 26 Lu# 26 $pciJ1 Lu# 26 Lu# 26 $pciJ1 Lu# 26 Lu# 26 $pciJ1 Lu# 26 Lu# 26 $pciJ1 Lu# 26 1/:)9:3/ Ginnie scsi: -8D 10/833 kern.Garning. E3:2821: !0$pciJ1$scsiJ)$sdJ1!0 'sd1(: 1/:)9:3/ Ginnie ;5;8 transport ai#ed: reason Hincomp#eteH: retrying command 1/:)9:3/ Ginnie scsi: -8D 10/833 kern.Garning. E3:2821: !0$pciJ1$scsiJ)$sdJ)!0 'sd5(: 1/:)9:3/ Ginnie ;5;8 transport ai#ed: reason Hincomp#eteH: retrying command 1/:)9:)0 Ginnie scsi: -8D 10/833 kern.Garning. E3:2821: !0$pciJ1$scsiJ)$sdJ1!0 'sd1(: 1/:)9:)0 Ginnie disk not responding to se#ection 1/:)9:)0 Ginnie scsi: -8D 10/833 kern.Garning. E3:2821: !0$pciJ1$scsiJ)$sdJ)!0 'sd5(: 1/:)9:)0 Ginnie disk not responding to se#ection 1/:)9:)2 Ginnie scsi: -8D 10/833 kern.Garning. E3:2821: !0$pciJ1$scsiJ)$sdJ1!0 'sd1(: 1/:)9:)2 Ginnie disk not responding to se#ection 1/:)9:)) Ginnie scsi: -8D 10/833 kern.Garning. E3:2821: !0$pciJ1$scsiJ)$sdJ1!0 'sd1(: 1/:)9:)) Ginnie disk not responding to se#ection

)he +eritas vxprint(Gm) utility %as also unable to display the disk group configuration (since the configuration database %as unavailable): Z vxprint $g oradg $%t Z I raced to the machine room to check the status of the hard%are and started to speculate that a (:(I cable or the storage array had failed. Then I checked the hard%are, I noticed that the storage array %as po%ered off, and the eRuipment in the surrounding racks %as %orking correctly. $ased on my findings, I thought that a po%er cord failed. 1nce I replaced the cord, the storage array came back to life, but the oradg disk group I needed to access %as in the disabled state: Z vxdg list
2304 oradg ;>3>4 disa"#ed 8D 1123603158.13.Ginnie

0 Ruick check of the disks sho%ed that they %ere online and associated %ith the disabled disk group: Z vxdisk list
D4?854 c0t0d0s2 c0t1d0s2 c1t1d0s2 c1t2d0s2 c1t3d0s2 c1t)d0s2 c1t5d0s2 c1t6d0s2 >@=4 auto:none auto:none auto:cdsdisk auto:cdsdisk auto:cdsdisk auto:cdsdisk auto:cdsdisk auto:cdsdisk D8;B , , c1t1d0 c1t2d0 c1t3d0 c1t)d0 c1t5d0 c1t6d0 1:6<= , , oradg oradg oradg oradg oradg oradg ;>3><; on#ine on#ine on#ine on#ine on#ine on#ine on#ine on#ine inva#id inva#id dgdisa"#ed dgdisa"#ed dgdisa"#ed dgdisa"#ed dgdisa"#ed dgdisa"#ed

In some situations, +eritas may report offline devices as Hfailed %as: c6t6d6sAH. Then this happens, the vxreattach(Gm) command can reconnect +eritas +olume Manager to HlostH devices. @uckily, in our case, +eritas %as able to reconnect to the devices so I unmounted the file system, deported the disk group, and imported the disk group to enable the oradg disk group: Z umount 1u68 Z vxdg deport oradg Z vxdg import oradg )he deport and import operations are reRuired to fix a disabled disk group and %ill validate that the disk group configuration records are consistent. 1nce the disk group %as imported, I ran vxinfo(Gm) to vie% the volume and plex status: Z vxinfo $g oradg $p
vo# p#e% vo# p#e% p#e% vo# p#e% p#e% vo# p#e% oravo#01 oravo#01,03 oravo#01,901 oravo#01,=01 oravo#01,=02 oravo#01,902 oravo#01,=03 oravo#01,=0) oravo#01,903 oravo#01,=05 sgen 35>8?4 sgen 35>8?4 35>8?4 sgen 35>8?4 35>8?4 sgen 35>8?4 ;tarta"#e ;tarta"#e ;tarta"#e ;tarta"#e

p#e% oravo#01,=06

35>8?4

)he H(tartableH flag indicates that the volume and plexes are in a startable state, so I executed the vxvol(Gm) utility to start the volume: Z vxvol $g oradg start oravol68 1nce the volume came online, I ran fsck(Gm) to replay the transactions in the +xQ( Oournal: Z fsck $( vxfs 1dev1vx1dsk1oof1oravol68
#og rep#ay in progress rep#ay comp#ete , marking super,"#ock as 59432

0fter fsck(Gm) finished the consistency check, I mounted the file system, applied the archive logs, and %as able to bring the database back up to an operational state. #ue to the recovery features built into +eritas volume manager, +eritas file system, and 1racle, %e %ere able to avoid a full file system restoreK (ince I received the failure notification immediately after +eritas detected a problem %ith the volume, I %as able to reduce the time it took to recover the faulted system.

"ase !tudy @BA Recovering a Deleted Volume

Thile debugging a : program on a Qriday afternoon (around Cpm), one of my colleagues came to my desk and told me he had accidentally destroyed the production disk group instead of the test disk group he thought he %as %orking on. 0 Ruick check of the +eritas configuration verified this: Z vxprint $%ft Z (ince %e used vxconfigbackup(Gm) to take regular backups of the disk group configuration, %e located the last kno%n good %orking backup and passed this location to vxconfigrestore(Gm)3s H*lH (location) option: Z 1usr1lib1vxvm1bin1vxconfigrestore $l 1etc1dgbackups1oradgC6D8BB65 oradg
Diskgroup oradg con iguration restoration started ...... 8nsta##ing 8nsta##ing 8nsta##ing 8nsta##ing 8nsta##ing 8nsta##ing vo#ume vo#ume vo#ume vo#ume vo#ume vo#ume manager manager manager manager manager manager disk disk disk disk disk disk header header header header header header or or or or or or c1t1d0s2 c1t2d0s2 c1t3d0s2 c1t)d0s2 c1t5d0s2 c1t6d0s2 ... ... ... ... ... ...

oradgHs diskgroup con iguration is restored 'in precommitted state(. Diskgroup can "e accessed in read on#y and can "e e%amined using v%print'1m( in this state. :un: v%con igrestore ,# $etc$dg"ackups$oradg*0612205 ,c oradg ==I to commit the restoration. v%con igrestore ,# $etc$dg"ackups$oradg*0612205 ,d oradg ==I to a"ort the restoration.

Te verified the configuration %as consistent by running vxprint(Gm), and committed the configuration to the disk headers and configuration database by invoking vxconfigrestore(Gm) %ith the H*cH option: Z 1usr1lib1vxvm1bin1vxconfigrestore $c $l 1var1tmp1back oradg
5ommitting con iguration restoration or diskgroup oradg ....

oradgHs diskgroup con iguration restoration is committed.

1nce %e committed the configuration, the layout matched our last kno%n good configuration backup: Z vxprint $%t
Disk group: oradg D1 ;> D0 :? :9 56 ?> ? =9 ;D ;? ;5 D5 ;= 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2304 2562781 ;>3>4 D4?854 :982B*52> :?1 535C4?69 2?69<04 :?1$?;4>$56 ?69<04 =94D =94D =94D =3:42>?69 ;23=?69 de au#t 2961 D0*52> >@=4 B;>3>4 B;>3>4 B;>3>4 B;>3>4 B;>3>4 B;>3>4 D8;B ?692304 535C4 961?69 D56 0826:; 1:6<=,8D ;=3:4*52> =:8?942 =<A942 ;>3>4 =:803:@ ;>3>4 :40*C6;> ;>3>4 ;>3>4 ;>3>4 9421>C ;>3>4 9421>C D8;B677; 9421>C 2?6993@: 9421>C D8;B677; 9421>C 3==?69*52> ;>3>4 D3>3?69; ;:9 :40*D1 :40*:92B :43D=69 93@6<> -569$.677 -569$.677 -569$.677 =:47=94D 2569$E8D D4?854 30$20 D4?854 <>@=4 06D4 06D4 06D4 06D4

dg oradg v p# sv sv sv v p# sd p# sd v p# sd p# sd v p# sd p# sd

de au#t

10000

112/2)0283.19.Ginnie oravo#01,03 sgen ;>:8=4 3$128 :E 0$0 2$2 423 1$0 2$2 423 2$0 2$2 423 ;4945> , 56253> , 0 c1t1d0 56253> , 0 c1t)d0 ;4945> , 56253> , 0 c1t2d0 56253> , 0 c1t5d0 ;4945> , 56253> , 0 c1t3d0 56253> , 0 c1t6d0 sgen :E 423 :E 423 sgen :E 423 :E 423 sgen :E 423 :E 423

oravo#01 , oravo#01,03 oravo#01,;01 oravo#01,;02 oravo#01,;03 oravo#01,901 oravo#01,=01 c1t1d0,02 oravo#01,=02 c1t)d0,02 oravo#01,902 oravo#01,=03 c1t2d0,02 oravo#01,=0) c1t5d0,02 oravo#01,903 oravo#01,=05 c1t3d0,02 oravo#01,=06 c1t6d0,02

423A94D 35>8?4 )19)30)0 ;4945> oravo#01 423A94D 35>8?4 )19)3168 oravo#01,03 oravo#01,901 1 13981056 oravo#01,03 oravo#01,902 1 13981056 oravo#01,03 oravo#01,903 1 13981056 , oravo#01,901 oravo#01,=01 oravo#01,901 oravo#01,=02 , oravo#01,902 oravo#01,=03 oravo#01,902 oravo#01,=0) , oravo#01,903 oravo#01,=05 oravo#01,903 oravo#01,=06 423A94D 423A94D c1t1d0 423A94D c1t)d0 423A94D 423A94D c1t2d0 423A94D c1t5d0 423A94D 423A94D c1t3d0 423A94D c1t6d0 ;@25 35>8?4 0 35>8?4 0 ;@25 35>8?4 0 35>8?4 0 ;@25 35>8?4 0 35>8?4 0 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056 13981056

)o ensure that no damage had occurred to the file system, %e used fsck(Gm) to consistency check the file system: Z fsck $( vxfs 1dev1vx1rdsk1oradg1oravol68 and got our 1racle #$0 to verify the integrity of each data file %ith 1racle3s database verification utilities. $ecause accidents and disasters can strike at any time, it is important to perform regular backups of the +eritas configuration and store this data in a safe location (having a configuration database backup saved us from having to restore the database or re*create the +eritas obOects by hand).

"onclusion
)his article provided an introduction to t%o important +eritas +olume Manager recovery techniRues and discussed %ays to ensure recoverability after a disaster. Then dealing %ith difficult +eritas situations, it is imperative to kno% %hat function a command performs before executing it. It is also helpful to engage +eritas support or the folks on the +eritas mailing lists %hen maOor disasters occur. )his %ill ensure that a second set of eyes is available to revie% the problem, and %ill ensure the Ruickest and safest methods are used to recover a faulted system. I tested all commands on a (olaris GB server running +eritas +olume Manager 9.G, and I highly recommend testing all configuration changes on non*production systems. If you have Ruestions or comments on the article, please feel free to *mail the author.

References

+eritas documentation: http:..%%%.veritas.com ((0 ?elpful ?ints: http:..%%%.eng.auburn.edu.pub.mail*lists.ssastuff. +eritas +olume Manager Mailing @ist 0rchives: http:..mailman.eng.auburn.edu.search.html

.ckno3ledgements
/yan %ould like to thank the individuals %ho maintain the ((0 helpful hints site, as %ell as the members of the +eritas +olume Manager list.

W 1riginally published in the Nanuary 3BC issue of (ys0dmin Maga=ine

Restore a lost volume group #Volume manager 0;6&

Obtain a configuration file (if it exists): # cd /etc/vx/cbr/bk/<diskgroup>.<diskgroupid>.<hostname> ou !ill see a file called "<diskgroupid>.<hostname>.cfgrec" this is the backed up configuration file for the disk group #ispla$ the configuration file in readable format: # cat <diskgroup>.<diskgroupid>.<hostname>.cfgrec % vxprint &# & &htr ('his should sho! $ou all the volumes under the group) 'o precommit the changes run the follo!ing: # /etc/vx/bin/vxconfigrestore &p <diskgroup> 'o commit the changes after the precommit: # /etc/vx/bin/vxconfigrestore &c <diskgroup> 'o abort the precommit: # /etc/vx/bin/vxconfigrestore &d <diskgroup> 'o confirm that disk group is ok: # vxprint &g <diskgroup>

+eritas +6#M< =%at is Dynamic Multipat%ing #DMP&E #M< is a method of providing t%o or more hard%are paths to a single drive. Qor example, the physical hard%are can have at least t%o paths, such as cGtGdB and cAtGdB, directing I.1 to the same drive. +x+M arbitrarily selects one of the t%o names and creates a single device entry, then transfers data across both paths to spread the I.1, #M< is enabled by default. +olume Manager detects multipath systems %ith the &niversal Torld%ide #evice Identifiers(TT# I#s) and manages multipath targets, such as disk arrays, %hich define policies for using more than one path. )enefits of DMP

7ig% availability * #M< provides greater reliability through a path failover mechanism. In the event of a loss of oneconnection to a disk, the system continues to access the critical data over the other sound connections until the failed pathis replaced. Improved performance * #M< provides greater I.1 throughput by balancing the I.1 load uniformly across multiple I.1 pathsto the disk device.

)he disk paths can be in a active.active (use both paths) or active.passive state (use one path). list controllers list enclosures #isplay <aths Tho controls the path :ontroller (enable.disable) (tatistics vxdmpadm listctlr all vxdmpadm listenclosure all vxdmpadm getsubpaths ctlr!fscsiA vxdmpadm getsubpaths dmpnodename!,enclosurevxdmpadm getdmpnode namename!c8tAdB vxdmpadm disable ctlr!,controllervxdmpadm enable ctlr!,controllervxdmpadm iostat start vxdmpadm iostat reset vxdmpadm iostat sho% all vxdmpadm setattr enclosure ,enclosure- iopolicy!,policyadaptive priority balanced (default) round*robin minimumR single*active vxdmpadm setattr path ,path*name- pathtype!,path*type<ath )ype active nomanual nopreffered primary secondary standby

I.1 balancing & load

#isplays the restore daemon

vxdmpadm stat restored

(tops the restore daemon vxdmpadm start restored (tops the restore daemon vxdmpadm stop restored