Documente Academic
Documente Profesional
Documente Cultură
an Introduction
By Bill Hassell
With acknowledgements to
David Totsch and
Chris Wong
1
LVM - Introduction
LVM Basics
HFS and VxFS filesystem
2
LVM - Disk Layout
LIF Header LVM Record
Lvol 3
1 4
2 5
3 6
5 Pvol 2
6 1
7 2
8 3
6 4
LVM Allocation
Mirroring Pvol 1
Lvol 3
1 4
2 5
3 6
5 Pvol 1
6 1
6
5
LVM Allocation
Striping Pvol 1
1
2
3
4
Lvol
5
6
1
Pvol 2
2
1
2
3
3
4
4
5
6
5
Pvol 3
6
1
2
3
4
5
6
6
LVM Allocation
"Chunking"
Pvol 1 Pvol 4
1 1
2 2
Lvol
3 3
4 4
1
5 5
6 6
2
Pvol 2 Pvol 5
3
1 1
2 2
4
3 3
4 4
5
5 5
6 6
6
Pvol 3 Pvol 6
1 1
2 2
3 3
4 4
5 5
6 6
7
LVM Allocation
fragmentation
before interim after
Pvol 1 Pvol 1 Pvol 1
lvol 1 lvol 1 lvol 1
lvol 1 lvol 1 lvol 1
8
LVM Performance
• Mirroring
– recovery policy
– cache
• Extent Size
– Is bigger really better?
• Number of Disks
9
LVM Convenience
• Mirrors
– Splitting
– Merging
• vgexport/vgimport
• pvmove (!)
10
LVM Don'ts
• Span disks w/o mirroring
• Leave /etc/lvmtab alone
• vgscan is a last resort
• backup w/o running vgcfgbackup
11
File System Types
• UFS a.k.a. HFS
• VXFS Veritas Journaled File System
• CDFS CD-ROM File System
• NFS Network File System
12
Boot Block
DATA
DATA
SuperBlock
Boot Block
Boot Block INODES
SuperBlock
SuperBlock
INODES
INODES
DATA
HFS Layout On Disk
DATA
DATA
13
Fragmentation
O 3k
X 9k
Z 18k
O O O X Z Z
Z Z Z Z Z Z Z Z
X X X X X X X X
Z Z Z Z Z Z Z Z 14
Inode
File Size
File Type • Direct - 96KB File
Mode
UID
• Single - 16.8MB
GID • Double - 34GB
Link Count
Time Stamps • Triple - 70TB
12 Direct
16
How HFS Works
SB
17
JFS (a.k.a. VxFS)
• Based on version from Veritas Software,
Inc.
• Journaling
• Quick recovery after failure
• Extent based allocation
• Mount options balance
performance/integrity
18
VXFS Layout
OLT
OLT
8Kb Super
Intent Log
Block
Extent
Map
Allocation Unit
AU
Data/Inode Extents
Header
Data/Inode Extents
19
File System Integrity
• Use proper shutdown procedures!
• What fsck(1M) does
– hfs
– vxfs
• bcheckrc
20
Mounting File Systems
• Methodology for adding a new file system
• MountPoint management
• Mounting for security
– nosuid (!)
– ro (?)
• What about the device files?
21
How JFS Allocation Works
22
JFS Extents
Extent Attributes
• Adjacent disk blocks treated as a unit
• Represented by starting block and count
• Objective: Instead of Many Small I/Os, Perform Fewer Large I/Os
File Growth
• Objective is to keep file contiguous on disk
• When file grows, JFS attempts to increase size of last extent
• Otherwise, double size of current extent and relocate data
• Otherwise, allocate a new extent
23
Extent Performance
Extent Read Ahead
• Fetch more current extent than immediately
needed
• Does NOT involve posting extra I/O
operations
24
File System
Consistency Check (fsck)
HFS JFS
25
Journaled File System
Base Features
JFS Features Beyond HFS
26
Creating JFS Manually
•lvcreate -L 100 -n lvol2 /dev/vg04
•mkdir /test
•newfs -F vxfs /dev/vg04/rlvol2
•mount /dev/vg04/lvol2 /test
27
Base JFS Mount Options
• Logging
– Full Logging
– Delayed Logging
– Temporary logging
• Cache
– closesync
• Others
– Block Clear
– No Data In Log
28
Resizing Base JFS
File Systems
• Extending
– lvextend(1m)
– unmount file system
– extendfs(1m)
• Reducing
– backup
– lvreduce(1m)
– newfs
– restore
29
Online JFS Features
On Mounted File Systems, you can:
• Defragment (reorganize)
– Remove unused spaces from directories
– Make small files contiguous
– Consolidate free blocks
• Resize file system while mounted
• Take a snapshot
• Modify extent attributes
30
Reporting Fragmentation
• fsadm -D directory
• fsadm -E extents
31
•
Defragmentation
-d directory
• -e extent
• -v verbose
• -p number of passes
• -s summarize each pass
• -t time limit
• -a how old is old
• -l how big is big
32
Defragmenation Guidelines
• As often as necessary
• Defragment directories first
– or both at the same time
33
Extending A File System
Extend the logical volume
•lvextend -L 440 /dev/vg00/lvol8
34
Reducing A File System
Backup the filesystem
•tar cvf /dev/rmt/0m /test
Perform/view directory & extent reorganization
•fsadm -F vxfs -d -D -e -E /test
Resize the JFS filesystem
fsadm -F vxfs -b <newsize> /test
Reduce the size of the logical volume
•lvreduce -L 1000 /dev/vg04/lvol2
36
Network File Systems
• Client
– biod
• Server
– /etc/exports
– nfsd
• Performance
– nfsstat
– number of daemons
37
NFS Misc.
• Mount options
– hard / soft
– ro
– never allow remote root access!
• nfs vs. ftp/rcp
• automount(1M)
38
Veritas Volume Manager
(VxVM)
– benefits of HP VERITAS Manager 3.1
– major features HP VERITAS Volume
Manager 3.1 (VxVM)
– Volume Manager Objects
– disks and disk groups
39
Volume Manager History
– Pre-10.0: Disk sections or partitions on 800’s
– Whole disk with data (+ optional swap) on 700’s
– HP-UX 9.0 = LVM on Series 800 only
– HP-UX 8.07 = Software Disk Striping (SDS) for
the Series 700 for 2 Gbyte disk or smaller
– HP-UX 10.0+, LVM added to 700
– HP-UX 11.11+, VxVM added to 800 only
40
Present and Future
• Present:
– HP-UX 11i, first release of Veritas VxVM 3.1.
• HP Workstations are not supported.
• VxVM 3.1 cannot control the root/boot disk.
– HP-UX 11.20 (IA-64) will support only VxVM .
• VxVM will then be able to control the root/boot disk.
• Future:
– Next major release - can choose either LVM or
VxVM to be the root/boot disk.
41
HP VERITAS Volume Manager 3.1
42
HP VERITAS Volume Manager
3.1
43
Base HP VERITAS Volume
Manager Overview
• Base product features:
– Java-based admin GUI
– Striping (RAID 0)
– Concatenation
– Path failover support (active/passive)
– Online resizing of volumes
– Task monitor
44
VxVM Overview
•VxVM includes all of the Base product features plus:
– Load-balancing -- DynamicMultiPathing (active/active)
– Hot-Relocation/unrelocation
– Mirroring (RAID-1)
– Supports up to 32 mirrors
– Mirrored Stripes (RAID 1+0)
– Striped Mirrors (RAID 0+1)
– RAID-5
– Online data migration
– Online relayout
45
Coexistence with LVM
– VxVM coexists with LVM.
– VxVM cannot be used to control root/boot
disk (use LVM)
– Both LVM and VxVM utilities are aware of
each other, and will not overwrite disks that
are being managed by the other
– Although VxVM is targeted toward new
installations, a conversion utility:
vxvmconvert is provided for converting LVM
volume groups to VxVM disk groups.
46
Limitations of VxVM
• There are limitations with the first release of
VxVM on HP-UX 11i:
– Cannot use VxVM to control the root/boot disk. LVM
must be the chosen volume manager for root/boot disk.
– Disk monitor integrated with EMS framework is not
available for disk being managed by VxVM.
– VxVM does not support HP Process Resource Manager
(PRM).
47
Supported Migration Scenarios
• VxVM 3.1 is the first HP Release of the Veritas
Volume Manager on HP-UX.
– No migration path from Veritas Volume Manager
3.0 on HP-UX 11.00 has been tested or is supported.
• Product was released by Veritas, and not HP
– VxVM 3.0 on HP-UX 11.00 users that want to
migrate to 11i VxVM 3.1: Backup their data.
• Upgrade to HP-UX 11i
• Install HP VERITAS Volume Manager 3.1.
48
HP or Veritas VxVM
– VxVM 3.0 – Contact Veritas for support
– HP/VeritasVxVM 3.1 on 11i+ supported by HP Response
Center.
– # swlist -l product | grep vxvm
HP Supported:
HPvxvm 3.1 HP VERITAS Volume Manager
Purchased from VERITAS:
VRTSvxvm 3.X VERITAS Volume Manager
– To verify that all files belong to HPvxvm are installed:
# swverify -v HPvxvm
49
What is VxVM?
• VxVM operates as a subsystem between
HP-UX and data management systems.
– manage physical disks as logical volumes
called volumes.
– Volumes can span more than one physical disk.
– provides enhanced recovery, data availability,
performance and storage configuration options.
50
VxVM Objects
•VxVM uses two objects to do storage management.
– Physical Objects
• Physical disks
• Partitions
– Virtual objects
• VM Disks
• Subdisks
• Plexes
• Volumes
• Disk Groups
51
Conceptual Comparison
VxVM Term LVM Term
•VxVM Disk •Physical Volume
•Subdisk •Physical Extent
•Volume •Logical Volume
•Disk Group •Volume Group
•Private Region •PVRA/BDRA/VGRA
•Free Space •Unused Physical Extent
•Plexes •Mirrors
•Dirty Region Logging •Mirror Write Cache
Volume
plex01 plex02
0 0
dg04-01
dg01-03 40
dg03-02
70
dg02-03
130 130
40m
dg04-01
60m dg02-03 30m
dg03-02
130m dg01-03
53
VM Disk
• When a physical disk is placed under VxVM, a
Volume Manager disk (VM Disk) is assigned to
the physical disk.
– A VM Disk has two regions.
• Private Region
– Area where VM internal configuration information is stored
• Public Region
– Area where storage space is allocated from
– Each VM Disk has a unique Disk Media Name.
• Default name that is assigned by VxVM is disk##.
• The disk will now be referred to by its Disk Media Name
instead of its physical address.
54
Example of a VM Disk
Disk under VxVM control
disk01
Private Region
c0t4d0
Public Region
55
Subdisks
– A subdisk is a set of contiguous disk blocks.
– Each subdisk represents a specific portion of a VM disk
which is mapped to a specific region of a physical disk.
– Allocated space from the VM Disk public region
– Default name of a subdisk is disk##-##
disk01-03 subdisk
56
Subdisks Example
VM Disk: disk01
Subdisks
disk01
disk01-01
disk01-01
disk01-02
disk01-02 Public Region
disk01-03 disk01-03
57
Plex
– Plex consists of one or more subdisks located
on one or more physical disks
– Plex also can be called a mirror (although it is
one copy of the data).
– Plexes have a variety of layouts:
• Concatenation
• Striping (RAID-0)
• RAID-5
58
Plex Example
VM Disk: disk01
Plex: vol01-01
disk01
disk01-01
disk01-01
Subdisks
disk01-02
disk01-02
59
Plex Layout - Concatenation
• Concatenation
– Maps data into a linear manner onto one or
more subdisks in a plex.
– Space is allocated from the first subdisk from
beginning to end. Then space is allocated from
the remaining subdisks in a similar fashion.
– Concatenation using subdisks that reside on
more than one VM disk is called spanning
60
Plex Layout - Striping
• Striping (RAID-0)
– Maps data so that data is interleaved among two
or more physical disks.
– Striped plex contains two or more subdisk
spread out over two or more physical disks.
– Data is allocated alternately and evenly to the
subdisks of a striped plex.
61
Plex Layout - RAID-5
• RAID-5
– Same as Striping but one additional column of
data is used for parity.
– Provides data redundancy by using Parity.
– Requires additional license.
62
Plex Example
disk01
Plex01
disk01-01
disk01-02 0
30m
disk01-03 disk01-03
disk01-04 30m
30 m
disk02-02
disk02
70m
100m
disk02-01
70m
disk02-02
disk2-03
disk2-04 63
Volume
– Volume is a virtual disk device, but does not have
the physical limitations of a physical disk device.
– Volume consists of one or more plexes, each
holding a copy of the selected data in the volume.
– Volume can consist of 32 plexes, each of which
contains one or more subdisks.
• Never use more than 31 plexes, VxVM uses one plex
for online (automatic or temporary) operations.
– When a volume has two or more plexes , it is a
mirrored volume.
64
Volume Example
Volume vol01
plex vol01-01
disk01-01 subdisk
65
Mirrored Volume Example
disk02-01
disk01-01
disk04-03
disk03-02
66
Disk Group
• A disk group is a collection of VM disks that share
a common configuration
– A disk group configuration is a set of records with
detailed information about related VxVM objects, their
attributes and their connections.
– A disk group and its components can be moved as a
unit from one host machine to another.
• Note: The rootdg (root disk group) cannot be removed
without disrupting volume manager service to the entire
system.
67
Disk Group Example
Disk Group
Volume
plex01 plex02
0 0
dg04-01
dg01-03 40
dg03-02 70
dg02-03
130 130
40m
dg04-01
60m dg02-03 30m
dg03-02
130m dg01-03
68
VxVM Installation Process
Overview
• Prior to installation:
– Check software & hardware requirements.
– Check disk space required to install.
– Prepare for installing on a system with LVM
Volume Groups
• Install VxVM:
– Non-interactively with swinstall or
– Interactively with swinstall
69
Software Requirements
• Software:
– Requires HP-UX 11i
– VxVM is on the HP-UX 11i Application
Release.
• It is not on the HP-UX 11i Core Media.
70
Hardware Requirements
• Supported Platforms:
– N-class
– V-class
– L-class
– D-class 32/64 bit
– R-class
– K-class 32/64 bit
71
Hardware Requirements (cont)
• Supported Peripherals:
– HP SureStore E Disk System FC10
– HP SureStore E Disk System SC10
– HP SureStore E Disk Array XP256
– HP SureStore Disk Array FC60
– EMC Symmetrix
– High Availability Storage System (HASS)
A3311A/12A
– HP SureStore E Disk Array 12H
– HP High Availability Fibre Channel Disk Array
Model 30/FC
– HP High Availability SCSI Disk Array Model 2072
Disk Space and Memory
Required
Additional disk space required:
File System Kbytes Needed
/ 21000
/stand 2600
/usr 11000
/var 800
/opt 25000
Memory:
Minimum: 128 MB, recommended: 256 MB
•
73
Preparing to install on LVM system
• Prior to installing VxVM on an LVM system
– Review the contents of all the disks on the
system.
– Select at least one disk to be placed under
VxVM control.
– Decide whether to put any existing LVM disks
under VxVM control.
– Decommission any unused disks under LVM.
74
VxVM Product
• VxVM consists of two products:
– B7961AA - Base HP VERITAS Volume
Manager
• Free of charge with the HP-UX 11i Application
Release.
– B9116AA - HP VERITAS Volume Manager
• The full product requires an additional purchase to
obtain the codeword required to unlock the software.
75
Installing VxVM
77
First VxVM Disk Group: rootdg
• VxVM cannot be used until rootdg is set up
and at least one disk is assigned.
– One disk must remain under rootdg at all times
while VxVM is running.
– Despite its name, rootdg will not be the system
root disk, as VxVM cannot be used to control
the root/boot disk.
– LVM will control the root/boot disk.
78
Verify VxVM has initialized
79
VxVM Utilities
• VxVM provides the following user
interfaces that can be used to manage disks:
– VMSA - graphical user interface
– vxdiskadm - interactive menu-based interface
– CLI - Command-Line Interface
80
Setting up the Storage
Administrator GUI
• The Storage Administrator, VMSA, is the Java
based interface for VxVM.
– Installed as part of the Base HP VxVM.
– After initializing VxVM with vxinstall, you must set
up VMSA.
– VMSA consists of a server and a client:
• The server (vmsa_server) must be installed and run on a HP-
UX 11i machine with VxVM.
• The client (vmsa) is the graphical user interface. It must be
installed and run on an HP-UX 11i machine that supports the
Java 1.1 Runtime Environment.
81
Starting VMSA
– Log in as root.
– Create and export VMSAHOME variable #
export VMSAHOME=/opt/HPvmsa
– Update PATH variable to include $VMSAHOME/bin
82
Customizing Security
– By default, only root can run VMSA.
– The system can be set up to allow other users to
run VMSA by:
• Adding a group name vrtsadm to /etc/group
• Specifying users, including root, as vrtsadm
members.
– VMSA can also be configured in read-only
mode.
• This mode is enabled via the properties file.
/opt/HPvmsa/vmsa/properties
83
VMSA commands
– Query the run state of the server:
• # vmsa_server -q
Grid
Object tree
Command
launcher
85
Useful CLI commands
86
Removing VxVM
• Before removing VxVM software using
swremove, you must:
– Do a full backup of your data.
– Move data out of VxVM control.
– Stop VxVM.
87
Summary
• VxVM is an LVM-like tool
• Much more powerful and complex
• Much more versatility
– Striping
– Mirroring (up to 31, not just 3)
– Stripe and mirror
– RAID 5 data protection
– And more
88