Documente Academic
Documente Profesional
Documente Cultură
DM Multipath
Legal Notice
Co pyright 20 15 Red Hat, Inc. and o thers.
This do cument is licensed by Red Hat under the Creative Co mmo ns Attributio n-ShareAlike 3.0
Unpo rted License. If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro vide
attributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all Red
Hat trademarks must be remo ved.
Red Hat, as the licenso r o f this do cument, waives the right to enfo rce, and agrees no t to assert,
Sectio n 4 d o f CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shado wman lo go , JBo ss, MetaMatrix, Fedo ra, the Infinity
Lo go , and RHCE are trademarks o f Red Hat, Inc., registered in the United States and o ther
co untries.
Linux is the registered trademark o f Linus To rvalds in the United States and o ther co untries.
Java is a registered trademark o f Oracle and/o r its affiliates.
XFS is a trademark o f Silico n Graphics Internatio nal Co rp. o r its subsidiaries in the United
States and/o r o ther co untries.
MySQL is a registered trademark o f MySQL AB in the United States, the Euro pean Unio n and
o ther co untries.
No de.js is an o fficial trademark o f Jo yent. Red Hat So ftware Co llectio ns is no t fo rmally
related to o r endo rsed by the o fficial Jo yent No de.js o pen so urce o r co mmercial pro ject.
The OpenStack Wo rd Mark and OpenStack Lo go are either registered trademarks/service
marks o r trademarks/service marks o f the OpenStack Fo undatio n, in the United States and o ther
co untries and are used with the OpenStack Fo undatio n's permissio n. We are no t affiliated with,
endo rsed o r spo nso red by the OpenStack Fo undatio n, o r the OpenStack co mmunity.
All o ther trademarks are the pro perty o f their respective o wners.
Abstract
This bo o k pro vides info rmatio n o n using the Device-Mapper Multipath feature o f Red Hat
Enterprise Linux 7.
T able of Contents
. .hapt
C
. . . .er
. .1. .. Device
. . . . . . Mapper
. . . . . . . Mult
. . . . ipat
. . . .hing
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . . .
1.1. New and Chang ed Features fo r Red Hat Enterp ris e Linux 7.1
2
1.2. O verview o f DM Multip ath
2
1.3. Sto rag e Array Sup p o rt
5
1.4. DM Multip ath Co mp o nents
5
1.5. DM Multip ath Setup O verview
6
. .hapt
C
. . . .er
. .2. .. Mult
. . . . ipat
. . . .h. Devices
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .
2 .1. Multip ath Devic e Id entifiers
7
2 .2. Co ns is tent Multip ath Devic e Names in a Clus ter
7
2 .3. Multip ath Devic e Attrib utes
8
2 .4. Multip ath Devic es in Lo g ic al Vo lumes
8
. .hapt
C
. . . .er
. .3.
. .Set
. . .t.ing
. . . Up
. . . DM
. . . Mult
. . . . ipat
. . . .h. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 0. . . . . . . . . .
3 .1. Setting Up DM Multip ath
3 .2. Ig no ring Lo c al Dis ks when G enerating Multip ath Devic es
3 .3. Co nfig uring Sto rag e Devic es
3 .4. Setting Up Multip athing in the initramfs File Sys tem
10
11
13
14
. .hapt
C
. . . .er
. .4. .. T. he
. . . DM
. . . Mult
. . . . ipat
. . . .h. .Configurat
. . . . . . . . . ion
. . . File
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. 5. . . . . . . . . .
4 .1. Co nfig uratio n File O verview
15
4 .2. Co nfig uratio n File Blac klis t
16
4 .3. Co nfig uratio n File Defaults
18
4 .4. Multip aths Devic e Co nfig uratio n Attrib utes
25
4 .5. Co nfig uratio n File Devic es
27
. .hapt
C
. . . .er
. .5.
. .DM
. . . Mult
. . . . ipat
. . . .h. Administ
. . . . . . . . rat
. . .ion
. . . and
. . . .T. roubleshoot
. . . . . . . . . . . ing
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
...........
5 .1. Res iz ing an O nline Multip ath Devic e
33
5 .2. Mo ving ro o t File Sys tems fro m a Sing le Path Devic e to a Multip ath Devic e
33
5 .3. Mo ving s wap File Sys tems fro m a Sing le Path Devic e to a Multip ath Devic e
35
5 .4. The Multip ath Daemo n
35
5 .5. Is s ues with Larg e Numb er o f LUNs
36
5 .6 . Is s ues with q ueue_if_no _p ath feature
36
5 .7. Multip ath Co mmand O utp ut
36
5 .8 . Multip ath Q ueries with multip ath Co mmand
37
5 .9 . Multip ath Co mmand O p tio ns
38
5 .10 . Determining Devic e Map p er Entries with the d ms etup Co mmand
38
5 .11. Tro ub les ho o ting with the multip athd Interac tive Co ns o le
39
. . . . . . . . .Hist
Revision
. . . ory
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. 0. . . . . . . . . .
I.ndex
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. 1. . . . . . . . . .
1.1. New and Changed Feat ures for Red Hat Ent erprise Linux 7.1
Red Hat Enterprise Linux 7.1 includes the following documentation and feature updates and
changes.
Table 5.1, Useful mul ti path Command Options . now includes entries for the -w and -W
options of the mul ti path command, which allow you to better manage the wwids file.
Additional options for the val ues argument of the features parameter in the mul ti path. co nf
file are documented in Chapter 4, The DM Multipath Configuration File.
Table 4.1, Multipath Configuration D efaults . includes an entry for the fo rce_sync parameter,
which prevents path checkers from running in aync mode when set to " yes" .
In addition, small technical corrections and clarifications have been made throughout the document.
D escrip t io n
Reroutes I/O and supports failover for paths and path groups.
Configures and enables device mapper multipathing.
C o mp o n en t
D escrip t io n
kpartx command
Note
If you attempt to create an LVM physical volume on a whole device on which you have
configured partitions, the pvcreate command will fail. Note that the Anaconda and Kickstart
installation programs create empty partition tables if you do not specify otherwise for every
block device. If you wish to use the whole device rather than a partition, you must remove the
existing partitions from the device. You can remove existing partitions with the kpartx -d and
the fd i sk commands. If your system has block devices that are greater that 2Tb, you can use
the parted command to remove partitions.
When you create an LVM logical volume that uses active/passive multipath arrays as the underlying
physical devices, you should include filters in the /etc/l vm/l vm. co nf file to exclude the disks
that underlie the multipath devices. This is because if the array automatically changes the active path
to the passive path when it receives I/O, multipath will failover and failback whenever LVM scans the
passive path if these devices are not filtered. For active/passive arrays that require a command to
make the passive path active, LVM prints a warning message when this occurs.
To filter all SCSI devices in the LVM configuration file (l vm. co nf), include the following filter in the
d evi ces section of the file.
filter = [ "r/block/", "r/disk/", "r/sd.*/", "a/.*/" ]
10
Note
If you find that you need to edit the multipath configuration file after you have started the
multipath daemon, you must execute the servi ce mul ti pathd rel o ad command for the
changes to take effect.
3.2. Ignoring Local Disks when Generat ing Mult ipat h Devices
Some machines have local SCSI cards for their internal disks. D M Multipath is not recommended for
these devices. If you set the fi nd _mul ti paths configuration parameter to yes, you should not
have to blacklist these devices. For information on the fi nd _mul ti paths configuration parameter,
see Section 4.3, Configuration File D efaults .
If you do not set the fi nd _mul ti paths configuration parameter to yes, can use the following
procedure to modify the multipath configuration file to ignore the local disks when configuring
multipath.
1. D etermine which disks are the internal disks and mark them as the ones to blacklist.
11
In this example, /d ev/sd a is the internal disk. Note that as originally configured in the
default multipath configuration file, executing the mul ti path -v2 command shows the local
disk, /d ev/sd a, in the multipath map.
For further information on the mul ti path command output, see Section 5.7, Multipath
Command Output .
# mul ti path -v2
create: SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1 undef
WINSYS,SF2372
size=33 GB features="0" hwhandler="0" wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 0:0:0:0 sda 8:0 [--------device-mapper ioctl cmd 9 failed: Invalid argument
device-mapper ioctl cmd 14 failed: No such device or address
create: 3600a0b80001327d80000006d43621677 undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:0 sdb 8:16 undef ready running
`- 3:0:0:0 sdf 8:80 undef ready running
create: 3600a0b80001327510000009a436215ec undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:1 sdc 8:32 undef ready running
`- 3:0:0:1 sdg 8:96 undef ready running
create: 3600a0b80001327d800000070436216b3 undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:2 sdd 8:48 undef ready running
`- 3:0:0:2 sdg 8:112 undef ready running
create: 3600a0b80001327510000009b4362163e undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:3 sdd 8:64 undef ready running
`- 3:0:0:3 sdg 8:128 undef ready running
2. In order to prevent the device mapper from mapping /d ev/sd a in its multipath maps, edit the
blacklist section of the /etc/mul ti path. co nf file to include this device. Although you
could blacklist the sd a device using a d evno d e type, that would not be a safe procedure
since /d ev/sd a is not guaranteed to be the same on reboot. To blacklist individual devices,
you can blacklist using the WWID of that device.
Note that in the output to the mul ti path -v2 command, the WWID of the /d ev/sd a device
is SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1. To blacklist this device, include
the following in the /etc/mul ti path. co nf file.
blacklist {
wwid SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1
}
12
3. After you have updated the /etc/mul ti path. co nf file, you must manually tell the
mul ti pathd daemon to reload the file. The following command reloads the updated
/etc/mul ti path. co nf file.
# servi ce mul ti pathd rel o ad
4. Run the following command to remove the multipath device:
# mul ti path -f SIBM-ESXSST 336 732LC ____F3ET 0 EP 0 Q 0 0 0 0 724 28BX1
5. To check whether the device removal worked, you can run the mul ti path -l l command to
display the current multipath configuration. For information on the mul ti path -l l
command, see Section 5.8, Multipath Queries with multipath Command .
To check that the blacklisted device was not added back, you can run the mul ti path
command, as in the following example. The mul ti path command defaults to a verbosity
level of v2 if you do not specify specify a -v option.
# mul ti path
create: 3600a0b80001327d80000006d43621677 undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:0 sdb 8:16 undef ready running
`- 3:0:0:0 sdf 8:80 undef ready running
create: 3600a0b80001327510000009a436215ec undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:1 sdc 8:32 undef ready running
`- 3:0:0:1 sdg 8:96 undef ready running
create: 3600a0b80001327d800000070436216b3 undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:2 sdd 8:48 undef ready running
`- 3:0:0:2 sdg 8:112 undef ready running
create: 3600a0b80001327510000009b4362163e undef WINSYS,SF2372
size=12G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 2:0:0:3 sdd 8:64 undef ready running
`- 3:0:0:3 sdg 8:128 undef ready running
13
the /etc/mul ti path. co nf file and insert the appropriate device information.
For example, to add information about the HP Open-V series the entry looks like this. This example
sets the device to queue for a minute (or 12 retries and 5 seconds per retry) after all paths have failed.
devices {
device {
vendor "HP"
product "OPEN-V"
no_path_retry 12
}
}
For more information on the d evi ces section of the configuration file, see Section 4.5,
Configuration File D evices .
3.4 . Set t ing Up Mult ipat hing in t he init ramfs File Syst em
You can set up multipathing in the i ni tramfs file system. After configuring multipath, you can
rebuild the i ni tramfs file system with the multipath configuration files by executing the d racut
command with the following options:
# d racut --fo rce --ad d mul ti path --i ncl ud e /etc/mul ti path /etc/mul ti path
If you run multipath from the i ni tramfs file system and you make any changes to the multipath
configuration files, you must rebuild the i ni tramfs file system for the changes to take effect.
14
Note
You can run set up multipathing in the i ni tramfs file system. If you run multipath from the
i ni tramfs file system and you make any changes to the multipath configuration files, you
must rebuild the i ni tramfs file system for the changes to take effect. For information on
rebuilding the i ni tramfs file system with multipath, refer to Section 3.4, Setting Up
Multipathing in the initramfs File System .
This chapter provides information on parsing and modifying the mul ti path. co nf file. It contains
sections on the following topics:
Configuration file overview
Configuration file blacklist
Configuration file defaults
Configuration file multipaths
Configuration file devices
In the multipath configuration file, you need to specify only the sections that you need for your
configuration, or that you wish to change from the default values. If there are sections of the file that
are not relevant to your environment or for which you do not need to override the default values, you
can leave them commented out, as they are in the initial file.
The configuration file allows regular expression description syntax.
Further information about the configuration file can be found on the mul ti path. co nf(5) man
page.
15
16
/3600d0230000000000e13955cc3757800/
/3600d02300069c9ce09d41c31f29d4c00/
/SWINSYS SF2372
0E13955CC3757802/
/3600d0230000000000e13955cc3757803/
With the fi nd _mul ti paths parameter set to yes, you need to blacklist only the devices with
multiple paths that you do not want to be multipathed. Because of this, it will generally not be
necessary to blacklist devices.
If you do need to blacklist devices, you can do so according to the following criteria:
By WWID , as described in Section 4.2.1, Blacklisting by WWID
By device name, as described in Section 4.2.2, Blacklisting By D evice Name
By device type, as described in Section 4.2.3, Blacklisting By D evice Type
By default, a variety of device types are blacklisted, even after you comment out the initial blacklist
section of the configuration file. For information, see Section 4.2.2, Blacklisting By D evice Name .
17
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^(td|ha)d[a-z]"
}
#DS4200 Product 10
}
device {
vendor "HP"
product "*"
}
}
18
defaults {
user_friendly_names yes
}
This overwrites the default value of the user_fri end l y_names parameter.
The configuration file includes a template of configuration defaults. This section is commented out,
as follows.
#defaults {
#
udev_dir
#
polling_interval
#
selector
#
path_grouping_policy
#
uid_attribute
#
prio
#
path_checker
#
rr_min_io
#
max_fds
#
rr_weight
#
failback
#
no_path_retry
#
user_friendly_names
#}
/dev
10
"round-robin 0"
multibus
ID_SERIAL
alua
readsector0
100
8192
priorities
immediate
fail
yes
To overwrite the default value for any of the configuration parameters, you can copy the relevant line
from this template into the d efaul ts section and uncomment it. For example, to overwrite the
path_g ro upi ng _po l i cy parameter so that it is mul ti bus rather than the default value of
fai l o ver, copy the appropriate line from the template to the initial d efaul ts section of the
configuration file, and uncomment it, as follows.
defaults {
user_friendly_names
path_grouping_policy
}
yes
multibus
Table 4.1, Multipath Configuration D efaults describes the attributes that are set in the d efaul ts
section of the mul ti path. co nf configuration file. These values are used by D M Multipath unless
they are overwritten by the attributes specified in the d evi ces and mul ti paths sections of the
mul ti path. co nf file.
T ab le 4 .1. Mu lt ip at h C o n f ig u rat io n D ef au lt s
At t rib u t e
po l l i ng _i nterval
ud ev_d i r
mul ti path_d i r
D escrip t io n
Specifies the interval between two path checks in seconds. For
properly functioning paths, the interval between checks will
gradually increase to (4 * po l l i ng _i nterval ). The default
value is 5.
The directory where ud ev device nodes are created. The default
value is /d ev.
The directory where the dynamic shared objects are stored. The
default value is system dependent, commonly /l i b/mul ti path.
19
At t rib u t e
fi nd _mul ti paths
D escrip t io n
D efines the mode for setting up multipath devices. If this
parameter is set to yes, then multipath will not try to create a
device for every non-blacklisted path. Instead multipath will
create a device only if one of three conditions are met:
- There are at least two non-blacklisted paths with the same
WWID .
- The user manually forces the creation of the device by
specifying a device with the mul ti path command.
- A path has the same WWID as a multipath device that was
previously created. Whenever a multipath device is created with
fi nd _mul ti paths set, multipath remembers the WWID of the
device so that it will automatically create the device again as
soon as it sees a path with that WWID . This allows you to have
multipath automatically choose the correct paths to make into
multipath devices, without having to edit the multipath blacklist.
For instructions on the procedure to follow if you have previously
created multipath devices when the fi nd _mul ti paths
parameter was not set, see Section 4.2, Configuration File
Blacklist .
The default value is yes.
verbo si ty
path_sel ecto r
20
At t rib u t e
path_g ro upi ng _po l i cy
D escrip t io n
Specifies the default path grouping policy to apply to unspecified
multipaths. Possible values include:
fai l o ver: 1 path per priority group.
mul ti bus: all valid paths in 1 priority group.
g ro up_by_seri al : 1 priority group per detected serial number.
g ro up_by_pri o : 1 priority group per path priority value.
Priorities are determined by callout programs specified as global,
per-controller, or per-multipath options.
g ro up_by_no d e_name: 1 priority group per target node name.
Target node names are fetched in
/sys/cl ass/fc_transpo rt/targ et*/no d e_name.
The default value is fai l o ver.
pri o
21
At t rib u t e
features
D escrip t io n
The default extra features of multipath devices, using the format:
" number_of_features_plus_arguments feature1 ..." .
Possible values for features include:
q ueue_i f_no _path, which is the same as setting
no _path_retry to q ueue. For information on issues that may
arise when using this feature, see Section 5.6, Issues with
queue_if_no_path feature .
retai n_attached _hw_hand l er: If this parameter is set to yes
and the SCSI layer has already attached a hardware handler to
the path device, multipath will not force the device to use the
hard ware_hand l er specified by the mul ti path. co nf file. If
the SCSI layer has not attached a hardware handler, multipath
will continue to use its configured hardware handler as usual.
The default value is no .
pg _i ni t_retri es n: Retry path group initialization up to n
times before failing where 1 <= n <= 50.
pg _i ni t_d el ay_msecs n: Wait n milliseconds between path
group initialization retries where 0 <= n <= 60000.
path_checker
22
At t rib u t e
fai l back
D escrip t io n
Manages path group failback.
A value of i mmed i ate specifies immediate failback to the highest
priority path group that contains active paths.
A value of manual specifies that there should not be immediate
failback but that failback can happen only with operator
intervention.
A value of fo l l o wo ver specifies that automatic failback should
be performed when the first path of a path group becomes active.
This keeps a node from automatically failing back when another
node requested the failover.
A numeric value greater than zero specifies deferred failback,
expressed in seconds.
The default value is manual .
rr_mi n_i o
rr_wei g ht
no _path_retry
23
At t rib u t e
fl ush_o n_l ast_d el
max_fd s
checker_ti meo ut
d ev_l o ss_tmo
D escrip t io n
If set to yes, the mul ti pathd daemon will disable queuing when
the last path to a device has been deleted. The default value is
no .
Sets the maximum number of open file descriptors that can be
opened by multipath and the mul ti pathd daemon. This is
equivalent to the ul i mi t -n command. As of the Red Hat
Enterprise Linux 6.3 release, the default value is max, which sets
this to the system limit from /pro c/sys/fs/nr_o pen. For earlier
releases, if this is not set the maximum number of open file
descriptors is taken from the calling process; it is usually 1024.
To be safe, this should be set to the maximum number of paths
plus 32, if that number is greater than 1024.
The timeout to use for prioritizers and path checkers that issue
SCSI commands with an explicit timeout, in seconds. The default
value is taken from sys/bl o ck/sd x/d evi ce/ti meo ut.
The number of seconds the SCSI layer will wait after a problem
has been detected on an FC remote port before failing I/O to
devices on that remote port. This value should be smaller than
the value of d ev_l o ss_tmo . Setting this to o ff will disable the
timeout. The default value is determined by the OS.
The number of seconds the SCSI layer will wait after a problem
has been detected on an FC remote port before removing it from
the system. Setting this to infinity will set this to 2147483647
seconds, or 68 years. The default value is determined by the OS.
Each device configuration in the d evi ces section of the
mul ti path. co nf file will either create its own device
configuration or it will modify one of the built-in device
configurations. If hwtabl e_stri ng _match is set to yes, then if
the vendor, product, and revision strings in a user's device
configuration exactly match those strings in a built-in device
configuration, the built-in configuration is modified by the options
in the user's configuration. Otherwise, the user's device
configuration is treated as a new configuration. If
hwtabl e_stri ng _match is set to no , a regular expression
match is used instead of a string match.
The hwtabl e_stri ng _match is set to no by default.
d etect_pri o
ui d _attri bute
24
If this parameter is set to yes and the SCSI layer has already
attached a hardware handler to the path device, multipath will not
force the device to use the hard ware_hand l er specified by the
mul ti path. co nf file. If the SCSI layer has not attached a
hardware handler, multipath will continue to use its configured
hardware handler as usual. The default value is no .
If this is set to yes, multipath will first check if the device supports
ALUA, and if so it will automatically assign the device the al ua
prioritizer. If the device doesn't support ALUA, it will determine the
prioritizer as it always does. The default value is no .
If this is set to yes, the mul ti pathd daemon will listen to path
device change events and if a device has become read/write it will
reload the multipath device.
Provides a unique path identifier. The default value is
ID _SER IAL.
At t rib u t e
fo rce_sync
D escrip t io n
(Red Hat Enterprise Linux Release 7.1 and later) If this is set to
" yes" , it prevents path checkers from running in async mode.
al i as
D escrip t io n
Specifies the WWID of the multipath device to which the
mul ti path attributes apply. This parameter is mandatory for this
section of the mul ti path. co nf file.
Specifies the symbolic name for the multipath device to which the
mul ti path attributes apply. If you are using
user_fri end l y_names, do not set this value to mpathn; this
may conflict with an automatically assigned user friendly name
and give you incorrect device node names.
Specifies the default path grouping policy to apply to unspecified
multipaths. Possible values include:
fai l o ver = 1 path per priority group
mul ti bus = all valid paths in 1 priority group
g ro up_by_seri al = 1 priority group per detected serial number
g ro up_by_pri o = 1 priority group per path priority value
g ro up_by_no d e_name = 1 priority group per target node name
path_sel ecto r
25
At t rib u t e
fai l back
D escrip t io n
Manages path group failback.
A value of i mmed i ate specifies immediate failback to the highest
priority path group that contains active paths.
A value of manual specifies that there should not be immediate
failback but that failback can happen only with operator
intervention.
A value of fo l l o wo ver specifies that automatic failback should
be performed when the first path of a path group becomes active.
This keeps a node from automatically failing back when another
node requested the failover.
A numeric value greater than zero specifies deferred failback,
expressed in seconds.
pri o
no _path_retry
A numeric value for this attribute specifies the number of times the
system should attempt to use a failed path before disabling
queuing.
A value of fai l indicates immediate failure, without queuing.
A value of q ueue indicates that queuing should not stop until the
path is fixed.
rr_mi n_i o
26
At t rib u t e
rr_mi n_i o _rq
rr_wei g ht
D escrip t io n
Specifies the number of I/O requests to route to a path before
switching to the next path in the current path group, using
request-based device-mapper-multipath. This setting should be
used on systems running current kernels. On systems running
kernels older than 2.6.31, use rr_mi n_i o . The default value is 1.
If set to pri o ri ti es, then instead of sending rr_mi n_i o
requests to a path before calling path_sel ecto r to choose the
next path, the number of requests to send is determined by
rr_mi n_i o times the path's priority, as determined by the pri o
function. If set to uni fo rm, all path weights are equal.
If set to yes, then multipath will disable queuing when the last
path to a device has been deleted.
If set to yes, specifies that the system should use the
/etc/mul ti path/bi nd i ng s file to assign a persistent and
unique alias to the multipath, in the form of mpathn. If set to no ,
specifies that the system should use use the WWID as the alias
for the multipath. In either case, what is specified here will be
overridden by any device-specific aliases you specify in the
mul ti paths section of the configuration file.
The following example shows multipath attributes specified in the configuration file for two specific
multipath devices. The first device has a WWID of 36 0 0 50 8b4 0 0 0 156 d 70 0 0 120 0 0 0 0 b0 0 0 0 and
a symbolic name of yel l o w.
The second multipath device in the example has a WWID of 1D EC _____321816 7584 74 and a
symbolic name of red . In this example, the rr_wei g ht attributes is set to pri o ri ti es.
multipaths {
multipath {
wwid
alias
path_grouping_policy
path_selector
failback
rr_weight
no_path_retry
}
multipath {
wwid
alias
rr_weight
}
}
3600508b4000156d70001200000b0000
yellow
multibus
"round-robin 0"
manual
priorities
5
1DEC_____321816758474
red
priorities
27
Many devices that support multipathing are included by default in a multipath configuration. For
information on the default configuration value, including supported devices, run either of the
following commands.
# mul ti pathd sho w co nfi g
# mul ti path -t
You probably will not need to modify the values for these devices, but if you do you can overwrite the
default values by including an entry in the configuration file for the device that overwrites those
values. You can copy the device configuration defaults for the device that the mul ti pathd sho w
co nfi g command displays and override the values that you want to change.
To add a device that is not configured automatically by default to this section of the configuration
file, you need to set the vend o r and pro d uct parameters. You can find these values by looking at
/sys/bl o ck/device_name/d evi ce/vend o r and
/sys/bl o ck/device_name/d evi ce/mo d el where device_name is the device to be multipathed,
as in the following example:
# cat /sys/bl o ck/sd a/d evi ce/vend o r
WINSYS
# cat /sys/bl o ck/sd a/d evi ce/mo d el
SF2372
The additional parameters to specify depend on your specific device. If the device is active/active,
you will usually not need to set additional parameters. You may want to set
path_g ro upi ng _po l i cy to mul ti bus. Other parameters you may need to set are
no _path_retry and rr_mi n_i o , as described in Table 4.3, D evice Attributes .
If the device is active/passive, but it automatically switches paths with I/O to the passive path, you
need to change the checker function to one that does not send I/O to the path to test if it is working
(otherwise, your device will keep failing over). This almost always means that you set the
path_checker to tur; this works for all SCSI devices that support the Test Unit Ready command,
which most do.
If the device needs a special command to switch paths, then configuring this device for multipath
requires a hardware handler kernel module. The current available hardware handler is emc. If this is
not sufficient for your device, you may not be able to configure the device for multipath.
T ab le 4 .3. D evice At t rib u t es
At t rib u t e
vend o r
pro d uct
revi si o n
pro d uct_bl ackl i st
28
D escrip t io n
Specifies the vendor name of the storage device to which the
d evi ce attributes apply, for example C O MP AQ .
Specifies the product name of the storage device to which the
d evi ce attributes apply, for example HSV110 (C )C O MP AQ .
Specifies the product revision identifier of the storage device.
Specifies a regular expression used to blacklist devices by
product.
At t rib u t e
hard ware_hand l er
D escrip t io n
Specifies a module that will be used to perform hardware specific
actions when switching path groups or handling I/O errors.
Possible values include:
1 emc: hardware handler for EMC storage arrays.
1 al ua: hardware handler for SCSI-3 ALUA arrays.
1 hp_sw: hardware handler for Compaq/HP controllers.
1 rd ac: hardware handler for the LSI/Engenio RD AC controllers.
path_sel ecto r
path_checker
29
At t rib u t e
features
D escrip t io n
The default extra features of multipath devices, using the format:
" number_of_features_plus_arguments feature1 ..." .
Possible values for features include:
q ueue_i f_no _path, which is the same as setting
no _path_retry to q ueue. For information on issues that may
arise when using this feature, see Section 5.6, Issues with
queue_if_no_path feature .
retai n_attached _hw_hand l er: If this parameter is set to yes
and the SCSI layer has already attached a hardware handler to
the path device, multipath will not force the device to use the
hard ware_hand l er specified by the mul ti path. co nf file. If
the SCSI layer has not attached a hardware handler, multipath
will continue to use its configured hardware handler as usual.
pg _i ni t_retri es n: Retry path group initialization up to n
times before failing where 1 <= n <= 50.
pg _i ni t_d el ay_msecs n: Wait n milliseconds between path
group initialization retries where 0 <= n <= 60000.
pri o
30
At t rib u t e
fai l back
D escrip t io n
Manages path group failback.
A value of i mmed i ate specifies immediate failback to the highest
priority path group that contains active paths.
A value of manual specifies that there should not be immediate
failback but that failback can happen only with operator
intervention.
A value of fo l l o wo ver specifies that automatic failback should
be performed when the first path of a path group becomes active.
This keeps a node from automatically failing back when another
node requested the failover.
A numeric value greater than zero specifies deferred failback,
expressed in seconds.
rr_wei g ht
no _path_retry
A numeric value for this attribute specifies the number of times the
system should attempt to use a failed path before disabling
queuing.
A value of fai l indicates immediate failure, without queuing.
A value of q ueue indicates that queuing should not stop until the
path is fixed.
rr_mi n_i o
d ev_l o ss_tmo
31
At t rib u t e
user_fri end l y_names
d etect_pri o
D escrip t io n
If set to yes, specifies that the system should use the
/etc/mul ti path/bi nd i ng s file to assign a persistent and
unique alias to the multipath, in the form of mpathn. If set to no ,
specifies that the system should use use the WWID as the alias
for the multipath. In either case, what is specified here will be
overridden by any device-specific aliases you specify in the
mul ti paths section of the configuration file. The default value is
no .
If this parameter is set to yes and the SCSI layer has already
attached a hardware handler to the path device, multipath will not
force the device to use the hard ware_hand l er specified by the
mul ti path. co nf file. If the SCSI layer has not attached a
hardware handler, multipath will continue to use its configured
hardware handler as usual. The default value is no .
If this is set to yes, multipath will first check if the device supports
ALUA, and if so it will automatically assign the device the al ua
prioritizer. If the device doesn't support ALUA, it will determine the
prioritizer as it always does.
The following example shows a d evi ce entry in the multipath configuration file.
# }
# device {
# vendor
"COMPAQ "
# product
"MSA1000
"
# path_grouping_policy multibus
# path_checker tur
# rr_weight priorities
# }
#}
32
5.2. Moving root File Syst ems from a Single Pat h Device t o a Mult ipat h
Device
33
If you have installed your system on a single-path device and later add another path to the root file
system, you will need to move your root file system to a multipathed device. This section documents
the procedure for moving from a single-path to a multipathed device.
After ensuring that you have installed the d evi ce-mapper-mul ti path package, perform the
following procedure:
1. Execute the following command to create the /etc/mul ti path. co nf configuration file,
load the multipath module, and set chkco nfi g for the mul ti pathd to o n:
# mpathco nf --enabl e
For further information on using the mpathco nf command to set up multipathing, see
Section 3.1, Setting Up D M Multipath .
2. If the fi nd _mul ti paths configuration parameter is not set to yes, edit the bl ackl i st and
bl ackl i st_excepti o ns sections of the /etc/mul ti path. co nf file, as described in
Section 4.2, Configuration File Blacklist .
3. In order for multipath to build a multipath device on top of the root device as soon as it is
discovered, run the following command. This command also ensures that
fi nd _mul ti paths will allow the device, even if it only has one path.
# mul ti path -a root_devname
For example, if the root device is /d ev/sd b, run the following command.
# mul ti path -a /d ev/sd b
wwid '3600d02300069c9ce09d41c4ac9c53200' added
4. To confirm that your configuration file is set up correctly, you can run the mul ti path
command and search the output for a line of the following format. This indicates that the
command failed to create the multipath device.
date
34
5.3. Moving swap File Syst ems from a Single Pat h Device t o a Mult ipat h
Device
By default, swap devices are set up as logical volumes. This does not require any special procedure
for configuring them as multipath devices as long as you set up multipathing on the physical
volumes that constitute the logical volume group. If your swap device is not an LVM volume, however,
and it is mounted by device name, you may need to edit the /etc/fstab file to switch to the
appropriate multipath device name.
1. D etermine the WWID number of the swap device by running the /sbi n/mul ti path
command with the -v3 option. The output from the command should show the swap device in
the paths list.
You should look in the command output for a line of the following format, showing the swap
device:
WWID
For example, if your swap file system is set up on sd a or one of its partitions, you would see a
line in the output such as the following:
===== paths list =====
...
1ATA
WDC WD800JD-75MSA3
1:0:0:0 sda 8:0
...
WD-WMAM9F
2. Set up an alias for the swap device in the /etc/mul ti path. co nf file:
multipaths {
multipath {
wwid WWID_of_swap_device
alias swapdev
}
}
3. Edit the /etc/fstab file and replace the old device path to the root device with the multipath
device.
For example, if you had the following entry in the /etc/fstab file:
/dev/sda2 swap
ext4
defaults
0 0
ext4
defaults
0 0
35
36
Note
When a multipath device is being created or modified, the path group status, the dm device
name, the write permissions, and the dm status are not known. Also, the features are not
always correct.
37
# mul ti path -l l
3600d0230000000000e13955cc3757801 dm-10 WINSYS,SF2372
size=269G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 19:0:0:1 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
`- 18:0:0:1 sdh 8:112 active ready running
3600d0230000000000e13955cc3757803 dm-2 WINSYS,SF2372
size=125G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
|- 19:0:0:3 sde 8:64 active ready running
`- 18:0:0:3 sdj 8:144 active ready running
D escrip t io n
-l
-l l
-f device
-F
-w device
-W
Some mul ti pathd commands include a fo rmat option followed by a wildcard. You can display a
list of available wildcards with the following command.
# mul ti pathd sho w wi l d card s
5.10. Det ermining Device Mapper Ent ries wit h t he dmset up Command
You can use the d msetup command to find out which device mapper entries match the multipathed
devices.
The following command displays all the device mapper devices and their major and minor numbers.
The minor numbers determine the name of the dm device. For example, a minor number of 3
corresponds to the multipathed device /d ev/d m-3.
# d msetup l s
mpathd (253:4)
mpathep1
mpathfp1
mpathb (253:3)
mpathgp1
mpathhp1
mpatha (253:2)
38
(253:12)
(253:11)
(253:14)
(253:13)
mpathh (253:9)
mpathg (253:8)
VolGroup00-LogVol01
(253:1)
mpathf (253:7)
VolGroup00-LogVol00
(253:0)
mpathe (253:6)
mpathbp1
(253:10)
mpathd (253:5)
5.11. T roubleshoot ing wit h t he mult ipat hd Int eract ive Console
The mul ti pathd -k command is an interactive interface to the mul ti pathd daemon. Entering this
command brings up an interactive multipath console. After executing this command, you can enter
hel p to get a list of available commands, you can enter a interactive command, or you can enter
C T R L-D to quit.
The mul ti pathd interactive console can be used to troubleshoot problems you may be having with
your system. For example, the following command sequence displays the multipath configuration,
including the defaults, before exiting the console.
# mul ti pathd -k
> > sho w co nfi g
> > C T R L-D
The following command sequence ensures that multipath has picked up any changes to the
mul ti path. co nf,
# mul ti pathd -k
> > reco nfi g ure
> > C T R L-D
Use the following command sequence to ensure that the path checker is working properly.
# mul ti pathd -k
> > sho w paths
> > C T R L-D
39
Revision History
R evisio n 0.2- 7
Version for 7.1 GA release
Mo n Feb 16 2015
St even Levin e
R evisio n 0.2- 6
Version for 7.1 Beta release
T h u D ec 11 2014
St even Levin e
R evisio n 0.2- 2
T u e D ec 2 2014
Resolves: #1162514, #1085979, #1129897
Small updates for technical clarification and correction.
St even Levin e
Resolves: #1117037
D ocuments -w and -W options of the multipath command.
Resolves: #1117043
D ocuments new options for the features configuration parameter.
Resolves: #1117499
D ocuments -force_sync configuration parameter.
R evisio n 0.2- 1
T h u O ct 30 2014
Build to update procedure for moving root file system for 7.0
St even Levin e
R evisio n 0.2- 0
Wed O ct 22 2014
Updated procedure for moving root file system
St even Levin e
R evisio n 0.1- 22
Version for 7.0 GA release
Mo n Ju n 2 2014
St even Levin e
R evisio n 0.1- 20
Rebuild for style changes
T u e May 20 2014
St even Levin e
Wed Ap r 9 2014
St even Levin e
R evisio n 0.1- 3
Version for 7.0 pre-Beta
T u e N o v 26 2013
St even Levin e
R evisio n 0.1- 2
T u e N o v 12 2013
Removing RHEL 6 references.
St even Levin e
R evisio n 0.1- 1
Wed Jan 16 2013
St even Levin e
Branched from the Red Hat Enterprise Linux 6 version of the document
40
Index
Index
Symbols
/et c/mu lt ip at h .co n f p ackag e, Set t in g U p D M Mu lt ip at h
A
act ive/act ive co n f ig u rat io n
- definition, Overview of D M Multipath
- illustration, Overview of D M Multipath
act ive/p assive co n f ig u rat io n
- definition, Overview of D M Multipath
- illustration, Overview of D M Multipath
alias p aramet er , Mu lt ip at h s D evice C o n f ig u rat io n At t rib u t es
- configuration file, Multipath D evice Identifiers
B
b lacklist
-
C
ch ecker_t imeo u t p aramet er, C o n f ig u rat io n File D ef au lt s
co n f ig u rat io n f ile
- alias parameter, Multipaths D evice Configuration Attributes
- blacklist, Configuration File Blacklist
- checker_timeout parameter, Configuration File D efaults
- detect_prio parameter, Configuration File D efaults, Multipaths D evice Configuration
Attributes
- dev_loss_tmo parameter, Configuration File D efaults, Configuration File D evices
- failback parameter, Configuration File D efaults, Multipaths D evice Configuration
Attributes, Configuration File D evices
- fast_io_fail_tmo parameter, Configuration File D efaults, Configuration File D evices
- features parameter, Configuration File D efaults, Configuration File D evices
- flush_on_last_del parameter, Configuration File D efaults, Multipaths D evice
Configuration Attributes, Configuration File D evices
- force_sync parameter, Configuration File D efaults
- hardware_handler parameter, Configuration File D evices
- hwtable_string_match parameter, Configuration File D efaults
- max_fds parameter, Configuration File D efaults
- no_path_retry parameter, Configuration File D efaults, Multipaths D evice
Configuration Attributes, Configuration File D evices
- overview, Configuration File Overview
- path_checker parameter, Configuration File D efaults, Configuration File D evices
- path_grouping_policy parameter, Configuration File D efaults, Multipaths D evice
Configuration Attributes, Configuration File D evices
41
D
d ef au lt s sect io n
- multipath.conf file, Configuration File D efaults
d et ect _p rio p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice
C o n f ig u rat io n At t rib u t es
d ev/map p er d irect o ry, Mu lt ip at h D evice Id en t if iers
d evice n ame, Mu lt ip at h D evice Id en t if iers
d evice- map p er- mu lt ip at h p ackag e, Set t in g U p D M Mu lt ip at h
d evices
- adding, Configuring Storage D evices, Configuration File D evices
d evices sect io n
- multipath.conf file, Configuration File D evices
d ev_lo ss_t mo p aramet er, C o n f ig u rat io n File D ef au lt s, C o n f ig u rat io n File D evices
D M Mu lt ip at h
- and LVM, Multipath D evices in Logical Volumes
- components, D M Multipath Components
- configuration file, The D M Multipath Configuration File
- configuring, Setting Up D M Multipath
- definition, D evice Mapper Multipathing
- device name, Multipath D evice Identifiers
- devices, Multipath D evices
- failover, Overview of D M Multipath
- overview, Overview of D M Multipath
- redundancy, Overview of D M Multipath
- setup, Setting Up D M Multipath
- setup, overview, D M Multipath Setup Overview
42
Index
F
f ailb ack p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice C o n f ig u rat io n
At t rib u t es, C o n f ig u rat io n File D evices
f ailo ver, O verview o f D M Mu lt ip at h
f ast _io _f ail_t mo p aramet er, C o n f ig u rat io n File D ef au lt s, C o n f ig u rat io n File
D evices
f eat u res p aramet er, C o n f ig u rat io n File D ef au lt s, C o n f ig u rat io n File D evices
f lu sh _o n _last _d el p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice
C o n f ig u rat io n At t rib u t es, C o n f ig u rat io n File D evices
H
h ard ware_h an d ler p aramet er, C o n f ig u rat io n File D evices
h wt ab le_st rin g _mat ch p aramet er, C o n f ig u rat io n File D ef au lt s
I
in it ramf s
- starting multipath, Setting Up Multipathing in the initramfs File System
K
kp art x co mman d , D M Mu lt ip at h C o mp o n en t s
L
lo cal d isks, ig n o rin g , Ig n o rin g Lo cal D isks wh en G en erat in g Mu lt ip at h D evices
LVM p h ysical vo lu mes
- multipath devices, Multipath D evices in Logical Volumes
lvm.co n f f ile , Mu lt ip at h D evices in Lo g ical Vo lu mes
M
max_f d s p aramet er, C o n f ig u rat io n File D ef au lt s
mp at h co n f co mman d , D M Mu lt ip at h C o mp o n en t s
mu lt ip at h co mman d , D M Mu lt ip at h C o mp o n en t s
- options, Multipath Command Options
- output, Multipath Command Output
- queries, Multipath Queries with multipath Command
mu lt ip at h d aemo n ( mu lt ip at h d ) , T h e Mu lt ip at h D aemo n
mu lt ip at h d evices, Mu lt ip at h D evices
- logical volumes, Multipath D evices in Logical Volumes
- LVM physical volumes, Multipath D evices in Logical Volumes
mu lt ip at h .co n f f ile, St o rag e Array Su p p o rt , T h e D M Mu lt ip at h C o n f ig u rat io n File
- blacklist_exceptions section, Blacklist Exceptions
- defaults section, Configuration File D efaults
- devices section, Configuration File D evices
43
N
n o _p at h _ret ry p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice
C o n f ig u rat io n At t rib u t es, C o n f ig u rat io n File D evices
P
p at h _ch ecker p aramet er, C o n f ig u rat io n File D ef au lt s, C o n f ig u rat io n File D evices
p at h _g ro u p in g _p o licy p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice
C o n f ig u rat io n At t rib u t es, C o n f ig u rat io n File D evices
p at h _select o r p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice
C o n f ig u rat io n At t rib u t es, C o n f ig u rat io n File D evices
p o llin g _in t erval p aramet er, C o n f ig u rat io n File D ef au lt s
p rio p aramet er, C o n f ig u rat io n File D ef au lt s, C o n f ig u rat io n File D evices
p ro d u ct p aramet er, C o n f ig u rat io n File D evices
p ro d u ct _b lacklist p aramet er, C o n f ig u rat io n File D evices
Q
q u eu e_wit h o u t _d aemo n p aramet er, C o n f ig u rat io n File D ef au lt s
R
resiz in g a mu lt ip at h d evice, R esiz in g an O n lin e Mu lt ip at h D evice
ret ain _at t ach ed _h w_h an d ler p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s
D evice C o n f ig u rat io n At t rib u t es
revisio n p aramet er, C o n f ig u rat io n File D evices
ro o t f ile syst em, Mo vin g ro o t File Syst ems f ro m a Sin g le Pat h D evice t o a Mu lt ip at h
D evice
rr_min _io p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice C o n f ig u rat io n
At t rib u t es
rr_weig h t p aramet er, C o n f ig u rat io n File D ef au lt s, Mu lt ip at h s D evice C o n f ig u rat io n
At t rib u t es, C o n f ig u rat io n File D evices
S
set u p
- D M Multipath, Setting Up D M Multipath
44
Index
U
u d ev_d ir p aramet er, C o n f ig u rat io n File D ef au lt s
u id _at t rib u t e p aramet er, C o n f ig u rat io n File D ef au lt s
u ser_f rien d ly_n ames p aramet er , Mu lt ip at h D evice Id en t if iers, C o n f ig u rat io n File
D ef au lt s, Mu lt ip at h s D evice C o n f ig u rat io n At t rib u t es, C o n f ig u rat io n File D evices
V
ven d o r p aramet er, C o n f ig u rat io n File D evices
verb o sit y p aramet er, C o n f ig u rat io n File D ef au lt s
W
Wo rld Wid e Id en t if ier ( WWID ) , Mu lt ip at h D evice Id en t if iers
wwid p aramet er, Mu lt ip at h s D evice C o n f ig u rat io n At t rib u t es
45