How to install ASM Filter Driver in a Linux

Environment without having Previously

Installed ASMLIB
Contents
Overview....................................................................................................................................................... 2
Installation Phases ................................................................................................................................... 2
Deploying udev in a Grid Infrastructure Environment (Step 1) ........................................ 2
Cluster Considerations ..................................................................................................................... 4
Migrate ASM Diskgroup (disks) to ASMFD (step 4) ................................................................ 5
Configure ASMFD ................................................................................................................................ 5
Labeling ASMFD Disks ...................................................................................................................... 6
Starting Clusterware .......................................................................................................................... 6
Cluster Configurations ...................................................................................................................... 6
Final Step ................................................................................................................................................. 7
Creating additional Diskgroups......................................................................................................... 7

Overview
This document describes the deployment of ASM Filter Driver in a Grid
Infrastructure environment for first time users. Installations of ASM Filter Driver,
otherwise known, as ASMFD requires that the disks managed by ASMFD are write
accessible by Oracle ASM before configuring ASMFD. In the Linux environment, the
root user normally owns all disks, and ownership of the disks cannot easily be
changed without first configuring another component such as udev, a multi-path
driver, or ASMLIB. If the user is configuring ASMFD in a preexisting Grid
Infrastructure environment, disk ownership is less of an issue because some
procedure had to have been taken in the first place to install Grid Infrastructures
component of ASM. The configuration procedure for ASMFD in an environment
where Grid Infrastructure is already installed is documented in the ASM Admin
Guide. The procedures outlined here provide the steps for configuring ASMFD
without having previously installed Grid Infrastructure or ASMLIB.

Installation Phases
Deploying Grid Infrastructure and configuring ASMFD involves the following phases.
1. Some means must be used to persistently change the ownership of the Grid
Infrastructure disks to the Oracle user. There are several ways to do this and
this document provides general guidelines for using Linux udev. This
document explains one method for achieving this first step.
2. Initially install Grid Infrastructure 12.1.0.2, or later without ASMLIB or
ASMFD. This procedure configures the initial ASM Diskgroup required for
Grid Infrastructure. The Grid Infrastructure document describes the process
for installation.
3. Upgrade the Grid Infrastructure to the current required patch level as
documented in MOS Doc ID 2034681.1 and is available from
http://support.oracle.com. The PSU listed in this MOS note corrects a
number of early issues discovered in ASMFD.
4. Migrate disks used in the Grid Infrastructure Diskgroup to ASMFD. If a
cluster environment is being configured, then this step has these substeps.
a. Configure ASMFD for the first server and label the disks
b. Configure ASMFD for the remaining cluster servers and issue
ASMFD_scan commands on each server to discover the disks
previously labeled.
5. Additional Diskgroups can be created through a combination of steps 1 and
4.
The remainder of this document describes steps 1 and 4. Installing Grid
Infrastructure and patching are documented in their respective documents.

Deploying udev in a Grid Infrastructure Environment (Step 1)

Providing the exact sequence of steps for setting up Linux udev is not practical
because udev configuration procedures have changed across Linux releases and
subtle differences exist between various Linux distributions. Additionally, the exact
steps are dependent on an individual customers storage configuration making
Configuring ASM Filter Driver without ASMLIB

Page 2

describing the precise steps impractical. The following procedure is for reference
only and applicable to an Oracle Linux 6.6 distribution. There are many published
articles for different environments for configuring udev including Oracle MOS note
Doc ID 1528148.1.
In this example, a system in which ASMFD is to be deployed has three disks that will
be used for the initial Grid Infrastructure Diskgroup. Disk sda and its partitions 1
and 2 are system disks. Disks sdb, sdc, and sdd are to be used for the initial ASM
Diskgroup as shown in the following command box.
[root@RAC1
brw-rw---brw-rw---brw-rw---brw-rw---brw-rw---brw-rw----

/]# ls
1 root
1 root
1 root
1 root
1 root
1 root

-l /dev/sd*
disk 8, 0 Sep
disk 8, 1 Sep
disk 8, 2 Sep
disk 8, 16 Sep
disk 8, 32 Sep
disk 8, 48 Sep

9
9
9
9
9
9

08:53
08:53
08:53
08:54
08:54
08:54

/dev/sda
/dev/sda1
/dev/sda2
/dev/sdb
/dev/sdc
/dev/sdd

In the following command box, an udev control file called /etc/udev/rules.d/99oracle-asmdevices.rules is created. Then udev command udevadm is used to invoke
this new udev rule as shown below. This rule causes the system to create a disk
reference in the form of /dev/asm-diskX when one of the designed ASM disks has
a partition added. These new disk paths are owned by user Oracle. It should be
noted, that this simple rule only works when the storage configuration is static and
the ASM disks will always be presented as /dev/sdb through /dev/sdf. If this is not
the case because of a particular storage environment, then a more complex udev file
utilizing scsi_id that maps disks to WWN references must be used.
[root@RAC1 rules.d]# cat 99-oracle-asmdevices.rules
KERNEL=="sdb1",NAME="asm-disk1",OWNER="oracle",GROUP="dba",MODE="0660"
KERNEL=="sdc1",NAME="asm-disk2",OWNER="oracle",GROUP="dba",MODE="0660"
KERNEL=="sdd1",NAME="asm-disk3",OWNER="oracle",GROUP="dba",MODE="0660"
KERNEL=="sde1",NAME="asm-disk4",OWNER="oracle",GROUP="dba",MODE="0660"
KERNEL=="sdf1",NAME="asm-disk5",OWNER="oracle",GROUP="dba",MODE="0660"
[root@RAC1 rules.d]# udevadm control --reload-rules

Configuring ASM Filter Driver without ASMLIB

Page 3

The following command adds a partition to /dev/sdb

[root@RAC1 rules.d]# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to sectors(command
'u').
Command (m for help): n
Command action
e
extended
p
primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): <return>
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): <return>
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table. Syncing disks.

When using udev as described here, a newly created partition (e.g. /dev/sdb1) does
not show up as /dev/sdb1, but rather /dev/asm-disk1. Partitions for the other disks
(sdc, sdd) are added likewise as well. Now the Oracle accessible disk paths have
been created.
[root@RAC1
brw-rw---brw-rw---brw-rw----

/]# ls -l /dev/asm*
1 oracle dba 8, 17 Sep
1 oracle dba 8, 33 Sep
1 oracle dba 8, 49 Sep

9 09:26 /dev/asm-disk1
9 09:34 /dev/asm-disk2
9 09:35 /dev/asm-disk3

Grid infrastructure can now be installed using the new Oracle accessible disks. The
required ASM diskstring is /dev/asm*. After installing Grid Infrastructure, it
should be patched to the appropriate level as documented in MOS Doc ID 2034681.1.
Cluster Considerations
When configuring ASMFD for a cluster, all severs in the cluster need to be made
aware of the partition tables created on the shared disks. Additionally, there needs
to be an udev configuration file, such as the one created above, active on every
server. The udev configuration file created above can be propagated to the other
servers and udevadm control --reload-rules issued to effect the change. After making
the udev change on all the servers, the partition table can be read on these servers
using partprobe /dev/sdX for each shared disk. These changes must be made before
beginning the Grid Infrastructure installation.

Configuring ASM Filter Driver without ASMLIB

Page 4

Migrate ASM Diskgroup (disks) to ASMFD (step 4)

Configure ASMFD
As the Oracle Grid Infrastructure owner update the Oracle ASM disk discovery string
to enable Oracle ASMFD to discover disks in the future. For example, check the
current value of the Oracle ASM disk discovery string and then update the value.
$ORACLE_HOME/bin/asmcmd dsget
$ORACLE_HOME/bin/asmcmd dsset old_diskstring, AFD:*

The value of old_diskstring is the current Oracle ASM disk discovery string value.
For information about updating the Oracle ASM discovery string, refer to "Updating
the Oracle ASM ASM_DISKSTRING Parameter for Oracle ASM Filter Driver Disks".
As the Oracle Grid Infrastructure owner list the nodes and node roles in your
cluster.
$ORACLE_HOME/bin/olsnodes -a

On the server log in as the root user and stop Oracle Grid Infrastructure:
# $ORACLE_HOME/bin/crsctl stop crs

If the command returns an error, then stop Oracle Grid Infrastructure forcibly as
follows:
# $ORACLE_HOME/bin/crsctl stop crs -f

As root user, configure Oracle ASMFD to filter at the server level.

# $ORACLE_HOME/bin/asmcmd afd_configure

As the Oracle Grid Infrastructure owner verify the status of Oracle ASMFD.
$ $ORACLE_HOME/bin/asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host
'myhost'

For information about checking on the state of the Oracle ASM Filter Driver, refer to
"Determining Whether Oracle ASM Filter Driver Has Been Configured".

Configuring ASM Filter Driver without ASMLIB

Page 5

Labeling ASMFD Disks

Each disks in the GI Diskgroup require the ASMFD label to be made. On the first
server, label each of the disks in the Diskgroup by running the following command.
The disks being labeled in this example are the disk references created as
/dev/asm-diskX
[oracle@rac1~]$ $ORACLE_HOME/bin/asmcmd afd_label disk1 /dev/asm-disk1 --migrate
[oracle@rac1~]$ $ORACLE_HOME/bin/asmcmd afd_label disk2 /dev/asm-disk2 --migrate
[oracle@rac1~]$ $ORACLE_HOME/bin/asmcmd afd_label disk3 /dev/asm-disk3 --migrate
[oracle@rac1~]$ $ORACLE_HOME/bin/asmcmd afd_lsdsk
Label
Filtering
Path
================================================================================
DISK1
ENABLED
/dev/asm-disk1
DISK2
ENABLED
/dev/asm-disk2
DISK3
ENABLED
/dev/asm-disk3

Starting Clusterware
As root user, start the Oracle Clusterware stack.
# $ORACLE_HOME/bin/crsctl start crs -wait

You can verify that the Diskgroups are mounted using ASMFD disks
$ORACLE_HOME/bin/asmcmd lsdsk

As the Oracle Grid Infrastructure user set the Oracle ASMFD discovery disk string to
the original Oracle ASM disk discovery string value that was retrieved in Step 1. The
value of old_diskstring is the old disk discovery string value used without the
ASMFD: (Oracle ASMFD) path. It can now be removed, as it is no longer required
for the ASMFD discovery path.
$ORACLE_HOME/bin/asmcmd afd_dsset old_diskstring

Cluster Configurations
At this point, ASMFD is configured on a server and Clusterware is running. If this is a
single server environment, then your job is complete. If this is the first server of a
cluster with more than one server, then the remaining servers in the cluster need
ASMFD configured as well. This is achieved by repeating the procedures above from
the section labeled Configure ASMFD through and including the section Starting
Clusterware. However on the secondary cluster servers, rather than labeling the
disks as described, issue the following command on the secondary servers.
# $ORACLE_HOME/bin/asmcmd afd_scan old_diskstring

Configuring ASM Filter Driver without ASMLIB

Page 6

Final Step
At this point, if one wants all disks managed by ASMFD, then the ASM discovery
string should be set to just use ASMFD string as the non-ASMFD string is no longer
required.
$ORACLE_HOME/bin/asmcmd dsset AFD:*

Creating additional Diskgroups

Additionally Diskgroups can now be created using the procedures outlined above. In
particular, follow these steps for creating new Diskgroups.
1. Provision disks and create the first partition on each of the disks
2. Add new entries in udev rules file mapping new disks as an ASM Disks (e.g.
/dev/asm-diskX)
3. Label the new disks (e.g. asmcmd ASMFD_label disk-name /dev/asm-diskX)
4. Create the new Diskgroup using provisioned disks

Configuring ASM Filter Driver without ASMLIB

Page 7