How to Make Fibre Channel Storage Available to

Oracle Solaris
With the Sun ZFS Storage Appliance
by Andrew Ness, February 2012

How to configure Fibre Channel LUNs in Oracle's Sun ZFS Storage Appliance and integrate them into an
Oracle Solaris 10 or Oracle Solaris 11 environment.


This article covers the following topics:
Introduction
Configuring the Oracle Solaris FC Stack
Configuring the Sun ZFS Storage Appliance Using the Browser User Interface
Configuring the LUN for Use by the Oracle Solaris Server
Conclusion
For More Information
Introduction
Oracle Solaris 10 and Oracle Solaris 11 come with a Fibre Channel initiator stack you can configure to integrate Fibre
Channel (FC) LUNs presented by the Sun ZFS Storage Appliance into the Oracle Solaris environment. This article
describes how to configure the Oracle Solaris Fibre Channel stack and how to configure the Sun ZFS Storage
Appliance to set up FC LUNs for access by Oracle Solaris servers. These configurations can be accomplished using
the browser user interface (BUI).
The following assumptions have been made in this article:
The root password for the Sun ZFS Storage Appliance is known.
The IP address or host name of the Sun ZFS Storage Appliance is known.
The network used by the Sun ZFS Storage Appliance has been configured.
The Sun ZFS Storage Appliance has pools configured with sufficient free space available.
The root password for the Oracle Solaris server is known.
The Sun ZFS Storage Appliance is already connected to a Fibre Channel switch.
The appropriate zoning has been configured on the FC switch to allow the Oracle Solaris host to access the
Sun ZFS Storage Appliance.
Configuring the Oracle Solaris FC Stack
To identify the Sun ZFS Storage Appliance and the Oracle Solaris server to each other, the FC World Wide Number
(WWN) for each must be registered with the other. You must determine the WWNs for some forms of FC zoning
implemented on the FC switch.
The FC WWN of the host is used to identify the host to the Sun ZFS Storage Appliance and is needed to complete
the configuration procedure in this article.
The WWNs are taken from the FC Host Bus Adapters (HBAs) installed in the Oracle Solaris host and the Sun ZFS
Storage Appliance.
In order to configure the Oracle Solaris FC stack, you will need to know the WWNs of the Sun ZFS Storage
Appliance. In a traditional dual-fabric Storage Area Network (SAN), the Sun ZFS Storage Appliance will have at least
one FC port attached to each fabric. Thus, you must determine at least two FC WWNs.
Identifying the Sun ZFS Storage Appliance FC WWNs
First, you need to establish a management session with the Sun ZFS Storage Appliance.
1. Enter an address in the address field of a Web browser that includes the IP address or host name of the Sun
ZFS Storage Appliance as shown in the following URL:
2. https://<ip-address or host name>:215
The login dialog box is displayed.


Figure 1. Sun ZFS Storage Appliance login
3. Enter a username and password and click LOGIN.
Once you have successfully logged in to the BUI, you can identify the WWNs through the Configuration tab.
4. Click on Configuration > SAN > Fibre Channel Ports.



Figure 2. Fibre Channel port configuration for SAN
The FC ports installed in the Sun ZFS Storage Appliance are shown. Since there is only one discovered port
per HBA channel, this must be the HBA channel itself.
In the preceding case, port 1 has the WWN 21:00:00:e0:8b:92:a1:cf and port 2 has the WWN
21:01:00:e0:8b:b2:a1:cf.
The FC channel ports should be set to Target in the list box to the right of each FC port box. If this is not the
case, the FC ports may be in use for another purpose. Do not change the setting until you have investigated
the reason. (One possible reason may be for NDMP backups.)
Identifying the Oracle Solaris Host HBA WWNs
If the Oracle Solaris host is already attached to the FC switch by the appropriate cables, use the following command
to identify the WWNs.
1. To find the WWNs for the host, enter:


2. root@solaris:~# cfgadm -al -o show_FCP_dev
3. Ap_Id Type Receptacle Occupant Condition
4. [...]
5. c8 fc-fabric connected unconfigured unknown
6. c9 fc-fabric connected unconfigured unknown
7. root@solaris:~#
From this output, the controller numbers you need are c8 and c9. You can also see that both ports are
attached to an FC switch, as the port type is fc-fabric. Next, interrogate these controllers to determine
what WWNs are discovered.
8. If the HBA ports are not being used to access any other FC-connected devices, the following command can
be used to determine the WWNs.


9. root@solaris:~# prtconf -vp | grep port-wwn
10. port-wwn: 210000e0.8b89bf8e
11. port-wwn: 210100e0.8ba9bf8e
12. root@solaris:~#
Alternatively, if FC devices are being accessed, the following command will show the FC HBA WWNs.
root@solaris:~# luxadm -e dump_map /dev/cfg/c8
Pos Port_ID Hard_Ad Port WWN Node WWN Type
0 2000 0 xxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxx 0x0 (Disk
device)
[...]
3 25600 0 2100000e08b89bf8e 200000e08b89bf8e 0x1f (Unknown
type, Host Bus Adapter)
root@solaris:~#
The last entry shown as type 0x1f (Unknown type, Host Bus Adapter) gives the appropriate WWN
under the port WWN entry. Repeat the command substituting /dev/cfg/c8 for the other controllers
identified in step 1.
13. From the output, you can see that c8 has the WWN 21:00:00:00:e0:8b:89:bf:8e and c9 has WWN
21:01:00:e0:8b:a9:bf:8e.
The Sun ZFS Storage Appliance HBA and the Oracle Solaris host HBA WWNs can then be used to
configure any FC switch zoning.
Once this has been completed, you can verify the proper zoning by running the following command:
root@solaris:~# cfgadm -al -o show_FCP_dev c8 c9
Ap_Id Type Receptacle Occupant Condition
[...]
c8 fc-fabric connected configured unknown
c8::210100e08bb2a1cf unknown connected unconfigured unknown
c9 fc-fabric connected configured unknown
c9::210000e08b92a1cf unknown connected unconfigured unknown
root@solaris:~#
You can now see the WWNs presented by the Sun ZFS Storage Appliance that are accessible by the Oracle
Solaris host.
Configuring the Sun ZFS Storage Appliance Using the Browser User Interface
As a unified storage platform, the Sun ZFS Storage Appliance supports access to block-protocol LUNs using iSCSI
and Fibre Channel protocols. This section describes how to use the Sun ZFS Storage Appliance BUI to configure the
Sun ZFS Storage Appliance to recognize the Oracle Solaris host and present FC LUNs to it.
Defining an FC Target Group
A target group is created on the Sun ZFS Storage Appliance to define the ports and the protocol by which the LUN to
be presented to the Oracle Solaris server is accessed. For this example, an FC target group is created.
To define an FC target group on the Sun ZFS Storage Appliance, complete these steps:
1. Click Configuration > SAN to display the Storage Area Network (SAN) screen, as shown in Figure 3.
2. Click the Targets tab at the right and then select Fibre Channel Ports at the top of the left panel, as shown
in Figure 3.
3. Place the mouse in the Fibre Channel Ports box and a Move icon ( ) will appear in the far left, as shown in
Figure 3.



Figure 3. Selecting the Fibre Channel port
4. Click on the Move icon and drag the box to the Fibre Channel Target Groups box, as demonstrated in
Figure 4.



Figure 4. Dragging the Fibre Channel port entry
5. Drop the entry in the orange box to create a new target group. The group is created and named
automatically as targets-n where n is an integer. See the example targets-0 in Figure 5.



Figure 5. Newly created Fibre Channel target group
6. Move the cursor over the entry for the new target group. Two icons appear to the right of the Fibre Channel
Target Groups box, as shown in Figure 6.



Figure 6. Selecting the FC target group
7. To rename the new target group targets-0, click the Edit icon ( ) to display the dialog box shown in
Figure 7.



Figure 7. Renaming the FC Target Group
8. In the Name field, replace the default name with your preferred name for the new FC target group and click
OK. For this example, the name FC-PortGroup replaces targets-0. In this window, you can also add
the second FC target port by clicking in the box to the left of the WWN you are choosing. In the example in
Figure 7, this second port is identified as PCIe 1: Port 2.
9. Click OK to save the changes.
10. Click APPLY. The changes are shown in the Fibre Channel Target Groups panel in Figure 8.



Figure 8. New FC target group with changes applied
Defining an FC Initiator
An FC initiator is defined to allow access from one or more servers to particular volumes. Permissions to volumes
should be configured so that a minimum number of FC initiators are allowed to access a particular volume. If more
than one host can write to a given volume concurrently and a non-shared file system is used, inconsistencies may
occur in file system caches on the hosts that can ultimately lead to corruption of the on-disk image. Typically a single
initiator is given access to a volume, unless a specialized cluster file system is being used.
The FC initiator serves to define the "host" from the point of view of the Sun ZFS Storage Appliance. In a traditional
dual-fabric SAN, the host will be defined by at least two FC initiators. The FC initiator definition contains the host
WWNs. To identify the Oracle Solaris server to the Sun ZFS Storage Appliance, the Oracle Solaris FC initiator
WWNs must then be registered with the appliance by completing the following steps.
1. Click Configuration > SAN to display the Storage Area Network (SAN) screen shown in Figure 9.
2. Click the Initiators tab at the right and then select Fibre Channel Initiators at the top of the left panel, as
shown in Figure 9.



Figure 9. Selecting the SAN configuration
3. Click the icon to the left of Fibre Channel Initiators to display the New Fibre Channel Initiator dialog box
shown in Figure 10.
If the zoning has been configured on the FC switches, the WWNs of the Oracle Solaris host should be
displayed (assuming they are not assigned to an alias already).
4. Click on one of the WWNs (if they are displayed) at the bottom of the dialog box to prepopulate the World
Wide Name or type the appropriate WWN in to the World Wide Name box.
5. Enter a more meaningful symbolic name in the Alias box.



Figure 10. New Fibre Channel initiator definition
6. Click OK.
7. Repeat the preceding steps for the other WWN(s) that refer to the Oracle Solaris host.
Defining the FC Initiator Group
Related FC initiators are combined into logical groups to allow single commands to execute on multiple FC initiators,
for example, assigning LUN access to all FC initiators in a group with one command. For the following example, the
FC initiator group will contain two initiators. Note that in a cluster, where multiple servers are treated as a single
logical entity, the initiator group may contain many more initiators.
To create an FC initiator group, complete these steps:
1. Select Configuration > SAN to display the Storage Area Network (SAN) screen.
2. Select the Initiators tab at the right and then click Fibre Channel Initiators at the top of the left panel.
3. Place the cursor over the entry for one of the FC initiators created in the previous section. The Move icon (
) appears to the left of the entry, as shown in Figure 11.



Figure 11. Displaying the Move icon for the new FC initiator
4. Click the Move icon and drag it to the Fibre Channel Initiator Groups panel on the right. A new entry
appears (highlighted in yellow) at the bottom of the Fibre Channel Initiators Groups panel shown in Figure
12.



Figure 12. Creating the FC initiator group
5. Move the cursor over the new entry box and release the mouse button. A new FC initiator group is created
with the name initiators-n, where n is an integer, as shown in Figure 13.
6. Move the cursor over the entry for the new initiator group. Several icons appear to the right of the target
initiator group box, as shown in Figure 13.



Figure 13. Selecting the Fibre Channel initiator group
7. Click the Edit icon ( ) to display the dialog box shown in Figure 14.
8. In the Name field, replace the new initiator group's default name with a chosen name and click OK. For this
example, the name sol-server is used.
In this dialog box, you can add the other FC initiator(s) to the group by clicking the checkbox to the left of the
WWN.


Figure 14. Renaming and completing the FC initiator group
9. Click APPLY on the SAN configuration screen to confirm all the modifications, as shown in Figure 15.



Figure 15. Completed FC initiator configuration
Defining a Sun ZFS Storage Appliance Project
To group related volumes, you can define a project in the Sun ZFS Storage Appliance. A project allows inheritance of
properties for file systems and LUNs presented from the project. A project also allows you to apply quotas and
reservations.
To create a project, complete the steps below:
1. Select Shares > Projects to display the Projects screen, as shown in Figure 16.



Figure 16. Viewing a project
2. Click the icon to the left of Projects at the top of the left panel to display the Create Project dialog box
shown in Figure 17.



Figure 17. Create Project dialog box
3. To create a new project, enter a name for the project and click APPLY. A new project appears in the
Projects list in the left panel.
4. Select the new project to view the components that comprise the project, as shown in Figure 18.



Figure 18. Displaying the new project sol-project
Defining a Sun ZFS Storage Appliance LUN
Next, you will create a LUN from an existing pool of storage resources that the Oracle Solaris server will access. In
the following example, a thin-provisioned 64-GB LUN called DocArchive1 is created.
The target group will be the FC target group, FC-PortGroup, which was created in the section Defining an FC Target
Group, to ensure that this LUN can be accessed using the FC protocol. The initiator group defined in the section
Defining the FC Initiator Group, sol-server, will be used to ensure that only the server(s) defined in the sol-
server group can access this LUN. (In this example, this group contains only one server.)
To create a target group, complete the following steps:
1. Select Shares > Projects to display the Projects screen.
2. In the Projects panel at the left, select the project. Then select LUNs at the top of the panel at the right, as
shown in Figure 19.



Figure 19. Selecting Shares > Project > LUNs
3. Click the icon to the left of LUNs to display the Create LUN dialog box shown in Figure 20.
4. Enter values as appropriate to set up the LUN. For this example, set Name to DocArchive1, Volume size
to 64 G, and select the Thin provisioned checkbox. Set Target Group to the FC target group FC-
PortGroup and set Initiator Group to sol-server. Set Volume block size to 32k because the volume
will hold an Oracle Solaris ZFS file system.



Figure 20. New LUN dialog box
5. Click APPLY to create the LUN and make it available to the Oracle Solaris server.
Configuring the LUN for Use by the Oracle Solaris Server
Now that the LUN is prepared and available to the FC initiator group, the LUN must be configured for use by the
Oracle Solaris server by completing the following steps:
1. Initiate an Oracle Solaris FC session with the Sun ZFS Storage Appliance, as shown in Listing 1. Since the
LUN was created prior to initiating the FC session, it will be automatically enabled.

Listing 1. Initiating an Oracle Solaris FC session
2. root@solaris:~# cfgadm -al c8 c9
3. root@solaris:~# cfgadm -c configure c8::210100e08bb2a1cf
4. root@solaris:~# cfgadm -c configure c9::210000e08b92a1cf
5. root@solaris:~# cfgadm -al -o show_FCP_dev c8 c9
6. Ap_Id Type Receptacle Occupant Condition
7. c8 fc-fabric connected configured unknown
8. c8::210100e08bb2a1cf,0 disk connected configured unknown
9. c9 fc-fabric connected configured unknown
10. c9::210000e08b92a1cf,0 disk connected configured unknown
11. root@solaris:~#
12. Verify access to the FC LUN, as shown in Listing 2.

Listing 2. Verifying access to the FC LUN
13. root@solaris:~# devfsadm -c ssd
14. root@solaris:~# tail /var/adm/messages
15. [...]
16. Nov 30 06:30:00 solaris last message repeated 2 times
17. Nov 30 06:32:22 solaris scsi: [ID 799468 kern.info] ssd42 at
18. scsi_vhci0: name g600144f0f05e906c00004ed6096d0001, bus address
19. g600144f0f05e906c00004ed6096d0001
20. Nov 30 06:32:22 solaris genunix: [ID 936769 kern.info] ssd42 is
21. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001
22. Nov 30 06:32:22 solaris scsi: [ID 107833 kern.warning] WARNING:
23. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42):
24. Nov 30 06:32:22 solaris Corrupt label; wrong magic number
25. Nov 30 06:32:22 solaris genunix: [ID 408114 kern.info]
26. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42) online
27. Nov 30 06:32:22 solaris genunix: [ID 834635 kern.info]
28. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42)
29. multipath status: degraded, path
30. /pci@1e,600000/SUNW,qlc@2/fp@0,0 (fp1) to target address:
31. w210100e08bb2a1cf,0 is online Load balancing: round-robin
32. Nov 30 06:32:22 solaris scsi: [ID 107833 kern.warning] WARNING:
33. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42):
34. Nov 30 06:32:22 solaris Corrupt label; wrong magic number
35. Nov 30 06:32:49 solaris genunix: [ID 834635 kern.info]
36. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42)
37. multipath status: optimal, path
38. /pci@1e,600000/SUNW,qlc@2,1/fp@0,0 (fp3) to target address:
39. w210000e08b92a1cf,0 is online Load balancing: round-robin
40. [...]
In this example, the multipath status is shown as degraded initially because only one path had been
identified at that point. Further on, the multipath status changes to optimal due to the presence of multiple
paths to the volume.
The disk device is now similarly available to an internal server disk.
41. Format the LUN as shown in Listing 3.

Listing 3. Formatting the LUN
42. root@solaris:~# format
43. Searching for disks...done
44.
45. c1t600144F0F05E906C00004ED6096D0001d0: configured with capacity of
63.93GB
46.
47.
48. AVAILABLE DISK SELECTIONS:
49. [...]
50. 4. c1t600144F0F05E906C00004ED6096D0001d0 <SUN-SunStorage7210-
1.0 cyl 2078 alt 2 hd 254 sec 254>
51. /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001
52. [...]
53. Specify disk (enter its number): 4
54. selecting c1t600144F0F05E906C00004ED6096D0001d0
55. [disk formatted]
56. Disk not labeled. Label it now? y
57.
58.
59. FORMAT MENU:
60. disk - select a disk
61. type - select (define) a disk type
62. partition - select (define) a partition table
63. current - describe the current disk
64. format - format and analyze the disk
65. repair - repair a defective sector
66. label - write label to the disk
67. analyze - surface analysis
68. defect - defect list management
69. backup - search for backup labels
70. verify - read and display labels
71. save - save new disk/partition definitions
72. inquiry - show vendor, product and revision
73. volname - set 8-character volume name
74. !<cmd> - execute <cmd>, then return
75. quit
76. format> q
77. Build an Oracle Solaris ZFS file system on the prepared LUN by creating a new ZFS pool, adding this device
to the ZFS pool, and creating the ZFS file system, as shown in the example in Listing 4.

Listing 4. Building an Oracle Solaris ZFS file system
78. root@solaris:~# zfs createzpool create docarchive1 \
79. c1t600144F0F05E906C00004ED6096D0001d0
80. root@solaris:~# zfs list
81. NAME USED AVAIL REFER MOUNTPOINT
82. docarchive1 72K 62.5G 21K /docarchive1
83. rpool 25.4G 41.5G 97K /rpool
84. rpool/ROOT 22.3G 41.5G 21K legacy
85. [...]
86. root@solaris:~# zfs create docarchive1/index
87. root@solaris:~# zfs create docarchive1/data
88. root@solaris:~# zfs create docarchive1/logs
89. root@solaris:~# zfs list
90. NAME USED AVAIL REFER MOUNTPOINT
91. docarchive1 159K 62.5G 24K /docarchive1
92. docarchive1/data 21K 62.5G 21K /docarchive1/data
93. docarchive1/index 21K 62.5G 21K /docarchive1/index
94. docarchive1/logs 21K 62.5G 21K /docarchive1/logs
95. rpool 25.4G 41.5G 97K /rpool
96. rpool/ROOT 22.3G 41.5G 21K legacy
97. [...]
The final two lines of the output from the df(1) command show that approximately 64 GB of new space is
now available.
Conclusion
This article has described how to configure the FC initiator stack supplied with Oracle Solaris 10 and Oracle Solaris
11 to enable an Oracle Solaris server to access Fibre Channel LUNs presented by the Sun ZFS Storage Appliance.
For More Information
Here are resources for Oracle's Sun ZFS Storage Appliance and Oracle Solaris ZFS:
Sun ZFS Storage Appliance Administration Guide: Oracle Unified Storage Systems
documentation libraries or http://download.oracle.com/docs/cd/E22471_01/index.html