TERiX #576014, Sun T5240, Disk Issues in pool: ldoms

zpool corruption
The general recommendation is to rebuild the pool and then restore the data in o
rder to ensure you have an error free zpool.

We will like to proceed further to replicate data from existing disk to new disk
.
Once that completes, then replace them in zfs on tus1cmsldmdin05 via:
http://docs.oracle.com/cd/E19253-01/819-5461/gbcet/
# zfs replace ldoms c1t2d0 NEWDISK1
# zfs replace ldoms c1t3d0 NEWDISK2
-------------------------------------------
We have failed disk on zfs fs and ldoms configured on it. We want to make sure t
hat, after disk replacement under zfs, ldoms is configuration and there is minim
al data loss. Also We need steps to perform disk replacement under zfs.
We have review oracle document but still unclear on data loss and configuration
is loss. Will it be possible for you to take remote session and perform the acti
vity?
root@tus1cmsldmdin05:/> ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 20 8G 20% 113d 3h
tus1cmsappdin05 active -n---- 5000 32 13G 0.3% 112d 2h
tus1cmsappsin05 active -n---- 5001 32 10G 10% 107d 13h
root@tus1cmsldmdin05:/>

root@tus1cmsldmdin05:/> zpool status -v

pool: ldoms
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://www.sun.com/msg/ZFS-8000-8A
scrub: scrub completed with 2 errors on Tue Dec 17 14:52:49 2013
config:

NAME STATE READ WRITE CKSUM

ldoms ONLINE 82 0 0
c1t2d0 ONLINE 82 0 0
c1t3d0 ONLINE 0 0 0

errors: Permanent errors have been detected in the following files:

/ldoms/tus1cmsappdin05/appsdisk-vdisk1.img
/ldoms/tus1cmsappsin05/appsdisk-vdisk1.img
root@tus1cmsldmdin05:/>
root@tus1cmsldmdin05:/> iostat -en
---- errors ---
s/w h/w trn tot device
0 0 0 0 c1t0d0
0 0 0 0 c1t1d0
1 0 0 1 c0t0d0
0 56 124 180 c1t3d0
503 48503 32672 81678 c1t2d0
root@tus1cmsldmdin05:/>
-----------------------------------------------------
The zpool status shows it has corruption at the data level. The zpool will have
to be destroyed, recreated and all data restored. To recreate the zfs file syste
ms on the zpool, you will want to gather the current configuration so you can re
create it the file systems.
zfs list
zfs get all
zpool status -v
To backup the structure of the ldoms you can follow this procedure.
http://docs.oracle.com/cd/E19608-01/html/821-1485/savingldomsconfigsforfuturereb
uilding.html
This will only be needed if something goes wrong on the primary ldom and you los
e the configuration.
This output will also help, when you rebuild the guest ldom's virtual disk devic
es.
Once you have this data you can more forward with replacement of the hardware, a
nd rebuilding.

To prepare the disks for physical replacement.

1) destroy the zpool called ldom. You WILL LOSE ALL data on this zpool. If you d
o not have a backup, you should try to backup up as much as you can, although yo
u may be backing up corrupted data.

2) offline the hard drives at the OS level so the FE can remove them
http://docs.oracle.com/cd/E19910-01/E21500-01/z4001fde1390126.html
Use the cfgadm utility to list the available components
cfgadm -alv
Use the cfgadm utility to unconfigure the disks. (confirm the ap-id needed using
the command above)
cfgadm -c unconfigure AP-ID
for disks c1t2d0 and c1t3d0 the ap-id's should be:
c1::dsk/c1t2d0
c1::dsk/c1t3d0
The FE should see the blue LED turn on. He can now remove these disks.

3) Install the new disks.

http://docs.oracle.com/cd/E19910-01/E21500-01/z4001fde1390774.html
The FE can now physically install the new drives.
list components including drives that are not configured.
cfgadm -al
Configure the new drives at the OS level.
cfgadm -c configure AP-ID
Again the ap-id's should be:
c1::dsk/c1t2d0
c1::dsk/c1t3d0
Confirm the drives are seen by the OS.
cfgadm -al
echo | format
4) Prepare the drives for use with zpool.
use the format utility to select each of the new disks (c1t2d0 and c1t3d0) and m
ake sure they have a label.
If the drive does not have a label, you will be told when you select it, and you
will be asked to label the drive.
Answer - yes - if you are asked to label the drives.
5) Create the zpool (mirrored pool of these two new disks).
http://docs.oracle.com/cd/E19253-01/816-5166/zpool-1m/
zpool create ldom mirror c1t2d0 c1t3d0
6) create the zfs file systems as required using the zfs get all output from abo
ve. This step is part of OS level recovery. Your administrators will be responsi
ble for this portion of the activities.
7) restore the data to the zfs file systems. Your team will also be responsible
for this step.
8) create the vdisks needed for the guest ldoms to use (this may have been compl
eted if you are able to restore them in step 7.)
If you were not able to restore them, you may have to recreate them.
9) check the ldom configuration for the guest ldoms and confirm the vdisk associ
ations are correct.
10) boot the ldom's.
If you were able to restore the originals in step 7, the guest ldoms should boot
up.
If you had to create new files for use as vdisks by the guest ldoms, then you wi
ll have to reload the operating systems.