SunCluster. Migrate Oracle from ufs to zfs.

GOAL

Migrate oracle resource group from ufs file system to zfs

Initial environment

metaset ora01-disk

• /dev/did/rdsk/d3
• /dev/did/rdsk/d6

metadevices

• /dev/md/ora01-disk/d10
• /dev/md/ora01-disk/d20

resources

• ora01-listener
• dependencies: ora01-disk,ora01-host
• ora01-oracle
• dependencies: ora01-disk,ora01-host
• ora01-disk
• dependencies: -
• mountpoints: /u01, /u02
• ora01-host
• dependencies: -

Plan

Migrate procedure

1. Map disk to your cluster
• on both nodes run cfgadm -al
• on one node format -e <disk_wwn> and label it with EFI
• on both nodes run scdidadm -r; scdidadm -u; scdidadm -i
2. Create zpool
• on active cluster node zpool create ora01-pool01 /dev/did/dsk/d9s0 /dev/did/dsk/d10s0 /dev/did/dsk/d11s0 /dev/did/dsk/d12s0
• create datasets, like describe in this document
• chown oracle:dba /ora01-pool01/*
3. Add zpool to cluster
• if you using SunCluster 3.3,  SunPatch 145333-11 or later must be installed
• add zpool to cluster clresource create -g ora01 -t SUNW.HAStoragePlus -p Zpools=ora01-pool01 -p ZpoolsSearchDir=/dev/did/dsk ora01-pool01
• add dependencies clrs set -p Resource_dependencies=ora01-disk,ora01-pool01,ora01-host ora01-oracle
4. Migrate oracle database
• unmonitor oracle database: clrs unmonitor ora01-oracle
• copy datafiles to new location
• edit pfile(spfile), set new paths to control files
• startup mount and rename files using alter database rename file 'file_name' to 'new_file_name';
• open database: alter database open;
5. Remove unused datasets and disks
• check that nobody used our disk: fuser -c /u01; fuser -c /u02
• remove cluster dependencies: clrs set -p Resource_dependencies=ora01-pool01,ora01-host ora01-oracle;  clrs set -p Resource_dependencies=ora01-pool01,ora01-host ora01-listener
• remove resource from resource group: clrs disable ora01-disk; clrs delete ora01-disk
• remove dataset
• clear metadevices: metaclear -s ora01-disk -a
• metaset -s ora01-disk -d /dev/did/rdsk/d3
• last one is removed with -f option: metaset -s ora01-disk -f -d /dev/did/rdsk/d6
• removing nodes. For all nodes (but the node where the diskset is primary last) perform:  metaset -s ora01-disk -d -h sc01-n2.labs.localnet && metaset -s ora01-disk -d -h sc01-n1.labs.localnet
• remove records in /etc/vfstab

ADD new disk to clustered zpool 

• map disk to cluster nodes
• label it with EFI label using: format -e <disk>
• update global devices on both nodes: scdidadm -r; scdidadm -u; scdidadm -i
• zpool add <poolname> </dev/did/dsk/new_disk_s0>