Using EMC PowerPath with Oracle ASM
Introduction
This post explains how to configure and use EMC PowerPath with Oracle ASM for clustered Oracle RAC servers running on RHEL 6.4 or higher servers.
Oracle ASM ensures consistent naming of devices across RAC clusters, and also maintains permissions on devices across reboots, a feature that was important until UDEV rules were added to Linux with the 2.5 kernel.
EMC PowerPath is an advanced multi-pathing host based solution that works with EMC arrays to intelligently load balance I/O across all available paths as well as provide fault tolerance by automatically re-routing traffic around failed paths. EMC PowerPath is significantly more powerful and robust that native Linux MPIO.
A feature of ASM is that devices stamped for ASM are assigned an alias. Each disk will be added to the /dev/oracleasm/disks/ device directory, and presented to ASM using the alias ORCL:diskname.
When Linux sees multiple paths to the same disk or LUN, it creates an entry in the SCSI device table for each path. Therefore a single LUN with two paths may appear to Linux as both /dev/sdg and /dev/sdh.
This is problematic for ASM, since ASM cannot natively handle two or more devices mapping to the same LUN.
Like ASM, EMC PowerPath also creates aliases, called pseudo-devices. One pseudo-device is assigned to cover all SCSI devices or paths that map back to the same LUN. EMC PowerPath pseudo-devices will have the naming convention /dev/emcpowerX where X is the device letter.
Process
One of the main issues to address with RHEL servers in a cluster is the EMC pseudo names need to match. We can see the pseudo names by running powermt display dev=all command.
[ linuxodb02 scsi_host ] # powermt display dev=all
Pseudo name=emcpowera
Symmetrix ID=000195702256
Logical device ID=1B79
Device WWN=60000970000195702256533031423739
state=alive; policy=SymmOpt; queued-IOs=0
==============================================================================
————— Host ————— – Stor – — I/O Path — — Stats —
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
1 lpfc sde FA 7e:00 active alive 0 0
1 lpfc sdd FA 5e:00 active alive 0 0
0 lpfc sdc FA 10e:00 active alive 0 0
0 lpfc sdb FA 12e:00 active alive 0 0
Pseudo name=emcpowerb
Symmetrix ID=000195702256
Logical device ID=1B8A
Device WWN=60000970000195702256533031423841
state=alive; policy=SymmOpt; queued-IOs=0
==============================================================================
————— Host ————— – Stor – — I/O Path — — Stats —
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
1 lpfc sdm FA 7e:00 active alive 0 0
1 lpfc sdk FA 5e:00 active alive 0 0
0 lpfc sdi FA 10e:00 active alive 0 0
0 lpfc sdg FA 12e:00 active alive 0 0
If we run the command on both hosts we will see the pseudo names are different. Here is how we address that issue.
Shutdown PowerPath on one of the clustered nodes.
$ sudo /etc/init.d/PowerPath stop
Now that PowerPath is stop we need to copy PowerPath configuration files from node one to node two. Copy /etc/emcp_devicesDB.dat and /etc/emcp_devicesDB.idx to the /etc directory on the node. Once the copy is complete start PowerPath (sudo /etc/init.d/PowerPath start) up and run the powermt display dev=all command and the pseudo names will match.
Now we match and are ready to make changes to /etc/sysconfig/oracleasm. The directives ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE need to be modified as follows:
# ORACLEASM_ENABLED: ‘true’ means to load the driver on boot.
ORACLEASM_ENABLED=true
# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=grid
# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=oinstall
# ORACLEASM_SCANBOOT: ‘true’ means scan for ASM disks on boot.
ORACLEASM_SCANBOOT=true
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER=”emcpower”
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE=”sd”
Now when ASMLib starts, it will scan for ASM disks among the PowerPath pseudo-devices instead of the SCSI devices.
Leave a Reply