SETUP ASM&ASMLib on
Multipath Disks
1. Install Correct
Packages
We must be careful while downloading
correct rpms. Before download the rpms, check your kernel version and
processor(32 bit or 64 bit):
oracleasm-(kernel) --- must be same version with the kernel
oracleasmlib ---
can be the latest version of correct 32 or 64 bit version
oracleasm-support --- can be the latest version of
correct 32 or 64 bit version
[root@racnode1 ~]# uname -a
Linux racnode1.msb.bak 2.6.18-194.el5 #1
SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
[root@racnode1 ~]# rpm -qa | grep asm
oracleasm-support-2.1.8-1.el5
oracleasmlib-2.0.4-1.el5
oracleasm-2.6.18-194.el5-2.0.5-1.el5
[root@racnode1 ~]#
if you downloaded and installed wrong
“oracleasm-(kernel)” rpm, you can get the following errors:
[root@racnode1 bin]# /etc/init.d/oracleasm
start
Initializing the Oracle ASMLib driver:
[FAILED]
[root@racnode1 bin]# more
/var/log/oracleasm
"/var/log/oracleasm"
"/var/log/oracleasm" 145L,
5999CCreating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm":
failed
Unable to load module "oracleasm"
Creating /dev/oracleasm mount point:
/dev/oracleasm
Loading module "oracleasm":
failed
Unable to load module "oracleasm"
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
oracleasm-read-label: Unable to open device
"/dev/sdb1": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-0": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-0": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-0": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-0": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-1": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-1": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-1": No such file or directory
oracleasm-read-label: Unable to open device
"/dev/dm-1": No such file or directory
2. Check your
disks
[oracle@racnode1 ~]$ ls -lrt
/dev/mapper/
total 0
brw-rw---- 1 root disk 253, 8 May 15 12:47 oradata2p1
brw-rw---- 1 root disk 253, 9 May 15 12:47 oradata3p1
brw-rw---- 1 root disk 253, 10 May
15 12:47 oradata1p1
3. Create ASM
disks
[root@racnode1 ~]#
/etc/init.d/oracleasm createdisk ASM1 /dev/mapper/oradata1p1
[root@racnode1 ~]# /etc/init.d/oracleasm
createdisk ASM2 /dev/mapper/oradata2p1
4. For RAC
systems, run the command for every node
[root@racnode1 ~]#
/etc/init.d/oracleasm scandisks
Scanning the system for Oracle
ASMLib disks: [ OK ]
[root@racnode1 ~]#
/etc/init.d/oracleasm listdisks
ASM1
5. Configure
ASMLib for multipath (do the following operations for every node)
Modify /etc/sysconfig/oracleasm
ORACLEASM_SCANORDER=”dm”
ORACLEASM_SCANEXCLUDE=”sd”
Restart asmlib
/etc/init.d/oracleasm stop
/etc/init.d/oracleasm start
6. Check disk
configuration
ASMLib checks the
/proc/partitions to detect the disks:
[root@racnode1 ~]# cat
/proc/partitions | grep dm-
253
0 90963968 dm-0
253
1 51478528 dm-1
253
2 536870912 dm-2
253
3 536870912 dm-3
253
4 536870912 dm-4
253
5 536870912 dm-5
253
6 104857600 dm-6
253
7 536868171 dm-7
253 8
536868171 dm-8
253
9 536868171 dm-9
253
10 536868171 dm-10
253
11 104856223 dm-11
7. ASM disks are
mounted under the /dev/oracleasm/disks
root@racnode1 ~]# ls -la
/dev/oracleasm/disks
brw-rw---- 1 oracle oinstall 253,
10 May 15 12:48 ASM1
brw-rw---- 1 oracle oinstall
253, 8 May 15 12:48 ASM2
8. Major and Minor
numbers of the disks have to be same in both /proc/partitions/ and /dev/oracleasm/disks/
/proc/partitions 253
10 536868171 dm-10
brw-rw---- 1 oracle oinstall 253,
10 May 15 12:48 ASM1
[root@racnode1 ~]# /etc/init.d/oracleasm
querydisk -d ASM1
Disk "ASM1" is a valid ASM disk
on device /dev/dm-10[253,10]
