The SAN administrator has informed us that they have extended the disks. This is the information I had from our client. The disks were labelled:
mpath_compellent_oraarch mpath_compellent_oraarch02 mpath_compellent_oraarch03
The original size of the disks were 300GB and they had been extended to 600GB. These were multipath disks belonging to the disk diskgroup ARCH, which was being used to store archive logs in ASM. The database was 11.2.0.3 and was in a 2-node RAC configuration. The server was Red Hat Linux 5.9 – 2.6.18-406.el5 – 64bit.
I checked the disks using fdisk (as the root user) and got the following:
fdisk -l /dev/mpath/mpath_compellent_oraarch Disk /dev/mpath/mpath_compellent_oraarch: 322.1 GB, 322122547200 bytes 255 heads, 63 sectors/track, 39162 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/mpath/mpath_compellent_oraarch doesn't contain a valid partition table
This confirmed that the OS was not aware of the size extension.
Firstly, I wanted to confirm that the correct disks had been extended. So the first place to look is in ASM:
select g.name , d.path , d.os_mb , d.total_mb from v$asm_diskgroup g , v$asm_disk d where g.group_number = d.group_number and g.name = 'ARCH' / NAME PATH OS_MB TOTAL_MB ---------- ------------------------- ---------- ---------- ARCH ORCL:ASMDISK_NEW_ARCH03 307200 307200 ARCH ORCL:ASMDISK_NEW_ARCH01 307200 307200 ARCH ORCL:ASMDISK_NEW_ARCH02 307200 307200
Now we need to match these names to those provided by the SAN administrator.
Check the directory:
/dev/oracleasm/disks
ls -l /dev/oracleasm/disks/ASMDISK_NEW_ARCH* brw-rw---- 1 oracle dba 253, 30 Oct 6 00:35 /dev/oracleasm/disks/ASMDISK_NEW_ARCH01 brw-rw---- 1 oracle dba 253, 29 Oct 6 00:35 /dev/oracleasm/disks/ASMDISK_NEW_ARCH02 brw-rw---- 1 oracle dba 253, 32 Oct 6 00:35 /dev/oracleasm/disks/ASMDISK_NEW_ARCH03
This gives is the major and minor numbers for the disks – major number is 253 and minor numbers are 30,29 and 32.
Then compare these numbers against the devices listed in:
/dev/mapper
ls -l /dev/mapper/mpath_compellent_oraarch* brw-rw---- 1 root disk 253, 30 Oct 6 00:34 /dev/mapper/mpath_compellent_oraarch brw-rw---- 1 root disk 253, 29 Oct 6 00:34 /dev/mapper/mpath_compellent_oraarch02 brw-rw---- 1 root disk 253, 32 Oct 6 00:34 /dev/mapper/mpath_compellent_oraarch03
The numbers match showing that they are the same devices.
Now we need to find the actual disks that make up the multipath devices.
multipath -l Output truncated for brevity mpath_compellent_oraarch03 (36000d310009aa700000000000000002b) dm-32 COMPELNT,Compellent Vol [size=300G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=0][active] \_ 13:0:0:3 sdfm 130:128 [active][undef] \_ 11:0:0:3 sdgd 131:144 [active][undef] mpath_compellent_oraarch02 (36000d310009aa700000000000000002a) dm-29 COMPELNT,Compellent Vol [size=300G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=0][active] \_ 12:0:0:2 sdfi 130:64 [active][undef] \_ 14:0:0:2 sdfk 130:96 [active][undef] mpath_compellent_oraarch (36000d310009aa7000000000000000026) dm-30 COMPELNT,Compellent Vol [size=300G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=0][active] \_ 13:0:0:1 sdfj 130:80 [active][undef] \_ 11:0:0:1 sdgc 131:128 [active][undef]
From here we can see the disks:
/dev/sdfm /dev/sdgd /dev/sdfi /dev/sdfk /dev/sdfj /dev/sdgc
We need to find this information on the other node as well, as the underlying disk names will very likely be different on the other server.
Now for each disk we need to rescan the disk to register the new size. To do this we need to the following for each disk on both nodes:
echo 1 > /sys/block/sdfm/device/rescan
Then we can check each disk to make sure it has successfully been extended:
fdisk -l /dev/sdfm Disk /dev/sdfm: 644.2 GB, 644245094400 bytes 255 heads, 63 sectors/track, 78325 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdfm doesn't contain a valid partition table
Looks good – once done on all nodes we can then extend the multipath devices for each device name on both nodes:
multipathd -k'resize map mpath_compellent_oraarch' ok
Then we can check the multipath device disk size:
fdisk -l /dev/mpath/mpath_compellent_oraarch Disk /dev/mpath/mpath_compellent_oraarch: 644.2 GB, 644245094400 bytes 255 heads, 63 sectors/track, 78325 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/mpath/mpath_compellent_oraarch doesn't contain a valid partition table
Looks good – once done on both nodes we can then resize the ASM disks within ASM:
SQL> select g.name , d.path , d.os_mb , d.total_mb from v$asm_diskgroup g , v$asm_disk d where g.group_number = d.group_number and g.name = 'ARCH' / NAME PATH OS_MB TOTAL_MB ---------- ------------------------- ---------- ---------- ARCH ORCL:ASMDISK_NEW_ARCH03 614400 307200 ARCH ORCL:ASMDISK_NEW_ARCH01 614400 307200 ARCH ORCL:ASMDISK_NEW_ARCH02 614400 307200 SQL> alter diskgroup ARCH resize all; Diskgroup altered. SQL> select g.name , d.path , d.os_mb , d.total_mb from v$asm_diskgroup g , v$asm_disk d where g.group_number = d.group_number and g.name = 'ARCH' / NAME PATH OS_MB TOTAL_MB ---------- ------------------------- ---------- ---------- ARCH ORCL:ASMDISK_NEW_ARCH03 614400 614400 ARCH ORCL:ASMDISK_NEW_ARCH01 614400 614400 ARCH ORCL:ASMDISK_NEW_ARCH02 614400 614400
The disks and diskgroup were successfully resized.
Discover more about our expertise in Database Management.
11 Comments. Leave new
HI Luke,
The article is very clear and I did all steps in our Oracle 2 node RAC but I still see the size of volume in ASM.
I must say that I noticed there is a partition in the volume looks like this, as well as each disk in the mpath for example for mpathd there is mpathdp1 and for that mpath each disk /dev/sdc have /dev/sdc1.
should I resize the partitions sdc1 and so on for each partition in the mpathd before the ASM can see the new size?
Can I do it live or should I shutdown the DB for the partition resize?
See below example –
fdisk -l /dev/mapper/mpathd
Disk /dev/mapper/mpathd: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8
Device Boot Start End Blocks Id System
/dev/mapper/mpathdp1 22 1295173 976543744 83 Linux
Hi Steve
What was the size prior to extending and what size are you extending to?
Does the /dev/mapper/mpathd now show the expected extended size?
Can you post the following?
Relevant multipath -l output to show the underlying disks
The fdisk -l output for those disks
The output from SQL given stating the os_mb and total_mb.
Then maybe I can see what has happened.
Hi Luke
Yes the /dev/mapper/mpathd show me the new size after I extended the LUN in the storage.
It was from 1TB to 2TB LUN resize.
below is the multipath -l output –
mpathd (3600c0ff0002795675418855701000000) dm-3 HP,MSA 2040 SAN
size=1.8T features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’round-robin 0′ prio=0 status=active
| `- 1:0:0:31 sdc 8:32 active undef running
|-+- policy=’round-robin 0′ prio=0 status=enabled
| `- 2:0:0:31 sdo 8:224 active undef running
|-+- policy=’round-robin 0′ prio=0 status=enabled
| `- 1:0:1:31 sdi 8:128 active undef running
`-+- policy=’round-robin 0′ prio=0 status=enabled
`- 2:0:1:31 sdu 65:64 active undef running
fdisk -l output –
[[email protected] ~]# fdisk -l /dev/sdc
Disk /dev/sdc: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8
Device Boot Start End Blocks Id System
/dev/sdc1 22 1295173 976543744 83 Linux
[[email protected] ~]# fdisk -l /dev/sdo
Disk /dev/sdo: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8
Device Boot Start End Blocks Id System
/dev/sdo1 22 1295173 976543744 83 Linux
[[email protected] ~]# fdisk -l /dev/sdi
Disk /dev/sdi: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8
Device Boot Start End Blocks Id System
/dev/sdi1 22 1295173 976543744 83 Linux
[[email protected] ~]# fdisk -l /dev/sdu
Disk /dev/sdu: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8
Device Boot Start End Blocks Id System
/dev/sdu1 22 1295173 976543744 83 Linux
SQL Output –
SQL> select g.name
, d.path
, d.os_mb
, d.total_mb
from v$asm_diskgroup g
, v$asm_disk d
where g.group_number = d.group_number
and g.name = ‘DATA’
/ 2 3 4 5 6 7 8 9
NAME
——————————
PATH
——————————————————————————–
OS_MB TOTAL_MB
———- ———-
DATA
ORCL:MPATHD
953656 953656
Thanks
Hmm – seems like the disks all show the extension but ASM is not picking it up.
Is this part of a RAC system?
Have you done the required changes on all other nodes?
Hi Steve
I had a look around and see that for your disks you have the new cylinders (2590345) showing for the disk but for the partition the old number of cylinders -> 1295173.
It seems you have to recreate partition 1.
There are some instructions as per this blog -> https://jarneil.wordpress.com/2008/04/22/expanding-an-asm-disk/
This blog suggests that you may have to take down ASM for this to write the partition table.
Good luck and let me know.
Thanks
Luke
Yes it is RAC with 2 nodes and I made the changes on both nodes.
What do you think about the fact there is a partition in the disks, could that be the issue?
I may have to resize the partition…
Thanks,
We are thinking the same thing :)
See previous comment.
Thanks Luke, I’ll update you once done resizing the partitions.
Hi Luke, just want to update you that ASM volumes resize finished successfully.
Thanks for the support, Steve.
HI Luke,
First of all thanks for all the information.
I got a problem. I followed all your instructions, and also I’m able to see space extended on OS_MB. When I try to run alter diskgroup command, it says SQL Error: ORA-15000: command disallowed by current instance type. I have sysdba credentials. How to connect to an ASM instance. Pls advice.
Try SYSASM rather than SYSDBA