Online resizing of ASM disks

Posted in: Technical Track

 

The SAN administrator has informed us that they have extended the disks. This is the information I had from our client. The disks were labelled:

mpath_compellent_oraarch
mpath_compellent_oraarch02
mpath_compellent_oraarch03

 

The original size of the disks were 300GB and they had been extended to 600GB. These were multipath disks belonging to the disk diskgroup ARCH, which was being used to store archive logs in ASM. The database was 11.2.0.3 and was in a 2-node RAC configuration. The server was Red Hat Linux 5.9 – 2.6.18-406.el5 – 64bit.

I checked the disks using fdisk (as the root user) and got the following:

fdisk -l /dev/mpath/mpath_compellent_oraarch
Disk /dev/mpath/mpath_compellent_oraarch: 322.1 GB, 322122547200 bytes
255 heads, 63 sectors/track, 39162 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/mpath/mpath_compellent_oraarch doesn't contain a valid partition table

This confirmed that the OS was not aware of the size extension.

 

Firstly, I wanted to confirm that the correct disks had been extended. So the first place to look is in ASM:

select g.name
     , d.path
     , d.os_mb
     , d.total_mb
from v$asm_diskgroup g
   , v$asm_disk      d
where g.group_number = d.group_number
and   g.name = 'ARCH'
/
NAME       PATH                           OS_MB   TOTAL_MB
---------- ------------------------- ---------- ----------
ARCH       ORCL:ASMDISK_NEW_ARCH03       307200     307200
ARCH       ORCL:ASMDISK_NEW_ARCH01       307200     307200
ARCH       ORCL:ASMDISK_NEW_ARCH02       307200     307200

 

Now we need to match these names to those provided by the SAN administrator.

Check the directory:

/dev/oracleasm/disks
ls -l /dev/oracleasm/disks/ASMDISK_NEW_ARCH*
brw-rw---- 1 oracle dba 253, 30 Oct  6 00:35 /dev/oracleasm/disks/ASMDISK_NEW_ARCH01
brw-rw---- 1 oracle dba 253, 29 Oct  6 00:35 /dev/oracleasm/disks/ASMDISK_NEW_ARCH02
brw-rw---- 1 oracle dba 253, 32 Oct  6 00:35 /dev/oracleasm/disks/ASMDISK_NEW_ARCH03

This gives is the major and minor numbers for the disks – major number is 253 and minor numbers are 30,29 and 32.

 

Then compare these numbers against the devices listed in:

/dev/mapper
ls -l /dev/mapper/mpath_compellent_oraarch*
brw-rw---- 1 root disk 253, 30 Oct  6 00:34 /dev/mapper/mpath_compellent_oraarch
brw-rw---- 1 root disk 253, 29 Oct  6 00:34 /dev/mapper/mpath_compellent_oraarch02
brw-rw---- 1 root disk 253, 32 Oct  6 00:34 /dev/mapper/mpath_compellent_oraarch03

The numbers match showing that they are the same devices.

 

Now we need to find the actual disks that make up the multipath devices.

multipath -l
Output truncated for brevity
mpath_compellent_oraarch03 (36000d310009aa700000000000000002b) dm-32 COMPELNT,Compellent Vol
[size=300G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 13:0:0:3  sdfm       130:128 [active][undef]
 \_ 11:0:0:3  sdgd       131:144 [active][undef]
mpath_compellent_oraarch02 (36000d310009aa700000000000000002a) dm-29 COMPELNT,Compellent Vol
[size=300G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 12:0:0:2  sdfi       130:64  [active][undef]
 \_ 14:0:0:2  sdfk       130:96  [active][undef]
mpath_compellent_oraarch (36000d310009aa7000000000000000026) dm-30 COMPELNT,Compellent Vol
[size=300G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 13:0:0:1  sdfj       130:80  [active][undef]
 \_ 11:0:0:1  sdgc       131:128 [active][undef]

From here we can see the disks:

/dev/sdfm
/dev/sdgd
/dev/sdfi
/dev/sdfk
/dev/sdfj
/dev/sdgc

We need to find this information on the other node as well, as the underlying disk names will very likely be different on the other server.

 

Now for each disk we need to rescan the disk to register the new size. To do this we need to the following for each disk on both nodes:

echo 1 > /sys/block/sdfm/device/rescan

Then we can check each disk to make sure it has successfully been extended:

fdisk -l /dev/sdfm
Disk /dev/sdfm: 644.2 GB, 644245094400 bytes
255 heads, 63 sectors/track, 78325 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdfm doesn't contain a valid partition table

Looks good – once done on all nodes we can then extend the multipath devices for each device name on both nodes:

multipathd -k'resize map mpath_compellent_oraarch'
ok

Then we can check the multipath device disk size:

fdisk -l /dev/mpath/mpath_compellent_oraarch
Disk /dev/mpath/mpath_compellent_oraarch: 644.2 GB, 644245094400 bytes
255 heads, 63 sectors/track, 78325 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/mpath/mpath_compellent_oraarch doesn't contain a valid partition table

Looks good – once done on both nodes we can then resize the ASM disks within ASM:

SQL> select g.name
     , d.path
     , d.os_mb
     , d.total_mb
from v$asm_diskgroup g
   , v$asm_disk      d
where g.group_number = d.group_number
and   g.name = 'ARCH'
/
NAME       PATH                           OS_MB   TOTAL_MB
---------- ------------------------- ---------- ----------
ARCH       ORCL:ASMDISK_NEW_ARCH03       614400     307200
ARCH       ORCL:ASMDISK_NEW_ARCH01       614400     307200
ARCH       ORCL:ASMDISK_NEW_ARCH02       614400     307200
SQL> alter diskgroup ARCH resize all;
Diskgroup altered.
SQL> select g.name
     , d.path
     , d.os_mb
     , d.total_mb
from v$asm_diskgroup g
   , v$asm_disk      d
where g.group_number = d.group_number
and   g.name = 'ARCH'
/
NAME       PATH                           OS_MB   TOTAL_MB
---------- ------------------------- ---------- ----------
ARCH       ORCL:ASMDISK_NEW_ARCH03       614400     614400
ARCH       ORCL:ASMDISK_NEW_ARCH01       614400     614400
ARCH       ORCL:ASMDISK_NEW_ARCH02       614400     614400

The disks and diskgroup were successfully resized.

 

Discover more about our expertise in Database Management.

email

Author

Want to talk with an expert? Schedule a call with our team to get the conversation started.

About the Author

I have been working as a DBA since Oracle V6 for over 20 years.

11 Comments. Leave new

HI Luke,
The article is very clear and I did all steps in our Oracle 2 node RAC but I still see the size of volume in ASM.
I must say that I noticed there is a partition in the volume looks like this, as well as each disk in the mpath for example for mpathd there is mpathdp1 and for that mpath each disk /dev/sdc have /dev/sdc1.

should I resize the partitions sdc1 and so on for each partition in the mpathd before the ASM can see the new size?
Can I do it live or should I shutdown the DB for the partition resize?

See below example –

fdisk -l /dev/mapper/mpathd

Disk /dev/mapper/mpathd: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8

Device Boot Start End Blocks Id System
/dev/mapper/mpathdp1 22 1295173 976543744 83 Linux

Reply

Hi Steve

What was the size prior to extending and what size are you extending to?
Does the /dev/mapper/mpathd now show the expected extended size?
Can you post the following?
Relevant multipath -l output to show the underlying disks
The fdisk -l output for those disks
The output from SQL given stating the os_mb and total_mb.

Then maybe I can see what has happened.

Reply

Hi Luke

Yes the /dev/mapper/mpathd show me the new size after I extended the LUN in the storage.
It was from 1TB to 2TB LUN resize.
below is the multipath -l output –

mpathd (3600c0ff0002795675418855701000000) dm-3 HP,MSA 2040 SAN
size=1.8T features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’round-robin 0′ prio=0 status=active
| `- 1:0:0:31 sdc 8:32 active undef running
|-+- policy=’round-robin 0′ prio=0 status=enabled
| `- 2:0:0:31 sdo 8:224 active undef running
|-+- policy=’round-robin 0′ prio=0 status=enabled
| `- 1:0:1:31 sdi 8:128 active undef running
`-+- policy=’round-robin 0′ prio=0 status=enabled
`- 2:0:1:31 sdu 65:64 active undef running

fdisk -l output –

[root@db01 ~]# fdisk -l /dev/sdc

Disk /dev/sdc: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8

Device Boot Start End Blocks Id System
/dev/sdc1 22 1295173 976543744 83 Linux

[root@db01 ~]# fdisk -l /dev/sdo

Disk /dev/sdo: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8

Device Boot Start End Blocks Id System
/dev/sdo1 22 1295173 976543744 83 Linux

[root@db01 ~]# fdisk -l /dev/sdi

Disk /dev/sdi: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8

Device Boot Start End Blocks Id System
/dev/sdi1 22 1295173 976543744 83 Linux

[root@db01 ~]# fdisk -l /dev/sdu

Disk /dev/sdu: 2000.0 GB, 1999995142144 bytes
29 heads, 52 sectors/track, 2590345 cylinders
Units = cylinders of 1508 * 512 = 772096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disk identifier: 0x6e81b0c8

Device Boot Start End Blocks Id System
/dev/sdu1 22 1295173 976543744 83 Linux

SQL Output –

SQL> select g.name
, d.path
, d.os_mb
, d.total_mb
from v$asm_diskgroup g
, v$asm_disk d
where g.group_number = d.group_number
and g.name = ‘DATA’
/ 2 3 4 5 6 7 8 9

NAME
——————————
PATH
——————————————————————————–
OS_MB TOTAL_MB
———- ———-
DATA
ORCL:MPATHD
953656 953656

Thanks

Reply

Hmm – seems like the disks all show the extension but ASM is not picking it up.
Is this part of a RAC system?
Have you done the required changes on all other nodes?

Reply

Hi Steve

I had a look around and see that for your disks you have the new cylinders (2590345) showing for the disk but for the partition the old number of cylinders -> 1295173.
It seems you have to recreate partition 1.
There are some instructions as per this blog -> https://jarneil.wordpress.com/2008/04/22/expanding-an-asm-disk/
This blog suggests that you may have to take down ASM for this to write the partition table.
Good luck and let me know.
Thanks
Luke

Reply

Yes it is RAC with 2 nodes and I made the changes on both nodes.
What do you think about the fact there is a partition in the disks, could that be the issue?
I may have to resize the partition…

Thanks,

Reply

We are thinking the same thing :)

See previous comment.

Reply

Thanks Luke, I’ll update you once done resizing the partitions.

Reply

Hi Luke, just want to update you that ASM volumes resize finished successfully.
Thanks for the support, Steve.

Reply

HI Luke,

First of all thanks for all the information.
I got a problem. I followed all your instructions, and also I’m able to see space extended on OS_MB. When I try to run alter diskgroup command, it says SQL Error: ORA-15000: command disallowed by current instance type. I have sysdba credentials. How to connect to an ASM instance. Pls advice.

Reply

Try SYSASM rather than SYSDBA

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *