Oracle 12.2 ASM Filter Driver Installation

Posted in: DBA Lounge, Oracle, Technical Track

Recently I needed to install Oracle 12.2 with the ASM Filter Driver for some research on an upcoming presentation. With the new image based Oracle Grid installation I expected this to be easy.

To be fair, it was fairly easy, once all the necessary instructions were assembled in the correct order; that was the hard part.

While the documentation is fairly good, it is somewhat difficult to read the docs and have a go at the installation and expect it to all work the first time through. This is a concept that is likely not new to many of you.

Not only did it not work on the first attempt, but not on the second, third, fourth… Honestly, I lost track. At that point I decided to start over and re-install. Initially I had intended to do just the software only install. As this is a test environment I am building on VirtualBox, I wanted a VM that I could snapshot a few times for different configurations as needed during testing, and then just configure ASM and the RDBMS for the new test.

After failing a few times to get that to work properly, it seemed a good idea to just go through a full installation to find out exactly what was needed to make it all work.

I will create the snap-shottable software only environment at some other time now that I know how to make it all work.

Some particulars: this is to be an environment using Oracle Restart via Grid Standalone Server. Grid will be owned by Linux user ‘grid’.

The RDBMS will be owned by Linux user ‘oracle’.

The initial ASM DiskGroup will be 6 2GiB VBOX disks assigned to the VM. There are 10 disks in all, with 4 held in reserve for some ASM specific tests. Following is the documentation used during this install.

Installing Oracle Grid Infrastructure for a Standalone Server with a New Database Installation

Groups and User accounts

This is the typical list of groups that must be created for oracle accounts on the server. Of particular importance is the ‘racdba’ group.

Identifying an Oracle Software Owner User Account

From the previously noted documentation: For Oracle Restart installations, to successfully install Oracle Database, ensure that the grid user is a member of the racdba group

The documentation is correct on that point, so for good measure, I added the ‘racdba’ group to both the ‘grid’ and ‘oracle’ accounts.

The following steps are all done as root.

 

ra12cR2-asm asm-disk-prep]# groupadd racdba
[root@ora12cR2-asm asm-disk-prep]# usermod -G  dba,asmdba,racdba,asmadmin -a oracle
[root@ora12cR2-asm asm-disk-prep]# usermod -g oinstall oracle 
[root@ora12cR2-asm asm-disk-prep]# id oracle
uid=300(oracle) gid=302(oinstall) groups=302(oinstall),301(dba),303(osoper),304(asmdba),306(asmadmin),503(racdba)

[root@ora12cR2-asm asm-disk-prep]# usermod -g oinstall grid
[root@ora12cR2-asm asm-disk-prep]# usermod -G  dba,asmdba,racdba,asmadmin,racdba -a grid
[root@ora12cR2-asm asm-disk-prep]# 
[root@ora12cR2-asm asm-disk-prep]# id grid
uid=301(grid) gid=302(oinstall) groups=302(oinstall),301(dba),304(asmdba),305(asmoper),306(asmadmin),503(racdba)

To avoid inflating the size of my VM, the oracle software was located in VBox shared folder. Here’s how to mount the shared folder so the installing user has full access:

 

# id grid 
 uid=301(grid) gid=302(oinstall) groups=302(oinstall),301(dba),304(asmdba),305(asmoper),306(asmadmin),503(racdba)

mount -t vboxsf shared /media/sf_shared -o uid=300,gid=302

Now login to the grid user and get the Grid image:

 

[grid@ora12cR2-asm ~]$ cd /u01/app/oracle/product/12.2.0/grid
[grid@ora12cR2-asm grid]$ unzip -q  /media/sf_shared/oracle//12.2/V840012-01.zip
[grid@ora12cR2-asm grid]$ pwd
/u01/app/oracle/product/12.2.0/grid

Back to root now, install the cvuqdisk RPM, and remove ASMLib

 

[root@ora12cR2-asm asm-disk-prep]# rpm -Uvh /u01/app/oracle/product/12.2.0/grid/cv/rpm/cvuqdisk-1.0.10-1.rpm
Preparing...                ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]

[root@ora12cR2-asm asm-disk-prep]# rpm -qa | grep asmlib
oracleasmlib-2.0.4-1.el5.x86_64
[root@ora12cR2-asm asm-disk-prep]# rpm -e oracleasmlib

The installers will not work without X11 access, this provides it:
( I used the GUI installers and created response files for future use)

 

[root@ora12cR2-asm asm-disk-prep]# xhost +local:
non-network local connections being added to access control list

Unless you have setup the server name in DNS (I did not for this test environment), then you need to add the server name to the /etc/hosts file. The oracle installer will fail if a lookup on hostname fails.

Please substitute your own IP Address and hostname:

 

[root@ora12cR2-asm asm-disk-prep]# hostname
ora12cR2-asm.jks.com


[root@ora12cR2-asm asm-disk-prep]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:84:0a:40 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.128/24 brd 192.168.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe84:a40/64 scope link 
       valid_lft forever preferred_lft forever

[root@ora12cR2-asm asm-disk-prep]# cat >> /etc/hosts
192.168.1.128   ora12cR2-asm.jks.com ora12cR2-asm

[root@ora12cR2-asm asm-disk-prep]# ping -c1 ora12cR2-asm
PING ora12cR2-asm.jks.com (192.168.1.128) 56(84) bytes of data.
64 bytes from ora12cR2-asm.jks.com (192.168.1.128): icmp_seq=1 ttl=64 time=0.028 ms

--- ora12cR2-asm.jks.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.028/0.028/0.028/0.000 ms

The ‘parted’ utility was used to partition disks. parted is fairly simple to use from a script, and can inform if a partition is correctly aligned on sector boundaries:

Please do not run this script as is without first modifying for the correct disks.

 

#!/bin/bash

# parts.sh

# aligned
for disk in /dev/sd[b-g]
do

        echo "#############  $disk ################"

        yes | parted $disk -- mklabel msdos  
        parted $disk -- rm 1
        parted $disk -- mkpart primary ext4 0% 100%
        parted $disk align-check optimal 1

done | tee parts.log

The next step is to prep the disks for ASM Filter Driver as root.
As before, please do not run this script as is without first modifying for the correct disks.

 

#!/bin/bash

# setup-disks.sh

set -v

export ORACLE_HOME=/u01/app/oracle/product/12.2.0/grid
export ORACLE_BASE=/tmp

cd $ORACLE_HOME/bin

: <<'COMMENT'

# These commands necessary only if re-attempting to config

$ORACLE_HOME/bin/asmcmd afd_deconfigure

$ORACLE_HOME/bin/asmcmd afd_unlabel  /dev/sdb1
$ORACLE_HOME/bin/asmcmd afd_unlabel  /dev/sdc1
$ORACLE_HOME/bin/asmcmd afd_unlabel  /dev/sdd1
$ORACLE_HOME/bin/asmcmd afd_unlabel  /dev/sde1
$ORACLE_HOME/bin/asmcmd afd_unlabel  /dev/sdf1
$ORACLE_HOME/bin/asmcmd afd_unlabel  /dev/sdg1

COMMENT

$ORACLE_HOME/bin/asmcmd afd_label DISK1 /dev/sdb1 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK2 /dev/sdc1 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK3 /dev/sdd1 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK4 /dev/sde1 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK5 /dev/sdf1 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK6 /dev/sdg1 --init

$ORACLE_HOME/bin/asmcmd afd_lsdsk

Here are a few steps that I found necessary for a successful installation. As root, create directories as shown:

 

[root@ora12cR2-asm asm-disk-prep]# chmod g+wrx /u01/app/oracle

# Create oraInventory and fix ownership and permissions

[root@ora12cR2-asm asm-disk-prep]# mkdir -p /u01/app/oraInventory
[root@ora12cR2-asm asm-disk-prep]# chown grid:oinstall /u01/app/oraInventory
[root@ora12cR2-asm asm-disk-prep]# chmod g+rwx /u01/app/oraInventory

Check /etc/sysctl.conf for the net.core.wmem_max parameter. If not there, then add it.
(backup sysctl.conf first)

 

echo net.core.wmem_max = 1048576 >> /etc/sysctl.conf

[root@ora12cR2-asm asm-disk-prep]# sysctl -p

Now you are ready for the grid installion. As the grid user, run gridSetup.sh

 

[grid@ora12cR2-asm grid]$ pwd
/u01/app/oracle/product/12.2.0/grid
[grid@ora12cR2-asm grid]$ 
[grid@ora12cR2-asm grid]$ ./gridSetup.sh &

Once loaded, following are the choices I made during Grid installation:

  • Set 1M AU
  • DATA DiskGroup ( 6 disks as External Redundancy )
  • All passwords everywhere set the same
  • (it is a test environment without real data or connectivity to other systems)
  • Groups for grid:
  • OSASM: admadmin
  • OSDBA: asmdba
  • OSOPER: asmoper
  • ORACLE_BASE=/u01/app/oracle
  • Do not check ‘Automatically run root scripts”
  • Personal Preference: I like to run them to see what is happening

Success!


[root@ora12cR2-asm asm-disk-prep]# . oraenv <<< +ASM
ORACLE_SID = [root] ? The Oracle base has been changed from /tmp to /u01/app/oracle
[root@ora12cR2-asm asm-disk-prep]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       ora12cr2-asm             STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       ora12cr2-asm             STABLE
ora.asm
               ONLINE  ONLINE       ora12cr2-asm             Started,STABLE
ora.ons
               OFFLINE OFFLINE      ora12cr2-asm             STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       ora12cr2-asm             STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.driver.afd
      1        ONLINE  ONLINE       ora12cr2-asm             STABLE
ora.evmd
      1        ONLINE  ONLINE       ora12cr2-asm             STABLE
--------------------------------------------------------------------------------

RDBMS Install

What good is ASM without a database? Let’s install one now.

There are several steps that must be run as root before proceeding.

Remount the VBOX share as oracle:

 

[root@ora12cR2-asm asm-disk-prep]# umount /media/sf_shared
[root@ora12cR2-asm asm-disk-prep]# id oracle
  uid=300(oracle) gid=302(oinstall) groups=302(oinstall),301(dba),303(osoper),304(asmdba),306(asmadmin),503(racdba)
[root@ora12cR2-asm asm-disk-prep]# mount -t vboxsf shared /media/sf_shared -o uid=300,gid=302

Create the Oracle Home directory:

 

[root@ora12cR2-asm asm-disk-prep]# mkdir /u01/app/oracle/product/12.2.0/db1
[root@ora12cR2-asm asm-disk-prep]# chown oracle:oinstall /u01/app/oracle/product/12.2.0/db1

Create new Oracle Inventory location:
runInstaller refused to use the current one, even when ownership was changed to oracle. I did not spend anytime trying to learn why this was so, as it was new to me, but just created and new oracle inventory and forged ahead.

 
[root@ora12cR2-asm asm-disk-prep]# mkdir /u01/app/oraInventory-db
[root@ora12cR2-asm asm-disk-prep]# chown oracle:oinstall /u01/app/oraInventory-db

Create this dump directory now, as the install will fail otherwise

This was found in the install log following failed attempts

WARNING: Skipping line: [FATAL] [DBT-06608] The specified audit file destination (/u01/app/oracle/admin/asmafd/adump) is not writable.
INFO: Read: ACTION: Ensure that the audit file destination (/u01/app/oracle/admin/asmafd/adump) is writable.
INFO: ACTION: Ensure that the audit file destination (/u01/app/oracle/admin/asmafd/adump) is writable.

As root:

 

[root@ora12cR2-asm asm-disk-prep]# mkdir -p /u01/app/oracle/admin/asmafd
[root@ora12cR2-asm asm-disk-prep]# chown grid:oinstall /u01/app/oracle/admin/asmafd
[root@ora12cR2-asm asm-disk-prep]# chmod g+rwx  /u01/app/oracle/admin/asmafd/

Now you are ready to run the installer as the Oracle user:

 

$ /media/sf_shared/oracle/12.2/rdbms/database/runInstaller &

Following are the choices made from the GUI installer:

  • Create and Configure
  • Server class of Advanced
  • Enterprise Edition
  • starter database
  • database name: asmafd.jks.com
  • pluggable db:p1
  • defaults for memory, etc
  • Choose ASM
  • Do not check register with EM
  • Do not setup Recovery
  • Choose DATA DiskGroup
  • ‘no’ to request to setup TFA (do not want extra things running)

Success!

 

[oracle@ora12cR2-asm ~]$ srvctl status database -db asmafd
Database is running.

If you just want to get ASM Filter Driver up and running with database for some exploratory testing, this article should save you a large chunk of time spent poring over the manuals and doing test runs to find out what breaks.

So, what are you waiting for? Please leave comments if you find this useful or find any errors or omissions.

Please, also leave comments if you discover other issues while installing the software.

email

Interested in working with Jared? Schedule a tech call.

About the Author

Oracle experience: started with Oracle 7.0.13
Programming Experience: Perl, PL/SQL, Shell, SQL Also some other odds and ends that are no longer useful
Systems: Networking, Storage, OS to varying degrees.
Have fond memories of DG/UX

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *