Oracle 12.2 ASM filter driver installation
Recently I needed to install Oracle 12.2 with the ASM Filter Driver for some research on an upcoming presentation. With the new image based Oracle Grid installation I expected this to be easy. To be fair, it was fairly easy, once all the necessary instructions were assembled in the correct order; that was the hard part. While the documentation is fairly good, it is somewhat difficult to read the docs and have a go at the installation and expect it to all work the first time through. This is a concept that is likely not new to many of you. Not only did it not work on the first attempt, but not on the second, third, fourth... Honestly, I lost track. At that point I decided to start over and re-install. Initially I had intended to do just the software only install. As this is a test environment I am building on VirtualBox, I wanted a VM that I could snapshot a few times for different configurations as needed during testing, and then just configure ASM and the RDBMS for the new test. After failing a few times to get that to work properly, it seemed a good idea to just go through a full installation to find out exactly what was needed to make it all work. I will create the snap-shottable software only environment at some other time now that I know how to make it all work. Some particulars: this is to be an environment using Oracle Restart via Grid Standalone Server. Grid will be owned by Linux user 'grid'. The RDBMS will be owned by Linux user 'oracle'. The initial ASM DiskGroup will be 6 2GiB VBOX disks assigned to the VM. There are 10 disks in all, with 4 held in reserve for some ASM specific tests. Following is the documentation used during this install.
Installing Oracle Grid Infrastructure for a Standalone Server with a New Database Installation
Groups and User accounts
This is the typical list of groups that must be created for oracle accounts on the server. Of particular importance is the 'racdba' group. Identifying an Oracle Software Owner User Account From the previously noted documentation: For Oracle Restart installations, to successfully install Oracle Database, ensure that the grid user is a member of the racdba group The documentation is correct on that point, so for good measure, I added the 'racdba' group to both the 'grid' and 'oracle' accounts. The following steps are all done as root. [code language="bash"] ra12cR2-asm asm-disk-prep]# groupadd racdba [root@ora12cR2-asm asm-disk-prep]# usermod -G dba,asmdba,racdba,asmadmin -a oracle [root@ora12cR2-asm asm-disk-prep]# usermod -g oinstall oracle [root@ora12cR2-asm asm-disk-prep]# id oracle uid=300(oracle) gid=302(oinstall) groups=302(oinstall),301(dba),303(osoper),304(asmdba),306(asmadmin),503(racdba) [root@ora12cR2-asm asm-disk-prep]# usermod -g oinstall grid [root@ora12cR2-asm asm-disk-prep]# usermod -G dba,asmdba,racdba,asmadmin,racdba -a grid [root@ora12cR2-asm asm-disk-prep]# [root@ora12cR2-asm asm-disk-prep]# id grid uid=301(grid) gid=302(oinstall) groups=302(oinstall),301(dba),304(asmdba),305(asmoper),306(asmadmin),503(racdba) [/code] To avoid inflating the size of my VM, the oracle software was located in VBox shared folder. Here's how to mount the shared folder so the installing user has full access: [code language="bash"] # id grid uid=301(grid) gid=302(oinstall) groups=302(oinstall),301(dba),304(asmdba),305(asmoper),306(asmadmin),503(racdba) mount -t vboxsf shared /media/sf_shared -o uid=300,gid=302 [/code] Now login to the grid user and get the Grid image: [code language="bash"] [grid@ora12cR2-asm ~]$ cd /u01/app/oracle/product/12.2.0/grid [grid@ora12cR2-asm grid]$ unzip -q /media/sf_shared/oracle//12.2/V840012-01.zip [grid@ora12cR2-asm grid]$ pwd /u01/app/oracle/product/12.2.0/grid [/code] Back to root now, install the cvuqdisk RPM, and remove ASMLib [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# rpm -Uvh /u01/app/oracle/product/12.2.0/grid/cv/rpm/cvuqdisk-1.0.10-1.rpm Preparing... ########################################### [100%] Using default group oinstall to install package 1:cvuqdisk ########################################### [100%] [root@ora12cR2-asm asm-disk-prep]# rpm -qa | grep asmlib oracleasmlib-2.0.4-1.el5.x86_64 [root@ora12cR2-asm asm-disk-prep]# rpm -e oracleasmlib [/code] The installers will not work without X11 access, this provides it: ( I used the GUI installers and created response files for future use) [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# xhost +local: non-network local connections being added to access control list [/code] Unless you have setup the server name in DNS (I did not for this test environment), then you need to add the server name to the /etc/hosts file. The oracle installer will fail if a lookup on hostname fails. Please substitute your own IP Address and hostname: [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# hostname ora12cR2-asm.jks.com [root@ora12cR2-asm asm-disk-prep]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:84:0a:40 brd ff:ff:ff:ff:ff:ff inet 192.168.1.128/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe84:a40/64 scope link valid_lft forever preferred_lft forever [root@ora12cR2-asm asm-disk-prep]# cat >> /etc/hosts 192.168.1.128 ora12cR2-asm.jks.com ora12cR2-asm [root@ora12cR2-asm asm-disk-prep]# ping -c1 ora12cR2-asm PING ora12cR2-asm.jks.com (192.168.1.128) 56(84) bytes of data. 64 bytes from ora12cR2-asm.jks.com (192.168.1.128): icmp_seq=1 ttl=64 time=0.028 ms --- ora12cR2-asm.jks.com ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.028/0.028/0.028/0.000 ms [/code] The 'parted' utility was used to partition disks. parted is fairly simple to use from a script, and can inform if a partition is correctly aligned on sector boundaries: Please do not run this script as is without first modifying for the correct disks. [code language="bash"] #!/bin/bash # parts.sh # aligned for disk in /dev/sd[b-g] do echo "############# $disk ################" yes | parted $disk -- mklabel msdos parted $disk -- rm 1 parted $disk -- mkpart primary ext4 0% 100% parted $disk align-check optimal 1 done | tee parts.log [/code] The next step is to prep the disks for ASM Filter Driver as root. As before, please do not run this script as is without first modifying for the correct disks. [code language="bash"] #!/bin/bash # setup-disks.sh set -v export ORACLE_HOME=/u01/app/oracle/product/12.2.0/grid export ORACLE_BASE=/tmp cd $ORACLE_HOME/bin : <<'COMMENT' # These commands necessary only if re-attempting to config $ORACLE_HOME/bin/asmcmd afd_deconfigure $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdb1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdc1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdd1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sde1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdf1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdg1 COMMENT $ORACLE_HOME/bin/asmcmd afd_label DISK1 /dev/sdb1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK2 /dev/sdc1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK3 /dev/sdd1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK4 /dev/sde1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK5 /dev/sdf1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK6 /dev/sdg1 --init $ORACLE_HOME/bin/asmcmd afd_lsdsk [/code] Here are a few steps that I found necessary for a successful installation. As root, create directories as shown: [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# chmod g+wrx /u01/app/oracle # Create oraInventory and fix ownership and permissions [root@ora12cR2-asm asm-disk-prep]# mkdir -p /u01/app/oraInventory [root@ora12cR2-asm asm-disk-prep]# chown grid:oinstall /u01/app/oraInventory [root@ora12cR2-asm asm-disk-prep]# chmod g+rwx /u01/app/oraInventory [/code] Check /etc/sysctl.conf for the net.core.wmem_max parameter. If not there, then add it. (backup sysctl.conf first) [code language="bash"] echo net.core.wmem_max = 1048576 >> /etc/sysctl.conf [root@ora12cR2-asm asm-disk-prep]# sysctl -p [/code] Now you are ready for the grid installion. As the grid user, run gridSetup.sh [code language="bash"] [grid@ora12cR2-asm grid]$ pwd /u01/app/oracle/product/12.2.0/grid [grid@ora12cR2-asm grid]$ [grid@ora12cR2-asm grid]$ ./gridSetup.sh & [/code] Once loaded, following are the choices I made during Grid installation:- Set 1M AU
- DATA DiskGroup ( 6 disks as External Redundancy )
- All passwords everywhere set the same
- (it is a test environment without real data or connectivity to other systems)
- Groups for grid:
- OSASM: admadmin
- OSDBA: asmdba
- OSOPER: asmoper
- ORACLE_BASE=/u01/app/oracle
- Do not check 'Automatically run root scripts"
- Personal Preference: I like to run them to see what is happening
RDBMS Install
What good is ASM without a database? Let's install one now. There are several steps that must be run as root before proceeding. Remount the VBOX share as oracle: [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# umount /media/sf_shared [root@ora12cR2-asm asm-disk-prep]# id oracle uid=300(oracle) gid=302(oinstall) groups=302(oinstall),301(dba),303(osoper),304(asmdba),306(asmadmin),503(racdba) [root@ora12cR2-asm asm-disk-prep]# mount -t vboxsf shared /media/sf_shared -o uid=300,gid=302 [/code] Create the Oracle Home directory: [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# mkdir /u01/app/oracle/product/12.2.0/db1 [root@ora12cR2-asm asm-disk-prep]# chown oracle:oinstall /u01/app/oracle/product/12.2.0/db1 [/code] Create new Oracle Inventory location: runInstaller refused to use the current one, even when ownership was changed to oracle. I did not spend anytime trying to learn why this was so, as it was new to me, but just created and new oracle inventory and forged ahead. [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# mkdir /u01/app/oraInventory-db [root@ora12cR2-asm asm-disk-prep]# chown oracle:oinstall /u01/app/oraInventory-db [/code] Create this dump directory now, as the install will fail otherwise This was found in the install log following failed attempts WARNING: Skipping line: [FATAL] [DBT-06608] The specified audit file destination (/u01/app/oracle/admin/asmafd/adump) is not writable. INFO: Read: ACTION: Ensure that the audit file destination (/u01/app/oracle/admin/asmafd/adump) is writable. INFO: ACTION: Ensure that the audit file destination (/u01/app/oracle/admin/asmafd/adump) is writable. As root: [code language="bash"] [root@ora12cR2-asm asm-disk-prep]# mkdir -p /u01/app/oracle/admin/asmafd [root@ora12cR2-asm asm-disk-prep]# chown grid:oinstall /u01/app/oracle/admin/asmafd [root@ora12cR2-asm asm-disk-prep]# chmod g+rwx /u01/app/oracle/admin/asmafd/ [/code] Now you are ready to run the installer as the Oracle user: [code language="bash"] $ /media/sf_shared/oracle/12.2/rdbms/database/runInstaller & [/code] Following are the choices made from the GUI installer:- Create and Configure
- Server class of Advanced
- Enterprise Edition
- starter database
- database name: asmafd.jks.com
- pluggable db:p1
- defaults for memory, etc
- Choose ASM
- Do not check register with EM
- Do not setup Recovery
- Choose DATA DiskGroup
- 'no' to request to setup TFA (do not want extra things running)
