Recently I needed to install Oracle 12.2 with the ASM Filter Driver for some research on an upcoming presentation. With the new image based Oracle Grid installation I expected this to be easy.
To be fair, it was fairly easy, once all the necessary instructions were assembled in the correct order; that was the hard part.
While the documentation is fairly good, it is somewhat difficult to read the docs and have a go at the installation and expect it to all work the first time through. This is a concept that is likely not new to many of you.
Not only did it not work on the first attempt, but not on the second, third, fourth… Honestly, I lost track. At that point I decided to start over and re-install. Initially I had intended to do just the software only install. As this is a test environment I am building on VirtualBox, I wanted a VM that I could snapshot a few times for different configurations as needed during testing, and then just configure ASM and the RDBMS for the new test.
After failing a few times to get that to work properly, it seemed a good idea to just go through a full installation to find out exactly what was needed to make it all work.
I will create the snap-shottable software only environment at some other time now that I know how to make it all work.
Some particulars: this is to be an environment using Oracle Restart via Grid Standalone Server. Grid will be owned by Linux user ‘grid’.
The RDBMS will be owned by Linux user ‘oracle’.
The initial ASM DiskGroup will be 6 2GiB VBOX disks assigned to the VM. There are 10 disks in all, with 4 held in reserve for some ASM specific tests. Following is the documentation used during this install.
Installing Oracle Grid Infrastructure for a Standalone Server with a New Database Installation
Groups and User accounts
This is the typical list of groups that must be created for oracle accounts on the server. Of particular importance is the ‘racdba’ group.
Identifying an Oracle Software Owner User Account
From the previously noted documentation: For Oracle Restart installations, to successfully install Oracle Database, ensure that the grid user is a member of the racdba group
The documentation is correct on that point, so for good measure, I added the ‘racdba’ group to both the ‘grid’ and ‘oracle’ accounts.
The following steps are all done as root.
ra12cR2-asm asm-disk-prep]# groupadd racdba [[email protected] asm-disk-prep]# usermod -G dba,asmdba,racdba,asmadmin -a oracle [[email protected] asm-disk-prep]# usermod -g oinstall oracle [[email protected] asm-disk-prep]# id oracle uid=300(oracle) gid=302(oinstall) groups=302(oinstall),301(dba),303(osoper),304(asmdba),306(asmadmin),503(racdba) [[email protected] asm-disk-prep]# usermod -g oinstall grid [[email protected] asm-disk-prep]# usermod -G dba,asmdba,racdba,asmadmin,racdba -a grid [[email protected] asm-disk-prep]# [[email protected] asm-disk-prep]# id grid uid=301(grid) gid=302(oinstall) groups=302(oinstall),301(dba),304(asmdba),305(asmoper),306(asmadmin),503(racdba)
To avoid inflating the size of my VM, the oracle software was located in VBox shared folder. Here’s how to mount the shared folder so the installing user has full access:
# id grid uid=301(grid) gid=302(oinstall) groups=302(oinstall),301(dba),304(asmdba),305(asmoper),306(asmadmin),503(racdba) mount -t vboxsf shared /media/sf_shared -o uid=300,gid=302
Now login to the grid user and get the Grid image:
[[email protected] ~]$ cd /u01/app/oracle/product/12.2.0/grid [[email protected] grid]$ unzip -q /media/sf_shared/oracle//12.2/V840012-01.zip [[email protected] grid]$ pwd /u01/app/oracle/product/12.2.0/grid
Back to root now, install the cvuqdisk RPM, and remove ASMLib
[[email protected] asm-disk-prep]# rpm -Uvh /u01/app/oracle/product/12.2.0/grid/cv/rpm/cvuqdisk-1.0.10-1.rpm Preparing... ########################################### [100%] Using default group oinstall to install package 1:cvuqdisk ########################################### [100%] [[email protected] asm-disk-prep]# rpm -qa | grep asmlib oracleasmlib-2.0.4-1.el5.x86_64 [[email protected] asm-disk-prep]# rpm -e oracleasmlib
The installers will not work without X11 access, this provides it:
( I used the GUI installers and created response files for future use)
[[email protected] asm-disk-prep]# xhost +local: non-network local connections being added to access control list
Unless you have setup the server name in DNS (I did not for this test environment), then you need to add the server name to the /etc/hosts file. The oracle installer will fail if a lookup on hostname fails.
Please substitute your own IP Address and hostname:
[[email protected] asm-disk-prep]# hostname ora12cR2-asm.jks.com [[email protected] asm-disk-prep]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:84:0a:40 brd ff:ff:ff:ff:ff:ff inet 192.168.1.128/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe84:a40/64 scope link valid_lft forever preferred_lft forever [[email protected] asm-disk-prep]# cat >> /etc/hosts 192.168.1.128 ora12cR2-asm.jks.com ora12cR2-asm [[email protected] asm-disk-prep]# ping -c1 ora12cR2-asm PING ora12cR2-asm.jks.com (192.168.1.128) 56(84) bytes of data. 64 bytes from ora12cR2-asm.jks.com (192.168.1.128): icmp_seq=1 ttl=64 time=0.028 ms --- ora12cR2-asm.jks.com ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.028/0.028/0.028/0.000 ms
The ‘parted’ utility was used to partition disks. parted is fairly simple to use from a script, and can inform if a partition is correctly aligned on sector boundaries:
Please do not run this script as is without first modifying for the correct disks.
#!/bin/bash
# parts.sh
# aligned
for disk in /dev/sd[b-g]
do
echo "############# $disk ################"
yes | parted $disk -- mklabel msdos
parted $disk -- rm 1
parted $disk -- mkpart primary ext4 0% 100%
parted $disk align-check optimal 1
done | tee parts.log
The next step is to prep the disks for ASM Filter Driver as root.
As before, please do not run this script as is without first modifying for the correct disks.
#!/bin/bash # setup-disks.sh set -v export ORACLE_HOME=/u01/app/oracle/product/12.2.0/grid export ORACLE_BASE=/tmp cd $ORACLE_HOME/bin : <<'COMMENT' # These commands necessary only if re-attempting to config $ORACLE_HOME/bin/asmcmd afd_deconfigure $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdb1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdc1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdd1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sde1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdf1 $ORACLE_HOME/bin/asmcmd afd_unlabel /dev/sdg1 COMMENT $ORACLE_HOME/bin/asmcmd afd_label DISK1 /dev/sdb1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK2 /dev/sdc1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK3 /dev/sdd1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK4 /dev/sde1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK5 /dev/sdf1 --init $ORACLE_HOME/bin/asmcmd afd_label DISK6 /dev/sdg1 --init $ORACLE_HOME/bin/asmcmd afd_lsdsk
Here are a few steps that I found necessary for a successful installation. As root, create directories as shown:
[[email protected] asm-disk-prep]# chmod g+wrx /u01/app/oracle # Create oraInventory and fix ownership and permissions [[email protected] asm-disk-prep]# mkdir -p /u01/app/oraInventory [[email protected] asm-disk-prep]# chown grid:oinstall /u01/app/oraInventory [[email protected] asm-disk-prep]# chmod g+rwx /u01/app/oraInventory
Check /etc/sysctl.conf for the net.core.wmem_max parameter. If not there, then add it.
(backup sysctl.conf first)
echo net.core.wmem_max = 1048576 >> /etc/sysctl.conf [[email protected] asm-disk-prep]# sysctl -p
Now you are ready for the grid installion. As the grid user, run gridSetup.sh
[[email protected] grid]$ pwd /u01/app/oracle/product/12.2.0/grid [[email protected] grid]$ [[email protected] grid]$ ./gridSetup.sh &
Once loaded, following are the choices I made during Grid installation:
- Set 1M AU
- DATA DiskGroup ( 6 disks as External Redundancy )
- All passwords everywhere set the same
- (it is a test environment without real data or connectivity to other systems)
- Groups for grid:
- OSASM: admadmin
- OSDBA: asmdba
- OSOPER: asmoper
- ORACLE_BASE=/u01/app/oracle
- Do not check ‘Automatically run root scripts”
- Personal Preference: I like to run them to see what is happening
Success!
[[email protected] asm-disk-prep]# . oraenv <<< +ASM ORACLE_SID = [root] ? The Oracle base has been changed from /tmp to /u01/app/oracle [[email protected] asm-disk-prep]# crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE ora12cr2-asm STABLE ora.LISTENER.lsnr ONLINE ONLINE ora12cr2-asm STABLE ora.asm ONLINE ONLINE ora12cr2-asm Started,STABLE ora.ons OFFLINE OFFLINE ora12cr2-asm STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 ONLINE ONLINE ora12cr2-asm STABLE ora.diskmon 1 OFFLINE OFFLINE STABLE ora.driver.afd 1 ONLINE ONLINE ora12cr2-asm STABLE ora.evmd 1 ONLINE ONLINE ora12cr2-asm STABLE --------------------------------------------------------------------------------
RDBMS Install
What good is ASM without a database? Let’s install one now.
There are several steps that must be run as root before proceeding.
Remount the VBOX share as oracle:
[[email protected] asm-disk-prep]# umount /media/sf_shared [[email protected] asm-disk-prep]# id oracle uid=300(oracle) gid=302(oinstall) groups=302(oinstall),301(dba),303(osoper),304(asmdba),306(asmadmin),503(racdba) [[email protected] asm-disk-prep]# mount -t vboxsf shared /media/sf_shared -o uid=300,gid=302
Create the Oracle Home directory:
[[email protected] asm-disk-prep]# mkdir /u01/app/oracle/product/12.2.0/db1 [[email protected] asm-disk-prep]# chown oracle:oinstall /u01/app/oracle/product/12.2.0/db1
Create new Oracle Inventory location:
runInstaller refused to use the current one, even when ownership was changed to oracle. I did not spend anytime trying to learn why this was so, as it was new to me, but just created and new oracle inventory and forged ahead.
[[email protected] asm-disk-prep]# mkdir /u01/app/oraInventory-db [[email protected] asm-disk-prep]# chown oracle:oinstall /u01/app/oraInventory-db
Create this dump directory now, as the install will fail otherwise
This was found in the install log following failed attempts
WARNING: Skipping line: [FATAL] [DBT-06608] The specified audit file destination (/u01/app/oracle/admin/asmafd/adump) is not writable.
INFO: Read: ACTION: Ensure that the audit file destination (/u01/app/oracle/admin/asmafd/adump) is writable.
INFO: ACTION: Ensure that the audit file destination (/u01/app/oracle/admin/asmafd/adump) is writable.
As root:
[[email protected] asm-disk-prep]# mkdir -p /u01/app/oracle/admin/asmafd [[email protected] asm-disk-prep]# chown grid:oinstall /u01/app/oracle/admin/asmafd [[email protected] asm-disk-prep]# chmod g+rwx /u01/app/oracle/admin/asmafd/
Now you are ready to run the installer as the Oracle user:
$ /media/sf_shared/oracle/12.2/rdbms/database/runInstaller &
Following are the choices made from the GUI installer:
- Create and Configure
- Server class of Advanced
- Enterprise Edition
- starter database
- database name: asmafd.jks.com
- pluggable db:p1
- defaults for memory, etc
- Choose ASM
- Do not check register with EM
- Do not setup Recovery
- Choose DATA DiskGroup
- ‘no’ to request to setup TFA (do not want extra things running)
Success!
[[email protected] ~]$ srvctl status database -db asmafd Database is running.
If you just want to get ASM Filter Driver up and running with database for some exploratory testing, this article should save you a large chunk of time spent poring over the manuals and doing test runs to find out what breaks.
So, what are you waiting for? Please leave comments if you find this useful or find any errors or omissions.
Please, also leave comments if you discover other issues while installing the software.
No comments