Upgrading grid infrastructure from 12.1 to 12.2 in Exadata

1.2 Upgrade OPatch if OPatch is not already at the latest version:

 [[email protected] ~]$ unzip -q /software/27475857/Database/OPatch/12.2/12.2.0.1.13/p6880880_12*_Linux-x86-64.zip -d /u01/app/12.2.0.1/grid

1.3 Check that Oracle has a 10240 limits for the “soft stack” (if not, set it, log off and log on).

[[email protected]~]$ dcli -g ~/dbs_group -l root grep stack /etc/security/limits.conf | grep soft
exadatadb01: * soft stack 10240
exadatadb02: * soft stack 10240
exadatadb03: * soft stack 10240
exadatadb04: * soft stack 10240

1.4 Verify that 1500 huge pages are available.

[[email protected]~]$ dcli -g ~/dbs_group -l root grep -i huge /proc/meminfo
....
AnonHugePages: 0 kB
HugePages_Total: 200000
HugePages_Free: 132171
HugePages_Rsvd: 38338
HugePages_Surp: 0
Hugepagesize: 2048 kB
....

 1.5 Fix the ntp failures in prerequisites check.

— Verifying NTP daemon is synchronized with at least one external time source … FAILED.

1.6 Run the prerequisites.

sudo su - oracle
cd /u01/app/12.2.0.1/grid
./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/12.1.0.2/grid -dest_crshome /u01/app/12.2.0.1/grid -dest_version 12.2.0.1 -fixup -verbose

2 Upgrade GI to 12.2

2.1 Set the ASM memory prerequisites.

sqlplus / as sysasm
alter system set sga_max_size = 3G scope=spfile sid='*';
alter system set sga_target = 3G scope=spfile sid='*';
alter system set memory_target=0 sid='*' scope=spfile;
alter system set memory_max_target=0 sid='*' scope=spfile /* required workaround */;
alter system reset memory_max_target sid='*' scope=spfile;
alter system set use_large_pages=true sid='*' scope=spfile /* 11.2.0.2 and later(Linux only) */;

2.2 Reset mis-count.

sudo su - oracle
. oraenv <<< +ASM1
crsctl unset css misscount

2.3 Check for any rebalance operation (there should be none).

sudo su - oracle
. oraenv <<< +ASM1
sqlplus / as sysasm
select count(*) from gv$asm_operation;

2.4 Gridsetup – verify the response file.

[[email protected] ~]$ egrep -v "^#|^$" /tmp/giresponse.rsp | head -10
oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v12.2.0
INVENTORY_LOCATION=
oracle.install.option=UPGRADE
ORACLE_BASE=/u01/app/oracle
oracle.install.asm.OSDBA=dba
oracle.install.asm.OSOPER=dba
oracle.install.asm.OSASM=dba
oracle.install.crs.config.gpnp.scanName=
oracle.install.crs.config.gpnp.scanPort=
oracle.install.crs.config.ClusterConfiguration=
[[email protected] ~]$

2.5 Gridsetup – run from exadatadb01.

sudo su - oracle
cd /u01/app/12.2.0.1/grid
./gridSetup.sh -silent -responseFile /tmp/giresponse.rsp -J-Doracle.install.mgmtDB=false -J-Doracle.install.crs.enableRemoteGIMR=false -applyPSU /software/27475857/Database/12.2.0.1.0/12.2.0.1.180417GIRU/27850694

2.6 rootupgrade.sh on exadatadb01/2/3:

/u01/app/12.2.0.1/grid/rootupgrade.sh

2.7 rootupgrade.sh on exadatadb04, it will be the last one.

/u01/app/12.2.0.1/grid/rootupgrade.sh

2.8 Check if GI is relinked with RDS:

sudo su - oracle
dcli -g /tmp/dbs_group -l oracle /u01/app/12.2.0.1/grid/bin/skgxpinfo

=> the command should return "rds" for each node

# If not, do it:

dcli -g /tmp/dbs_group -l oracle ORACLE_HOME=/u01/app/12.2.0.1/grid \ make -C /u01/app/12.2.0.1/grid/rdbms/lib -f ins_rdbms.mk ipc_rds ioracle

2.9 Check the status of the cluster and the installed version.

sudo su - oracle
/u01/app/12.2.0.1/grid/bin/crsctl check cluster -all
/u01/app/12.2.0.1/grid/bin/crsctl query crs activeversion

2.10 Modify ASM.

sudo su - oracle
srvctl modify asm -count ALL

2.11 Update compatible.asm on the diskgroups.

$ . oraenv <<< +ASM1
$ sqlplus / as sysasm
SQL> select name, COMPATIBILITY from v$asm_diskgroup ;
SQL> ALTER DISKGROUP DATA SET ATTRIBUTE 'compatible.asm' = '12.2.0.1.0';
SQL> ALTER DISKGROUP DBFS_DG SET ATTRIBUTE 'compatible.asm' = '12.2.0.1.0';
SQL> ALTER DISKGROUP RECO SET ATTRIBUTE 'compatible.asm' = '12.2.0.1.0';
SQL> select name, COMPATIBILITY from v$asm_diskgroup ;

2.12 Update inventory.

$ . oraenv <<< +ASM1
$ /u01/app/12.2.0.1/grid/oui/bin/runInstaller -ignoreSysPrereqs -updateNodeList ORACLE_HOME=/u01/app/12.2.0.1/grid "CLUSTER_NODES={exadatadb01,exadatadb02,exadatadb03,exadatadb04}" CRS=true LOCAL_NODE=exadatadb01

3 Known issues

3.1 The spfile was not being used by the ASM instances.

# If the command above doesn’t return anything, which means the spfile is not being used. We need to set the spfile, restart the cluster if needed.

3.2 The password file of ASM instances was not in ASM diskgroup, you will see the failures in prerequisites check.

3.3 The patch 21255373 had been applied to GI, while when doing the prerequisites, it always reports the failures. According to Oracle support, this can be ignored. So the binary installation will have the option “-skipPrereqs”.

3.4 The postscripts was not successful as the SCAN and SCAN_LISTENERS were not up.

3.5 Add the ASM instance entry into /etc/oratab.

For some reason, the ASM instance entry had been removed from /etc/oratab and never been added back. We need to add it manually to all nodes.