There may be many reasons for downgrading from Grid 19c to 12.1; however, the reason I am downgrading is to rerun a test requested by Oracle support from the 12.1 clusterware.
Here are the steps performed.
### Check cluster:
[email protected] ~]$ crsctl check cluster -all ************************************************************** ol7-121-rac1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** ol7-121-rac2: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** [[email protected] ~]$
### Downgrade Node2:
(Can run the downgrade script in parallel on all cluster nodes, but one.)
[[email protected] ~]# env|egrep -i 'ORA|HOME|TNS' HOME=/root [[email protected] ~]# ls /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh [[email protected] ~]# /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh -downgrade Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params The log of current session can be found at: /u01/app/oracle/crsdata/ol7-121-rac2/crsconfig/crsdowngrade_ol7-121-rac2_2019-12-09_10-58-49PM.log 2019/12/09 23:04:14 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector. 2019/12/09 23:05:04 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector. 2019/12/09 23:05:04 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node [[email protected] ~]#
### Downgrade Node1:
(Downgrade the last node after downgrade all other nodes)
[[email protected] ~]# env|egrep -i 'ORA|HOME|TNS' HOME=/root [[email protected] ~]# ls /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh [[email protected] ~]# /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh -downgrade Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params The log of current session can be found at: /u01/app/oracle/crsdata/ol7-121-rac1/crsconfig/crsdowngrade_ol7-121-rac1_2019-12-09_11-06-04PM.log CRS-4123: Oracle High Availability Services has been started. CRS-2672: Attempting to start 'ora.evmd' on 'ol7-121-rac1' CRS-2672: Attempting to start 'ora.mdnsd' on 'ol7-121-rac1' CRS-2676: Start of 'ora.evmd' on 'ol7-121-rac1' succeeded CRS-2676: Start of 'ora.mdnsd' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'ol7-121-rac1' CRS-2676: Start of 'ora.gpnpd' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol7-121-rac1' CRS-2672: Attempting to start 'ora.gipcd' on 'ol7-121-rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'ol7-121-rac1' succeeded CRS-2676: Start of 'ora.gipcd' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'ol7-121-rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'ol7-121-rac1' CRS-2676: Start of 'ora.diskmon' on 'ol7-121-rac1' succeeded CRS-2676: Start of 'ora.cssd' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.crf' on 'ol7-121-rac1' CRS-2672: Attempting to start 'ora.ctssd' on 'ol7-121-rac1' CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol7-121-rac1' CRS-2676: Start of 'ora.crf' on 'ol7-121-rac1' succeeded CRS-2676: Start of 'ora.ctssd' on 'ol7-121-rac1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'ol7-121-rac1' CRS-2676: Start of 'ora.asm' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'ol7-121-rac1' CRS-2676: Start of 'ora.storage' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'ol7-121-rac1' CRS-2676: Start of 'ora.crsd' on 'ol7-121-rac1' succeeded 2019/12/09 23:18:00 CLSRSC-338: Successfully downgraded OCR to version 12.1.0.2.0 CRS-2672: Attempting to start 'ora.crf' on 'ol7-121-rac1' CRS-2672: Attempting to start 'ora.storage' on 'ol7-121-rac1' CRS-2676: Start of 'ora.storage' on 'ol7-121-rac1' succeeded CRS-2676: Start of 'ora.crf' on 'ol7-121-rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'ol7-121-rac1' CRS-2676: Start of 'ora.crsd' on 'ol7-121-rac1' succeeded 2019/12/09 23:18:23 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector. 2019/12/09 23:18:49 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector. 2019/12/09 23:18:50 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node 2019/12/09 23:18:51 CLSRSC-640: To complete the downgrade operation, ensure that the node inventory on all nodes points to the configured Grid Infrastructure home '/u01/app/12.1.0.2/grid'. 2019/12/09 23:18:52 CLSRSC-592: Run 'crsctl start crs' from home /u01/app/12.1.0.2/grid on each node to complete downgrade. 2019/12/09 23:19:13 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. [[email protected] ~]#
### Remove Grid 19c from Clusterware:
[[email protected] ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -nowait -waitforcompletion -ignoreSysPrereqs \ > -updateNodeList -silent CRS=false ORACLE_HOME=/u01/app/19.3.0.0/grid \ > "CLUSTER_NODES=ol7-121-rac1, ol7-121-rac2" -doNotUpdateNodeList Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed The inventory pointer is located at /etc/oraInst.loc You can find the log of this install session at: /u01/app/oraInventory/logs/UpdateNodeList2019-12-09_11-57-44PM.log 'UpdateNodeList' was successful. [[email protected] ~]$
### Configure Grid 12c with Clusterware:
[[email protected] ~]$ /u01/app/12.1.0.2/grid/oui/bin/runInstaller -nowait -waitforcompletion -ignoreSysPrereqs \ > -updateNodeList -silent CRS=true ORACLE_HOME=/u01/app/12.1.0.2/grid \ > "CLUSTER_NODES=ol7-121-rac1, ol7-121-rac2" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed The inventory pointer is located at /etc/oraInst.loc 'UpdateNodeList' was successful. [[email protected] ~]$
### Start crs for Node1
[[email protected] ~]# . oraenv <<< +ASM1 ORACLE_SID = [root] ? The Oracle base has been set to /u01/app/oracle [[email protected] ~]# crsctl start crs CRS-4123: Oracle High Availability Services has been started. [[email protected] ~]#
### Start crs for Node2:
[[email protected] ~]# . oraenv <<< +ASM2 ORACLE_SID = [root] ? The Oracle base has been set to /u01/app/oracle [[email protected] ~]# crsctl start crs CRS-4123: Oracle High Availability Services has been started. [[email protected] ~]#
### Validate Cluster:
[[email protected] ~]$ olsnodes ol7-121-rac1 ol7-121-rac2 [[email protected] ~]$ crsctl check cluster -all ************************************************************** ol7-121-rac1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** ol7-121-rac2: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** [[email protected] ~]$ crsctl query crs softwareversion Oracle Clusterware version on node [ol7-121-rac2] is [12.1.0.2.0] [[email protected] ~]$ crsctl query crs softwarepatch Oracle Clusterware patch level on node ol7-121-rac2 is [365281826]. [[email protected] ~]$ crsctl query crs releaseversion Oracle High Availability Services release version on the local node is [12.1.0.2.0] [[email protected] ~]$ crsctl query crs releasepatch Oracle Clusterware release patch level is [365281826] and the complete list of patches [28553832 ] have been applied on the local node. [[email protected] ~]$ crsctl query crs activeversion -f Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [365281826]. [[email protected] ~]$
### Validate inventory:
[[email protected] ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50 Oracle Interim Patch Installer version 12.2.0.1.18 Copyright (c) 2019, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/12.1.0.2/grid Central Inventory : /u01/app/oraInventory from : /u01/app/12.1.0.2/grid/oraInst.loc OPatch version : 12.2.0.1.18 OUI version : 12.1.0.2.0 Log file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2019-12-10_00-22-52AM_1.log Lsinventory Output file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-22-52AM.txt -------------------------------------------------------------------------------- Local Machine Information:: Hostname: ol7-121-rac2.localdomain ARU platform id: 226 ARU platform description:: Linux x86-64 List of Oracle Homes: Name Location OraGI12Home1 /u01/app/12.1.0.2/grid OraDB12Home1 /u01/app/oracle/product/12.1.0.2/dbhome_1 OraGI19Home1 /u01/app/19.3.0.0/grid Installed Top-level Products (1): Oracle Grid Infrastructure 12c 12.1.0.2.0 There are 1 products installed in this Oracle Home. Interim patches (1) : Patch 28553832 : applied on Fri Dec 06 16:38:26 UTC 2019 Unique Patch ID: 22732107 Patch description: "OCW Interim patch for 28553832" Created on 11 Feb 2019, 02:30:42 hrs PST8PDT Bugs fixed: 21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987 19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932 19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017 19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795 19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490 20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947 19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793 18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930 19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695 19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799 25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156 21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642 19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834 [[email protected] ~]$
### Deinstall Grid 19c Clusterware for Node1:
[[email protected] ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -detachHome -silent ORACLE_HOME=/u01/app/19.3.0.0/grid Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed The inventory pointer is located at /etc/oraInst.loc You can find the log of this install session at: /u01/app/oraInventory/logs/DetachHome2019-12-10_12-24-09AM.log 'DetachHome' was successful. [[email protected] ~]$
### Deinstall Grid 19c Clusterware for Node2:
[[email protected] ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -detachHome -silent ORACLE_HOME=/u01/app/19.3.0.0/grid Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed The inventory pointer is located at /etc/oraInst.loc You can find the log of this install session at: /u01/app/oraInventory/logs/DetachHome2019-12-10_12-24-39AM.log 'DetachHome' was successful. [[email protected] ~]$
### Check inventory for Node1:
[[email protected] ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50 Oracle Interim Patch Installer version 12.2.0.1.18 Copyright (c) 2019, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/12.1.0.2/dbhome_1 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/12.1.0.2/dbhome_1/oraInst.loc OPatch version : 12.2.0.1.18 OUI version : 12.1.0.2.0 Log file location : /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2019-12-10_00-34-03AM_1.log Lsinventory Output file location : /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-34-03AM.txt -------------------------------------------------------------------------------- Local Machine Information:: Hostname: ol7-121-rac1.localdomain ARU platform id: 226 ARU platform description:: Linux x86-64 List of Oracle Homes: Name Location OraGI12Home1 /u01/app/12.1.0.2/grid OraDB12Home1 /u01/app/oracle/product/12.1.0.2/dbhome_1 Installed Top-level Products (1): Oracle Database 12c 12.1.0.2.0 There are 1 products installed in this Oracle Home. Interim patches (1) : Patch 28553832 : applied on Fri Dec 06 16:54:30 UTC 2019 Unique Patch ID: 22732107 Patch description: "OCW Interim patch for 28553832" Created on 11 Feb 2019, 02:30:42 hrs PST8PDT Bugs fixed: 21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987 19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932 19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017 19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795 19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490 20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947 19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793 18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930 19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695 19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799 25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156 21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642 19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834 18893463, 21242118, 19514085, 25496699, 20758997, 27738131, 27006180 [[email protected] ~]$
### Check inventory for Node2:
[[email protected] ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50 Oracle Interim Patch Installer version 12.2.0.1.18 Copyright (c) 2019, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/12.1.0.2/grid Central Inventory : /u01/app/oraInventory from : /u01/app/12.1.0.2/grid/oraInst.loc OPatch version : 12.2.0.1.18 OUI version : 12.1.0.2.0 Log file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2019-12-10_00-24-54AM_1.log Lsinventory Output file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-24-54AM.txt -------------------------------------------------------------------------------- Local Machine Information:: Hostname: ol7-121-rac2.localdomain ARU platform id: 226 ARU platform description:: Linux x86-64 List of Oracle Homes: Name Location OraGI12Home1 /u01/app/12.1.0.2/grid OraDB12Home1 /u01/app/oracle/product/12.1.0.2/dbhome_1 Installed Top-level Products (1): Oracle Grid Infrastructure 12c 12.1.0.2.0 There are 1 products installed in this Oracle Home. Interim patches (1) : Patch 28553832 : applied on Fri Dec 06 16:38:26 UTC 2019 Unique Patch ID: 22732107 Patch description: "OCW Interim patch for 28553832" Created on 11 Feb 2019, 02:30:42 hrs PST8PDT Bugs fixed: 21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987 19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932 19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017 19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795 19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490 20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947 19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793 18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930 19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695 19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799 25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156 21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642 19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834 18893463, 21242118, 19514085, 25496699, 20758997, 27738131, 27006180 [[email protected] ~]$
### Remove Grid 19c directory for Node1:
[[email protected] ~]# ls -ld /u01/app/19.3.0.0 drwxr-xr-x. 3 root oinstall 18 Dec 6 17:30 /u01/app/19.3.0.0 [[email protected] ~]# rm -rf /u01/app/19.3.0.0 [[email protected] ~]# ls -ld /u01/app/19.3.0.0 ls: cannot access /u01/app/19.3.0.0: No such file or directory [[email protected] ~]# mkdir -p /u01/app/19.3.0.0 [[email protected] ~]# chmod -R 775 /u01/app/19.3.0.0 [[email protected] ~]# chown -R oracle:oinstall /u01/app/19.3.0.0 [[email protected] ~]# ls -ld /u01/app/19.3.0.0 drwxrwxr-x. 2 oracle oinstall 6 Dec 10 00:26 /u01/app/19.3.0.0 [[email protected] ~]#
### Remove Grid 19c directory for Node2:
[[email protected] ~]# ls -ld /u01/app/19.3.0.0 drwxr-xr-x. 3 root oinstall 18 Dec 9 15:47 /u01/app/19.3.0.0 [[email protected] ~]# rm -rf /u01/app/19.3.0.0 [[email protected] ~]# ls -ld /u01/app/19.3.0.0 ls: cannot access /u01/app/19.3.0.0: No such file or directory [[email protected] ~]# mkdir -p /u01/app/19.3.0.0 [[email protected] ~]# chmod -R 775 /u01/app/19.3.0.0 [[email protected] ~]# chown -R oracle:oinstall /u01/app/19.3.0.0 [[email protected] ~]# ls -ld /u01/app/19.3.0.0 drwxrwxr-x. 2 oracle oinstall 6 Dec 10 00:29 /u01/app/19.3.0.0 [[email protected] ~]#
Hopefully the steps provided will be useful in the event that a downgrade is necessary.
2 Comments. Leave new
thx – your approach helped me with a GI upgrade gone bad in the middle of the rootupgade.sh script .. lost the connection and had reverse the changes to start again. great help.
Luke
Good to hear and thanks.