How To Downgrade Grid 19c Clusterware to 12.1

Posted in: Technical Track

There may be many reasons for downgrading from Grid 19c to 12.1; however, the reason I am downgrading is to rerun a test requested by Oracle support from the 12.1 clusterware.

Here are the steps performed.

### Check cluster:

[email protected] ~]$ crsctl check cluster -all
**************************************************************
ol7-121-rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
ol7-121-rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[[email protected] ~]$

### Downgrade Node2:

(Can run the downgrade script in parallel on all cluster nodes, but one.)

[[email protected] ~]# env|egrep -i 'ORA|HOME|TNS'
HOME=/root

[[email protected] ~]# ls /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh
/u01/app/19.3.0.0/grid/crs/install/rootcrs.sh

[[email protected] ~]# /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh -downgrade
Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/ol7-121-rac2/crsconfig/crsdowngrade_ol7-121-rac2_2019-12-09_10-58-49PM.log
2019/12/09 23:04:14 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.
2019/12/09 23:05:04 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.
2019/12/09 23:05:04 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node
[[email protected] ~]#

### Downgrade Node1:

(Downgrade the last node after downgrade all other nodes)

[[email protected] ~]# env|egrep -i 'ORA|HOME|TNS'
HOME=/root

[[email protected] ~]# ls /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh
/u01/app/19.3.0.0/grid/crs/install/rootcrs.sh

[[email protected] ~]# /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh -downgrade
Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/ol7-121-rac1/crsconfig/crsdowngrade_ol7-121-rac1_2019-12-09_11-06-04PM.log
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'ol7-121-rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.evmd' on 'ol7-121-rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.gpnpd' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol7-121-rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ol7-121-rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ol7-121-rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.diskmon' on 'ol7-121-rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'ol7-121-rac1'
CRS-2672: Attempting to start 'ora.ctssd' on 'ol7-121-rac1'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.crf' on 'ol7-121-rac1' succeeded
CRS-2676: Start of 'ora.ctssd' on 'ol7-121-rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.asm' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.storage' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.crsd' on 'ol7-121-rac1' succeeded
2019/12/09 23:18:00 CLSRSC-338: Successfully downgraded OCR to version 12.1.0.2.0
CRS-2672: Attempting to start 'ora.crf' on 'ol7-121-rac1'
CRS-2672: Attempting to start 'ora.storage' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.storage' on 'ol7-121-rac1' succeeded
CRS-2676: Start of 'ora.crf' on 'ol7-121-rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ol7-121-rac1'
CRS-2676: Start of 'ora.crsd' on 'ol7-121-rac1' succeeded
2019/12/09 23:18:23 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.
2019/12/09 23:18:49 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.
2019/12/09 23:18:50 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node
2019/12/09 23:18:51 CLSRSC-640: To complete the downgrade operation, ensure that the node inventory on all nodes points to the configured Grid Infrastructure home '/u01/app/12.1.0.2/grid'.
2019/12/09 23:18:52 CLSRSC-592: Run 'crsctl start crs' from home /u01/app/12.1.0.2/grid on each node to complete downgrade.
2019/12/09 23:19:13 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
[[email protected] ~]#

### Remove Grid 19c from Clusterware:

[[email protected] ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -nowait -waitforcompletion -ignoreSysPrereqs \
> -updateNodeList -silent CRS=false ORACLE_HOME=/u01/app/19.3.0.0/grid \
> "CLUSTER_NODES=ol7-121-rac1, ol7-121-rac2" -doNotUpdateNodeList
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 18431 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
You can find the log of this install session at:
 /u01/app/oraInventory/logs/UpdateNodeList2019-12-09_11-57-44PM.log
'UpdateNodeList' was successful.
[[email protected] ~]$

### Configure Grid 12c with Clusterware:

[[email protected] ~]$ /u01/app/12.1.0.2/grid/oui/bin/runInstaller -nowait -waitforcompletion -ignoreSysPrereqs \
> -updateNodeList -silent CRS=true ORACLE_HOME=/u01/app/12.1.0.2/grid \
> "CLUSTER_NODES=ol7-121-rac1, ol7-121-rac2"
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 18431 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
'UpdateNodeList' was successful.
[[email protected] ~]$

### Start crs for Node1

[[email protected] ~]# . oraenv <<< +ASM1
ORACLE_SID = [root] ? The Oracle base has been set to /u01/app/oracle

[[email protected] ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[[email protected] ~]#

### Start crs for Node2:

[[email protected] ~]# . oraenv <<< +ASM2
ORACLE_SID = [root] ? The Oracle base has been set to /u01/app/oracle

[[email protected] ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[[email protected] ~]#

### Validate Cluster:

[[email protected] ~]$ olsnodes
ol7-121-rac1
ol7-121-rac2

[[email protected] ~]$ crsctl check cluster -all
**************************************************************
ol7-121-rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
ol7-121-rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

[[email protected] ~]$ crsctl query crs softwareversion
Oracle Clusterware version on node [ol7-121-rac2] is [12.1.0.2.0]

[[email protected] ~]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node ol7-121-rac2 is [365281826].

[[email protected] ~]$ crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [12.1.0.2.0]

[[email protected] ~]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [365281826] and the complete list of patches [28553832 ] have been applied on the local node.

[[email protected] ~]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [365281826].
[[email protected] ~]$

### Validate inventory:

[[email protected] ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50
Oracle Interim Patch Installer version 12.2.0.1.18
Copyright (c) 2019, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/12.1.0.2/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/12.1.0.2/grid/oraInst.loc
OPatch version    : 12.2.0.1.18
OUI version       : 12.1.0.2.0
Log file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2019-12-10_00-22-52AM_1.log

Lsinventory Output file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-22-52AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: ol7-121-rac2.localdomain
ARU platform id: 226
ARU platform description:: Linux x86-64

List of Oracle Homes:
  Name          Location
   OraGI12Home1         /u01/app/12.1.0.2/grid
   OraDB12Home1         /u01/app/oracle/product/12.1.0.2/dbhome_1
   OraGI19Home1         /u01/app/19.3.0.0/grid

Installed Top-level Products (1):

Oracle Grid Infrastructure 12c                                       12.1.0.2.0
There are 1 products installed in this Oracle Home.


Interim patches (1) :

Patch  28553832     : applied on Fri Dec 06 16:38:26 UTC 2019
Unique Patch ID:  22732107
Patch description:  "OCW Interim patch for 28553832"
   Created on 11 Feb 2019, 02:30:42 hrs PST8PDT
   Bugs fixed:
     21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987
     19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932
     19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017
     19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795
     19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490
     20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947
     19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793
     18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930
     19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695
     19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799
     25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156
     21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642
     19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834
[[email protected] ~]$

### Deinstall Grid 19c Clusterware for Node1:

[[email protected] ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -detachHome -silent ORACLE_HOME=/u01/app/19.3.0.0/grid
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 18431 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
You can find the log of this install session at:
 /u01/app/oraInventory/logs/DetachHome2019-12-10_12-24-09AM.log
'DetachHome' was successful.
[[email protected] ~]$

### Deinstall Grid 19c Clusterware for Node2:

[[email protected] ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -detachHome -silent ORACLE_HOME=/u01/app/19.3.0.0/grid
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 18431 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
You can find the log of this install session at:
 /u01/app/oraInventory/logs/DetachHome2019-12-10_12-24-39AM.log
'DetachHome' was successful.
[[email protected] ~]$

### Check inventory for Node1:

[[email protected] ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50
Oracle Interim Patch Installer version 12.2.0.1.18
Copyright (c) 2019, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/12.1.0.2/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.1.0.2/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.18
OUI version       : 12.1.0.2.0
Log file location : /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2019-12-10_00-34-03AM_1.log

Lsinventory Output file location : /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-34-03AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: ol7-121-rac1.localdomain
ARU platform id: 226
ARU platform description:: Linux x86-64

List of Oracle Homes:
  Name          Location
   OraGI12Home1         /u01/app/12.1.0.2/grid
   OraDB12Home1         /u01/app/oracle/product/12.1.0.2/dbhome_1

Installed Top-level Products (1):

Oracle Database 12c                                                  12.1.0.2.0
There are 1 products installed in this Oracle Home.


Interim patches (1) :

Patch  28553832     : applied on Fri Dec 06 16:54:30 UTC 2019
Unique Patch ID:  22732107
Patch description:  "OCW Interim patch for 28553832"
   Created on 11 Feb 2019, 02:30:42 hrs PST8PDT
   Bugs fixed:
     21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987
     19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932
     19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017
     19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795
     19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490
     20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947
     19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793
     18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930
     19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695
     19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799
     25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156
     21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642
     19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834
     18893463, 21242118, 19514085, 25496699, 20758997, 27738131, 27006180
[[email protected] ~]$

### Check inventory for Node2:

[or[email protected] ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50
Oracle Interim Patch Installer version 12.2.0.1.18
Copyright (c) 2019, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/12.1.0.2/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/12.1.0.2/grid/oraInst.loc
OPatch version    : 12.2.0.1.18
OUI version       : 12.1.0.2.0
Log file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2019-12-10_00-24-54AM_1.log

Lsinventory Output file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-24-54AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: ol7-121-rac2.localdomain
ARU platform id: 226
ARU platform description:: Linux x86-64

List of Oracle Homes:
  Name          Location
   OraGI12Home1         /u01/app/12.1.0.2/grid
   OraDB12Home1         /u01/app/oracle/product/12.1.0.2/dbhome_1

Installed Top-level Products (1):

Oracle Grid Infrastructure 12c                                       12.1.0.2.0
There are 1 products installed in this Oracle Home.


Interim patches (1) :

Patch  28553832     : applied on Fri Dec 06 16:38:26 UTC 2019
Unique Patch ID:  22732107
Patch description:  "OCW Interim patch for 28553832"
   Created on 11 Feb 2019, 02:30:42 hrs PST8PDT
   Bugs fixed:
     21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987
     19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932
     19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017
     19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795
     19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490
     20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947
     19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793
     18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930
     19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695
     19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799
     25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156
     21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642
     19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834
     18893463, 21242118, 19514085, 25496699, 20758997, 27738131, 27006180
[[email protected] ~]$

### Remove Grid 19c directory for Node1:

[[email protected] ~]# ls -ld /u01/app/19.3.0.0
drwxr-xr-x. 3 root oinstall 18 Dec  6 17:30 /u01/app/19.3.0.0
[[email protected] ~]# rm -rf /u01/app/19.3.0.0
[[email protected] ~]# ls -ld /u01/app/19.3.0.0
ls: cannot access /u01/app/19.3.0.0: No such file or directory
[[email protected] ~]# mkdir -p /u01/app/19.3.0.0
[[email protected] ~]# chmod -R 775 /u01/app/19.3.0.0
[[email protected] ~]# chown -R oracle:oinstall /u01/app/19.3.0.0
[[email protected] ~]# ls -ld /u01/app/19.3.0.0
drwxrwxr-x. 2 oracle oinstall 6 Dec 10 00:26 /u01/app/19.3.0.0
[[email protected] ~]#

### Remove Grid 19c directory for Node2:

[[email protected] ~]# ls -ld /u01/app/19.3.0.0
drwxr-xr-x. 3 root oinstall 18 Dec  9 15:47 /u01/app/19.3.0.0
[[email protected] ~]# rm -rf /u01/app/19.3.0.0
[[email protected] ~]# ls -ld /u01/app/19.3.0.0
ls: cannot access /u01/app/19.3.0.0: No such file or directory
[[email protected] ~]# mkdir -p /u01/app/19.3.0.0
[[email protected] ~]# chmod -R 775 /u01/app/19.3.0.0
[[email protected] ~]# chown -R oracle:oinstall /u01/app/19.3.0.0
[[email protected] ~]# ls -ld /u01/app/19.3.0.0
drwxrwxr-x. 2 oracle oinstall 6 Dec 10 00:29 /u01/app/19.3.0.0
[[email protected] ~]#

Hopefully the steps provided will be useful in the event that a downgrade is necessary.

email

Author

Want to talk with an expert? Schedule a call with our team to get the conversation started.

2 Comments. Leave new

thx – your approach helped me with a GI upgrade gone bad in the middle of the rootupgade.sh script .. lost the connection and had reverse the changes to start again. great help.
Luke

Reply

Good to hear and thanks.

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *