How to Find Cluster Resources Where Target!=State

Posted in: Technical Track

DBA typically checks cluster resources as part of patching efforts. However, when checking cluster resources returns 200+ results, it’s not feasible to verify the results and it’s too time-consuming.

A better solution is to find a cluster resource where the state does not match the intended target.

First, kill the database instance process (please don’t do this in production):

[oracle@racnode-dc2-1 patch]$ ps -ef|grep pmon
oracle 13542 1 0 16:09 ? 00:00:00 asm_pmon_+ASM1
oracle 27663 1 0 16:39 ? 00:00:00 ora_pmon_hawk1
oracle 29401 18930 0 16:40 pts/0 00:00:00 grep --color=auto pmon
[oracle@racnode-dc2-1 patch]$
[oracle@racnode-dc2-1 patch]$ kill -9 27663
[oracle@racnode-dc2-1 patch]$

Check the cluster resource. Syntax works but provides a false positive:

[oracle@racnode-dc2-1 patch]$ crsctl stat res -t -w '((TARGET != ONLINE) or (STATE != ONLINE)'
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.proxy_advm
OFFLINE OFFLINE racnode-dc2-1 STABLE
OFFLINE OFFLINE racnode-dc2-2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
3 OFFLINE OFFLINE STABLE
ora.hawk.db
1 ONLINE OFFLINE racnode-dc2-1 Instance Shutdown,ST
ARTING
--------------------------------------------------------------------------------
[oracle@racnode-dc2-1 patch]$

Check cluster resource. The cluster resource for database instance Target is supposed to be ONLINE but State is OFFLINE:

[oracle@racnode-dc2-1 patch]$ crsctl stat res -t -w '((TARGET = ONLINE) and (STATE != ONLINE)'
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.hawk.db
1 ONLINE OFFLINE racnode-dc2-1 Instance Shutdown,ST
ARTING
--------------------------------------------------------------------------------
[oracle@racnode-dc2-1 patch]$

Check the cluster resource to be sure everything is up and running as it should be:

[oracle@racnode-dc2-1 patch]$ crsctl stat res -t -w '((TARGET = ONLINE) and (STATE != ONLINE)'
[oracle@racnode-dc2-1 patch]$
[oracle@racnode-dc2-1 patch]$ crsctl stat res -t -w 'TYPE = ora.database.type'
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.hawk.db
1 ONLINE ONLINE racnode-dc2-1 Open,HOME=/u01/app/o
racle/12.2.0.1/db1,S
TABLE
2 ONLINE ONLINE racnode-dc2-2 Open,HOME=/u01/app/o
racle/12.2.0.1/db1,S
TABLE
--------------------------------------------------------------------------------
[oracle@racnode-dc2-1 patch]$

In conclusion, it’s more efficient to check cluster resource accordingly to avoid having to review false positive or inapplicable results.

email

Interested in working with Michael? Schedule a tech call.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *