A Simple Command That Saved Me

Posted in: Technical Track

So there I was, reviewing the environment to create an action plan for patching. To my surprise, I noticed the following when checking DBFS:

Oracle Instance not alive for sid "DBFS2"

I will demonstrate the simple command I used to ensure that all services that are supposed to be running are running.

What happened? It turns out there was a network outage and when starting resources, DBFS was missing.

It’s much harder to know what are all the resources that need to be started for a complex environment with RAC, GoldenGate, DataGuard, ACFS, DBFS.

crsctl stat res -t -w ‘((TARGET != ONLINE) or (STATE != ONLINE)’

--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.xxxxxxxx.yy_users_dr.svc
      1        OFFLINE OFFLINE                               STABLE
      2        OFFLINE OFFLINE                               STABLE
--------------------------------------------------------------------------------	

From the command above, Target=State=OFFLINE which is good.

crsctl stat res -t -w ‘((TARGET != ONLINE) or (STATE != ONLINE)’



crsctl stat res -t -w '((TARGET != ONLINE) or (STATE != ONLINE)'
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
dbfs_mount
               OFFLINE ONLINE       hostxxxx01              STABLE
               OFFLINE OFFLINE      hostxxxx02              STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.dbfs.db
      2        OFFLINE OFFLINE                               Instance Shutdown,STABLE
ora.xxxxxxxx.yy_users_dr.svc
      1        OFFLINE OFFLINE                               STABLE
      2        OFFLINE OFFLINE                               STABLE
xag.ggue_src.goldengate
      1        OFFLINE ONLINE       hostxxxx01              STABLE
--------------------------------------------------------------------------------

The above is not good because State=ONLINE but Target=OFFLINE for dbfs_mount at hostxxxx02.

It seems Oracle is seldom intuitive as checking the condition (TARGET != STATE) does not return any results.

crsctl stat res -t -w '((TARGET != STATE)'

In conclusion, always check TARGET=STATE for CRS to avoid having services down that are supposed to be running.

Alternatively, I suppose just restarting CRS versus starting each service individually might be better, provided resources are integrated with CRS.

email

Interested in working with Michael? Schedule a tech call.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *