In this post:
- General – the presentation file
- Australia and Europe Webinar is scheduled (July 27th 2011)
- SCAN Questions and Answers
General
This is a follow up post to answer questions you asked during my “Oracle 11G SCAN: RAC SIG webcast” at June 16th.
You can get the full presentation HERE – “Oracle 11G SCAN: Concepts and implementation experience sharing”. If you are not a member please feel free to join the RAC SIG. The membership is free and it will take you just couple of minutes to get registered (hint: check your spam folder if you don’t see password email from RAC SIG).
Australia and Europe Webinar is scheduled (July 27th 2011)
If you missed the “Oracle11G SCAN: Sharing successful implementation experience” Webinar then you have a chance to see it. For Australia – Europe time zones I am repeating it at July 27th.
If you are interested please feel free to register:
Australia – I would love you to go through Sydney Oracle Meetup site
For All others – please register HERE
Questions & Answers
Thanks for all your questions guys you asked during the presentation! Some of them deserve a whole presentation to make the answer complete. Let see if I will find enough motivation and time to follow up with a SCAN2 presentations. Meanwhile let me give you some short answers and possible hints. Below are ALL questions I recorded during the Webinar. I tried my best to provide the best answers I can give in limited time frame I had. If you have any further questions or related comment please be my guest and post a comment under this post.
Question from Rick Blythe:
Q: What happens during failure events ? – How scan connection still works when node dies? When local listener dies/when scan listener dies… ?
A: The main component that makes the solution agnostic to nodes’ failures are 3 SCAN listeners and related IP addresses. Those listeners are Cluster wide services meaning that any of 3 listeners are not bind to any particular node and can run on any of the nodes. If one of the nodes crashes than the following happens:
– Survived nodes will have at least 1 SCAN listener running
– Short time (a minute) after the crash SCAN Listener records got refreshed and new client request are not forwarded to the crashed node anymore
– Failed SCAN listeners got started on survived nodes
– After a minute (default refresh time) new SCAN listener’s instances know about the DB instances that serves certain services configured and start function as before.
I think during the crash itself there are a small timeframe when client connections may get some errors alike the following. However in 1-2 minutes it all gets to normal:
"ORA-12514: TNS:listener does not currently know of service requested in connect descriptor"
The exact SCAN information refresh process is to be investigated. I am guessing Oracle Process and Notification service might be part of the process as each of the listeners keeps connection to “127.0.0.1:6100” socket.
Question from Anand Sathyamurthy:
Q: Yury great presentation, great thoughts. How to connections in ebsusinessuite in tns_admin
A: Thank you Anand for the great question. Since certain ATG autoconfig templates version Oracle officially supports SCAN. Please see the following note:
Using Oracle 11g Release 2 Real Application Clusters with Oracle E-Business Suite Release 12 [ID 823587.1]
If you for one reason or another are not on the right ATG level you may be interested to have a look on the following my blog post.
EBS DBA: Integrating EBS with Oracle 11G Cluster Infrastructure (TNS_ADMIN).
Questions from Pascal:
Hello Yury, thank you for a Great Presentation!
Q1: I have a Question regarding SCAN and Static Listeners. How to configure or integrate Static Listeners in a RAC Environment which is using SCAN? This is required for instance when using Data Guard. Do we have to configure the Static Listener SID_LIST Entries under the LISTENER_SCAN* entries in the listener.ora File also?
A: My first guess would be to make sure that your local “static” listener is listed in “local_listener” init.ora parameter. This way Oracle instance would automatically register the local listener to SCAN listeners. A local DB Instance users remote_listener parameter to contact SCAN listeners.
Q2: Another Question regarding EZConnect and Connect Time Failover Is it possible to use EZConnect Method with Load Balancing/Connect Time Failover when using SCAN?
A: I would say that Failover and Load Balancing options are build-in into SCAN concept. You do not need to specify those anymore. You just specify the SCAN connection string as described in my presentation and both features are enabled for your clients by default. For more information on EZConnect Method click HERE. See “Understanding Failover, Load Balancing, and SCAN” for additional information on what is going on the server side of things.
Q3: Is it supported to use the old style connect strings using VIPs in the tnsnames.ora File for connecting to RAC 11gR2?
REF: Understanding the Oracle Real Application Clusters Installed Configuration
“Clients configured to use VIP addresses for Oracle RAC releases prior to Oracle Database 11g release 2 can continue to use their existing connection addresses; using SCAN is not required.”
Question from Yong:
Q: Yury: Do regular and SCAN listeners both have to know node load stats? If yes, isn’t it duplicate?
A: REF Database Services for an Oracle RAC Database.
I think that you are right and the load stats (workload) got registered in both listeners. The only listener that is using the information provided in “proper” SCAN configuration is SCAN listener. However!!! (REF: previous answer) SCAN doesn’t limit the previous VIP address load balancing configuration on the client side. Oracle registers workload on both SCAN and VIP listeners using service registration process to make both load balancing options available.
Question from Babu Srinivasan:
Q: so if i stop and start .. will it interrrupt the new connections?
A: If you stop start a SCAN listener it will not interrupt existing or new connections. There may be a short time-frame (less than a minute) when SCAN listener got the services related information from local listeners (DB instances). During that time-frame your clients might see the following errors:
"ORA-12514: TNS:listener does not currently know of service requested in connect descriptor"
Question from Babu Srinivasan:
Q: endpoints listener.ora for clusters .. what should be the entries in them .. how it is managed?
A: REF: How Oracle Database Uses the Listener File (listener.ora)
In ideal case you do not edit the listener.ora file nor related local_listener parameter. The endpoints are the ADDRESS part of the listener.ora file. You can create an additional listeners using the following syntax or modify and add additional endpoints to your configuration using srvctl utility.
[sode]srvctl add listener -p TCP:1522 -o /u01/app/oracle/product/11.2.0/dbhome_1
srvctl modify listener -n mynode1 -p “TCP:1521,1522”
srvctl modify listener -l crmlistener -p TCP:1522[/code]
However sometimes in emergency cases you may want to implement a “dirty fix” and make the manual adjustments (I did it on couple of occasions). However after got the emergency under control I would strongly suggest you to research on the issue you faced and come with an action plan to convert the setup to the officially supported. Otherwise you may end up with a bigger problems (mess).
Question from Deepak:
Q: What is the problem using “Oracle” user instead of “Grid” – which I thought is really an optionall user during GI install?
A: Well. Strictly speaking there is no problem running both components under the same OS user. However starting from 11GR2 there are much cleaner boundaries in between Grid and Oracle components. If you isolate the Grid services under separate user you protect it from everything else. On the other hand if you have multiple RAC databases (and instances) running on the same host you may want to run those under a separate oracle users. Keeping Grid and Oracle separated makes sense to me after I worked with it on several occasions. However the solution that works in one case may not be suitable for others. If you run a small configuration or/and your DBA/Maintenance team is small (no need to separate responsibilities) you may find that running all under the same user is a better idea. In any circumstances I would keep Cluster and RDBMS oracle homes separate (I am guessing that it isn’t possible to keep those in the same oracle home starting from 11GR12).
Question from David Start:
Q: Have you run into any issues with clients on windows caching the first ip address that dns returns and not round robining
—————————————————————-
A: Yes I do remember some issues related to the fact that some Windows versions cached DNS entries. However I just tested it on my patched XP workstation and don’t see it anymore. I am guessing it is very OS specific. This is a very good point tho and WARNING for others. Please keep this in mind. Some OS in certain circumstances can cache DNS entries. This may prevent your and users from connection to Oracle services in case of one of the 3 SCAN failures.
11 Comments. Leave new
Hi Yury,
thanks for this post, as I’m just researching various issues with SCAN let me also share some experience.
reg Q1: I think what Pascal meant is adding static registration for your standby database in case you need to connect to it while it’s in nomount status i.e. for standby rebuild.
The way I’ve done it is adding manual entry into listener.ora for a SID_LIST_LISTENER_SCANn where n is the scan listener process which is running on the host where my standby database (waiting to be refreshed) sits.
Another point I’d like to bring up is to make sure that clients connecting via SCAN listener can still resolve the VIP hostnames. We faced an issue when MS Service (using ODAC 11gR2) connecting to Oracle Database via SCAN listener was throwing ORA-12545. Eventually we found that SCAN listener is probably returning host name of the node where the client should connect to back to the client. Inability to resolve the VIP hostname (it’s not in DNS in our case) resulted into this error. After adding the cluster VIP hostnames to client hosts files, issue was resolved.
I also have question if anyone faced issues with SCAN in EMGC? Recently our SCAN listener status in EMGC turned to “down” (after being in blackout for some time).
Although the listener is running, connection to DB using SCAN are possible, and the usual techniques (agent clearstate) does not help it. Therefore I think EMGC 11g still has some reserves in detecting and managing SCAN listeners.
Cheers,
Lukas
We have faced some issue regarding to DNS. I would appreciate if you can help us to conclude the scenario –
We have two node cluster.
1. On second node, the server utilization went high SYS 40-50%, user 50%, Idle 0%. Sessions were waiting on “latch free”, “enq: HW contention”, CBC etc.
2. When brought down the Local Listener of node then within a minute, everything came back to normal.
3. Again we started up Listener issue re-occured again.
4. We stopped the listener, again everything came back to normal.
5. During this time, Issue in 2 DNS servers (out of 4 DNS servers configured) were observed, so we have shutdown two dns servers.
6. Then we stared the listener till then everything is fine.
During all this time period, there is no issue observed on first node. It was running perfectly fine.
Would like to co-relate this scenario, why DNS would have create issue in one server only? We dont know the root cause of this scenario. Please help us to understand if DNS will really cause this issue.
Hi Yury,
Great article.
I have a question for you which is bugging me.
I have a 2 node 11G RAC cluster with 3 Scan ip’s, what happens when both the node listeners are stopped? is it expected behavior to receive the error below even though the 3 scan listeners are okay? I’m assuming this is expected because the SCAN listeners are not able to forward requests to the local node listener.
Please advise a simple yes no will suffice.
SP2-0751: Unable to connect to Oracle. Exiting SQL*Plus
ERROR:
ORA-12516: TNS:listener could not find available handler with matching protocol
stack
Thanks.
Hi Zak,
You are correct. SCAN listeners are just a front line of listeners. They do not connect sessions to Instances. SCAN listeners just forward connections to a local listener. If there is not local listener sessions can’t connect.
Yury
Hi Yury
I have one question on the DNS caching issue. Do I have to manually flushdns on the client machine or any other way? How to resovle this issue…
becuase whenever node eviction happens, this is the problem for my application to restart their web services (like apache etc).
Hello Kannan,
thanks for a very good question. I must say I don’t have direct answer now. The SCAN GNS setup is still kind of exotic setup and there are no too much experience around.
I am guessing that the Oracle 11g client should address this issue automagicaly (if it doesn’t we need to talk to Oracle development, I can coordinate if needed).
The older Oracle clients most probably rely on OS functions and I would try look into cliets’ OS configuration.
Yury
Yury, Thanks for the article and nice Q & A post, however i still dont understand why do I have 2 types of listener on my cluster node, LISTENER , SID_LISTENER & SCAN_LISTENERn ?
appreciate if anyone can shade lights on this.
i mean 3 types of listeners. sorry for the typo in previous post.
Hi Yuri,
How do i restrict certain group of users to connect to specific NODE using SCAN? should i need to attach service_name to SCAN listener or using UCP? This is required especially on EBS environment to restrict particular module transactions to particular node
Thanks in advance
Hi ,
I have a doubt about services running on single instace. here is scenario.
My databases running on 2 instances and my service is running on instance 1. We are on 11gr2 so scan in place. In this situation when a connection comes to SCAN whether it checks to local listener for load stats or it directly handover this to server process on instance 1.
can you please explain
i think it will directly give the connection to instance 1, bcz service is not configured on instance 2, and SCAN will also find the instance 1 for this service . bcz remote listener on 2nd node will not have this service details..and remote listenre is for scan..that’s why scan will not check for load on 2nd node, it will directly handover to instance 1