Unveiling the OLTP Oracle Database Machine & Exadata v2

Update, July 9, 2010: Pythian has now announced our range of services for Oracle Exadata, along with successful implementations and reference customers.

Now that I, apparently successfully, predicted OLTP Database Machine on Sun hardware, I had to wake up before 6AM in Sydney to tune into Larry’s joined with Sun Microsystems webcast (just to learn that he is late, by the way – 8 minutes so far…). As the follow up post’s comments show, people are interested in the role of SPARC platform in the new OLTP Oracle Database Machine (turns our there is no role for SPARC as of now).

Waiting… Waiting… ah here it comes — yachts, BMW (yeah love it as well) and Larry walks in — he starts by mentioning lo-o-o-ong partnership with Sun and announced —
“Oracle Exadata version 2 – hardware by Sun and software by Oracle.” Funny, I heard exactly the same sitting at the Oracle Open World last year but with HP. He then proceeds — “It is the *First* Database Machine that does OLTP. All the other machines, Teradata, Netezza, etc. are designed just for data warehousing.”

Interesting that Larry’s speech was very harsh on competition and where it comes to data warehousing, it’s Netezza and Teradata, while in hardware it’s IBM. I need to count how many times Larry said “better/cheaper/faster than IBM” during his announcement.

Ellison also seems to freely use Exadata term for the Database Machine itself which is a bit confusing but the more confusion, the more hype and interest so I guess he must have been asked to do it.

Let’s go into the nitty-gritty… ODBM (Oracle Database Machine) v2 is twice as fast as version 1 when it comes to data warehousing. Compare to version 1, ODBM v2 runs OLTP traffic — “something that Netezza and Teradata can’t do at all… but we can do both [i.e. data warehousing] and we do both very well” said Ellison.

So what’s inside?

Database Servers

Sun Fire X4170 with two Xeon Nehalem processors and this is the state of the art. Nehalem processors are the fastest CPU’s for database workload in case you didn’t know. Each server also has 40 Gb connectivity to InfiniBand switch.

Memory is 3 times faster in the new database server and there more the two times memory now — 72GB per database server. For those of you who still use 5 GB for their SGA’s when the server has 128GB — wake up! The prohibitive overhead of larger SGA is long time gone and all modern releases of Oracle handle large SGA really well.

What’s important is that each server has 4 10Gbit Ethernet ports. OLTP traffic now has bigger pipes to connect application tiers which what was one the main problems why ODBM v1 wasn’t suitable for OLTP traffic.

Exadata Storage Servers

Storage Servers are based on Sun Fire X4275. Again top of the shelf Intel Xeon 5500 Nehalem processors.

Capacity increased — SAS disks are 600 GB (+33%) and SATA disks are 2 TB which doubles what ODBM v1 offered. We all know that big disks are bad but that’s where Sun FlashWire technology comes into play with flash cache on the controllers (5+TB for the full ODBM).

This is what Ellison said about Flash cache — “These are not Flash disks. Make no mistake — these are no Flash disks. This is a memory hierarchy made up of DRAM in our database servers and Flash in our storage servers with very sophisticated algorithms, not simple LRU — we know for doing a sequential scan, we don’t blow out the cache.”

Database Machine — all together

8 Compute Servers — note that they are not Database Servers anymore. That means, customers are expected to be able to run their application and web tiers on those compute servers just as well as Oracle databases. Of course, it’s called the server grid. You have in total 64 Nehalem cores and 400 GB of memory.

14 Storage Servers feature 300+TB of disks (using SATA disks) and more than 5TB of flash cache. The flash cache is built into the disk controllers. Each disk controller has 96 GB of flash. Each Exadata Storage server has 4 of those controllers so you do math (we must be at least as good as Larry in math — “1 million IO’s per second (IOPS) from a single cabinet. Obviously, 2 cabinets is 2 million IOPS. 8 Cabinets is… I can do the multiplications in my head — it’ll be 8 million random IOPS.”). So it’s 5+ TB of disk controllers flash cache — Sun’s FlashWire technology.

Ellison mentioned 880 Gb/sec aggregate throughput for InfiniBand. In reality, 8 compute servers can only do 320 Gb/sec which is a lot and if you are going to use this machine for OLTP — you don’t care as all you interested is random IO’s and the machine is sized to deliver one million random IO’s.

It’s not clear whether customers have complete flexibility on configuration of the database machine but based on Larry’s scale-out ideas, it’s possible. He said — “just plug in another compute server or another storage server or just plug in another InfiniBand networking switch”. The latter can let you expand beyond one single database machine and rack together 8, 16, 32 machine and according to Ellison “you can just go on and on and on”. Petabyte era comes indeed, we are in the Petabyte territory with 3 database machine racks.

1 million random IO’s per second — this is what you can achieve if your whole database fits into Flash Cache. The bandwidth of Flash cache is only 2.5 times faster than disks (50GB/sec vs 21 GB/sec) but don’t forget that we need Flash Cache only for random IO’s. Don’t forget that this is random reads performance. Right now, it’s not clear how write IO’s impact the number.

Ellison also mentioned compression and then flash can fit something like 50TB of the raw data after compression (10x compression for data warehouse). It’s not free but the results are impressive and based on the current adoption, customers are very happy with the new compression that Oracle database 11g provides.

11gR2 with Oracle Database Machine is running in-memory-only faster than all new start-ups specialized on in-memory databases. This is an interesting claim — I’m keen to verify it. According to Ellison, “Exadata is faster than the best of the specialized in-memory-only database query processing systems”. Oracle ran TPC-H data warehousing benchmark for 1TB data in memory and… won, according to Ellison.

IBM

Building the similar system on IBM Power6 Model 595 and 8 DS8300 Turbo storage, will cost you US$10 million.

Two racks of database machine cost $2.3 million only which is less than a quarter of what you would pay for IBM hardware. Putting it another way around — you can spend the same dollars and get 4 times faster hardware. Interesting note from Ellison to this IBM set — “we are fault tolerant, they are not”. Hm… I was quite sure IBM hardware is damn reliable but OK, if Larry says so…

One slide said that Oracle Database Machine is 20 times faster than IBM fastest computer. Not sure how that was calculated?

Ellison also claimed that Massively Parallel Architecture is what Oracle has been pursuing over the last 20 years. Interesting that many keep blaming Oracle for focusing on shared-everything while other vendors using shared-nothing with massively parallel processing. Well, if nobody else can do shared-everything — why not to blame this technology and stamp it as non-scalable. Here is where Ellison mentioned IBM DB2 and SQL Server and that they, obviously, can’t do what Oracle database can.

Pricing

• 1 Exadata Server + 1 Compute Server — $110K
• Quarter Rack — $350K
• Half Rack — $650K
• Full Rack — $1.15M

Obviously, that doesn’t include licensing but you can take over you existing Oracle database licenses. Update 16-Sep-09: Beware of Exadata Software licensing – it’s $10K per disk! Official price list has been updated.

How do you migrate

There is no changes to application that need to be done to migrate to Oracle Database Machine — all your existing applications will just work. The migration time-frame can be very aggressive — up to days really.

And if you need help migrating your application to the new Oracle Database Machine — Pythian, of course, can help you in doing so: migration with minimal downtime, optimal layout and capacity planning, help you transfer your licenses, evaluate potential compression benefits, upgrade your database version up to the latest 11g Release 2 (we were part of the beta program as usual) and all the tasks involved including backups and business continuity as well as monitoring.

Learn more about Pythian’s services for evaluation, migration to and operational support for Oracle Exadata.