My First Experience Running SLOB – Status Update 2 (first results)

NOTE: Other SLOB-related posts are listed in “My SLOB IO testing index“.

I think the results we got so far may surprise you. At lease they don’t seem to be the results +Alex Gorbachev and +Kevin Closson expected to see. You can find the first related blog post here. It will give you the necessary context for further reading. Just to recap: +Kevin Closson says that “Orion may give It’s VERY easy to get huge Orion nums but reasonable SLOB” and +Alex Gorbachev  says that “lots of the system IO bound below the CPU level so you should see similar number with Orion or SLOB”. Let’s see what the first results revealed.

I would say that the Orion results seem to be the expected numbers :

ran (small): my  12 oth   0 iops 1378 size 8 K lat   8.71 ms bw =  10.77 MBps dur  59.97 s READ
...
ran (small): my  24 oth   0 iops 2114 size 8 K lat  11.35 ms bw =  16.52 MBps dur  59.96 s READ
...
ran (small): my 240 oth   0 iops 4305 size 8 K lat  55.74 ms bw =  33.63 MBps dur  59.81 s READ

We ran the OLTP 8k Orion test in read-only mode, providing 12 x 500 GB disks. It executed 20 tests, increasing the number of parallel threads from 12 to 240. The IOPS numbers grew slowly along with increased response time. The best response time we got was 8.71 ms with 12 processes harming storage and the best IOPS 4305 with correspondent latency 55.74ms running 240 processes. It gave us from 115 to 359 IOPS per IO spindle.

The SLOB results surprised us a bit:

We ran the “runit.sh 0 24” test, where there were 24 Oracle sessions executing single table’s block reads via index access (db file sequential read). We made sure that most IO issues by foreground processes would be PIOs. AWR data confirmed that we succeeded: Logical reads Per Second 4,541.6, Physical reads Per Second 4,524.4. It looked like most block reads (99.6%) have been processed via storage requests. But hold on a second. 4,524.4 is nothing else but IOPS. If we compare it with the Orion numbers, we see that SLOB shows better IO performance than Orion does. On top of that, the SLOB latency is significantly less than Orion’s – 5ms vs 55.74ms. I don’t think anybody expected that type of result. Just to give you a bit more input, here the most relevant parts of the SLOB AWR report:

              Snap Id      Snap Time      Sessions Curs/Sess
            --------- ------------------- -------- ---------
Begin Snap:       104 15-May-12 01:00:13        59       1.2
  End Snap:       105 15-May-12 01:43:04        36        .7
   Elapsed:               42.86 (mins)
   DB Time:              990.84 (mins)

Load Profile              Per Second    Per Transaction   Per Exec   Per Call
~~~~~~~~~~~~         ---------------    --------------- ---------- ----------
      DB Time(s):               23.1            5,404.6       1.25     499.58
       DB CPU(s):                0.3               68.0       0.02       6.28
   Logical reads:            4,541.6        1,061,707.7
  Physical reads:            4,524.4        1,057,677.6

Top 5 Timed Foreground Events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                           Avg
                                                          wait   % DB
Event                                 Waits     Time(s)   (ms)   time Wait Class
------------------------------ ------------ ----------- ------ ------ ----------
db file sequential read          11,630,768      58,928      5   99.1 User I/O
DB CPU                                              748           1.3
resmgr:cpu quantum                1,854,672          79      0     .1 Scheduler
resmgr:internal state change              1           0    101     .0 Concurrenc
undo segment extension                    2           0      5     .0 Configurat

The status is still a work-in-progress. We are running some other test now, and there is the “runit.sh 0 128” SLOB test on the way. Surprise, surprise! The server didn’t melt down with 128 processes running at the same time :). Speaking of the results, we have gotten so far that I suspect that SLOB doesn’t have objects big enough to hit over the HDDs caches. Some of the SLOB request are served from HDDs’ cache. If this is correct, then in this round Orion should be bigger than SLOB in terms of providing a “cleaner” IO test results.

+Kevin Closson said “Orion may give It’s VERY easy to get huge Orion nums but reasonable SLOB” – kind of FALSE in this case.

+Alex Gorbachev said “lots of the system IO bound below the CPU level so you should see similar number with Orion or SLOB.” – FALSE again in this case. :)

Stay tuned folks,

More results on the way…

Yury & Co

Well, right after I published this blog post while on the way to my English Pronunciation course, I thought about another probable reason for SLOB giving better results than Orion. It still is related to the size of the SLOB data set. Since we are using 12 x 500GB big HDDs for testing and the SLOB data set is relatively small, all the data is located close to the fastest (outer) edge of the disk. Orion, however, read from all the devices’ address space. This may be a good explanation for the discrepancy in the SLOB & Orion results. We will test it and update you on the results.