Physical Disk, Cell Disk, Grid Disk, and ASM Disk in Exadata

Exadata is the Ferrari of the technology world. It’s considered the technology for the select few, but everyone dreams about it. There is very little information available on the Internet right now about the architecture and the administration of Exadata, though it’s a huge subject in itself, which demand numerous high quality books, dedicated blogs, websites and special interest groups.

This blog post just mentions some of very important components of Exadata Storage Server, which are physical disks, cell disks, grid disks and ASM disks and their correspondence.

The Exadata Storage Server contains 12 physical disks.

There is one to one relationship between a physical disk and the cell disk. One physical disk corresponds to a single cell disk.

Then a Grid disk comprises of many cell disks.

And a Grid disk corresponds to one ASM disk.

An ASM diskgroup comprises of many ASM disks.

On the Exadata Storage Server, We can use cellcli command line utility in Exadata to see the information about physical disks, cell disks, grid disks and the cell.

The cellcli utility works from the root, celladmin and cellmonitor (read-only) users. The best practice is actually to run it from the last two less-privileged users, and not from the root user.

Now let’s have look at some of the disk management commands using cellcli utility.

[root@mycell-net0 ~]# cellcli

CellCLI: Release 11.2.1.3.1 - Production on Fri Oct 29 07:47:26 GMT 2010

Copyright (c) 2007, 2009, Oracle. All rights reserved.

Cell Efficiency Ratio: 140

Just to give you an idea about what cellcli has on offer, look at the output of the help command:

CellCLI> help

HELP [topic]
Available Topics:

ALTER

ALTER ALERTHISTORY

ALTER CELL

ALTER CELLDISK

ALTER GRIDDISK

ALTER IORMPLAN

ALTER LUN

ALTER THRESHOLD

ASSIGN KEY

CALIBRATE

CREATE

CREATE CELL

CREATE CELLDISK

CREATE FLASHCACHE

CREATE GRIDDISK

CREATE KEY

CREATE THRESHOLD

DESCRIBE

DROP

DROP ALERTHISTORY

DROP CELL

DROP CELLDISK

DROP FLASHCACHE

DROP GRIDDISK

DROP THRESHOLD

EXPORT CELLDISK

IMPORT CELLDISK

LIST

LIST ACTIVEREQUEST

LIST ALERTDEFINITION

LIST ALERTHISTORY

LIST CELL

LIST CELLDISK

LIST FLASHCACHE

LIST FLASHCACHECONTENT

LIST GRIDDISK

LIST IORMPLAN

LIST KEY

LIST LUN

LIST METRICCURRENT

LIST METRICDEFINITION

LIST METRICHISTORY

LIST PHYSICALDISK

LIST THRESHOLD

SET

SPOOL

START

Let’s see the output of some of the commands listed above:

CellCLI> list physicaldisk detail

name:                   [5:3:2:0]
diskType:               FlashDisk

id:                     00000200000000000000

luns:                   1_2

makeModel:              "MARVELL SD88SA02"

physicalFirmware:       D20R

physicalInsertTime:     2010-10-04T21:05:46+00:00

physicalInterface:      sas

physicalSerial:         0000000000000000000

physicalSize:           22.8880615234375G

slotNumber:             "PCI Slot: 1; FDOM: 2"

status:                 normal

<pre>

CellCLI> list cell detail

name:                   mycellnet0

bmcType:                IPMI

cellVersion:            OSS_11.2.0.1.0_LINUX.X64_100818.1

cpuCount:               16

fanCount:               12/12

fanStatus:              normal

id:                     0000X00000

interconnectCount:      3

interconnect1:          bond0

iormBoost:              0.0

ipaddress1:

kernelVersion:          2.6.18-194.3.1.0.2.el5

makeModel:              SUN MICROSYSTEMS SUN FIRE X4275 SERVER SAS

metricHistoryDays:      7

notificationMethod:     snmp

notificationPolicy:     critical,warning,clear

offloadEfficiency:      47,485.1

powerCount:             2/2

powerStatus:            normal

smtpFrom:

smtpFromAddr:

smtpPort:

smtpServer:

smtpToAddr:

smtpUseSSL:

snmpSubscriber:

status:                 online

temperatureReading:     27.0

temperatureStatus:      normal

upTime:                 16 days, 14:01

cellsrvStatus:          running

msStatus:               running

rsStatus:               running

<pre>

CellCLI> list celldisk detail

name:                   FD_15_mycellnet0

comment:

creationTime:           2010-07-14T02:05:06+00:00

deviceName:             /dev/sdy

devicePartition:        /dev/sdy

diskType:               FlashDisk

errorCount:             0

freeSpace:              0

id:                     00000000-0000-0000-0000-000000000000

interleaving:           none

lun:                    5_3

size:                   22.875G

status:                 normal

<pre>

CellCLI> list griddisk detail

name:                   CD_15_mycellnet0

availableTo:

cellDisk:               CD_15_mycellnet0

comment:

creationTime:           2010-07-14T02:12:55+00:00

diskType:               FlashDisk

errorCount:             0

id:                     00000000-0000-000-0000-000000000000

offset:                 528.734375G

size:                   29.125G

status:                 active

In the future posts, I will be touching the administration of celldisks, as how to perform operations like (import/export/create/drop/alter) on the celldisks.

Interested in working with Fahd? Schedule a tech call.