IOT Internals

Julian Dyke Independent Consultant

Web Version
1

2005 Julian Dyke

juliandyke.com

Agenda

 Heap

Organized Tables  Index Organized Tables  IOT Overflow  IOT Secondary Indexes  IOT Mapping Tables

2005 Julian Dyke

juliandyke.com

Heap Organized Tables Introduction



Tables can be  ORGANIZATION HEAP (default)  ORGANIZATION INDEX (8.0 and above)  ORGANIZATION EXTERNAL (9.0 and above) By default all tables are heap organized Heap organized tables are not sorted on any column Rows can be located using a physical ROWID which comprises  Relative tablespace number  Block number  Slot number

2005 Julian Dyke

juliandyke.com

Heap Organized Tables Indexes



Indexes on heap organized tables can be  B*Tree indexes  Bitmap indexes B*Tree indexes contain  Key (one or more table columns)  Physical ROWID Bitmap indexes contain  Key (one or more table columns)  Start ROWID  End ROWID  Bitmap

2005 Julian Dyke

juliandyke.com

Heap Organized Tables Execution Plans

CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3), CONSTRAINT team_pk PRIMARY KEY (team_key) ) ORGANIZATION HEAP; INSERT INTO team VALUES (FER, Ferrari, ITA); SELECT team_name FROM team WHERE team_key = FER; SELECT STATEMENT TABLE ACCESS (BY INDEX ROWID) OF TEAM INDEX (UNIQUE SCAN) OF TEAM_PK

2005 Julian Dyke

juliandyke.com

Heap Organized Tables Example Index Block

INSERT INTO team VALUES (FER,Ferrari,ITA); Row#0[8024] flag: -----, lock: 2, data 01 00 00 72 00 00 col 0; len 3; (3): 46 45 52

Index Row Header

Table ROWID

Index Col 0

00 02
Flag Lock

01 00 00 72 00 00
Length

F E R 03 46 45 52
Length

2005 Julian Dyke

juliandyke.com

Heap Organized Tables Example Table Block

INSERT INTO team VALUES (FER,Ferrari,ITA); Tab tl: col col col 0, 19 0: 1: 2: row 0,@0x1f8d fb: --H-FL lb: 0x1 cc: 3 [ 3] 46 45 52 [ 7] 46 65 72 72 61 72 69 [ 3] 49 54 41

-- FER -- Ferrari -- ITA

Table Row Header

Table Col 0

Table Col 0

Table Col 2

2C 01 03
Flag Lock

F E R 03 46 45 52
#Cols Length

F e r r a r i 07 46 65 72 72 61 72 69
Length

I T A 03 49 54 41
Length

2005 Julian Dyke

juliandyke.com

Heap Organized Tables Summary



Tables include  Three byte row header  Table columns (including key columns) Indexes include  Two byte row header  Redundant key columns  Six byte ROWID

2005 Julian Dyke

juliandyke.com

Index Organised Tables Introduction



Introduced in Oracle 8.0 Must be declared as ORGANIZATION INDEX Must have primary key constraint  held in index row Can contain non-key columns  held in table row  may be stored in overflow segment Do not have table segment Do not store physical ROWIDs

 

2005 Julian Dyke

juliandyke.com

Index Organised Tables Introduction

      

Can be compressed Can be range or hash partitioned (8.1.5 and above) Can have secondary B*tree indexes (8.1.5 and above) Can have secondary bitmap indexes (9.2 and above) Can include LOBs in non-key columns Can include ADTs in non-key columns Can include ROWID columns Cannot be reversed Cannot be list or composite partitioned Cannot contain LONG columns

  

10

2005 Julian Dyke

juliandyke.com

Index Organised Tables Restrictions

  

Maximum number of primary key columns is 33 Maximum number of columns is 1000 Maximum primary key lengths in Oracle 9.2
Block Size 2048 4096 8192 16384 Maximum Key Length 755 1575 3215 3800

Non-key column lengths are not restricted

11

2005 Julian Dyke

juliandyke.com

Index Organized Tables Inline versus Out-of-Line Constraints

 

Primary key constraint can be inline or out-of-line Inline

CREATE TABLE team ( team_key VARCHAR2(3) PRIMARY KEY, team_name VARCHAR2(50), country_key VARCHAR2(3) ) ORGANIZATION INDEX;

Out-of-line
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3), CONSTRAINT team_pk PRIMARY KEY (team_key) ) ORGANIZATION INDEX;

12

2005 Julian Dyke

juliandyke.com

Index Organized Tables Implementation



The statement
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3), CONSTRAINT team_pk PRIMARY KEY (team_key) ) ORGANIZATION INDEX;

generates the following recursive statement (10046)

CREATE UNIQUE INDEX TEAM_PK ON TEAM (TEAM_KEY) INDEX ONLY TOPLEVEL TABLESPACE TS01 NOPARALLEL;

13

2005 Julian Dyke

juliandyke.com

Index Organised Tables Heap tables versus IOTs

Heap-organised table
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3) CONSTRAINT team_pk PRIMARY KEY (team_key); ) ORGANIZATION HEAP; SELECT team_name FROM team WHERE team_key = FER;
SELECT STATEMENT TABLE ACCESS (BY INDEX ROWID) OF TEAM INDEX (UNIQUE SCAN) OF TEAM_PK

Index-organised table
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3) CONSTRAINT team_pk PRIMARY KEY (team_key); ) ORGANIZATION INDEX; SELECT team_name FROM team WHERE team_key = FER;
SELECT STATEMENT INDEX (UNIQUE SCAN) OF TEAM_PK

14

2005 Julian Dyke

juliandyke.com

Index Organised Tables Heap tables versus IOTs

Heap-organised table
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3) CONSTRAINT team_pk PRIMARY KEY (team_key); ) ORGANIZATION HEAP; Root Branch Leaf

Index-organised table

CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3) CONSTRAINT team_pk PRIMARY KEY (team_key); ) ORGANIZATION INDEX;

Table

15

2005 Julian Dyke

juliandyke.com

Index Organized Tables Segments



IOTs do not have table segments Index segment name depends on primary key constraint Out-of-line constraints  Index segment name same as constraint name Inline constraints  Index segment name is
SYS_IOT_TOP_<objectid>


e.g.
SYS_IOT_TOP_6650

where objectid is object ID of IOT / IOT partition

16

2005 Julian Dyke

juliandyke.com

Index Organized Tables Example Index Block

INSERT INTO team VALUES (FER,Ferrari,ITA); Row#0[8015] flag: K----, lock: 2 col 0; len 3; (3): 46 45 52 tl: 15 fb: --H-FL lb: 0x0 cc: 2 col 0: [ 7] Dump of memory from 0x05C92FB5 to 0x05C92FBC 5C92FB0 72654607 69726172 [.Ferrari] col 1: [ 3] Dump of memory from 0x05C92FBD to 0x05C92FC0 5C92FB0 41544903 [.ITA] Index Row Header Index Col 0 Table Row Header Table Col 0 Table Col 1

F E R F e r r a r i I T A 04 02 03 46 45 52 2C 00 02 07 46 65 72 72 61 72 69 03 49 54 41
Flag
17

Lock

Length

Flag

Lock

#Cols

Length

Length

2005 Julian Dyke

juliandyke.com

Index Organized Tables Heap organised tables versus IOTs

Heap-organised table
Index Row Header Index Row Table ROWID Index Col 0

00 02
Table Row Header

F E R 01 00 01 E2 00 00 03 46 45 52
Table Col 0 Table Col 1 Table Col 2

Table Row

F E R 2C 01 03 03 46 45 52

F e r r a r 07 46 65 72 72 61 72 69i

I T A 03 49 54 41

Index-organised table
Index Row Header Index Row
18

Index Col 0

Table Row Header

Table Col 0

Table Col 1

04 02 03 46 45 52 2C 00 02 07 46 65 72 72 61 72 69 03 49 54 41 F E R I T A F e r r a r i

2005 Julian Dyke

juliandyke.com

Index Organized Tables Physical ROWIDs

Heap-organised table
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3) CONSTRAINT team_pk PRIMARY KEY (team_key); ) ORGANIZATION HEAP; SELECT ROWID FROM team; ROWID AAABvFAAEAAAADgAAA AAABvFAAEAAAADgAAB AAABvFAAEAAAADgAAC AAABvFAAEAAAADgAAD

Index-organised table
CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3) CONSTRAINT team_pk PRIMARY KEY (team_key); ) ORGANIZATION INDEX; SELECT ROWID FROM team; ROWID *BAEAAOoDQUdT/g *BAEAAOoDQUxG/g *BAEAAOoDQU1P/g *BAEAAOoDQU5E/g

19

2005 Julian Dyke

juliandyke.com

Index Organized Tables Column Ordering



If primary key columns are not first in table declaration, Oracle will reorder columns within the segment
CREATE TABLE team ( team_name VARCHAR2(50), country_key VARCHAR2(3), team_key VARCHAR2(3) PRIMARY KEY ) ORGANIZATION INDEX

DESCRIBE team
Column Name team_name country_key team_key Column Type VARCHAR2(50) VARCHAR2(3) VARCHAR2(3) NAME team_name country_key team_key

SYS.COL$
COL# INTCOL# SEGCOL# 1 2 3 1 2 3 2 3 1

20

2005 Julian Dyke

juliandyke.com

Index Organized Tables Column Ordering

INSERT INTO team VALUES (Ferrari ,ITA, FER); Row#0[8015] flag: K----, lock: 2 col 0; len 3; (3): 46 45 52 tl: 15 fb: --H-FL lb: 0x0 cc: 2 col 0: [ 7] Dump of memory from 0x05C92FB5 to 0x05C92FBC 5C92FB0 72654607 69726172 [.Ferrari] col 1: [ 3] Dump of memory from 0x05C92FBD to 0x05C92FC0 5C92FB0 41544903 [.ITA] Index Row Header Index Col 0 Table Row Header Table Col 0 Table Col 1

F E R F e r r a r i I T A 04 02 03 46 45 52 2C 00 02 07 46 65 72 72 61 72 69 03 49 54 41
Flag
21

Lock

Length

Flag

Lock

#Cols

Length

Length

2005 Julian Dyke

juliandyke.com

Index Organised Tables Summary



For SELECT statements  useful for lookup tables  avoid additional logical I/O for non-key data  reduce latching Do not store physical ROWID  Six bytes less storage required Optionally can store non-key data  Requires three byte table row header

22

2005 Julian Dyke

juliandyke.com

IOT Overflow Introduction



Introduced in Oracle 8.0 Inefficient to store long rows in index segment Store rarely accessed columns in overflow segment Primary key rows always stored in index segment Non-key rows can stored in index segment or overflow Overflow segment is additional table segment Index row includes six byte ROWID for overflow row

 

   

23

2005 Julian Dyke

juliandyke.com

IOT Overflow PCTTHRESHOLD Clause

  

Specifies percentage of space reserved in index block for IOT row Must be large enough to hold primary key All columns causing threshold to be exceeded are stored in overflow segment Syntax is
PCTTHRESHOLD threshold

 

where threshold is integer in the range 1..50 Default value is 50

CREATE TABLE team ( team_key VARCHAR2(3) PRIMARY KEY, team_name VARCHAR2(50), history VARCHAR2(4000) ) ORGANIZATION INDEX OVERFLOW PCTTHRESHOLD 5;

24

2005 Julian Dyke

juliandyke.com

IOT Overflow INCLUDING Clause

  

Can be used to specify which columns are held on index block Syntax is
INCLUDING column_name

where column_name can be  last column of primary key  any non-key column
CREATE TABLE team ( team_key VARCHAR2(3) PRIMARY KEY, team_name VARCHAR2(50), history VARCHAR2(4000) ) ORGANIZATION INDEX OVERFLOW INCLUDING team_name; Column Name team_key team_name history Column Type Primary Key Non-Key Non-Key Block Type Index Block Index Block Overflow Block

25

2005 Julian Dyke

juliandyke.com

IOT Overflow Tables Segments



IOT Overflow Table segments are created automatically when OVERFLOW clause is used Overflow segment name is
SYS_IOT_OVER_<object_id>

e.g.
SYS_IOT_OVER_6650

where object_id is the object id of the IOT / IOT partition Overflow segment cannot be directly accessed by SELECT, INSERT, UPDATE or DELETE operations For partitioned IOTs, overflow segment is locally partitioned

26

2005 Julian Dyke

juliandyke.com

IOT Overflow Execution Plans

CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), history VARCHAR2(4000), CONSTRAINT team_pk PRIMARY KEY (team_key) ) ORGANIZATION INDEX OVERFLOW INCLUDING team_name; INSERT INTO team VALUES (FER,Ferrari,History); SELECT team_name, history FROM team WHERE team_key = FER; 0 1 SELECT STATEMENT 0 INDEX (UNIQUE SCAN) OF TEAM_PK

27

2005 Julian Dyke

juliandyke.com

IOT Overflow Example Index Block

INSERT INTO team VALUES (FER,Ferrari,ITA); Row#0[8013] flag: K----, lock: 2 col 0; len 3; (3): 46 45 52 tl: 17 fb: --H-F- lb: 0x0 cc: 1 nrid: 0x010000ea.0 col 0: [ 7] Dump of memory from 0x04B92FB9 to 0x04B92FC0 4B92FB0 72654607 69726172 [.Ferrari]

Index Row Header

Index Col 0

Table Row Header

Pointer to Overflow

Table Col 0

F E R 04 02 03 46 45 52
Flag Lock Length

28 00 01
Lock

F e r r a r i 01 00 00 EA 00 00 07 46 65 72 72 61 72 69
#Cols ROWID Length

Flag

28

2005 Julian Dyke

juliandyke.com

IOT Overflow Example Overflow Block

INSERT INTO team VALUES (FER,Ferrari,ITA); tab 0, row 0, @0x1fb1 tl: 7 fb: -----L lb: 0x1 col 0: [ 3] 49 54 41 cc: 1

Table Row Header

Table Col 0

04 01 01 03 49 54 41 I T A
Flag Lock #Cols Length

29

2005 Julian Dyke

juliandyke.com

IOT Overflow Summary

 

Store frequently accessed columns in index segment Store rarely accessed columns in overflow segments Overflow segments  Useful for variable length columns e.g  Useful for nullable columns  May reduce index height  May increase row density Statements accessing overflow segments  must access index segment first  require additional logical I/Os

30

2005 Julian Dyke

juliandyke.com

IOT Secondary Indexes Introduction



Introduced in Oracle 8.1.5 Allow additional B*tree indexes to be created on IOTs Allow new access paths Can be created on  Primary key columns  Non key columns  Overflow columns Can be compressed Can be partitioned Cannot be reversed

 

  

31

2005 Julian Dyke

juliandyke.com

IOT Secondary Indexes Execution Plans

CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3), CONSTRAINT team_pk PRIMARY KEY (team_key) ) ORGANIZATION INDEX; CREATE INDEX team_country ON team (country_key); SELECT team_key,team_name FROM team WHERE country_key = ITA; 0 1 2 SELECT STATEMENT 0 INDEX (UNIQUE SCAN) OF TEAM_PK 1 INDEX (RANGE_SCAN) OF TEAM_COUNTRY;

32

2005 Julian Dyke

juliandyke.com

IOT Secondary Indexes Row Structure



Each secondary index row contains  key values  physical ROWID guess  primary key Not the same as UROWIDs As insertions occur IOT rows can move to different blocks Physical ROWID guesses are not updated in secondary index Guesses degrade over time Percentage of rows with valid guesses is collected by ANALYZE and DBMS_STATS and recorded in DBA_INDEXES.PCT_DIRECT_ACCESS

  

33

2005 Julian Dyke

juliandyke.com

IOT Secondary Indexes Example Leaf Row

INSERT INTO team VALUES (FER,Ferrari,ITA); row#0[8018] flag: K----, lock: 2 col 0; len 3; (3): 49 54 41 col 1; len 3; (3): 46 45 52 tl: 8 fb: --H-FL lb: 0x0 cc: 1 col 0: [ 4] Dump of memory from 0x05B12FBC to 0x05B12FC0 5B12FB0 EA010001

Index Row Header

Index Col 0

Index Col 1

Table Row Header

Pointer to IOT Block

I T A 04 02 03 49 54 41
Flag Lock Length

F E R 03 46 45 52 2C 00 01
Length Flag Lock

04 01 00 01 EA
#Cols Length

34

2005 Julian Dyke

juliandyke.com

IOT Mapping Tables Introduction

  

Introduced in Oracle 9.2.0 Required to support IOT bitmap secondary indexes IOT rows subject to block splits  Can move from block to block Bitmap indexes rows reference contiguous set of slots IOT must be created with MAPPING TABLE clause Bitmap index stores mapping table blocks Read access via IOT bitmap secondary indexes as follows  Obtain mapping table block number from bitmap index  Obtain primary key from mapping table block  Use primary key to access IOT

 

35

2005 Julian Dyke

juliandyke.com

IOT Mapping Tables Segments



IOT Mapping Table segments are created automatically when MAPPING TABLE clause is used Segment name is
SYS_IOT_MAP_<object_id>

e.g.
SYS_IOT_MAP_6650

Where object_id is the object id of the IOT / IOT partition Mapping table contains one column
Column Name SYS_NC_01 Column Type UROWID (Type 208)

36

2005 Julian Dyke

juliandyke.com

IOT Mapping Tables Execution Plan

CREATE TABLE team ( team_key VARCHAR2(3), team_name VARCHAR2(50), country_key VARCHAR2(3), CONSTRAINT team_pk PRIMARY KEY (team_key) ) ORGANIZATION INDEX MAPPING TABLE; CREATE BITMAP INDEX team_country ON team (country_key); SELECT team_key,team_name FROM team WHERE country_key = ITA; 0 1 2 3 SELECT STATEMENT 0 INDEX (UNIQUE SCAN) OF SYS_IOT_TOP_6650 1 BITMAP CONVERSION (TO ROWIDS) 2 BITMAP INDEX (SINGLE VALUE) OF TEAM_COUNTRY;

37

2005 Julian Dyke

juliandyke.com

IOT Mapping Tables Example Bitmap Index Leaf Block

INSERT INTO team VALUES (FER,Ferrari,ITA); Row#0[8013] flag: -----, Col 0: len 3; (3): 49 54 Col 1: len 6; (6): 01 00 Col 2: len 6; (6): 01 00 Col 3: len 1; (2): 00 Index Row Header Index Key Column Start ROWID lock 0 41 0a 62 00 00 0a 62 00 07

End ROWID

Bitmap

00 00 03 49 54 41 I T A
Flag Lock Length

06 01 00 0A 62 00 00
Length

06 01 00 0A 62 00 07
Length

01 00
Length

Start ROWID and end ROWID refer to blocks in mapping table

38

2005 Julian Dyke

juliandyke.com

IOT Mapping Tables Example Mapping Table Block

INSERT INTO team VALUES (FER,Ferrari,ITA); tab 0, row 0, @0x1f91 tl: 15 fb: --HFL- lb: 0x1 cc:1 col 0: [11] 02 04 00 00 00 00 03 46 45 52 FE Table Row Header UROWID

2C 01 01 0B 02
Flag Lock #Cols

F E R 04 00 00 00 00 03 46 45 52
Block Pointer ? Index Col 0

FE

0B 02
Length
39

F E R 04 00 00 00 00 03 46 45 52 FE
Length Length Terminator

#Cols

2005 Julian Dyke

juliandyke.com

IOT Mapping Indexes Example IOT Block

INSERT INTO team VALUES (FER,Ferrari,ITA); Row#0[8009] flag: K----, lock: 2, data:(6) 01 00 0a 62 00 00 col 0; len 3; (3): 46 45 52 tl: 15 fb: --H-FL lb: 0x0 cc: 2 col 0: [ 7] Dump of memory from 0x05C92FB5 to 0x05C92FBC 5C92FB0 72654607 69726172 [.Ferrari] col 1: [ 3] Dump of memory from 0x05C92FBD to 0x05C92FC0 5C92FB0 41544903 [.ITA] Index Row Header Mapping Table ROWID Index Col 0 Table Row Header Table Col 0 Table Col 1

F E R I 54 41 04 02 01 00 0A 62 00 00 03 46 45 52 2C 00 02 07 46 65 72 72 61 72 69 03 49 T A F e r r a r i
Flag
40

Lock

Length

Flag

Lock

#Cols

Length

Length

2005 Julian Dyke

juliandyke.com

IOT Mapping Tables Summary



Bitmap indexes can be much more efficient than B*Tree indexes requiring  Less physical storage  Fewer logical I/Os Mapping tables require  Additional physical disk usage  Additional logical I/Os for SELECT and DML Each IOT leaf row requires additional six bytes for mapping table ROWID

41

2005 Julian Dyke

juliandyke.com

Thank you for your interest

For more information and to provide feedback please contact me My e-mail address is: info@juliandyke.com My website address is: www.juliandyke.com

42

2005 Julian Dyke

juliandyke.com