Sunteți pe pagina 1din 44

Transform Batch to Real Time

with the Oracle Database


In-Memory 12c Option on
Oracle Systems
Vince Carbone
Hui Dong
David Lutz
Oracle Performance Engineering
October 1, 2014

Copyright 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Internal/Restricted/Highly Restricted

Safe Harbor Statement


The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracles products remains at the sole discretion of Oracle.

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda
1

Configuring the In-Memory Column Store

Populating the In-Memory Column Store

Managing the In-Memory Column Store on RAC

Querying the In-Memory Column Store

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda
1

Configuring the In-Memory Column Store

Populating the In-Memory Column Store

Managing the In-Memory Column Store on RAC

Querying the In-Memory Column Store

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Memory Bandwidth Enables Batch to Real-Time


DRAM

PCI FLASH

Oracle SuperCluster T5-8


(Full Rack)

DISK

~1280 GB/sec

58 GB/sec

In-memory delivers

Over 100x more throughput than disk

Over 14x more throughput than flash

What was batch can now be run


real time

12 GB/sec

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Breakthrough: Dual Format Database


Memory

Memory

SALES

SALES

Row
Format

Column
Format

BOTH row and column


formats for same table
Simultaneously active and
transactionally consistent
Analytics & reporting use new
in-memory Column format
OLTP uses proven row format

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

SPARC Systems For Maximum In-Memory Performance


Large Memory/High Memory Bandwidth
IM performs best when
complete working set is in
memory
M6-32 32TB

SuperCluster T5-8 and M6-32

Scale-up on large SMPs


Scale-out on SuperCluster

T5-8 4TB
T4-1 512GB

T5-2 1TB

T5-4 2TB

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Configuring : In-Memory Column Store


System Global Area SGA
Buffer Cache

Large Pool

Shared Pool

Other shared
Memory Components

Redo Buffer

In-Memory Area

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Configuring : In-Memory Column Store


SELECT * FROM V$SGA;

NAME
-----------------Fixed Size
Variable Size
Database Buffers
Redo Buffers
In-Memory Area

VALUE
--------2927176
570426808
4634022912
13848576
1024483648

Controlled by INMEMORY_SIZE
parameter
Minimum size of 100MB
Must be large enough to
accommodate both column store and
meta-data area

SGA_TARGET must be large


enough to accommodate
Implemented using Optimized
Shared Memory (OSM)
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Configuring : In-Memory Column Store


Oracle Multitenant with Pluggable Databases
INMEMORY_SIZE=1600G
INMEMORY_SIZE=400G
Example:

INMEMORY_SIZE=0G
ERP

IMPDB1

Container Database

IMPDB2

SQL> alter session set container = IMPDB1;


SQL> alter system set inmemory_size = 400g;
SQL> alter session set container = IMPDB2;
SQL> alter system set inmemory_size = 1600g;

INMEMORY_SIZE=2000G

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

11

Configuring : In-Memory Column Store


Oracle Multitenant with Pluggable Databases
INMEMORY_SIZE=1600G
INMEMORY_SIZE=1000G
INMEMORY_SIZE=0G
ERP

IMPDB1

IMPDB2

Over subscription
is possible!

INMEMORY_SIZE=2000G

Container Database

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

12

Configuring : In-Memory Column Store


New INMEMORY ATTRIBUTE
ALTER TABLE sales INMEMORY;

ALTER TABLE sales NO INMEMORY;

Following segment types are

eligible
Tables
Partitions

CREATE TABLE
PARTITION BY
(PARTITION
(PARTITION

customers
LIST
p1 INMEMORY,
p2 NO INMEMORY);

Subpartition
Materialized views

Following segment types not

eligible
IOTs

Hash clusters
Out of line LOBs
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Pure OLTP
Features

Oracle Compression Advisor And In-Memory


Easy way to determine

memory requirements
Use DBMS_COMPRESSION
Applies MEMCOMPRESS to
sample set of data from a table
Returns estimated
compression ratio

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda
1

Configuring the In-Memory Column Store

Populating the In-Memory Column Store

Managing the In-Memory Column Store on RAC

Querying the In-Memory Column Store

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Oracle In-Memory Columnar Technology


Pure In-Memory Columnar

Pure in-memory column format


Not persistent, and no logging
Quick to change data: fast OLTP

2x to 20x compression

Enabled at table or partition


level
SALES

Brings existing data into


memory and formats it in an
optimized columnar format
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Populating : In-Memory Column Store


Memory
IMCU

IMCU

IMCU

IMCU

IMCU

IMCU

Population is highly parallel and


scalable
Completed by new set of
background processes

ora_wNNN_orcl

Number of processes controlled by


Column Format

INMEMORY_MAX_POPULATE_SERVERS

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Populating : In-Memory Column Store


Memory
IMCU

IMCU

IMCU

IMCU

IMCU

IMCU

Column Format

Each object populated in the


Column Store is actually made
up of multiple In-Memory
Compression Units
Each IMCU contains the column
entries for a subset of rows in
the object
Each column is encoded and/or
compressed individually

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Populating : In-Memory Column Store


ALTER TABLE sales INMEMORY;
ALTER TABLE sales NO INMEMORY;

ALTER TABLE sales INMEMORY


NO INMEMORY (PROD_ID);
CREATE TABLE orders
(c1 number,
c2 varchar(20),
c3 number)
INMEMORY PRIORITY CRITICAL
NO INMEMORY (c1);

New INMEMORY attribute


Possible to populate only

certain columns from a table


or partition
Order (not speed) in which
objects are populated
controlled by PRIORITY
subclause
Critical, high, medium, low

Default none (populate on

first access)
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Populating : In-Memory Column Store


CREATE TABLE ORDERS
PARTITION BY RANGE
(PARTITION p1
INMEMORY NO MEMCOMPRESS
PARTITION p2
INMEMORY MEMCOMPRESS FOR DML,
PARTITION p3
INMEMORY MEMCOMPRESS FOR QUERY,
:
PARTITION p200
INMEMORY MEMCOMPRESS FOR CAPACITY
);

Objects compressed during

population
Different levels
FOR DML - Use on tables or

partitions with very active DML


activity
FOR QUERY - Default mode for

most tables
FOR CAPACITY - For less

frequently accessed segments


Great for ILM strategy
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Identifying : Tables With INMEMORY Attribute


SELECT table_name, inmemory
FROM
USER_TABLES;
TABLE_NAME
-----------CHANNELS
COSTS
CUSTOMERS
PRODUCTS
SALES
TIMES

INMEMORY
-------DISABLED
DISABLED
ENABLED
DISABLED

New INMEMORY column in


*_TABLES dictionary views
INMEMORY is a segment

attribute
doesnt display
segment attributes for logical
objects
Both COSTS & SALES are
partitioned => logical objects
INMEMORY attribute also
reported in *_TAB_PARTITIONS
USER_TABLES

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Identifying : Tables With INMEMORY Attribute


SELECT segment_name name,
population_status status
FROM
v$IM_SEGMENTS;

NAME
-----------PRODUCTS
SALES

STATUS
-------COMPLETED
STARTED

New view V$IM_SEGMENTS


Indicate:
Objects populated in memory
Current population status
Can also be used to determine

compression ratio achieved

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Identifying : Columns Without the INMEMORY Attribute

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda
1

Configuring the In-Memory Column Store

Populating the In-Memory Column Store


DML and the In-Memory Column Store

Managing the In-Memory Column Store on RAC

Querying the In-Memory Column Store

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

DML and the In-Memory Column Store


Memory
IMCU
JOURNAL

A journal is used to keep the


column store transactionally
consistent
DML operations processed in
row store as they are today
Corresponding entry in
column store marked stale

Column
Format

Copy of changed row stored


in Transaction Journal
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

DML and the In-Memory Column Store


Memory
In-Memory Column Store is
never out of date

IMCU
JOURNAL

Read consistency achieved by


merging contents of column and
the transaction journal
When number of entries in
journal hits internal threshold
CU is refreshed

Column
Format

This is an online operation


columns store always available
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda
1

Configuring the In-Memory Column Store

Populating the In-Memory Column Store

Managing the In-Memory Column Store on RAC

Querying the In-Memory Column Store

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Scale-Out In-Memory Database to Any Size

Scale-Out across servers to grow


memory and CPUs

In-Memory queries parallelized across


servers to access local column data

Scale-out policy is a segment level


(table, partition, sub partition) property

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Scale-Out In-Memory Database to Any Size


ALTER TABLE sales INMEMORY
DISTRIBUTE BY PARTITION;
ALTER TABLE COSTS INMEMORY
DISTRIBUTE AUTO;

Policy is user-specifiable
Controlled by DISTRIBUTE
subclause

Distribute by rowid range

Distribute by partition

Distribute AUTO

Parallel Scan Affinity

Requires parallel_degree_policy=AUTO

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Oracle Database In-Memory: Unique Fault Tolerance


Similar to storage mirroring
Duplicate in-memory
columns on another node
Enabled per table/partition
Application transparent

In-Memory Column Store


remains available by using
duplicate after failure
Only Available on Engineered Systems

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Oracle Database In-Memory: Unique Fault Tolerance


ALTER TABLE sales INMEMORY
DUPLICATE;
ALTER TABLE COSTS INMEMORY
DISTRIBUTE AUTO DUPLICATE ALL;

Policy is user-specifiable
Controlled by DUPLICATE subclause

DUPLICATE
DUPLICATE ALL

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda
1

Configuring the In-Memory Column Store

Populating the In-Memory Column Store

Managing the In-Memory Column Store on RAC

Querying the In-Memory Column Store

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Why is an In-Memory scan faster than the buffer cache?


SELECT COL4 FROM MYTABLE;
Buffer Cache

IM Column Store
X
X
X
X
X

RESULT

Column Format

Row Format

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Oracle In-Memory Column Store Storage Index


Example: Find sales from stores with a store_id of 8 or higher
Memory
Min 1
Max 3

Min 4
Max 7

Min 8
Max 12

SALES
Column
Format

Min 13
Max 15

Each column is the made up


of multiple column units
Min / max value is recorded
for each column unit in a
storage index
Storage index provides
partition pruning like
performance for ALL queries

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Orders of Magnitude Faster Analytic Data Scans


Memory
PROMO_ID

Example:
Find all sales
With PROMO_ID 9999

Scans use super fast SIMD


vector instructions

CPU

PROMO_ID
values

VECTOR
REGISTER

9999

Load
multiple

Each CPU core scans local inmemory columns

9999
9999

9999

Vector
Compare
all values
in 1 cycle

Originally designed for


graphics & science
Billions of rows/sec scan rate
per CPU core
Row format is millions/sec

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Identifying : INMEMORY Table Scan


Optimizer fully aware
Cost model adapted to
consider INMEMORY scan
New access method
TABLE ACCESS IN MEMORY FULL

Can be disabled via new


parameter
INMEMORY_QUERY
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Identifying : INMEMORY Table Scan


NEW Session level statistics

Best way to determine if


In-Memory was used
Best way to measure the
benefits of In-Memory scan

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Joining and Combining Data Also Dramatically Faster


Example: Find all orders placed on Christmas eve
LINEORDER

Amount

Datekey is
24122013
DateKey

DateKey

Date

DATE_DIM

Converts joins of data in


multiple tables into fast column
scans
Joins tables 10x faster

Type=d.d_date='December 24, 2013'

Sum
Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

In-memory Joins
SELECT SUM(lo_extendedprice * lo_discount)
revenue
FROM
lineorder l,
date_dim d
WHERE l.lo_orderdate = d.d_datekey
AND
l.lo_discount BETWEEN 2 AND 3
AND
d.d_date='December 24, 2013';

Simple two-table join


between the
DATE_DIM and
LINEORDERS table
IM column store takes
advantage of Bloom
Filter to process table
joins very efficiently

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Creation and Use of Bloom Filter


Bloom filters enable joins to be
converted into fast column scans
Introduced in Oracle Database 10g.
Very efficiently applied to column
format data via SIMD.

Same technique used to offload joins


on Exadata

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Summary: Oracle Database In-Memory


Powering the Real-Time Enterprise

DATADRIVEN

AGILE

Extreme Performance: Analytics


& OLTP
Extreme Scale-Out & Scale-Up
Extreme Availability
Extreme Simplicity

EFFICIENT

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |

Copyright 2014, Oracle and/or its affiliates. All rights reserved. |