Adaptive Query Optimization

Oracle OpenWorld, 28 September 2014

Christian Antognini

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

2014 © Trivadis
1 Adaptive Query Optimization
28 September 2014
 @ChrisAntognini

 Senior principal consultant, trainer and partner

at Trivadis in Zurich (CH)
 christian.antognini@trivadis.com
 http://antognini.ch
 Focus: get the most out of Oracle Database
 Logical and physical database design
 Query optimizer
 Application performance management
 Author of Troubleshooting Oracle Performance (Apress, 2008/2014)
 OakTable Network, Oracle ACE Director

2014 © Trivadis
2 Adaptive Query Optimization
28 September 2014
 Introduction

 Over the years Oracle has extremely improved the capabilities of the query
optimizer
 Most of the improvements fall into one of the following areas
 Enhance the quality of the inputs (e.g. objects statistics)
 Make the gathering and management of object statistics easier and more efficient
 Use new data structures
 Implement or enhance query transformations
 Improve plan stability
 Cope with poor estimations that leads to poor execution plans

 12c is no exception, every one of these areas were improved

2014 © Trivadis
3 Adaptive Query Optimization
28 September 2014
 Adaptive Query Optimization Features

 Adaptive plans (Enterprise Edition only)

 Join methods
 Parallel distribution methods
 Star transformation

 Adaptive statistics
 SQL plan directives
 Automatic reoptimization
- Statistics feedback (evolution of cardinality feedback)
- Performance feedback
 Dynamic statistics (evolution of dynamic sampling)

2014 © Trivadis
4 Adaptive Query Optimization
28 September 2014
 OPTIMIZER_ADAPTIVE_FEATURES

 Enables or disables adaptive query optimization features

 Adaptive plans
 SQL plan directives
 Automatic reoptimization
- It isn’t the case in 12.1.0.1 (bug 16824474)

 The default value is TRUE

 OPTIMIZER_DYNAMIC_SAMPLING controls dynamic statistics

2014 © Trivadis
5 Adaptive Query Optimization
28 September 2014
 OPTIMIZER_ADAPTIVE_REPORTING_ONLY

 Enables or disables reporting mode for adaptive query optimization features

 Useful to assess how an execution plan would change
 Use DBMS_XPLAN to get detail information about the analysis
 Might fail with an ORA-1001 (bug 17270605)

SELECT *
FROM table(dbms_xplan.display_cursor(format=>'report'))

 The default value is FALSE

2014 © Trivadis
6 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Challenge

 Object statistics don’t always provide sufficient information to find an optimal

execution plan
 To get additional insights, the query optimizer can take advantage of features
like dynamic sampling and cardinality feedback
 Don’t solve all issues, though

2014 © Trivadis
7 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Concept

 As of 12c, the query optimizer can postpone some decisions until the execution
phase
 The idea is to leverage information collected while executing part of an
execution plan to determine how another part should be carried out
 The query optimizer uses adaptive plans in three situations:
 To switch the join method from a nested loops join to a hash join and vice versa
 To switch the PX distribution method from hash to broadcast/round-robin
 To disable the access to a dimension for execution plans using the star transformation

2014 © Trivadis
8 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Join Method Switch

 The query optimizer adds subplans to execution plans

 One represents the nested loops join, the other the hash join
 One of the alternatives is the default plan

 One of the subplans is chosen during the first execution

 The choice is based on the number of rows actually processed
 The query optimizer computes an inflection point

 A new row source operation is used to partially buffer and count the rows
 STATISTICS COLLECTOR
 Limitation: if a too large buffer is required, no adaptive plan is used

 The execution plan that is actually executed is called the final plan

2014 © Trivadis
9 Adaptive Query Optimization
28 September 2014
 adaptive_plan.sql
Adaptive Plans – Join Method Switch Example

SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666

-----------------------------------------------
| Id | Operation | Name |
-----------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | HASH JOIN | |
| 2 | NESTED LOOPS | |
| 3 | NESTED LOOPS | |
| 4 | STATISTICS COLLECTOR | |
| 5 | TABLE ACCESS FULL | T1 |
| 6 | INDEX UNIQUE SCAN | T2_PK |
| 7 | TABLE ACCESS BY INDEX ROWID| T2 |
| 8 | TABLE ACCESS FULL | T2 |
-----------------------------------------------

2014 © Trivadis
10 Adaptive Query Optimization
28 September 2014
 adaptive_plan.sql
Adaptive Plans – Join Method Switch Example

SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666

-----------------------------------------------
| Id | Operation | Name |
-----------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | HASH JOIN | |
| 2 | NESTED LOOPS | |
| 3 | NESTED LOOPS | |
| 4 | STATISTICS COLLECTOR | |
| 5 | TABLE ACCESS FULL | T1 |
| 6 | INDEX UNIQUE SCAN | T2_PK |
| 7 | TABLE ACCESS BY INDEX ROWID| T2 |
| 8 | TABLE ACCESS FULL | T2 |
-----------------------------------------------

2014 © Trivadis
11 Adaptive Query Optimization
28 September 2014
 adaptive_plan.sql
Adaptive Plans – Join Method Switch Example

SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666

-----------------------------------------------
| Id | Operation | Name |
-----------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | HASH JOIN | |
| 2 | NESTED LOOPS | |
| 3 | NESTED LOOPS | |
| 4 | STATISTICS COLLECTOR | |
| 5 | TABLE ACCESS FULL | T1 |
| 6 | INDEX UNIQUE SCAN | T2_PK |
| 7 | TABLE ACCESS BY INDEX ROWID| T2 |
| 8 | TABLE ACCESS FULL | T2 |
-----------------------------------------------

2014 © Trivadis
12 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Join Method Switch Inflection Point

 For both join methods, the cost associated to different cardinalities is estimated
 The cardinality of the outer table varies
 The cardinality of the inner table remains fixed

 The query optimizer uses a binary search

 The search takes place between a minimum and maximum cardinality

2014 © Trivadis
13 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Join Method Switch Inflection Point Example

20,000

2,000
Cost

Nested Loops Join

200 Hash Join

20

Cardinality
(In Order of Execution)

2014 © Trivadis
14 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Parallel Distribution Method

 New distribution method HYBRID HASH

 It helps avoiding some data skewing problems

 The actual distribution method is chosen at execution time

 STATISTICS COLLECTOR
 The decision takes place during each execution

 Number of rows < 2 * DOP

 Left input: BROADCAST / Right input: ROUND-ROBIN

 Number of rows ≥ 2 * DOP

 Left input: HASH / Right input: HASH

2014 © Trivadis
15 Adaptive Query Optimization
28 September 2014
 hybrid_hash.sql
Adaptive Plans – Parallel Distribution Method Example

------------------------------------------------------------------------
| Operation | Name | TQ |IN-OUT| PQ Distrib |
------------------------------------------------------------------------
| SELECT STATEMENT | | | | |
| SORT AGGREGATE | | | | |
| PX COORDINATOR | | | | |
| PX SEND QC (RANDOM) | :TQ10002 | Q1,02 | P->S | QC (RAND) |
| SORT AGGREGATE | | Q1,02 | PCWP | |
| HASH JOIN | | Q1,02 | PCWP | |
| PX RECEIVE | | Q1,02 | PCWP | |
| PX SEND HYBRID HASH | :TQ10000 | Q1,00 | P->P | HYBRID HASH|
| STATISTICS COLLECTOR | | Q1,00 | PCWC | |
| PX BLOCK ITERATOR | | Q1,00 | PCWC | |
| TABLE ACCESS FULL | T1 | Q1,00 | PCWP | |
| PX RECEIVE | | Q1,02 | PCWP | |
| PX SEND HYBRID HASH | :TQ10001 | Q1,01 | P->P | HYBRID HASH|
| PX BLOCK ITERATOR | | Q1,01 | PCWC | |
| TABLE ACCESS FULL | T2 | Q1,01 | PCWP | |
------------------------------------------------------------------------

2014 © Trivadis
16 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Star Transformation

 With the star transformation, the data of each dimension that has a restriction
applied to it might be “joined” to the corresponding bitmap index of the fact
 If the number of rowids returned by such a “join” is underestimated, applying the
filter can be detrimental to the performance
 With an adaptive plan the access to some dimensions can be disabled
 Decision takes place during the first execution only

2014 © Trivadis
17 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – Star Transformation Example

-----------------------------------------------------
| Operation | Name |
-----------------------------------------------------
| … | |
| VIEW | VW_ST_5497B905 |
| NESTED LOOPS | |
| BITMAP CONVERSION TO ROWIDS | |
| BITMAP AND | |
| BITMAP MERGE | |
| BITMAP KEY ITERATION | |
| TABLE ACCESS FULL | COLORS |
| BITMAP INDEX RANGE SCAN | CAR_COLOR_IDX |
| STATISTICS COLLECTOR | |
| BITMAP MERGE | |
| BITMAP KEY ITERATION | |
| TABLE ACCESS FULL | MODELS |
| BITMAP INDEX RANGE SCAN| CAR_MODEL_IDX |
| … | |
| TABLE ACCESS BY USER ROWID | CARS |
-----------------------------------------------------
2014 © Trivadis
18 Adaptive Query Optimization
28 September 2014
 Adaptive Plans – V$SQL.IS_RESOLVED_ADAPTIVE_PLAN

 NULL means that the execution plan associated to the cursor isn’t adaptive
 N means that the final execution plan hasn’t been determined
 Y means that the final execution plan was determined
 Also set if reporting mode is enabled

 Set for join method switches and star transformation only

2014 © Trivadis
19 Adaptive Query Optimization
28 September 2014
 Adaptive Statistics – SQL Plan Directives

 SQL plan directives are automatically created when misestimates occur

 They are temporarily stored in the SGA and flushed to disk every 15 min by a
background process

 They aren’t tied to a specific SQL statement, but to specific tables/columns

 Several of them can be used for a single SQL statement

 They instruct the database engine to automatically create extended statistics

 Only column groups are considered
 The limit of 20 extensions applies to them as well
 If creating extended statistics isn’t supported/possible/sensible, they instruct the query
optimizer to use dynamic sampling

2014 © Trivadis
20 Adaptive Query Optimization
28 September 2014
 Adaptive Statistics – Management of SQL Plan Directives

 The database engine automatically maintains SQL plan directives

 They are automatically created
 They are automatically purged if not used (by default after 53 weeks)

 They can be manually managed through DBMS_SPD

 Flush to disk SQL plan directives temporarily stored in the SGA
 Change retention period
 Immediate purge
 Alter attributes (ENABLED, AUTO_DROP)
 Pack/Unpack
 Requirement: ADMINISTER SQL MANAGEMENT OBJECT privilege

2014 © Trivadis
21 Adaptive Query Optimization
28 September 2014
 Adaptive Statistics – SQL Plan Directives in the Data Dictionary

 DBA_SQL_PLAN_DIRECTIVES
 DBA_SQL_PLAN_DIR_OBJECTS
SQL> SELECT type, reason, count(*)
2 FROM dba_sql_plan_directives
3 GROUP BY type, reason;

TYPE REASON COUNT(*)

---------------- ------------------------------------ ----------
DYNAMIC_SAMPLING SINGLE TABLE CARDINALITY MISESTIMATE 81
DYNAMIC_SAMPLING JOIN CARDINALITY MISESTIMATE 180
DYNAMIC_SAMPLING GROUP BY CARDINALITY MISESTIMATE 6

2014 © Trivadis
22 Adaptive Query Optimization
28 September 2014
 statistics_feedback1.sql
Adaptive Statistics – Statistics Feedback statistics_feedback2.sql

 Evolution of cardinality feedback

 Used for single-table cardinalities as well as join cardinalities
 Information about misestimates might be persisted through SQL plan directives
 For misestimates due to table functions no information is stored

 V$SQL.IS_REOPTIMIZABLE
 Y means that the next execution will trigger a reoptimization
 N means that no reoptimization is necessary
 R means that reoptimization information is available, but reporting mode was enabled

2014 © Trivadis
23 Adaptive Query Optimization
28 September 2014
 Adaptive Statistics – Performance Feedback

 Enabled when PARALLEL_DEGREE_POLICY = ADAPTIVE

 Assessment of the auto DOP after the first execution
 If auto DOP is suboptimal, the next execution will trigger a reoptimization

2014 © Trivadis
24 Adaptive Query Optimization
28 September 2014
 dynamic_statistics.sql
Adaptive Statistics – Dynamic Statistics

 Evolution of dynamic sampling

 Used for single-table cardinalities as well as join and group-by cardinalities
 The query optimizer decides when and how to use dynamic statistics
 The data collected through dynamic statistics is shared through the result cache
 The sampling can’t exceed a given amount of time
 The maximum is determined based on past executions (cursor cache, AWR)

 To use the new features OPTIMIZER_DYNAMIC_SAMPLING has to be set to 11

2014 © Trivadis
25 Adaptive Query Optimization
28 September 2014
 Summary

 Adaptive query optimization

introduced a number of very
interesting features
 The query optimizer is getting more
and more dynamic

2014 © Trivadis
26 Adaptive Query Optimization
28 September 2014
 Questions and answers ...
Christian Antognini
Senior Principal Consultant
christian.antognini@trivadis.com

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

2013 © Trivadis
2014
27 Adaptive Query Optimization
28 September 2014