IDUG 2013 Sheryl Larsen Tuning SQL

Platform: z/OS Application Development
Tuning SQL for Performance

Sheryl M. Larsen Sheryl M. Larsen, Inc.
2013
IDUG, World Wide
Sheryl M. Larsen, Inc.

DB2 SQL Consulting & Education
Sheryl Larsen is an internationally recognized researcher, consultant and lecturer, specializing in DB2 and is known for her extensive expertise in SQL. She co-authored a book, DB2 Answers, OsborneMcGraw-Hill, 1999. She was voted an inaugural IBM Information Champion 20092011, IDUG Speaker Hall of Fame and, a member of IBMs DB2 Gold Consultants program since 1994. Sheryl has 25 years experience in DB2, has published articles, white papers, webtv, animated access paths:
Google Sheryl Larsen @

Bmc.com Ca.com Ibm.com
Currently, she is President of the Midwest Database Users Group (mwdug.org), and owns Sheryl M. Larsen, Inc., a firm specializing in Advanced DB2 Consulting and Education.
WWW.SMLSQL.COM
Contact: smlsql@comcast.net
(630) 399-3330
Sheryl M. Larsen, Inc. 2000-2011
Table of Contents - Tuning

SQL Performance DB2 Engine Components Predicate Processing Intelligence Optimizer Details SQL Performance Rules Tuning Queries When? Why? How? Introduction to Proven SQL Tuning Methods When are Access Paths Good or Bad? Variations of index access Variations of table access Variations of join methods Reading the Optimizers Mind PLAN_TABLE Output DSN_STATEMENT Output Case Studies Using a Proven Method Tuning Example 1 OPTIMIZE FOR n ROWS/FETCH FIRST n ROWS ONLY Tuning Example 2, 3, 4 No Operations Tuning Example 5, 6 Fake Filtering Tuning Example 7 Index Design Extreme Tuning Tuning Example 8 Distinct Table Expressions Tuning Example 9 Anti-Joins Tuning Example 10 (Predicate OR 0 = 1) Tuning Example 11 Extreme Cross Query Block Optimization Designing Performance Structures Index Design for a Single Query Index Design for a Workload MQT Design for a Workload
SQL Performance
DB2 Engine Components Predicate Processing Intelligence Optimizer Details SQL Performance Rules
DB2 EngineSQL Components Execution

Result
Work Files
SQL
Optimizer
Stage 2 z/OS Residual LUW

RID Pool
Catalog
Meta Data
Stage 1 z/OS Sargable LUW
Directory
Access Plans
Buffer Pool
Dynamic Statement Cache
Buffer Manager
Data Index
RIDs
DB2 V8 Enhancements
8K 16K 32K Pages 64 Bit Addressing
Result
SQL
Optimizer
2M Statement Length Multi-row FETCH Multi-row INSERT SELECT INTO w/ ORDER BY Common Table Expressions Recursive SQL IS NOT DISCTINCT FROM Scalar Fullselect GROUP BY Expression Dynamic Scrollable Cursors INSERT with SELECT Multiple DISTINCT Clauses Sequences
Work Files
RID Pool
Stage 2
Catalog
Meta Data
Top & Bottom NUDS
Parallel Sort
Stage 1 Unlike Data Types IS NOT NULL IS NOT DISTINCT FROM
Directory Stage 1 Dynamic Access Plans Statement More Hybrid Joins Buffer Pool Cache
Buffer Manager
Data
SMLI 2009
Forward and Backward Index DPSI Indexes
Materialized Query Tables
Index
RIDs
Index only on VARCHAR Indexes

6
DB2 9 WOW!
In Memory GLOBAL TEMP TABLES Views Table Expressions
TRUNCATE MERGE SELECT FROM UPDATE/DELETE EXCEPT, INTERSECT OLAP Expressions ORDER BY / FETCH FIRST in Subselect
Result
SQL
Histogram Statistics
Optimizer
Work Files
Sparse Indexes for more Work Files Faster Sorts for GROUP BY DISTINCT, ORDER BY
Stage 2
In Memory
RID Pool
Catalog
Meta Data
Global Optimization
Stage 1 Dynamic Statement Buffer Pool Cache
Directory
Access Plans
More Parallelism
Page Range Index Screening For Partitioned Indexes
Buffer Manager
Data Index
RIDs
Index on Expression
7
4K Page Processing z/OS

Work Files
Stage 2
4K
Stage 1 Buffer Pool

Buffer Manager
Data Index
8
Indexable Stage 1 Predicates

Predicate Type COL = value COL = noncol expr COL IS NULL COL op value COL op noncol expr COL BETWEEN value1 AND value2 COL BETWEEN noncol expr1 AND noncol expr2 COL LIKE 'pattern' COL IN (list) COL LIKE host variable T1.COL = T2.COL T1.COL op T2.COL COL=(non subq) COL op (non subq) COL op ANY (non subq) COL op ALL (non subq) COL IN (non subq) COL = expression (COL1,...COLn) IN (non subq) (COL1, COLn) = (value1, valuen) T1.COL = T2.colexpr COL IS NOT NULL COL IS NOT DISTINCT FROM value COL IS NOT DISTINCT FROM noncol expression COL IS NOT DISTINCT FROM col expression COL IS NOT DISTINCT FROM non subq T1.COL IS NOT DISTINCT FROM T2.COL T1.COL IS NOT DISTINCT FROM T2.col expression Indexable Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Stage 1 Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
Stage 1 Predicates
Predicate Type COL <> value COL <> noncol expr COL NOT BETWEEN value1 AND value2 COL NOT BETWEEN noncol expr1 AND noncol expr2 COL NOT IN (list) COL NOT LIKE ' char' COL LIKE '%char' COL LIKE '_char' T1.COL <> T2.COL T1.COL1 = T1.COL2 COL <> (non subq) COL IS DISTINCT FROM
1.
Indexable N N N N N N N N N N N N
Stage 1 Y Y Y Y Y Y Y Y Y Y Y Y
Indexable = The predicate is a candidate for Matching Index access. When the optimizer chooses to use a predicate in the probe of the index, the condition is named Matching (matching the index). This is the first point that filtering is possible in DB2. Index Screening = The Stage 1 predicate is a candidate for filtering on the index leaf pages. This is the second point of filtering in DB2. Data Screening = The Stage 1 predicate is a candidate for filtering on the data pages. This is the third point of filtering in DB2. Stage 2 = The predicate is not listed as Stage 1 and will be applied on the remaining qualifying pages from Stage 1. This is the fourth and final point of filtering in DB2.
2.
3.
4.
Y
Stage 2 Predicates - z/OS

(Chapter 29, Table 123, page 734 of the Database Administration Guide)
V7 may promote join Between
Predicate Type
value BETWEEN COL1 AND COL2 COL BETWEEN COL1 AND COL2 COL BETWEEN expression1 AND expression2 value NOT BETWEEN COL1 AND COL2 T1.COL1 op T1.COL2 T1.COL1 <> T1.COL2 COL = ANY (non subq) COL = ALL (non subq) COL <> ANY (non subq) COL <> ALL (non subq) COL NOT IN (non subq) COL = (cor subq)
Indexable Stage 1
N N N N N N N N N N N N
Stage 1
N N N N N N N N N N N N
10
Stage 2 Predicates - z/OS

(Chapter 29, Table 123, page 734 of the Database Administration Guide)
Predicate Type
Not made Indexable, . . V7 just rewrites the query to a join
COL = (cor subq) COL = ANY (cor subq) COL = ALL (cor subq) COL op (cor subq) COL op ANY (cor subq) COL op ALL (cor subq) COL <> (cor subq) COL <> ANY (cor subq) COL <> ALL (cor subq) COL IN (cor subq) COL NOT IN (cor subq) EXISTS (subq) NOT EXISTS (subq) expression = value expression <> value expression op value expression op (subquery)
Indexable Stage 1
N N N N N N N N N N N N N N N N N
Stage 1
N N N N N N N N N N N N N N N N N
11
Four Points of Filtering DB2

1. 2. 3. 4. Stage 2
WHERE C.LAST_NM LIKE ? AND C.TOKEN_NR = Indexable Stage 1 Probe B.TOKEN_NR Stage 1 Index Filtering C.ROLE_CD > ? AND CASE C.SEX WHEN X Stage 1 Data Filtering Type 2 Index THEN ? END) = ABCDE
1
Non-Leaf Page O O O
C1.C2.C3 TOKEN_NR. O O O
Root Page
ROLE_CD
Non-Leaf Page O O Non-Leaf Page O O O O
2
Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o o
SQL Performance Rules ..
Promote Stage 2s/Residuals if possible

Value BETWEEN COL1 AND COL2 is Stage 2 Rewrite to: (Value >= COL1 AND value <= COL2) COL NOT IN (K, S, T) = COL IN (known values)
SELECT only the columns needed

Disallow SELECT *
SELECT only the rows needed

Disallow program filtering
Use constants and literals if the values will not change in the next 3 years
Increase optimizer accuracy for static values
13
Bad BETWEENs
WHERE 0001-01-01 BETWEEN B.SCND_ALOC_EFF_DATE AND B.SCND_ALOC_END_DATE Should be:
WHERE 0001-01-01 >= B.SCND_ALOC_EFF_DATE AND 0001-01-01 <= B.SCND_ALOC_END_DATE
SQL Performance Rules

Make WHERE clause data types match

Costs more to navigate indexes when not matching
Do not SELECT columns with known static values

Disallow COLn from SELECT list if COLn = value
Do not place any local filtering in the ON clause for OUTER JOINs
Disallow during join filtering, encourage before join
Sequence filtering from most restrictive to least restrictive by table, by predicate type
WHERE AND AND AND AND A.COL2 = abracadabra A.COL1 IN (:hv1, :hv2, :hv3) A.COL4 > :hvcol4 A.COL3 > :hvcol3 A.COL5 LIKE %SON
Ties Matter!
15
Data Type Conversions

WHERE DATE(col_TS) BETWEEN :date1 AND :date2
WHERE col_TS BETWEEN TIMESTAMP(:date1, 00:00:00) AND TIMESTAMP(:date2, 59:59:99)
Need to leave columns alone and translate the second argument

16
Host Variable Filter Factors

(DB2 Database Administration Guide- Properties of Predicates)
COLCARDF >=100,000,000 >=10,000,000 >=1,000,000 >=100,000 >=10,000 >=1,000 >=100 >=2 =1 <=0 Factor for <, <=, >, >= 1/10,000 1/3,000 1/1,000 1/300 1/100 1/30 1/10 1/3 1/1 1/3 Factor for LIKE or BETWEEN 3/100,000 1/10,000 3/10,000 1/1,000 3/1,000 1/100 3/100 1/10 1/1 1/10
COLCARD
Factor for <, <=, >, >= BETWEEN, LIKE 1/100,000,000 1/10,000,000 1/1,000,000 1/100,000 1/10,000 1/1,000 1/100 1/3
17
>=100,000,000 >=10,000,000 >=1,000,000 >=100,000 >=10,000 >=1,000 >=100 >=0
SQL Review Checklist

1.
2. 3. 4. 5.
6.
7. 8.
Examine Program logic Examine FROM clause Verify Join conditions Promote Stage 2s/Residuals Prune SELECT lists Verify local filtering sequence Analyze Access Paths Tune if necessary
18
SQL Portfolio- DB2 9 z/OS vs. DB2 9.5 LUW

Stage1 unlike data types, Multi-row INSERT, FETCH, Multi-row cursor UPDATE, Dynamic Scrollable Cursors, Multiple CCSIDs per statement, GET DIAGNOSTICS, Enhanced UNICODE, IS NOT DISTINCT FROM, VARBINARY, FETCH CONTINUE, MERGE
Inner and Outer Joins, Table Expressions, Subqueries, GROUP BY, Complex Correlation, Global Temporary Tables, CASE, 100+ Built-in Functions including SQL/XML, Limited Fetch, Insensitive Scrollable Cursors, UNION Everywhere, MIN/MAX Single Index Support, Self Referencing Updates with Subqueries, Sor Avoidance for ORDER BY, and Row Expressions 2M Statement Length, GROUP BY Expression, Sequences, Scalar Fullselect, Materialized Query Tables, Common Table Expressions, Recursive SQL, CURRENT PACKAGE PATH, VOLATILE Table Support, Star Join Sparse Index, Qualified Column names, Multiple DISTINCT clauses, ON COMMIT DROP, Transparent ROWID Column, call from trigger, statement isolation, FOR READ ONLY KEEP UPDATE LOCKS, SET CURRENT SCHEMA, client special registers, long SQL Object names, SELECT FROM INSERT, UPDATE, DELETE, MERGE, INSTEAD OF TRIGGER, Native SQL Procedure Language, BIGINT, file reference variables, XML, FETCH FIRST & ORDER BY IN subselect and fullselect, caseless comparisons, INTERSECT, EXCEPT, not logged tables, DECIMAL FLOAT, XQuery,TRUNCATE OLAP Functions, Session variables, OmniFind, Spatial, ROLE
GROUPING SETS, ROLLUP, CUBE, Many Built-in Functions, SET CURRENT ISOLATION , multi-site join, MERGE, ARRAY data type, global variables, Oracle syntax, XML enhancements
19
SQL Performance Rule Violations & Misuse

Cost
CPU & I/O Cost time to fix Cost $$$$$$$$$

How much does it cost to misuse SQL?
1. Use joins over subqueries when detail row information is required 2. Use subqueries over joins when detail row information is not required 3. Use INNER JOIN over LEFT JOIN when exceptions are not expected or needed 4. Use CREATE GLOBAL TEMPORARY TABLE when data is infrequently accessed 5. Use DECLARE GLOBAL TEMPORARY TABLE with a clustered index when DTT is large and data is frequently accessed
DB2 10 z/OS vs. DB2 10 LUW

Multi-row INSERT, FETCH & multi-row cursor UPDATE, Dynamic Scrollable Cursors, GET DIAGNOSTICS, Enhanced UNICODE SQL, join across encoding schemes, IS NOT DISTINCT FROM, VARBINARY, FETCH CONTINUE, MERGE, SELECT from MERGE, routine versioning, timestamps w/timezone
Inner and Outer Joins, Table Expressions, Subqueries, GROUP BY, Complex Correlation, Global Temporary Tables, CASE, 100+ Built-in Functions including SQL/XML, Limited Fetch, Insensitive Scroll Cursors, UNION Everywhere, MIN/MAX Single Index, Self Referencing Updates with Subqueries, Sort Avoidance for ORDER BY, and Row Expressions, 2M Statement Length, GROUP BY Expression, Sequences, Scalar Fullselect, Materialized Query Tables, Common Table Expressions, Recursive SQL, CURRENT PACKAGE PATH, VOLATILE Tables, Star Join Sparse Index, Qualified Column names, Multiple DISTINCT clauses, ON COMMIT DROP, Transparent ROWID Column, Call from trigger, statement isolation, FOR READ ONLY KEEP UPDATE LOCKS, SET CURRENT SCHEMA, Client special registers, long SQL object names, SELECT from INSERT, UPDATE or DELETE, INSTEAD OF TRIGGER, Native SQL Procedure Language, BIGINT, file reference variables, XML, FETCH FIRST & ORDER BY in subselect & fullselect, caseless comparisons, INTERSECT, EXCEPT, not logged tables, OmniFind, spatial, range partitions, data compression, session variables, DECIMAL FLOAT, optimistic locking, ROLE, TRUNCATE, index & XML compression, created temps, inline LOB, administrative privileges, implicit cast, date/time changes, currently committed, moving sum & average, index include columns, row and column access control, time travel query
Updateable UNION in Views, GROUPING SETS, ROLLUP, CUBE, more Built-in Functions, SET CURRENT ISOLATION, multi-site join, MERGE, MDC, XQuery, XML enhancements, additional data type (array, row, cursor), global variables, even more 21 vendor syntax, temp table compression, MODULEs
DB2 11 SQL Standard SQL support (not exhaustive, some features may be missing)
DB2 11 for z/OS and DB2 10.5 Linux, Unix & Windows
z
c o m m o n l u w
Multi-row INSERT, FETCH & multi-row cursor UPDATE, Dynamic Scrollable Cursors, GET DIAGNOSTICS, Enhanced UNICODE SQL, join across encoding schemes, IS NOT DISTINCT FROM, VARBINARY, FETCH CONTINUE, SELECT FROM MERGE, MERGE, routine versioning, transparent archive query
Inner and Outer Joins, Table Expressions, Subqueries, GROUP BY, Complex Correlation, Global Temporary Tables, CASE, 100+ Built-in Functions including SQL/XML, Limited Fetch, Insensitive Scroll Cursors, UNION Everywhere, MIN/MAX Single Index, Self Referencing Updates with Subqueries, Sort Avoidance for ORDER BY, and Row Expressions, 2M Statement Length, GROUP BY Expression, Sequences, Scalar Fullselect, Materialized Query Tables, Common Table Expressions, Recursive SQL, CURRENT PACKAGE PATH, VOLATILE Tables, Star Join Sparse Index, Qualified Column names, Multiple DISTINCT clauses, ON COMMIT DROP, Transparent ROWID Column, Call from trigger, statement isolation, FOR READ ONLY KEEP UPDATE LOCKS, SET CURRENT SCHEMA, Client special registers, long SQL object names, SELECT from INSERT, UPDATE or DELETE, INSTEAD OF TRIGGER, SQL PL in routines, BIGINT, file reference variables, XML, FETCH FIRST & ORDER BY in subselect & fullselect, caseless comparisons, INTERSECT, EXCEPT, MERGE not logged tables, OmniFind, spatial, range partitions, data compression, DECFLOAT, optimistic locking, ROLE, TRUNCATE, index & XML compression, created temps, inline LOB, administrative privileges, implicit cast, increased timestamp precision, currently committed, moving sum & average, index include columns, row and column access controls, time travel query, GROUPING SETS, ROLLUP, CUBE, global variables, Text Search functions, accelerated tables, DROP COLUMN, array data type, XML enhancements
Updateable UNION in Views, more Built-in Functions, SET CURRENT ISOLATION, multi-site join, full MERGE, MDC, XQuery, additional data type (row, cursor), even more vendor syntax, temp table compression, MODULEs
Name: __________________
Level
SQL Skill Self Assessment
Assessment = YOU CAN FULLY UNDERSTAND THE FEATURE AND PROPER USE OF:
Before:_______
0 1
You think SQL is a new energy drink Simple SELECT statements, WITH clause, ORDER BY WHERE clauses, BETWEEN, LIKE, IN(list), =, >=, >, <, <=, <>, NOT IN(list), NOT LIKE, NOT BETWEEN, DISTINCT Table joins (inner, outer, full), UNION, UNION ALL, CONCAT, static CURSORs, FOR UPDATE OF, ROW_NUMBER, EXCEPT, INTERCEPT noncorrelated and correlated subqueries, EXISTS, NOT EXISTS, FETCH FIRST x ROWS ONLY, OPTIMIZE FOR x ROWS, MERGE, TRUNCATE Indexable, Stage1 and Stage 2 predicate evaluation, multirow FETCH/INSERT, GET DIAGNOSTICS, Scalar full SELECT
Table expressions/common table expressions, GROUP BY, HAVING, IS NOT DISTINCT FROM, embedded dynamic SQL, ORDER BY/FETCH FIRST in subselect, FETCH CONTINUE
CASE expressions, Global Temporary Table (GTT), Declared Temporary Table (DTT), Dynamic Scrollable cursors, SEQUENCES/IDENTITY Queries involving > 10 tables, INSERT within (SELECT , UPDATE, DELETE, MERGE), Star Schema, GROUP BY expression MQT (Materialized Query Tables), Recursive SQL, UNION in Views, > 30 useful Built-in Functions, DENSE_RANK, RANK Codes effective and efficient SQL applying performance rules and knows when to use each SQL feature appropriately
10
After:________
When to Tune Queries
Not until the query is coded the best it can be All predicates are the best they can be
Promote Stage 2s if possible Promote Stage 1s if possible Apply performance rules
Check Access Paths of all Query Blocks Apply data knowledge and program knowledge to predict response time If, and only if, the predicted service levels are not met - TUNE!
24
Why Tune Queries
Performance! Host variables force the optimizer to guess IBM optimizer researchers and developers provide extremely good guesses Occasionally (2-7% of queries) they guess wrong Have no fear, changing the optimizers mind is easier than you think
25
How to Tune Queries
Do not change statistics, just keep accurate Do not panic Choose a proven, low maintenance, tuning technique IBMs list:
OPTIMIZE FOR n ROWS FETCH FIRST n ROWS ONLY No Op (+0, CONCAT ) TX.CX=TX.CX REOPT(VARS) ON 1=1
26
SQL Tuning Examples

WHERE S.SALES_ID > 44 AND S.MNGR = :hv-mngr AND S.REGION BETWEEN :hvlo AND :hvhi CONCAT
No Operation
SELECT S.QTY_SOLD, S.ITEM_NO , S.ITEM_NAME FROM SALE S WHERE S.ITEM_NO > :hv ORDER BY ITEM_NO Limited FETCH FIRST 22 ROWS ONLY
Fetch
WHERE B.BID BETWEEN :hvlo AND :hvhi AND B.BID = D.DID AND B.SID = S.SID AND B.COL2 >= :hv AND B.COL3 >= :hv AND B.COL4 >= :hv
Fake Filter
27
Tuning Tools
Sheryls Extended List

Fake Filtering
COL BETWEEN :hv1 AND :hv2 COL >= :hv
Table expressions with DISTINCT

FROM (SELECT DISTINCT COL1, COL2 ..)
Anti-Joins Extreme Experiments Index Changes MQT Design

28
What Used to Work z/OS
UNION queries for index access until V3 Multiple Index Access (Predicate OR 0 = 1) until V5 No Operations Layered Table Expressions until V6 smarter optimizer Correlated Subqueries until V7 query re-write No cross table indexes - until V8 Materialized Query Tables (more on that later)
29
Steps in Tuning Queries

Step 1 - Learn Access Paths Step 2 Interpret the Optimizer Decisions Step 3 Find Top CPU Consumers Step 4 Learn Tuning Techniques
Step 5 Tune if Necessary
Step 6 Designing Performance Structures

30
Step 1
Learn the DB2 Access Paths
www.smlsql.com
www.ca.com/ Google Sheryl Larsen
31
All the Possible Access Paths

One Fetch IN(list) Index Access Matching Index Access Matching Index Access Sparse Index Access Partition Scan Without Index
Nested Loop Hybrid Join: Hybrid Type C or Type N
Limited Partition Scan With Partitioning Using Partitioning Index Index

Limited Partition Scan Limited Partition Scan Using Data Partitioned With NPI Secondary Index Table Scan
NonMatching Index Access

List Prefetch Multiple Index Access
(Bold names use an Index)
Star Join: Merge Scan Cartesian or Pair-wise

Star Scan Merge
Partitioned Table Scan
Direct Row
32
Variations of Index Accesses

One Fetch IN(list) Index Access Matching Index Access

Limited Partition Scan With Partitioning Index Limited Partition Scan With NPSI
Limited Partition Scan With DPSI

Multidimensional Index Access
NonMatching Index Access Sparse Index Access

Non-Leaf Page O O O
All can be parallel All can omit data access Clustering index or nonclustering access
Root Page O O O
Non-Leaf Page O O
range
Non-Leaf Page O O O O
Leaf Page o o o o o o
Rid + value(s) Bid

33
One Fetch
SELECT MAX(C3) FROM T1 WHERE C1 = L ANDType C2 2 =Index 99 SELECT MIN(C1) FROM T1
C1.C2.C3 O O O
Root Page
1
Leaf Page o o oo oo
Non-Leaf Page O O O
Non-Leaf Page O O
3
Leaf Page o o oo o o
Leaf Page o o oo oo
Leaf Page o o oo o o
Leaf Page o o oo oo
Leaf Page o o oo oo
Leaf Page o o oo oo
Leaf Page o o oo oo
34
Matching Index Access

WHERE C1 = L AND C2 > 99 AND C3 = :hv Type 2 Index
1
Non-Leaf Page O O O
C1.C2.C3 O O O
Root Page
Non-Leaf Page O O
35

WHERE C2 > 99 AND C3 = :hv Type 2 Index
C1.C2.C3 O O O
Non-Leaf Page O O O Non-Leaf Page O O Non-Leaf Page O O O O Root Page
1
2
36
IN(list) Index Access

WHERE C1 IN (K, S, T) AND C2 > 99 AND C3 = :hv Type 2 Index
C1.C2.C3 O O O
1
2
3
37
IN(list) Index Access -Parallel

WHERE C1 IN (K, S, T) AND C2 > 99 AND C3 = :hv Type 2 Index
C1.C2.C3 O O O
1
1
1
38
List Prefetch
WHERE C1 > 50 AND C2 = :hv
COL1 Index
Root Page O O O
1
Non-Leaf Page O O O
2
Non-Leaf Page O O
3 4
39
Multiple Index Access

WHERE C1 > 50
COL1 Index
WHERE C2 = :hv
COL2 Index
Root Page O O O
1
Non-Leaf Page O O O
Root Page O O O
C1
4
Non-Leaf Page O O
C2
2
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O
AND/OR 3
7
8
40
Sparse Index Access
Use Any Single Table Access Method And Filter
41
Limited Partition Scan Using Clustered Partitioning Index WHERE C1 IN (K, S, T)

AND C2 > 99 AND C3 = :hv
Partitioning Index Clustered
C1 Parts
1
Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
1
Non-Leaf Page O O O O Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
Type 2 Index
Root Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O O
Non-Leaf Page O O
3
3
42
Limited Partition Scan Using Nonclustered Partitioning Index WHERE CORP IN (20,
Partitioning Index Nonclustered
1
CORP
40,60) AND TICKET_PREFIX = A AND CUST_NAME LIKE %CAMB%
Type 2 Index
Root Page O O O
1
Non-Leaf Page O O O O Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
Type 2 Index
Root Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O O
Non-Leaf Page O O
3
3
43
Scan for Last Name

Using Nonclustered Partitioning Index
Partitioning Index Nonclustered
TOKEN_NR
WHERE TOKEN_NR BETWEEN ? AND ? AND TICKET_PREFIX = A AND LAST_NM LIKE LAR%
1
Type 2 Index
Root Page O O O
1
Type 2 Index
Root Page O O O
Type 2 Index
Root Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O O
Non-Leaf Page O O
3
3
44
Limited Partition Scan Using Clustered Partitioning Index 1 WHERE TOKEN_NR

Partitioning Index Clustered
TOKEN_NR Parts
BETWEEN ? AND ? AND LAST_NM > 99 3 AND ROLE_CD = ?
1
Type 2 Index
Root Page O O O
1
Non-Leaf Page O O O O Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
Type 2 Index
Root Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O O
Non-Leaf Page O O
3
3
45
Limited Partition Scan Using DPSI

Partitioning by TOKEN_NR
WHERE TOKEN_NR BETWEEN ? AND ? AND LAST_NM LIKE %LAR
DPSI = Data Partitioned Secondary Index Clustered by TOKEN_NR LAST_NM

1
Type 2 Index
Root Page O O O
Type 2 Index
Root Page O O O
1
Type 2 Index
Root Page O O O
Non-Leaf Page O O O
Non-Leaf Page O O
Non-Leaf Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
3
3
46
Helps Improve Performance

old
Partition/Cluster Partition/Cluster by Date Partition/Cluster by Date Partition/Cluster by Date Partition/Cluster by Date Partition/Cluster by Date by TOKEN_NR
TOKEN_NR BETWEEN ? AND ? AND LAST_NM_LIKE ?
BP03
TOKEN_NR BETWEEN ? AND ? AND LAST_NM_LIKE ?
New
Partition by Date Partition by Date Cluster by Custid Partition by Date Cluster by Custid Partition by Date Cluster by Custid Partition Cluster by by TOKEN_NR Custid Cluster by LAST_NM
47
Limited Partition Scan Using DPSI

Data Partitioned Secondary Index DPSI
C1 Parts
Partitioning by Date 1
Type 2 Index
Root Page O O O Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Non-Leaf Page O O
Clustered by Customer 1 2
Type 2 Index
Root Page O O O Non-Leaf Page O O O Non-Leaf Page O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
Non-Leaf Page O O O
Non-Leaf Page O O
3
3
48
Variations of Table Access

All
can be parallel
If
not enough room in memory, at run time create sparse index instead
Segmented Partitioned Limited Partitioned In Memory Data Cache
49
Table Scan
WHERE C1 BETWEEN :lowest AND :highest
50

WHERE C1 BETWEEN :lowest AND :highest
1
Part 1
Part 2 Part 3 Part 4 Part 5 Part 6 Part 7 Part 8
51
Limited Partitioned Table Scan

WHERE C1 IN (1, 3, 4, 16, 17, 18)
1
Part 1
Part 3 Part 4 Part 16 Part 17 Part 18
52
Variations of Join Methods

All
choose outer table and filter first Nested Loop Hybrid Join Type C All can be parallel (Star Hybrid Join Type N CPU only ) Merge Scan Join Worry about join table Star Join Cartesian sequence instead of join Star Join Pair Wise method
53
Nested Loop Join

COL1 Index
WHERE T1.ORDER > 222 AND T1.ID = T2.ID AND T2.COLOR IN (BLUE, YELLOW, GREEN)
Root Page O O O
Non-Leaf Page O O O
Non-Leaf Page O O
Inner Index
Root Page O O O
ID.COLOR index
Non-Leaf Page O O O
Non-Leaf Page O O
3
4
54
Hybrid Join Type N

COL1 Index
Root Page O O O
1
Non-Leaf Page O O O
Non-Leaf Page O O
COL2 Index
Root Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O
Filtered outer
R I D S
3 6 5
55
Hybrid Join Type C

COL1 Index
1
Root Page O O O
Non-Leaf Page O O O
Non-Leaf Page O O
COL2 Index
Root Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
Non-Leaf Page O O
3 4
56
Merge Scan Join

1 2
WHERE T1.C1 = T2.CA AND T1.C2 = T2.CB AND T1.C3 = T2.CC
57
Star Join - Cartesian

Small/Dimension Tables
1 2 6
Non-Leaf Page O O O
3
Inner Index Big/Fact Table Index
Root Page O O O
Non-Leaf Page O O
5
58
Pair-Wise Star Join

6
Type 2 Index
Root Page O O O
1
Type 2 Index
Root Page O O O Non-Leaf Page O O O Non-Leaf Page O O Non-Leaf Page O O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Non-Leaf Page O O O
Non-Leaf Page O O
2
Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
Non-Leaf Page O O
Type 2 Index
Root Page O O O
3
Non-Leaf Page O O O O Non-Leaf Page O O O Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o Leaf Page o o o o o o
Type 2 Index
Root Page O O O
Non-Leaf Page O O O
Non-Leaf Page O O
Non-Leaf Page O O
4 5 8 9
59
Read Direct Access

1.
2.
3.
Create table with ROWID type column (DIRECT_ID) SELECT DIRECT_ID INTO :direct-id FROM TAB12 2 WHERE UKEY = AMX UPDATE TAB12 3 WHERE DIRECT_ID = :direct-id
1
DIRECT_ID AMX DIRECT_ID
DIRECT_ID
DIRECT_ID
60
All the Possible Access Paths

One Fetch IN(list) Index Access Matching Index Access Matching Index Access Sparse Index Access Partition Scan Without Index
Nested Loop Hybrid Join: Type C or Hybrid Type N
Limited Partition Scan With Partitioning Using Partitioning Index Index

Limited Partition Scan Limited Partition Scan Using Data Partitioned With NPI Secondary Index Table Scan

Star Join: Cartesian or Merge Scan Pair-wise Merge Star Scan

Direct Row
61
Step 2
Interpret the Optimizers Decisions
62
Determine If Tuning Is Necessary

Find out what the general access path is for each query block (subselect, union, table expression) Calculate execution frequencies of each query block
Correlated subselects execute bottom query block once for each qualifying row in bottom query
Identify join table sequence Identify materialization (intermediate or final) Identify sorts (even hidden Rid sorts an IN(list) Identify User Defined Table or Column functions
63
Using EXPLAIN
EXPLAIN PLAN SET QUERYNO = n FOR ..; SELECT * FROM PLAN_TABLE ORDER BY ..;
Q No
1 1 2 2 2
Qblock Step No No
1 1 1 1 1 1 2 1 2 3
Meth
0 4 0 2 4
Access Type
R I N I I
Match Cols
0 2 2 2 2
Pre Fetch
S L
Join Type
IX Only
N N Y
IX Name
IX22
F L
N N
IX27 IX42
3 3
1 2
1 1
0 2
I R
4 0 S
N N
IXF1
64
Using EXPLAIN
SORTN UJOG NNNN NNNN NYNN NNNN NNNN NNNN NNNN NNNN NNNN SORTC UJOG NNYN NNNN NNNN NNNN NNNN NNNN NNNN NNNN NNNN R S Y Y Y Y R R Page Col Range Func Y S S Corr Name AB CD EF XX GA SL ML KL TL Qblock Type SELECT UPDATE INSERT DELECT NCOSUB CORSUB DELCUR UPDCUR SELUPD SS SS XX XX XX Opt Hint GG NN Hint Used GG TS Remark s Good 1 Bad 1 +0 Bogus 3 Extra > parens help
. . . . . . . . .
65
Learning EXPLAIN
Need IBMs definition of the PLAN_TABLE Need a way to transform PLAN_TABLE into something readable (i.e. QMF Form or program) Review PLAN_TABLE contents with Access Path descriptions Study, practice, practice, practice
66
Using DSN_STATEMENT_TABLE
Explain Time Stmt Type Cost Cat PROCMS PROCSU Reason
Time Stamp
SEL UPD
A or B
Milli seconds
Service Units
For Cat B
A = No Defaults used in estimate B = Check Reason column

Host varibles Cardinality UDF Triggers RI
67
Service Units
http://www-03.ibm.com/servers/eserver/zseries/srm/
zSeries 990
Processor Model 2084-301 2084-302 2084-303 2084-304 STIDP Type 2084 2084 2084 2084 STSI Model Name 301 302 303 304 #C P 1 2 3 4 SU/SEC 21857.9235 20752.2698 20075.2823 19559.9022 SRMsec/RealSec 508.1298 508.1298 508.1298 508.1298
68
Service Units
IBM System z10
Processor Model 2097-401 2097-402 2097-501 2097-502 2097-503 2097-701 2097-702 2097-703 2097-704 STIDP Type 2097 2097 2097 2097 2097 2097 2097 2097 2097 STSI Model Name 401 402 501 502 503 701 702 703 704 #CP 1 2 1 2 3 1 2 3 4 SU/SEC 11291.4608 10680.9079 24427.4809 23021.5827 22222.2222 47619.0476 44692.7374 43010.7527 41666.6667 SRMsec/RealSec 262.6049 262.6049 568.1815 568.1815 568.1815 1106.1941 1106.1941 1106.1941 1106.1941
69
SQL Tuning using EXPLAIN

Type of Access
EXPLAIN
ALL Expensive Queries

# of Matching Columns
-- Query Joins PLAN_TABLE with DSN_STATEMENT
BLOCK PLAN METH TABLE_NAME PQB TYPE MC ANAME IO --+---------+---------+---------+---------+---------+---------+---------+------01 01 00 R10_CLM_FACT12 12 M 00 N 01 01 00 R10_CLM_FACT12 01 X011_10_12_F_IX07 Y Index Name 12 MX 01 01 00 R10_CLM_FACT12 12 MX 01 X011_10_12_F_IX04 Y 01 01 00 R10_CLM_FACT12 12 MI 00 N 01 01 00 R10_CLM_FACT12 12 MX 01 X011_10_12_F_IX04 Y 01 01 00 R10_CLM_FACT12 12 MX 01 X011_10_12_F_IX04 Y Index Only? 01 01 00 R10_CLM_FACT12 12 MU 00 N 01 01 00 R10_CLM_FACT12 12 MI 00 N 01 02 01 OTR_PRTY_LIAB_NET 12 N 03 OPL_NET_IX04 N Step # 01 03 01 MEMBER_ELIG_NET 12 N 04 MBR_ELIG_NET_IX06 Y 01 04 01 R10_CHG_FACT12 12 N 03 X007_10_12_F_IX19 N 01 05 01 R10_MULTI_CD_FAC12 12 I 02 X021_10_12_F_IX14 N Join Method 02 01 00 R10_CHG_FACT12 09 N 03 X007_10_12_F_IX19 Y
dave@davebeulke.com
Copyright Sheryl Larsen 2011
70 smlsql@comcast.co
Sort, Prefetch, & Partition Usage Columns Type of Query Block

Sort
N = New/Inner Sort Sort C = Outer/Composite Sort
SNU SNJ SNO SNG SCU SCJ SCO SCG PF QB_TYPE STMT_TYPE T_TP PR JT +---------+---------+---------+---------+---------+---------+---------+------N N N N N N N N L NCOSUB SELECT T N N N N N N S NCOSUB SELECT T O NG N U N N N Nr Nr N N N S NCOSUB SELECT T J n N N N N No N N N Limited NCOSUB SELECT Partition Scan? T d o N N N N Nu N N N S NCOSUB SELECT T i e i N N N Nr Np N N N S NCOSUB SELECT T q n N N N N N N N N NCOSUB SELECT T u N N N N N N NCOSUB SELECT T B NB N e N N N N N N N NCOSUB SELECT T y N y Prefetch N N N N N N N N NCOSUB SELECT T N N N N N N N N NCOSUB SELECT Type T N N N N N N N N NCOSUB SELECT T N N N N N N N N CORSUB SELECT T
Parallelism and Cost Columns

Sort
N = New/Inner Sort Sort C = Outer/Composite Sort

PM ADGREE JDGREE COST_CAT PROCMS PROCSU --+---------+---------+---------+---------+---------+-C 4 ------ B 710 21495 C ------ ------ B 710 21495 C ------ ------ B 710 21495 # of I/O Streams C ------ ------ B 710 21495 C ------ ------ B 710 21495 C ------ CPU ------ B 710 21495 Parallelism C ------ ------ B 710 21495 Contains :hostvars C ------ ------ B 710 21495 C 4 4 B 710 21495 or cardinality issue C 4 4 B 710 21495 C 4 4 B 710 21495 C 4 4 B 710 21495
Processor Service Units
smlsql@comcast.com
dave@davebeulke.com
72
Step 3
Fetch Top CPU Consumers
73
Tuning SQL
FIND
ALL Expensive Queries
---------+---------+--PROGNAME PROCSU ---------+---------+--EXPNPROG 121,059,664 EXPNPROG 21,059,664 ONESECPG 79,664 SUBSECPG 9,664 CHEEPPRG 64 FREEPROG 4
dave@davebeulke.com
PROCSU is To Expensive to Calculate!
2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647 2,147,483,647
Step 4
Learn Traditional and Extreme Tuning Techniques
76
Tuning Techniques to Apply When Necessary

Learn Traditional Tuning Techniques OPTIMIZE FOR n ROWS No Ops REOPT(ONCE), REOPT(ALWAYS) Fake Filtering ON 1 = 1 Index & MQT Design
Experiment with Extreme Tuning Techniques DISTINCT Table Expressions Correlated Table Expressions Odd/old Techniques Anti-Joins Manual Query Rewrite (X2QBOpt) covered in Extreme
77
OPTIMIZE FOR n ROWS FETCH FIRST n ROWS
Both clauses influence the Optimizer

To encourage index access and nested loop join To discourage list prefetch, sequential prefetch, and access paths with Rid processing Use FETCH n = total rows required for set Use OPTIMIZE n = number of rows to send across network for distributed applications Works at the statement level
78
Fetch First Example
SELECT S.QTY_SOLD , S.ITEM_NO , S.ITEM_NAME FROM SALE S WHERE S.ITEM_NO > :hv ORDER BY ITEM_NO
Optimizer choose List Prefetch Index Access + sort for ORDER BY for 50,000 rows All qualifying rows processed (materialized) before first row returned = .81 sec <.1sec response time required
Optimizer now chooses Matching Index Access (first probe .004 sec) No materialization Cursor closed after 22 items displayed (22 * .0008 repetitive access) .004 + .017 = .021 sec
SELECT S.QTY_SOLD, S.ITEM_NO , S.ITEM_NAME FROM SALE S WHERE S.ITEM_NO > :hv ORDER BY ITEM_NO FETCH FIRST 22 ROWS ONLY
79
No Operation (No Op)
+0, CONCAT also 0, *1, /1

Place no op next to predicate Use as many as needed Discourages index access, however, preserves Stage 1 Can Alter table join sequence Can fine tune a given access path Can request a table scan Does not Benefit Works at the predicate level DB2 on Linux,
UNIX or Windows
80
No Op Example CONCAT
SALES_ID.MNGR.REGION Index
SELECT S.QTY_SOLD , S.ITEM_NO , S.ITEM_NAME FROM SALE S WHERE S.SALES_ID > 44 AND S.MNGR = :hv-mngr AND S.REGION BETWEEN :hvlo AND :hvhi ORDER BY S.REGION . FROM SALE S WHERE S.SALES_ID > 44 AND S.MNGR = :hv-mngr AND S.REGION BETWEEN :hvlo AND :hvhi CONCAT ORDER BY R.REGION
MNGR Index
REGION Index
Optimizer chooses Multiple Index Access The table contains 100,000 rows and there are only 6 regions Region range qualifies 2/3 of table <.1sec response time required No Op allows Multiple Index Access to continue on first 2 indexes Two Matching index accesses, two small Rid sorts, & Rid intersection ____unique Rids/16 * .000375 sec = _____ + sort for Region = ______
81
No Op Example - Scan
SALES_ID.MNGR.REGION Index MNGR Index REGION Index
SELECT S.QTY_SOLD , S.ITEM_NO , S.ITEM_NAME FROM SALE S WHERE S.SALES_ID > 44 +0 AND S.MNGR = :hv-mngr CONCAT AND S.REGION BETWEEN :hvlo AND :hvhi CONCAT ORDER BY S.REGION FOR FETCH ONLY WITH UR
If you know the predicates do very little filtering, force a table scan Use a No Op on every predicate This forces a table scan FOR FETCH ONLY encourages parallelism WITH UR for read only tables to reduce CPU
Should this be Documented?

82
REOPT
Package/Plan level option

QUERYOPT-- optimization-level
REOPT ONCE REOPT ALWAYS REOPT AUTO DB2 9 Every querys access path gets re-optimized with host variable contents known only when needed System Parameter REOPTEXT Compares New Values For enables REOPT(AUTO) for all BIG differences unspecified dynamic queries with parameter markers
Dynamic Statement Cache
SELECT FROM T1 WHERE C1 > ? , 120
Long running queries/programs easily benefit High volume transactions will reflect the cost of the rebind (90% of a full BIND)
83
Fake Filtering
Fake Predicates
To encourage index access To alter table join sequence when nothing else works Works by decreasing filter factor on a certain table The filtering is fake and negligible cost Not effective for dynamic queries if the filter contains :host variables
84
Fake Filtering Example

SELECT B.BID, D.DID, S.SID, ,D.DESC, , S.DESC FROM BONDS B , DENOM D, SERIAL S WHERE B.BID BETWEEN :hvlo AND :hvhi AND B.DID = D.DID AND B.SID = S.SID ORDER BY B.BID
D.DID Index 12 rows
Nested
Loop
S.SID Index 20 rows B.BID Index 6 billion rows
Joins
Large report query with average of 400,000 row range of BID table Need to start nested loop with big table Tools required
85
Fake Filtering Example

SELECT B.BID, D.DID, S.SID, ,D.DESC, , S.DESC FROM BONDS B , DENOM D, SERIAL S WHERE B.BID BETWEEN :hvlo AND :hvhi AND B.BID = D.DID AND B.SID = S.SID AND B.COL2 >= :hv = B.COL2 AND B.COL3 >= :hv = B.COL3 AND B.COL4 >= :hv = B.COL4 = B.COL5 AND B.COL5 >= :hv = B.COL6 AND B.COL6 >= :hv ORDER BY B.BID
B.BID Index 6 billion rows
Nested Loop
D.DID Index 12 rows
Joins
S.SID Index 20 rows
Keep adding filters until table join sequence changes Start with index columns To preserve index-only access No limit! For Dynamic
86
ON 1 = 1
ON
1=1
To fill in a required join field To request a star join When table ratios are under the system specified number (starts at 1:25) Can benefit when large table has high selectivity
87
Experiment with Extreme Techniques

After Traditional Techniques Fail
88
DISTINCT Table Expressions
Table expressions with DISTINCT

FROM (SELECT DISTINCT COL1 FROM T1 ..) AS STEP1 JOIN T2 ON JOIN T3 ON .
Used for forcing creation of logical set of data

No physical materialization if an index satisfies DISTINCT
Can encourage sequential detection Can encourage a Merge Scan join

STEP1 Logical
Index STEP1 Physical
Buffer Pool Data

89
Work File
DISTINCT Table Expressions Example
SELECT Columns FROM TABX, TABY, (SELECT DISTINCT COL1, COL2 .. FROM BIG_TABLE Z WHERE local conditions) AS BIGZ WHERE join conditions
Optimizer is forced to analyze the table expression prior to joining TABX & TABY
90
Typical Join Problem

SELECT COL1, COL2 .. FROM ADDR, NAME, TAB3, TAB4, TAB5, TAB6, TAB7 WHERE join conditions AND TAB6.CODE = :hv
Cardinality 1
Result
is only 1,000 rows ADDR and NAME first two tables in join Index scan on TAB6 table
Not good because zero filter
91
Tuning Technique
SELECT COL1, COL2 .. FROM ADDR, NAME, (SELECT DISTINCT columns FROM TAB3, TAB4, TAB5, TAB6, TAB7 WHERE join conditions AND (TAB6.CODE = :hv OR 0 = 1)) AS TEMP WHERE join conditions Gets rid of Index Scan
Keeps large tables joined last
92
Correlated Table Expressions
Advantage over correlated subquery

Access both sets of data
Performance gains
Access path changes Encourages Nested Loop Join Reduced processing and/or materialization
SELECT Columns from either set of data FROM table1 T1 ,TABLE(SELECT T2.STUDENT_YEAR, SUM(C8) AS SUM8, MAX(C9) AS MAX9 FROM table2 T2 WHERE T1.column = T2.column GROUP BY T2.STUDENT_YEAR) AS T2
93
Correlated Table Expressions

Get
a students class load, and average & max class load of their peers
SELECT SR.NAME, SS.CLASS_LOAD, P.AVG_CLASS_LOAD, P.MAX_CLASS_LOAD FROM SMLU_STUDENT_ROSTER SR , TABLE(SELECT SR2.STUDENT_YEAR ,AVG(SR2.NUM_OF_CLASSES) AS AVG_CLASS_LOAD ,MAX(SR2.NUM_OF_CLASSES) AS MAX_CLASS_LOAD FROM SMLU_STUDENT_ROSTER SR2 WHERE SR.STUDENT_YEAR = SR2.STUDENT_YEAR GROUP BY SR2.STUDENT_YEAR) AS P LEFT JOIN TABLE(SELECT SS.SID, COUNT(*) AS SS.CLASS_LOAD FROM SMLU_STUDENT_SCHED SS WHERE SS.SID = SR.SID GROUP BY SS.SID) as SS ON 1=1
94
Anti-Join
SELECT Columns FROM TABY Y LEFT JOIN TABX X ON X.COL1 = Y.COL1 WHERE X.COL1 IS NULL
SELECT Columns FROM TABX X WHERE NOT EXISTS (SELECT * FROM TABY Y WHERE X.COL1 = Y.COL1)
TABX Exceptions
TABY Exceptions
Inner Join SELECT Columns FROM TABX X, TABY Y WHERE X.COL1 = Y.COL1
95
Anti-Join
TABX Exceptions
SELECT Columns FROM TABX X WHERE NOT EXISTS (SELECT * FROM TABY Y WHERE X.COL1 = Y.COL1)
Stage 2 when correlated
Indexable Stage 1
Does not Benefit LUW
SELECT Columns FROM TABX X LEFT JOIN TABY Y ON X.COL1 = Y.COL1 WHERE Y.COL1 IS NULL
96
Step 5
Tune if Necessary
97
Hunting Query Ideas

WHERE AND AND WHERE AND WHERE AND WHERE PROGNAME LIKE online% ACCESSTYPE = I PREFETCH = L ACCESSTYPE = I MATCH_COLS = 0 METHOD = 4 SORTN_JOIN = Y ACCESSTYPE = R
List Onlines Doing List Prefetch List Index Scans List Nasty Hybrid Joins
List Table Scans Materialization Table Expression Materialization(V7)

98
WHERE WHERE AND
TNAME LIKE DSNWF% QBLOCK_TYPE = TABLEX TABLE_TYPE = W

SYSPACKSTMT
WHERE STMT LIKE %WHERE_:%BETWEEN%
List Ugly BETWEEN clauses
WHERE
STMT LIKE %WHERE%NOT%
List NOTs
WHERE
STMT LIKE %WHERE%COLX =%
Check data Types of literals and constants
WHERE
STMT LIKE any SQL code problem previously determined

99
Step 6
Designing Optimal Performance Structures for One Query or a Workload
100
One Query at a Time

1. 2. 3. 4.
Equal predicates first Sequencing columns next Predicates most restrictive to least restrictive All remaining SELECT Columns
Type 2 Index
Root Page C4.C5.C6.C1.C2.C3.C9.C7.C8.C12 O O O
SELECT C7, C8, C12, C2, C3 WHERE C4 = L AND C5 = 99 AND C6 = :hv AND C1 IN (:a,:b,:c) AND C2 NOT LIKE :hv AND C3 BETWEEN :hvstart AND :hvend AND C9 = (CASE expression) ORDER BY C1.C2.C3
Non-Leaf Page O O O
Non-Leaf Page O O
Leaf Page o o oo oo
Leaf Page o o oo oo
Leaf Page o o oo oo
Leaf Page o o oo oo
Leaf Page o o oo oo
Leaf Page o o oo oo
101
. To Build or not to Build

Generic
MQT strategies
HUGE MQT with many indexes Many small MQTs with few indexes
Optimal
MQT strategy
Most popular queries as MQTs

Common tables Common predicate sets Common data translations
Indexes on the popular ones to allow access to subset of MQT

102
Expensive Recursive Syntax

WITH ORG_CHART(BOSS, DIRECT_REPORT, N) AS (SELECT BOSS, DIRECT_REPORT, 1 FROM ORG_TABLE WHERE BOSS= SAM UNION ALL SELECT T1.BOSS, T2.DIRECT_REPORT, N+1 FROM ORG_CHART T2, ORG_TABLE T1 WHERE T2.DIRECT_REPORT = T2.BOSS AND N < 10 0,000) SELECT BOSS, DIRECT_REPORT, N Generated FROM ORG_CHART
SAM
at Execution
103
MQT INDEX: BOSS
MQT
CREATE TABLE ORG_LIST AS ( WITH ORG_CHART(BOSS, DIRECT_REPORT, N) AS (SELECT BOSS, DIRECT_REPORT, 1 FROM ORG_TABLE WHERE BOSS= SAM UNION ALL SELECT T1.BOSS, T2.DIRECT_REPORT, N+1 FROM ORG_CHART T2, ORG_TABLE T1 WHERE T2.DIRECT_REPORT = T2.BOSS ) SELECT BOSS, DIRECT_REPORT, N FROM ORG_CHART) DATA INITIALLY DEFERRED REFRESH DEFERRED MAINTAINED BY SYSTEM ENABLE QUERY OPTMIZATION;
104
Query Speed Up
WITH ORG_CHART (BOSS, DIRECT_REPORT, N) AS (SELECT BOSS, DIRECT_REPORT, 1 FROM ORG_TABLE WHERE BOSS= BETH UNION ALL SELECT T1.BOSS, T2.DIRECT_REPORT, N+1 FROM ORG_CHART T2, ORG_TABLE T1 WHERE T2.DIRECT_REPORT = T2.BOSS AND N < 10 0,000) SELECT BOSS, DIRECT_REPORT, N FROM ORG_CHART
Accessed at Execution
SELECT * FROM ORG_LIST WHERE BOSS = BETH
ORG_LIST
BOSS JOHN BETH BETH BETH BETH
DIRECT _REPORT ADAM JIM SUE MARK SAM
N 501 634 634 635 635 105
Designing Indexes
For a Workload
106
Table/Column Usage
Tran tran1
c1
q1 q2 q3
O1 G1
=T4.C2
c2
O2 G2
=:hv
c3 c4 c5
O3 G3
>800
c6 c7 c8 c9
S S S
O1 = Order By First Position G1 = Group By First Position S = SELECT list W = Local or Join WHERE/ON or Specify actual local/join filter Specify Group by Aggregates as: Aggr BY Column(s)
c10
c11
MAX by C1.C2. C3
=T2.C1
SUM BTW by 2and C1.C2. C3 7
AVG by C1.C2.C3
s
=T3.C1
w s
=22
o1 s
=:hv o2
o1
=T3.C11
=T3.C1
tran2
q1 q2
O1 s
O2 s
O3
=:hv
=:hv
>:hv
O1
IN(list)
LIKE :hv
107
Final Ranking
Weight
Weighting Based on Importance to the Business
tran 1
100
4200
q1 q2 q3
O4200 1 4200 G4200 1 W s 10
4200 O O4200 S w S S S 4200 2 3 4200 G4200 4200 4200 4200 4200 w 4200 G 2 3 4200 4200 w w s10 w 10 w 10 w o10 1 w 10 o2 10
10 9900
9900
o1 s w w 9900 9900 9900 9900
tran2
3 0 0
Frequency * Weight
108
Determining Weight
Copy
the frequency report Sort by business priorities

Use Work Load Manager Goals for guidance Use Business Unit Owner Priorities
Remove
the top 30% Lets say 5,500 is top Sort by Frequency frequent Pull of the top number Then 55,000 is the goal Multiply by 10 Now you have the frequency number to beat
Pulls out most critical
109
Grand Total
By Table, by Operation
c1
O1 55980 G1 4790 O2 2220 O6 70 W 3920
Total Activity
c2
c3
c4 c5
O1 8740 S 8740 w 22090
c6
O2 3940 S 480
c7
c8
O2 55980 O3 55980 S G2 4790 G3 4790 660 W 400 W3
S 4000 S 8390 W 7640 W 2020
Use
this information to justify making changes to current index design

110
Summary
Many Things Impact Query Speed

SQL exploitation and proper us Manual Query Rewrite when necessary Optimal Index and MQT design
It is Easy to Change the Optimizers Mind The Trick is to Know WHEN to use:
Extreme Tuning Techniques DISTINCT Table Expressions Correlated Table Expressions OR 0 = 1 Anti-Join Technique Index and MQT Design X2QBOpt
111
Traditional Tuning Techniques OPTIMIZE FOR n ROWS No Ops REOPT(VARS) Fake Filtering Index & MQT Redesign
Recommended SQL Reading

DB2 for z/OS: Design Guidelines for High Performance and Availability http://www.redbooks.ibm.com/abstracts/sg247134.html
DB2 for z/OS Version 8 Performance Topics Explore the book online at http://www.redbooks.ibm.com/abstracts/sg246465.html DB2 9 for z/OS Performance Topics, SG24-7473-00 Explore the book online at http://www.redbooks.ibm.com/abstracts/sg247473.html?Open
DB2 10 for z/OS Performance Topics http://www.redbooks.ibm.com/abstracts/sg247942.html
DB2 Developers Guide, Sixth Edition SAMS Publishing, Craig S. Mullins Covers DB2 9 & 10 for z/OS!!!!!
112

IDUG 2013 Sheryl Larsen Tuning SQL

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

IDUG 2013 Sheryl Larsen Tuning SQL

Încărcat de

Drepturi de autor:

Formate disponibile

Platform: z/OS Application Development

Tuning SQL for Performance

Sheryl M. Larsen, Inc.

Google Sheryl Larsen @

Sheryl M. Larsen, Inc. 2000-2011

Table of Contents - Tuning

Sheryl M. Larsen, Inc. 2000-2011

DB2 EngineSQL Components Execution

Stage 2 z/OS Residual LUW

Stage 1 z/OS Sargable LUW

Dynamic Statement Cache

Forward and Backward Index DPSI Indexes

Materialized Query Tables

Index only on VARCHAR Indexes

Stage 1 Dynamic Statement Buffer Pool Cache

4K Page Processing z/OS

Stage 1 Buffer Pool

Sheryl M. Larsen, Inc. 2000-2008

Indexable Stage 1 Predicates

Sheryl M. Larsen, Inc. 2000-2008

Stage 2 Predicates - z/OS

V7 may promote join Between

Sheryl M. Larsen, Inc. 2000-2008

Stage 2 Predicates - z/OS

Four Points of Filtering DB2

SQL Performance Rules ..

Promote Stage 2s/Residuals if possible

SELECT only the columns needed

SELECT only the rows needed

WHERE 0001-01-01 >= B.SCND_ALOC_EFF_DATE AND 0001-01-01 <= B.SCND_ALOC_END_DATE

SQL Performance Rules

Make WHERE clause data types match

Do not SELECT columns with known static values

Data Type Conversions

WHERE col_TS BETWEEN TIMESTAMP(:date1, 00:00:00) AND TIMESTAMP(:date2, 59:59:99)

Need to leave columns alone and translate the second argument

Host Variable Filter Factors

>=100,000,000 >=10,000,000 >=1,000,000 >=100,000 >=10,000 >=1,000 >=100 >=0

Sheryl M. Larsen, Inc. 2000-2008

SQL Review Checklist

SQL Portfolio- DB2 9 z/OS vs. DB2 9.5 LUW

SQL Performance Rule Violations & Misuse

CPU & I/O Cost time to fix Cost $$$$$$$$$

DB2 10 z/OS vs. DB2 10 LUW

SQL Skill Self Assessment

When to Tune Queries

Sheryl M. Larsen, Inc. 2000-2011

Why Tune Queries

How to Tune Queries

SQL Tuning Examples

Sheryls Extended List

Table expressions with DISTINCT

Anti-Joins Extreme Experiments Index Changes MQT Design

What Used to Work z/OS

Sheryl M. Larsen, Inc. 2000-2011

Steps in Tuning Queries

Step 6 Designing Performance Structures

All the Possible Access Paths

Nested Loop Hybrid Join: Hybrid Type C or Type N

Limited Partition Scan With Partitioning Using Partitioning Index Index

NonMatching Index Access

Star Join: Merge Scan Cartesian or Pair-wise

Partitioned Table Scan

Sheryl M. Larsen, Inc. 2000-2011