Documente Academic
Documente Profesional
Documente Cultură
by
Cardinality Feedback
Method and Examples
Wolfgang Breitling
www.centrexcc.com
Who am I
Independent consultant since 1996
specializing in Oracle and Peoplesoft setup,
administration, and performance tuning
Member of the Oaktable Network
25+ years in database management
DL/1, IMS, ADABAS, SQL/DS, DB2, Oracle
OCP certified DBA - 7, 8, 8i, 9i
Oracle since 1993 (7.0.12)
Mathematics major from University of Stuttgart
2
Focus
The empirical Basis for the Method
Explanation of the Method on a Test case
Real-life examples of using the Method
Comparing TCF to
Hints
Profiles
3
120
103.13
100
C. Re-execute SQL
D. Analyze 40,000 row
table
80
61.62
60
53.73
51.25
E. Re-execute SQL
F. Execute SQL with
OICA=25, OIC=90
40
20
17.01
provided
it is able or given some help - to accurately
estimate the cardinalities of the row sources in
the plan
6
But instead
Why is this plan, which the CBO chose as
p Look for the first (innermost) row source where the ratio of
actual/estimated cardinality is orders of magnitude
usually at least in the 100s
q Find the predicates in the SQL for the tables that contribute
to the row source with the miscalculated cardinality and look
for violated assumptions:
Uniform distribution
Predicate independence
Join uniformity
9
10
504.6
534.9
1.0
858.1
353.3
1.0
3.0
1.6
2.7
1.0
11
2
6,274
13,120
208,620
15
44,621
14,131
5
40,000
44,621
74,101
13,679
9,860
4,930
4,930
20,022
7,750
10,011
card
2
2
26
390
15
52
40
5
13,334
27,456
27,456
13,679
1
1
1
1
1
1
operation
SELECT STATEMENT
SORT GROUP BY
FILTER
HASH JOIN
HASH JOIN
TABLE ACCESS FULL PS_RETROPAYPGM_TBL
NESTED LOOPS
HASH JOIN
TABLE ACCESS FULL PS_PAY_CALENDAR
TABLE ACCESS FULL WB_JOB
TABLE ACCESS BY INDEX ROWID WB_RETROPAY_EARNS
INDEX RANGE SCAN WB0RETROPAY_EARNS
TABLE ACCESS FULL PS_RETROPAY_RQST
SORT AGGREGATE
FIRST ROW
INDEX RANGE SCAN (MIN/MAX) WB_JOB
SORT AGGREGATE
FIRST ROW
INDEX RANGE SCAN (MIN/MAX) WB_JOB
WB_JOB
12
column
COMPANY
PAYGROUP
PAY_END_DT
RUN_ID
PAY_OFF_CYCLE_CAL
PAY_CONFIRM_RUN
EMPLID
EMPL_RCD#
EFFDT
EFFSEQ
COMPANY
PAYGROUP
NDV
11
15
160
240
2
2
density
9.0909E-02
6.6667E-02
6.2500E-03
4.1667E-03
5.0000E-01
5.0000E-01
26,167
1
10
3
10
14
3.8216E-05
1.0000E+00
1.0000E-01
3.3333E-01
1.0000E-01
7.1429E-02
lo
ACE
ACA
1998-01-18
N
N
hi
TES
TEP
2004-02-22
PP2
Y
Y
bkts
1
1
1
1
1
1
000036
0
1995-01-01
1
ACE
ACA
041530
0
2004-02-01
3
TES
TEP
1
1
1
1
1
1
table
WB_JOB
13
column
EMPLID
EMPL_RCD#
EFFDT
EFFSEQ
COMPANY
PAYGROUP
NDV
26,167
1
10
3
10
14
density
3.8216E-05
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E-01
7.1429E-02
lo
000036
0
1995-01-01
1
ACE
ACA
hi
041530
0
2004-02-01
3
TES
TEP
bkts
1
1
1
1
1
1
17.5
1.0
28.1
29.8
9.9
1.0
1.0
4.5
1.0
14
2
6,274
13,120
15
42,054
44,621
14,130
5
40,000
122,813
13,679
11,212
5,606
5,606
17,374
6,418
8,687
card operation
2 SELECT STATEMENT
2
SORT GROUP BY
FILTER
750
HASH JOIN
15
TABLE ACCESS FULL PS_RETROPAYPGM_TBL
1,499
HASH JOIN
1,499
HASH JOIN
1,429
HASH JOIN
5
TABLE ACCESS FULL PS_PAY_CALENDAR
40,000
TABLE ACCESS FULL WB_JOB
27,456
TABLE ACCESS FULL WB_RETROPAY_EARNS
13,679
TABLE ACCESS FULL PS_RETROPAY_RQST
1
SORT AGGREGATE
1
FIRST ROW
1
INDEX RANGE SCAN (MIN/MAX) WB_JOB
1
SORT AGGREGATE
2
FIRST ROW
2
INDEX RANGE SCAN (MIN/MAX) WB_JOB
Tuning Examples
Unlike the demo case, which was an
artificial test case
albeit based on a real-life case
the following examples are actual,
recent tuning cases.
For the record, the examples are from a system running Oracle
9.2.0.6 with system statistics.
15
Example 1
SELECT R.PRCSINSTANCE ,R.ORIGPRCSINSTANCE ,R.RECURORIGPRCSINST,
R.MAINJOBINSTANCE ,R.PRCSJOBSEQ ,R.PRCSJOBNAME ,R.PRCSNAME ,R.PRCSTYPE, R.RECURNAME
FROM PSPRCSQUE R ,PS_PRCSRECUR S
WHERE ((R.RUNSTATUS IN (:"SYS_B_00", :"SYS_B_01") AND S.INITIATEWHEN = :"SYS_B_02")
OR (R.RUNSTATUS IN (:"SYS_B_03", :"SYS_B_04", :"SYS_B_05", :"SYS_B_06",
:"SYS_B_07",:"SYS_B_08", :"SYS_B_09") AND S.INITIATEWHEN = :"SYS_B_10"))
AND R.INITIATEDNEXT = :"SYS_B_11"
AND R.OPSYS = :1
AND R.RUNLOCATION = :"SYS_B_12"
AND R.RECURNAME <> :"SYS_B_13"
AND R.PRCSJOBSEQ = :"SYS_B_14"
AND R.SERVERNAMERUN = :2
AND R.RECURNAME = S.RECURNAME
COST
220
220
3
214
Rows
0
15
0
16
CARD operation
SELECT STATEMENT
12,516
HASH JOIN
15
TABLE ACCESS FULL PS_PRCSRECUR
34,057
TABLE ACCESS FULL PSPRCSQUE
ELAPSED
ROWS
CR_GETS
0.060
0.000
0.060
0
15
0
2,362
3
2,359
Example 2
SELECT Q.PRCSINSTANCE, Q.JOBINSTANCE, Q.MAINJOBINSTANCE, Q.SESSIONIDNUM
, Q.OPRID, Q.OUTDESTTYPE, Q.GENPRCSTYPE, Q.PRCSTYPE, P.PRCSOUTPUTDIR
FROM PSPRCSQUE Q, PSPRCSPARMS P
WHERE Q.RUNSTATUS = :1
AND Q.SERVERNAMERUN = :2
AND Q.RUNLOCATION = :"SYS_B_0"
AND Q.PRCSINSTANCE = P.PRCSINSTANCE
COST
546
546
201
341
Rows
1
1
38539
17
CARD operation
SELECT STATEMENT
1,065
HASH JOIN
1,101
TABLE ACCESS FULL PSPRCSQUE
36,284
TABLE ACCESS FULL PSPRCSPARMS
ELAPSED
ROWS
CR_GETS
0.240
1
0.030
1
0.080 38,539
6,922
2,359
4,563
Example 3
SELECT R.PRCSINSTANCE, R.ORIGPRCSINSTANCE, R.JOBINSTANCE, R.MAINJOBINSTANCE
, R.MAINJOBNAME, R.PRCSITEMLEVEL, R.PRCSJOBSEQ, R.PRCSJOBNAME, R.PRCSTYPE
, R.PRCSNAME, R.PRCSPRTY, TO_CHAR(R.RUNDTTM,:"SYS_B_00"), R.GENPRCSTYPE
, R.OUTDESTTYPE, R.RETRYCOUNT, R.RESTARTENABLED, R.SERVERNAMERQST, R.OPSYS
, R.SCHEDULENAME, R.PRCSCATEGORY, R.P_PRCSINSTANCE, C.PRCSPRIORITY
, S.PRCSPRIORITY, R.PRCSWINPOP, R.MCFREN_URL_ID
FROM PSPRCSQUE R, PS_SERVERCLASS S, PS_SERVERCATEGORY C
WHERE R.RUNDTTM <= SYSDATE
AND R.OPSYS = :1 AND R.RUNSTATUS = :2
AND (R.SERVERNAMERQST = :3 OR R.SERVERNAMERQST = :"SYS_B_01")
AND S.SERVERNAME = :4 AND R.PRCSTYPE = S.PRCSTYPE
AND R.PRCSCATEGORY = C.PRCSCATEGORY AND S.SERVERNAME = C.SERVERNAME
AND ((R.PRCSJOBSEQ = :"SYS_B_02" AND R.PRCSTYPE <> :"SYS_B_03")
OR (R.PRCSJOBSEQ > :"SYS_B_04" AND R.MAINJOBINSTANCE IN (
SELECT A.MAINJOBINSTANCE FROM PSPRCSQUE A WHERE A.MAINJOBINSTANCE > :"SYS_B_05"
AND A.PRCSTYPE=:"SYS_B_06" AND A.RUNSTATUS=:"SYS_B_07"
AND A.PRCSJOBSEQ = :"SYS_B_08" AND (A.SERVERNAMERUN = :"SYS_B_09" OR
A.SERVERNAMERUN = :5))))
AND C.MAXCONCURRENT > :"SYS_B_10"
ORDER BY C.PRCSPRIORITY DESC, R.PRCSPRTY DESC, S.PRCSPRIORITY DESC, R.RUNDTTM ASC
18
Example 3
COST
221
221
220
5
3
2
2
3
2
215
3
2
19
CARD operation
ELAPSED
ROWS
SELECT STATEMENT
108
SORT ORDER BY
0.040
0
FILTER
108
HASH JOIN
8
MERGE JOIN CARTESIAN
0.000
7
1
TABLE ACCESS BY INDEX ROWID PS_SERVERCATEGORY 1
1
INDEX RANGE SCAN PS_SERVERCATEGORY
8
BUFFER SORT
7
8
TABLE ACCESS BY INDEX ROWID PS_SERVERCLASS
8
INDEX RANGE SCAN PS_SERVERCLASS
108
TABLE ACCESS FULL PSPRCSQUE
0.040
0
FILTER
0.000
0
1
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
5
INDEX RANGE SCAN PSDPSPRCSQUE
CR_GETS
2,363
2,363
2,363
4
2
1
2
2
1
2,359
0
0
0
Statistics
TABLE_NAME
PSPRCSQUE
table
PSPRCSQUE
rows
38,539
index
PSAPSPRCSQUE
blks
2,326
empty
0
column
NDV
43
3
3
2
11
3.3333E-01
3.3333E-01
5.0000E-01
9.0909E-02
CLUF
4,998
0
0
0
0
SERVERNAMERUN
PRCSINSTANCE
38,539
3
38,539
242
0
0
3.3333E-01
2.5948E-05
1
0
0
1
0
0
3,735
0
0
PRCSINSTANCE
SESSIONIDNUM
OPRID
38,539
38,539
9,015
139
315
0
0
0
2.5948E-05
1.1093E-04
7.1942E-03
1
0
0
0
1
0
0
0
2,658
0
0
0
1.3795E-04
174
0
1
0
1
0
4,395
0
9.3188E-05
2.5000E-01
5.0000E-01
221
0
0
0
RECURORIGPRCSINST
RECURNAME
INITIATEDNEXT
10,735
10,731
4
2
1
0
0
0
1
0
0
0
3,121
0
0
0
U
PRCSINSTANCE
38,539
38,539
166
0
2.5948E-05
1
0
1
0
2,658
0
SERVERNAMERQST
SERVERNAMERUN
OPSYS
RUNSTATUS
PSBPSPRCSQUE
PSCPSPRCSQUE
PSDPSPRCSQUE
MAINJOBINSTANCE
PSEPSPRCSQUE
PS_PSPRCSQUE
20
row
204
7,249
7,249
DENS
Tuning Example 1
21
Tuning Steps
n Create an index on psprcsque
create index uc_psprcsque_ix1 on psprcsque(prcsjobseq,recurname)
22
Histogram Attempt
With index and histogram on prcsjobseq?
create index UC_PSPRCSQUE_IX1 on PSPRCSQUE (PRCSJOBSEQ, RECURNAME);
Table
PSPRCSQUE
COST
209
209
3
206
NDV
16
density
1.2974E-05
CARD operation
SELECT STATEMENT
236
HASH JOIN
15
TABLE ACCESS FULL PS_PRCSRECUR
641
TABLE ACCESS FULL PSPRCSQUE
Rows
0
15
0
23
column
PRCSJOBSEQ
nulls
0
lo
0
hi
15
ELAPSED
0.040
0.000
0.040
av lg
3
bkts
15
ROWS CR_GETS
0
15
0
2,362
3
2,359
Statistics Attempt A
set_column_stats(USER,'PSPRCSQUE','PRCSJOBSEQ',distcnt=>250);
Table
PSPRCSQUE
COST
40
40
37
3
3
Rows
0
0
112
0
24
column
PRCSJOBSEQ
NDV
250
density
4.0000E-03
nulls
0
CARD operation
SELECT STATEMENT
3
HASH JOIN
9
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
109
INDEX RANGE SCAN UC_PSPRCSQUE_IX1
15
TABLE ACCESS FULL PS_PRCSRECUR
lo
0
hi
15
ELAPSED
av lg
3
bkts
1
ROWS CR_GETS
0.010
0.000
112
0
112
112
49
0
Statistics Attempt B
set_column_stats(USER,'PSPRCSQUE','PRCSJOBSEQ',distcnt=>1000);
Table
PSPRCSQUE
COST
14
14
12
3
2
1
Rows
0
0
112
0
0
25
column
PRCSJOBSEQ
NDV
1,000
density
1.0000E-03
nulls
0
CARD operation
SELECT STATEMENT
1
NESTED LOOPS
2
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
27
INDEX RANGE SCAN UC_PSPRCSQUE_IX1
1
TABLE ACCESS BY INDEX ROWID PS_PRCSRECUR
1
INDEX UNIQUE SCAN PS_PRCSRECUR
lo
0
hi
15
ELAPSED
av lg
3
bkts
1
ROWS CR_GETS
0.010
0.000
112
0
112
112
49
0
0
Tuning Example 2
26
Tuning Steps
n There is a reasonably usable index on
psprcsque
But the optimizer doesnt use it
27
Tuned Example 2
Table
PSPRCSQUE
COST
133
133
132
131
2
1
Rows
1
1
1
1
1
28
column
RUNSTATUS
NDV
11
density
1.3764E-05
nulls
0
lo
1
hi
9
CARD operation
ELAPSED
SELECT STATEMENT
1
NESTED LOOPS
0.020
1
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
1
INDEX SKIP SCAN PSAPSPRCSQUE
1
TABLE ACCESS BY INDEX ROWID PSPRCSPARMS
0.000
1
INDEX UNIQUE SCAN PS_PSPRCSPARMS
av lg
3
bkts
10
ROWS
CR_GETS
16
13
12
3
2
us)
us)
us)
us)
us)
Tuning Example 3
29
Tuning Steps
n By the time we got to tuning this SQL,
there was nothing left to do.
The SQL got tuned as well by the actions to
tune the other two.
30
Tuned Example 3
COST
160
160
159
154
156
3
2
142
3
2
3
2
31
CARD operation
ELAPSED
ROWS CR_GETS
SELECT STATEMENT
5
SORT ORDER BY
0.010
0
2
FILTER
2
5
HASH JOIN
2
5
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
2
5
NESTED LOOPS
2
2
1
TABLE ACCESS BY INDEX ROWID PS_SERVERCATEGORY 0.000 1 2
1
INDEX RANGE SCAN PS_SERVERCATEGORY
1
INLIST ITERATOR
0
0
103
INDEX RANGE SCAN PSAPSPRCSQUE
0
8
TABLE ACCESS BY INDEX ROWID PS_SERVERCLASS
0
1
INDEX RANGE SCAN PS_SERVERCLASS
0
FILTER
0
1
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
0
5
INDEX RANGE SCAN PSDPSPRCSQUE
0
us)
us)
us)
us)
us)
us)
us)
us)
us)
Example 4
COST
144
144
137
135
134
132
131
122
3
118
3
2
1
2
1
1
4
3
3
2
1
32
CARD operation
SELECT STATEMENT
1 VIEW
FILTER
1
SORT GROUP BY
FILTER
1
NESTED LOOPS
1
NESTED LOOPS
1
NESTED LOOPS
1
NESTED LOOPS
3
HASH JOIN
2
TABLE ACCESS FULL PS_RT_RATE_TBL
9
TABLE ACCESS FULL PS_CUST_CREDIT
1
TABLE ACCESS BY INDEX ROWID PS_CUSTOMER
1
INDEX RANGE SCAN PSBCUSTOMER
1
TABLE ACCESS BY INDEX ROWID PS_RT_INDEX_TBL
1
INDEX UNIQUE SCAN PS_RT_INDEX_TBL
1
TABLE ACCESS BY INDEX ROWID PS_CUST_DATA
1
INDEX RANGE SCAN PSACUST_DATA
1
TABLE ACCESS BY INDEX ROWID PS_CUSTOMER
1
INDEX UNIQUE SCAN PS_CUSTOMER
1
SORT AGGREGATE
1
TABLE ACCESS BY INDEX ROWID PS_CUST_CREDIT
1
INDEX RANGE SCAN PS_CUST_CREDIT
1
SORT AGGREGATE
2
NESTED LOOPS
1
TABLE ACCESS FULL PS_RT_INDEX_TBL
2
INDEX RANGE SCAN PS_RT_RATE_TBL
Wolfgang Breitling, Centrex Consulting Corporation
ROWS
ELAPSED
CR_GETS
87
87
12,565
24,437
3,244,217
3,244,217
13,519,270
13,519,270
12,985,742
975
34,676
13,519,270
13,831,838
13,519,270
13,519,270
3,244,217
9,206,235
3,244,217
3,244,217
12,631
12,767
12,771
4
749
4
749
21.490
21.490
21.480
606.170
600.030
565.460
376.740
259.290
23.110
0.020
0.390
194.890
106.360
84.440
26.830
152.400
98.990
25.700
13.270
1.290
1.190
0.340
0.040
0.040
0.000
0.040
81,889,486
81,889,486
81,889,486
81,889,486
81,851,299
75,362,863
53,001,492
39,482,220
1,239
22
1,217
39,480,981
26,075,609
13,519,272
2
22,361,371
13,627,522
6,488,436
3,244,219
38,160
38,160
25,378
27
27
12
15
Tuning Example 4
33
Tuning Steps
n Adjust the density of the effdt column of tables
ps_rt_rate_tbl and ps_cust_credit:
set_column_stats(USER,'PS_CUST_CREDIT','EFFDT',density => 1);
set_column_stats(USER,'PS_RT_RATE_TBL','EFFDT',density => 1);
34
Tuned Example 4
COST
53825
53825
8409
756
2
748
3
742
570
492
338
118
48
152
4
3
3
2
1
35
CARD operation
ROWS
SELECT STATEMENT
6,488 VIEW
87
FILTER
87
6,488
SORT GROUP BY
12,565
FILTER
24,437
129,744
HASH JOIN
3,244,217
1
TABLE ACCESS FULL PS_RT_INDEX_TBL
1
129,744
HASH JOIN
3,244,217
975
TABLE ACCESS FULL PS_RT_RATE_TBL
975
3,726
HASH JOIN
24,578
2,651
HASH JOIN
36,097
2,651
HASH JOIN
36,097
5,267
TABLE ACCESS FULL PS_CUSTOMER
40,715
39,207
TABLE ACCESS FULL PS_CUST_CREDIT
34,676
42,133
INDEX FAST FULL SCAN PS0CUSTOMER
42,133
29,605
TABLE ACCESS FULL PS_CUST_DATA
29,609
1
SORT AGGREGATE
14,091
1
TABLE ACCESS BY INDEX ROWID PS_CUST_CREDIT
14,229
1
INDEX RANGE SCAN PS_CUST_CREDIT
14,233
1
SORT AGGREGATE
4
2
NESTED LOOPS
749
1
TABLE ACCESS FULL PS_RT_INDEX_TBL
4
2
INDEX RANGE SCAN PS_RT_RATE_TBL
749
ELAPSED
CR_GETS
17.720
17.720
17.710
17.430
13.610
0.000
7.570
0.030
2.800
1.980
1.210
0.170
0.210
0.070
0.040
0.890
0.860
0.220
0.020
0.020
0.000
0.020
49,408
49,408
49,408
49,408
6,807
3
6,804
22
6,782
5,212
4,724
3,507
1,217
488
1,570
42,574
42,574
28,326
27
27
12
15
CARD operation
54
142
1,703
1
1
SELECT STATEMENT
NESTED LOOPS
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
INDEX RANGE SCAN UC_PSPRCSQUE_IX1
TABLE ACCESS BY INDEX ROWID PS_PRCSRECUR
INDEX UNIQUE SCAN PS_PRCSRECUR
ELAPSED
ROWS
CR_GETS
0.010
0.010
0.010
0.000
0.000
0
0
112
0
0
111
111
48
0
0
ELAPSED
ROWS
CR_GETS
0.010
0.000
112
0
112
112
49
0
0
36
CARD operation
SELECT STATEMENT
1
NESTED LOOPS
2
TABLE ACCESS BY INDEX ROWID PSPRCSQUE
27
INDEX RANGE SCAN UC_PSPRCSQUE_IX1
1
TABLE ACCESS BY INDEX ROWID PS_PRCSRECUR
1
INDEX UNIQUE SCAN PS_PRCSRECUR
37
ATTR_VALUE
OPT_ESTIMATE(@"SEL$1",
OPT_ESTIMATE(@"SEL$1",
OPT_ESTIMATE(@"SEL$1",
OPT_ESTIMATE(@"SEL$1",
OPT_ESTIMATE(@"SEL$1",
OPT_ESTIMATE(@"SEL$1",
OPT_ESTIMATE(@"SEL$1",
38
References
Berg, Martin. Query Tuning by Eliminating Throwaway. 2000.
http://www.miracleas.dk/tools/throwaway2.pdf.
Breitling, W. (2003). Fallacies of the Cost Based Optimizer. Paper presented at the Hotsos
Symposium on Oracle Performance, Dallas, Texas.
Bruno, N. and S. Chaudhuri (2002). Exploiting Statistics on Query Expressions for Optimization.
Paper presented at the ACM SIGMOD international conference on Management of
data, Madison, Wisconsin.
Christodoulakis, S. (1984). Implications of Certain Assumptions in Database Performance Evaluation.
ACM Transactions on Database Systems (TODS), 9(2).
Markl, V. and G. Lohman (2002). Learning Table Access Cardinalities with LEO. Paper
presented at the ACM SIGMOD international conference on Management of data,
Madison, Wisconsin.
Millsap, C. and J. Holt (2003). Optimizing Oracle Performance. O'Reilly. ISBN: 0-596-00527-X.
39
My favorite websites
asktom.oracle.com
integrid.info
www.evdbt.com
www.go-faster.co.uk
www.ixora.com.au
www.jlcomp.demon.co.uk
www.juliandyke.com
www.hotsos.com
www.miracleas.dk
www.oracledba.co.uk
www.oraperf.com
www.orapub.com
www.scale-abilities.com
40
(Thomas Kyte)
(Tanel Pder)
(Tim Gorman)
(David Kurtz)
(Steve Adams)
(Jonathan Lewis)
(Julian Dyke)
(Cary Millsap)
(Mogens Nrgaard)
(Connor McDonald)
(Anjo Kolk)
(Craig Shallahamer)
(James Morle)
Hotsos Symposium March 5-9, 2006
Wolfgang Breitling
breitliw@centrexcc.com