September911,2013

Anaheim,California

WebIntelligenceReportValidationTipsandTechniques
MichaelWelter

LearningPoints

First,becomefamiliarwithwhatisright
Carefullyobservehowyourreportdivergencesfromright
Categorizethedivergence
Systematicallyfixtheproblem

ReturnonInvestment

Timesavedinvalidatingandfixingreports
ConsumerconfidenceinreliableBIreports
Properbusinessdecisionsbasedoncorrectanalysis
Reducedwearandtearonreportdeveloper(s)

BreakoutAbstract
WebIntelligenceReportValidationTipsandTechniques
HaveyoueverhadunexpectedresultswhendevelopingacomplexWeb
Intelligencereport?ThissessionisforWebIntelligencereportdesigners
andITstaffwhosupportthem,lookingforsometimesavingpointersand
frustrationalleviatingtoolsfortroubleshootingandvalidatingreports.See
howtoidentifyproblemsinthesourcedata,intheuniverse,andinthe
reportitself.Learntechniquesforverifyingaccuracy,andexplorethe
mysteriesofcalculationcontextsinWebIntelligence.Seethesemethodsin
actionwithliveexamples.

Agenda
1.
2.
3.
4.
5.

Whyarewetalkingaboutvalidation?
WhatsRight?
WhatsWrong?
Conclusion
Q&A

WhyAreWeTalkingAboutValidation?
1.
2.
3.
4.
5.

Whyarewetalkingaboutvalidation?
WhatsRight?
WhatsWrong?
Conclusion
Q&A

Afancy,beautifulreportthatswrongisuseless(orworse)

BusinessObjectsGivesYouthePowerofPerspective
Turnstoreddataintodecisions
Decisionmakingisamaze
Wehaveagroundlevel
perspective
Needanaerialperspective

Businessdecisionsrequireaccess
toinformation
Flexible
Dependable
Accurate

ABigJobThatLooksEasy
Queryingadatabaseisnotatrivialmatter
SQLcanbecomplex
Databasedesignsareoftencomplicated
BusinessObjectsWebIntelligenceisabiglever
Draganddroptobuildcomplexqueries
Navigatesthelabyrinthforyou
Knowstherulesofconnectingdatabasetables
Knowswhentoseparatestatementsforyou

Youdonthavetounderstandwhathappensbehindthescenes
Consistentlycorrectresultsamazinglyoften!

DetectandFixErrors
Stillplentyofroomforerror!
Onedaytobuild;Twoweeksto
debug
Errorscanbeelusive
Oftentakesmuchlongertofindand
fixthanoriginaldevelopment
Generallyfrustrating

Weneedawaytofindandfixthese
errors
Asystematicapproachtodetecting
problems
Amapforwhentofixandwhento
gethelp
Asetoftoolsforwhenitstimeto
divein

Assumptions
YouhavebasictointermediateWebIntelligenceskills
YouareusingWebIntelligencetoreportfromdatathat
Isbelievedtobecorrect
Hassomebenchmarks(perhapsapreviousreport)
You(oracolleague)knowthedataandthebusinessitrepresents
Thedatabaseyouarereportingfromiseither
Thesameastheoriginalwork,or
Adatawarehouse/martbasedontheoriginaldatabase(andisvalid)
YouhaveITsupport(orskills)
Youhaveextract/transform/loadsupport(ifwarehouse)
Youhaveuniversesupport(orskills)
YouhavesomeideahowtoreadSQL

10

WhatsRight?
1.
2.
3.
4.
5.

Whyarewetalkingaboutvalidation?
WhatsRight?
WhatsWrong?
Conclusion
Q&A

Whattargetareyoutryingtohit?

11

StartWithWhatisCorrect
Definebenchmarksfirst
Youllsavetimeifyouknowthe
answer
Ifyoudontknowwhatcorrectis,
youwontknowwhenyouhitormiss
it

Thenanalyzeyourreportsdeviation
fromthebenchmark
Thefollowingisastrategyfor
definingbenchmarks

12

CollectDataPointsFromBusinessExpert(s)
Collectbenchmarkdatapointsfrombusinessexperts
Perhapsthepersonwhowasresponsibleforthepreviousversionofa
reportyourereplacing
Someonewhoknowsthesubjectandthedata
Someonewhoknowshowitwasdone
Collectdatapointsatthelowestgrain
Mostspecific
Example:Sales_Qty,Sales_AmtofProduct1059on8/1/2007atStore21by
Employee99
andatvariouslevelsofaggregation
Increasinglymoresummarized
Acrossalldimensionsinvolvedinthereport
Example:Sales_Qty,Sales_AmtofallProductson8/1/2007atStore21by
allemployees
13

DiscoverHowTheseDataPointsWereProduced

Whatelementswereused?
Whatcalculationswereused?
Howwastimerelated?
Anyoutsidedatasources?
Isanydatasourcerequiredoutsidethedatabasefromwhichwere
reporting?
Ifso,howwasthatdatarelatedtotherest?
GettheSQL
Ifpossible,getacopyoftheSQLused
Softcopyisbest

14

DiscoveryMethods
Interview
Theresnosubstitutefortalking
Dothisinpersonifpossible
Observation
Example1:reportscurrentlyproducedfromAccess

Notewhatdatabase,queries,macros(ifany)theyused
GetacopyoftheSQLused
Notetheprocessingflow:Whatstepsandinwhatorder?
Noteanytemporarytablesused

Example2:reportscurrentlycomingfromERPsystem
Notetheprocessingflow:Whatutilities/commands,inwhatorder?
AskITtoexplainhowthatappgetsitsreport
AskITforcopyofSQLtheappgeneratesifpossible

15

VerifyResults
VerifyyouorITisabletoduplicateresultsofbenchmarkSQL
Atalllevelsofaggregationprovidedinbenchmarkdatapoints
Insamesystemasoriginalreportcamefrom
IndatawarehousewithanalogousSQL
VerifyrequiredSQLispresentinuniverse
Askuniversedesignerforhelphere
Fragmentssimilartobenchmarkexamples,or
Analogousfragmentsbuilttoworkagainstdatawarehouse
Duplicatebenchmarknumberswithsimplereports
MakebarebonesWebIntelligencereport
CheckSQLagainstbenchmarkSQL
GetIT/designerhelpasnecessary

16

WhatsWrong?
1.
2.
3.
4.
5.

Whyarewetalkingaboutvalidation?
WhatsRight?
WhatsWrong?
Conclusion
Q&A

Howdoesyourreportdeviatefromthebenchmarks?

17

NowYouCanStarttoSeeWhatsWrong
Canadianbanktrainingtechnique
Plumblineexample
Nowwecanstarttoseewhats
wrong
Wehavesomeknowngooddata
points
Themoreyouknowthesourcedata,
themoreintuitiveyoullbeat
spottingdeviation

18

YourBusinessObjectsTeam
DatabaseproblemsgotoIT
Universeproblemsgotoyouruniversedesigner
Asermon/wordofcautionregardingyourteam
Whenaproblemisinanothersareaofexpertise
Itsoutofyourhands(relief)
Whydidnttheycatchthatproblem(frustration)
Bytheway,its,um,outofyourhands(morefrustration)

Youneedthewholeteam
Dataproblemsarearealityinyoungdatawarehouses
Avoidblame
Resentmentanddefensivenessdegradeteamworkquickly
Youworkhalfspeedwithaswordinonehand

19

TheTools
Thecureisofteneasyonceyouknow
whatswrong
Thetools
Thefollowingstrategyhelpsfocus
yourefforts
Thefollowingtechniquesgiveyou
leverage

Theapproach
Grouppossibleissuesbysymptom
Workthroughthepossibilities

20

NumberValuesConsistentlyTooHighorTooLow

Reportorblockfilter
Theresafilterinplacethatisntobvious
Demo
Toolowconditiononly
Queryconditionisntright
Youmayhavetoomanyortoofewwherestatements
Youmaybefilteringonthewrongelements
Wronguniversemeasure(unitsinsteadofdollars?)
Measureisdefinedincorrectlyorimprecisely
Ex:bookedordersvs.revisedbookings
Semanticproblemneedstobeaddressedinthebusinessandinthe
universe

21

CheckFilters
Showfilterpane

Viewstructure

22

CheckFilters
Selectfiltersonmapstab

23

CheckFilters
Clickthroughreportelementstofindfilters...

Aha...foundone!

24

CheckQueryConditions
Lookforunintendedconditions

25

WrongUniverseMeasure?
Shouldhave$insteadofunits

26

ObjectDefinedIncorrectly
Checkforwhatcorrectis...

27

ObjectDefinedIncorrectly
ChecktheSQLyourobjectisgenerating

28

NumberValuesConsistentlyTooHighorTooLow

Wrongaggregationtypeonauniversemeasure
Unusual,butpossible
Useobjectinbarebonesreport
Checkthemathonaggregation

29

Useobjectintableblockfooter(noagg function)
Comparetoobjectusedinfooterw/aggregatefunction

Designeraddressesinuniverse

WrongAggregationType
Valueseemstoolow

Especiallythetotal

30

WrongAggregationType
IDthemiscreantobject

Makeasimpledocwithit

31

WrongAggregationType
Insertasumonyoursimpletable

32

WrongAggregationType
...butwithnoaggregatefunction

33

WrongAggregationType
Thatlooksmorelikeanaveragethanasum...

34

WrongAggregationType
Ah,thebareobjectis projectingasanaverage

Askuniversedesignertofixthemeasuresaggregationfunction

35

NumberValuesAreInconsistentlyWrong

36

Symptom:somevaluesaretoohigh,otherstoolow

Universeisconnectedtotestdata
Youmayknowbydate,orsomeothertelltaledata
Askuniversedesignertocheckthis

Wrongdatekeyinfacttablejoinedtodatetable
Manydatawarehouseshaveadatelookup
Facttablemayhavemanydates designerpickedone
Example:Salesbydateshoulduseorderdate,notshipdate
Whenviewbydate,somemonthsaretoohigh,otherstoolow
ReviewSQLtoseewhatdateisbeingused

WrongDateKey
Resultsaretoolowsomequarters,toohighothers

37

WrongDateKey
ExamineSQLstatement:itappearstouseashipdate

Askdesignertofixthedate

38

SomeValuesTooHighbyEvenMultiples
Symptom
Somevaluestoohigh,othersmaybecorrect
Whentoohigh,itsbyanevenmultipleofthecorrectvalue
ThissuggestsaSQL(fan/chasm)trapintheuniverse
IDtheoffendingobject(s)bysystematicallypullingobjectsoutofthequery
untilthevaluesarecorrect
Sharefindingswithyouruniversedesignersotheycanfix

39

SomeValuesTooHighbyEvenMultiples
Somevaluestoohigh
Thosethataretoohigharesobyevenmultiples
($190,174*2=$380,348)

40

SomeValuesTooHighbyEvenMultiples
ToIDaSQLtrap,startpullingobjectsoutofthequery
IsitShowroomTown?

41

SomeValuesTooHighbyEvenMultiples
ToIDaSQLtrap,startpullingobjectsoutofthequery
IsitShowroomTown?

No.IsitShowroom?

42

SomeValuesTooHighbyEvenMultiples
ToIDaSQLtrap,startpullingobjectsoutofthequery
IsitShowroom?

Nope.HowaboutSaleQty?

43

SomeValuesTooHighbyEvenMultiples
ToIDaSQLtrap,startpullingobjectsoutofthequery
BINGO!

NowaddShowroomobjectsbackin

44

SomeValuesTooHighbyEvenMultiples
ToIDaSQLtrap,startpullingobjectsoutofthequery
Revenuestilllooksgood

Nowgotellyouruniversedesigner:
SalesRevenueistoohighforsomecustomers
Byevenmultiples
WhenyouaddSalesQtytothequery

Yoususpectafanorchasmtraptheyllbeimpressed(orannoyed)

45

HistorybyCertainDimensionisWrong
Example
TY/LYsalesbylineiswrongforlastyear(buttotalisOK)
WhenyoucompareTY/LYsalesbystate,everythingisfine

46

HistorybyCertainDimensionisWrong
Totalsareright,buttwolinesarewrongforlastyear

47

HistorybyCertainDimensionisWrong
Maybeslowlychangingdimensionindatawarehouse

48

DiscusswithbusinessandIT
MaybesomeproductshavechangedLinessincelastyear
Businessmustdecidetoviewbycurrentorhistoricalalignment
Solutionisindatawarehouse(ifextract/transform/loadpersonis
abletochangebehavior)

MissingDimensionValuesWhereNothingHappened
Example
Salesbyproductlacksnosaleproducts

49

MissingDimensionValuesWhereNothingHappened
Innerjoinusedinuniverse,butforgoodreason
Weneedtoseeallcustomers,salesornosales
Youcancheat:Demo
Forcereportsideouterjoin

50

MissingDimensionValuesWhereNothingHappened
Addquery:fulllistofdimensionvalues(listitemsonly)

51

MissingDimensionValuesWhereNothingHappened
Addtable:fullproductlist

52

MissingDimensionValuesWhereNothingHappened
Verifydimension(s)merged

53

MissingDimensionValuesWhereNothingHappened
Makemergedtable
Usedimensionfrom2nd queryorthemergeddimension

54

CalculatedVariablesAreWrong
1.
2.

Wrongcalculation(checkarguments)
Objectsusedbyvariablerenderingwrongresults
Duplicatereportusingreferencedobjects
Workthroughthetoolow/highsection
3. Calculationcontextisincorrect
Example:calculatedcolumnvaluenotcomputedright
Requiresspecificoutputcontext
Demoofcalculationcontexts

55

CalculatedVariablesAreWrong
Well,THATsnotright

56

CalculatedVariablesAreWrong
Clickcell noticeitcontainsavariable

57

CalculatedVariablesAreWrong
Editvariable(HmmformulalooksOK)

Butletscheckhelpanyway

58

CalculatedVariablesAreWrong
Help>FormatDate>IndextoDate&NumberFormats

ArgumentshouldbeMM/dd/yyyy(notmm/dd/yyyy)
59

CalculatedVariablesAreWrong
Fixthevariablesformula

60

CalculatedVariablesAreWrong
Thatsbetter!

61

ObjectsUsedbyCalculationareWrong
Businessanalystsaysmargingoalcalculationistoolow

MargingoalcalculationlogiclooksOK

62

ObjectsUsedbyCalculationareWrong
Whatabouttheunderlyingobject?
Margingoalsvariablesformulais=[Marginw]*1.1
Isthemarginwmeasuregivingustherightvalues?
Letstestit:

63

ObjectsUsedbyCalculationareWrong
Ah:marginwisnotgivingcorrectresults:

Calculationiswrongbecauseunderlyingobjectiswrong
Followpreviousstepstodiagnoseproblemwiththisobject
64

WrongCalculationContext
Storesshouldbeclassifiedbasedonyearsopen
Young,averageorold,relativetotheaverageforallstores
Butallofourssayaverage somethingisamiss...

65

WrongCalculationContext
Storeclasscolumnisempty

66

WrongCalculationContext
Analerterpopulatesthecolumn

67

WrongCalculationContext
Thealerterslogicdependsonvariables

68

WrongCalculationContext
whichdependonothervariables

69

WrongCalculationContext
Letsseewhatsgoingonwith.Avg YearsOpen
Duplicatethereport,andplaceanaverageonYearsOpen

Theaveragestorehasbeenopen14.6yearsatthistime

70

WrongCalculationContext
Letslookatour.AvgYearsOpenobjectinthetable:

.AvgYearsOpeniscalculatingtheaverageofeachrow
71

WrongCalculationContext
Whatwewantisforeachrowtohaveavalueof14.6
Theaverageofallrowsintheblock
Justlikeourformulasittinginthefooteroftheblock
Bydefault,BusinessObjectsmeasures:
Considerallthedimensionstheysee(intheirblockorsection)
Aggregatetothelevelappropriatetowheretheyareplaced
Wewantour.AvgYearsOpenmeasureto:
Consideralldimensionsforeachrow(default),but
Aggregateasifitwereitplacedintheblocksfooter

72

WrongCalculationContext
ThisisCalculationContext:
InputContext=thelistofdimensionsthecalculationconsiders
OutputContext=theleveltowhichthecalculationaggregates
Syntax:=AggregateFx([measure] input context) output context
Inourcase,weonlywanttochangetheoutputcontext:

73

WrongCalculationContext
Now.AvgYearsOpenbehaveslikeafooterformula
Butitdoessoforeachrow

74

WrongCalculationContext
Andnowthatourcontextforaverageiscorrect
Ouralertercompareseachstoretotheaverageofallstores

75

Conclusion
1.
2.
3.
4.
5.

76

Whyarewetalkingaboutvalidation?
WhatsRight?
WhatsWrong?
Conclusion
Q&A

KeyLearnings

77

Becomewellfamiliarwithwhatrightlookslike
Carefullyobservehowyourreportdivergencesfromright
Categorizethedivergence
Worktheproblemsystematically

BestPractices
Indeterminingwhatisright",collectnotonlyexistingreportsandSQL,but
alsoknownaccuratedatapointsatvariouslevelsofgranularity
Takethetroubletounderstandhowthebusinessworks
VerifyresultsofsuppliedSQL
Beoneofthoseplayersthatraisesyourteammates'game

78

Q&A
1.
2.
3.
4.
5.

79

Whyarewetalkingaboutvalidation?
WhatsRight?
WhatsWrong?
Conclusion
Q&A

Thank you for participating.

Please provide feedback on this session by
completing a short survey via the event
mobile application.
SESSION CODE: 1209
Learn more year-round at www.asug.com