1

More

NextBlog»

CreateBlog SignIn

Home

DatastageRelated

DatastageTraining

BigData

Unix

Database

InterviewRelated

Certifications

DiscussionForum

Feedback

ManythanksforvisitingmyBlog !!Pleasesharethisblogusin

SEARCHYOURPROBLEMSSOLUTIONINTHISBLOG

Search

Like

0

Tweet 0

Share

0

₁

4 ShareThisBlog !!

DatastageSlowlyChangingDimensions

DatastageImplementations–SlowlyChangingDimensions

BasicsofSCD

SlowlyChangingDimensions(SCDs)aredimensionsthathavedatathatchangesslowly,ratherthanchangingonatime­based,regularschedule.

Type1

TheType1methodologyoverwritesolddatawithnewdata,andthereforedoesnottrackhistoricaldataatall.

Hereisanexampleofadatabasetablethatkeepssupplierinformation:

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Supplier_Key Supplier_Code Supplier_Name			Supplier_State
123	ABC	AcmeSupplyCo	CA

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Inthisexample,Supplier_CodeisthenaturalkeyandSupplier_Keyisasurrogatekey.Technically,thesurrogatekeyisnotnecessary,sincethe

tablewillbeuniquebythenaturalkey(Supplier_Code).However,thejoinswillperformbetteronanintegerthanonacharacterstring.

NowimaginethatthissuppliermovestheirheadquarterstoIllinois.Theupdatedtablewouldsimplyoverwritethisrecord:

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Supplier_KeySupplier_Code Supplier_Name Supplier_State

123 ABC

AcmeSupplyCo IL

TRANSLATETHISBLOG

SelectLanguage ​▼

LogIn

VISITOR'SVIEWCOUNT

8 0 6 2 5 9

ABOUTME:CLICKONG+BUTTONTOFOLLOWME

Devendra Kumar Yadav

Follow

181 followers

OTHERDATASTAGEQUESTIONSSOLUTIONS

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Type2

TheType2methodtrackshistoricaldatabycreatingmultiplerecordsforagivennaturalkeyinthedimensionaltableswithseparatesurrogatekeys

and/ordifferentversionnumbers.WithType2,wehaveunlimitedhistorypreservationasanewrecordisinsertedeachtimeachangeismade.

Inthesameexample,ifthesuppliermovestoIllinois,thetablecouldlooklikethis,withincrementedversionnumberstoindicatethesequenceof

changes:

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Supplier_KeySupplier_CodeSupplier_Name Supplier_StateVersion

123 ABC	AcmeSupplyCoCA	0
124 ABC	AcmeSupplyCoIL	1

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Anotherpopularmethodfortupleversioningistoaddeffectivedatecolumns.

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

Supplier_KeySupplier_CodeSupplier_Name Supplier_StateStart_Date End_Date

123 ABC	AcmeSupplyCoCA	01­Jan­200021­Dec­2004
124 ABC	AcmeSupplyCoIL	22­Dec­2004

­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­

ThenullEnd_Dateinrowtwoindicatesthecurrenttupleversion.Insomecases,astandardizedsurrogatehighdate(e.g.9999­12­31)maybeused

asanenddate,sothatthefieldcanbeincludedinanindex,andsothatnull­valuesubstitutionisnotrequiredwhenquerying.

HowtoImplementSCDusingDataStage8.1–SCDstage?

Step1:Createadatastagejobwiththebelowstructure­

1. SourcefilethatcomesfromtheOLTPsources

2. Olddimesionreferncetablelink

3. TheSCDstage

4. TargetFactTable

5. DimesionUpdate/Insertlink

► 2014(34)

▼ 2013(48)

► Dec(8)

► Nov(15)

► Oct(12)

▼ Jan(13)

IBMDataStage8.5NewFeatures

UseofStagesinDatastage8.5or8xSeries

NewDebugfeatureinDataStage8.5

DSParallelProcessing&PartitionTechniques

DatastageTransformerStageLoopingconcept

DataModelingConceptinDatawarehouse

DW&BIConceptsInterviewQuestionsandAnswers

Datastage8xEngine/TiersStartandStopProcess

RT_SCTEMPError:NotabletorunDatastageJobs

ParametersUsingParameter/ValueSet/ValueFile

MYMOSTPOPULARFREQUENTLYACCESSEDPOSTS

Datastage8.5,8.7and9.1Differences

DatastageTransformerStageLoopingconcept

Datapartitioning&collectingmethodsExamples

IBMDatastage9.1NewlyAddedfeatures

DATASTAGEPerformanceTuningTipsV1.1

SurrogateKeyGeneratorImplementation

ParametersUsingParameter/ValueSet/ValueFile

UseofStagesinDatastage8.5or8xSeries

DatastageInterviewQuestionsandAnswersV1.4

DatastageScenarioBasedQuestion/Answer:1

LISTOFVISITOR'SCOUNTRIES

Figure1

Step2: TosetuptheSCDpropertiesintheSCDstage,openthestageandaccesstheFastPath

Figure2

Step3:Thetab2ofSCDstageisusedspecifythepurposeofeachofthepulledkeysfromthereferenceddimensiontables.

RECENTLYVISITEDUSER'SLOCATION

LiveTrafficFeed

AvisitorfromMontréal,Quebecviewed "DEV'SDATASTAGE TUTORIAL,GUIDES,TRAININGAND

ONLINEHELP4U.UNIX,ETL,DATABASE

AvisitorfromMumbai,Maharashtraleft

RELATEDSOLUTIONS:SurrogateKey

GeneratorImplementation"4minsago

TUTORIAL,GUIDES,TRAININGAND

"DEV'SDATASTAGE

ONLINEHELP4U.UNIX,ETL,DATABASE

RELATEDSOLUTIONS:DatastageRelated ProblemsandSolutions"viadatastageinfoguid AvisitorfromHyderabad,AndhraPradesh viewed"DEV'SDATASTAGE e.blogspot.com11minsago TUTORIAL,GUIDES,TRAININGAND

ONLINEHELP4U.UNIX,ETL,DATABASE

AvisitorfromHyderabad,AndhraPradeshleft

RELATEDSOLUTIONS:DatastageJobsBest

PracticesforTuning"12minsago

TUTORIAL,GUIDES,TRAININGAND

ONLINEHELP4U.UNIX,ETL,DATABASE

AvisitorfromBangalore,Karnatakaviewed

funnelstage"viadatastageinfoguid

e.blogspot.com.au12minsago

ONLINEHELP4U.UNIX,ETL,DATABASE

RELATEDSOLUTIONS:SequentialFileBest "DEV'SDATASTAGE PerformanceTips/Settings"17minsago AvisitorfromMumbai,Maharashtraleft TUTORIAL,GUIDES,TRAININGAND

ONLINEHELP4U.UNIX,ETL,DATABASE

RELATEDSOLUTIONS:DatastageRelated ProblemsandSolutions"viadatastageinfoguid AvisitorfromMumbai,Maharashtraviewed "DEV'SDATASTAGE e.blogspot.com24minsago TUTORIAL,GUIDES,TRAININGAND

ONLINEHELP4U.UNIX,ETL,DATABASE

TUTORIAL,GUIDES,TRAININGAND

"DEV'SDATASTAGE

RELATEDSOLUTIONS:Searchresultsfor

"DEV'SDATASTAGE

RELATEDSOLUTIONS:Conductor AvisitorfromHyderabad,AndhraPradesh Node,SectionLeadersandPlayers"35minsago viewed"DEV'SDATASTAGE

TUTORIAL,GUIDES,TRAININGAND

ONLINEHELP4U.UNIX,ETL,DATABASE

RELATEDSOLUTIONS:DatastageJobsBest

AvisitorfromGrandRapids,Michiganviewed

PracticesforTuning"37minsago

"DEV'SDATASTAGE

Figure3

Step4:Tab3isusedtoprovidetheseqencegeneratorfile/tablenamewhichisusedtogeneratethenewsurrogatekeysfortheneworlatest

dimesionrecords.Thesearekeyswhichalsogetpassedtothefacttablesfordirectload.

TUTORIAL,GUIDES,TRAININGAND RELATEDSOLUTIONS:IBMDatastage9.1 ONLINEHELP4U.UNIX,ETL,DATABASE DATASTAGE NewlyAddedfeatures"40minsago TUTORIAL,GUIDES,TRAININGAND AvisitorfromIndiaviewed"DEV'S

ONLINEHELP4U.UNIX,ETL,DATABASE Real­timeview·Menu

DAILYHOROSCOPES

DailyHoroscopes

Aries

3/21­4/19

Taurus

4/20­5/20

Gemini

5/21­6/21

Cancer

6/22­7/22

Leo

7/23­8/22

Virgo

8/23­9/22

Libra

9/23­10/22

Scorpio

10/23­11/21

Sagittarius

11/22­12/21

Capricorn

12/22­1/19

Aquarius

1/20­2/18

Pisces

2/19­3/20

Madebyalbinoblacksheep.comTerms

MYBLOGPOSTS

DEV'SDATAWAREHOUSINGHELPGUIDE

DataStageParalleljobsfailwithfailurecode11

Figure4

Step5: TheTab4isusedtosetthepropertiesforconfiguringthedatapopulationlogicforthenewandolddimensionrows.Thetypeofactivies

thatwecanconfigureasapartofthistabare:

1. GenerationthenewSurrogatekeyvaluestobepassedtothedimensionandfacttable

2. Mappingthesourcecolumnswiththesourcecolumn

3. Settingupoftheexpiredvaluesfortheoldrows

4. Definingthevaluestomarkthecurrentactiverowsoutofmultipletyperows

Figure5

Step6:Setthederivationlogicforthefactasapartofthelasttab.

Figure6

Step7:Completetheremainingsetup,runthejob

Figure7

_Reactions:

+1

Like (0)

Recommend this on Google

Useful (0)

Dislike (0)

AROUNDTHEWEB

PBHNetwork

IncrediblePicture

GalleriesOfLifeAround

TheWorld

Lifespan

Stack

WHAT'STHIS?

14FoodsThatWillMake 'X­Force'WillBetheNext

YouHappier

X­MenMovie

Web2Carz.com

TheTop5CoolestKit

Cars

ALSOONDK.DSXCHANGE­DWBITUTORIAL

DEV'SDATASTAGETUTORIAL,GUIDES,TRAINING

ANDONLINEHELP4U.UNIX,ETL,DATABASE

DEV'SDATASTAGETUTORIAL,GUIDES,TRAINING

ANDONLINEHELP4U.UNIX,ETL,DATABASE

DEV'SDATASTAGETUTORIAL,GUIDES,TRAINING

ANDONLINEHELP4U.UNIX,ETL,DATABASE

DEV'SDATASTAGETUTORIAL,GUIDES,TRAINING

ANDONLINEHELP4U.UNIX,ETL,DATABASE

1Comment

DK.DSXchange­DWBITutorial

 Login

SortbyNewest

Share _⤤ Favorite _★

Jointhediscussion…

Articlegivesgoodunderstandingofimplementation goodjob

maheshtanpure • 2yearsago

NewerPost

Home

Subscribeto:PostComments(Atom)

OlderPost

DISCLAIMER

Allcontentprovidedonthishttp://datastageinfoguide.blogspot.inblogisforinformationalpurposesonly.Theownerofthisblogmakesnorepresentationsastotheaccuracyorcompleteness

ofanyinformationonthissiteorfoundbyfollowinganylinkonthissite.Theownerofhttp://datastageinfoguide.blogspot.inwillnotbeliableforanyerrorsoromissionsinthisinformationnorfor

theavailabilityofthisinformation.Theownerwillnotbeliableforanylosses,injuries,ordamagesfromthedisplayoruseofthisinformation.

.

FINDYOURPROBLEMSSOLUTIONHERE

!

Search