GettingStarted Newsletters

Hi,Guest

LogOn

JoinUs

Products

Services&Support

AboutSCN

Downloads

Industries

Training&Education

Partnership

DeveloperCenter

Activity

LinesofBusiness

UniversityAlliances

Events&Webinars

Innovation

Browse

PerformanceImprovementofDataSelection
andDataTransferActivitiesinSAPTDMS
4.0

Version7

createdbyAmitSharmaonDec5,201310:43AM,lastmodifiedbyAmitSharmaonAug26,201510:58AM

Share

Tweet

Like

Store

SearchtheCommunity

Objective:
Thisdocumentisusedtospreadawarenessaboutvariousoptionsavailabletoimproveperformanceofdata
selectionanddatatransferactivityinTDMSmigrationpackages.Wecanclassifyallavailableoptionsin
followingtypes:
PerformanceimprovementsofDataTransferActivity
PerformanceimprovementsofDataSelectionActivit

1.PerformanceImprovementofDataTransferActivity:
Followingoptionsareavailabletoimproveperformanceofdatatransferactivity:
Indexdrop/recreatesolution
SystemSettings

1.1IndexDropandRecreateSolution:
Writingdataintothereceiversystemcanbesubstantiallyacceleratedifindexesaredroppedbeforethedata
istransferred,andrecreatedafterthetransferiscompleted.

Prerequisite:
1.Ifnootheruserisworkinginanyotherclientofreceiversystem.
2.Itmightevenbepossibletodroptheprimaryindex.This,however,isonlypossibleifINSERTistheonlywritebehavior
tobeused,andIfstrictlynoSELECTisdoneforthattableinparalleltothetransfer.

Implementation:
1.InTDMS,youcanusetroubleshooterDropIndexesinReceiverSystemofactivityStartDataTransfertodrop
indexesoftable.Youneedtopasstablenameandtypeofindexes(tobedropped)thatisall(alsoincludeprimary
index)oronlynonunique.
2.Oncetransferofanytableisfinished,youcanrecreatedroppedindexesusingtroubleshooterReCreatethe
DroppedIndexesinReceiverSystemofactivityStartDataTransferbyprovidingtablename.
3.OnenewactivityReCreatetheDroppedIndexesinReceiverSystemwouldalsobedynamicallyinsertedtoprocess
treeifyouhavedroppedanyindexusingabovestatedtroubleshooter.Itwilltakecareofallremainingindexeswhich
weredeletedusingTDMSbutnotyetrecreated.

Note:
IfOracleisusedasDBsystem,itmightalsobeconsideredtodefineapartitioningofaprimaryindexbytheclient
field.Ifthetransferonlyreferstoacertainclientthedataofwhicharenotreadinparalleltothetransfer,itis
helpfultodroponlytheindexpartitionreferringtothatclient,whilekeepingthosepartitionsreferringtotheother
clients.
Intheabsenceoftheprimaryandallsecondaryindexes,anydeleteormodifyaccesstothetablewouldbe
executedasfulltablescan.
IncaseofOracle,abovestatedTDMSimplementationusesthefollowingoptionsoftheDDLstatementto
recreateindexes:NOLOGGING,PARALLELandONLINE.
Incaseduplicateshaveoccurred,setparameterP_DUPL=XforactivityReCreatetheDroppedIndexesin
ReceiverSystemtodeleteduplicaterecordsbeforerecreatingindexes.Thisneedstobedonebeforeexecuting
indexrecreationactivity.
Incaseoforacledatabase,performanceofindexrecreationcanbeincreasedbyusingfollowingoptions:
multipleparalleldatabaseprocess,onlineindexrecreationandnologgingbysettingparametersP_HINTtoany
integerandP_ONLINEtoXinCNVMBTACTPARAMSforactivityTD05P_CREATE_INDEX_TSand
TD05P_CREATE_INDEXusingviewV_CNVMBTACTPAR.

1.2ReceiverSystemSettings:
Thereceiversystemisonlyaccessedbymeansofsynchronousrfccalls,whichrequiresahighnumberof
dialogworkprocesses.Recommendationsforprofileparameters:
rec/clientOFF

rdisp/btctime60
Foralllargertablesinthereceiversystem,theswitch"logdatachanges"inthetechnicalsettingsintheSAP
DDICshouldnotbeactive.Otherwise,thedatabasewillthenwritethedatarecordsinsinglerecordmode.
Ifitisnotpossible/notallowedtodeactivatethisloggingswitch,Itisthennecessarytoset,beforethe

Communications

Actions

runtimeobjectsaregenerated,parameterSUPPRESS_DBTABLOG_UPDATEinparametertable
DMC_RT_PARAMSneedstobesetto'X'inthecentralsystem.Fordetails,seenote1549208.
ThesystemshouldberuninNOARCHIVELOGMODEasperdatabase.
Themirroringoftheredologfilesshouldbeswitchedoff(thereshouldbeonlyonememberperredolog
group).

2.PerformanceImprovementofDataSelectionActivity:
Followingoptionsareavailabletoimproveperformanceofdataselectionactivity:
Parallelizationofdataselection
ReadingType7
UsesofHints
DifferentSelectionlogics
SystemSettings

2.1ParallelizationofDataSelection:
ForDMISandDMIS_CNTSP09andhigher,Pleaserefertoanotherdocument
"

ParallelizationofDataSelectioninTDMS4.0fromSP09onwards"

Bydefault,singlebatchjobrunsfordataselectionofonemigrationobject.Multiplebatchjobscanbe
scheduledforaccessplancalculation(dataselection)ofanymigrationobjectbasedonnumberofrecordsin
applicationtableincaseofflatmigrationobjectandnumberofrecordsinheadertableincaseofheader
migrationobject.

Prerequisites:
1.Resourcesofsystemswouldbeusedwithhigherdegreeatthesametimeintermsofnumberofbatch
processesinsenderandcentralsystem.
2.Incasesendersystemisproductivesystem,impactofparallelizationwouldbequitehighasweneedto
competewithotherapplicationsforsystemresource(CPUpowerandbatchprocesses).

Implementation:
Youcanperformparallelizationofanymigrationobjectin2ways:
Automaticoption
Manualoption

Automaticwaytoparallelizedataselection:
MaintainentryintableCNVMBTPRECALCOBJusingviewV_CNVMBTPRECALCformigrationobject.Youneedto
specifynameofheadertable(incaseitisstructuredmigrationobject),delimitervalue(maximumnumberofrecords
whichonebatchjobshouldhandle)andnameofmigrationobject.

FoForExample:ifnumberofrecordsfilledinTDMSheadertableCNVTDMS_05_BKPFis10million(Youcan
findthisinformationfromlogsofcorrespondingfillheaderactivity)andyouwanttoschedule5batchjobsfor
dataselectionofmigrationobjectsX_BKPF.
YouneedtomaintainentryinstatedtablewithCNVTDMS_05_BKPF,2000000andX_BKPFasnameof
headertable,delimitervalueandnameofmigrationlogic.
Incaseofflatmigrationobject,nameofheadertablewouldbeblank.
Incaseoffillheadermigrationobjects,setparameterP_SEL=Xforcorrespondingfillheaderaswell.

Note:
AbovestatedsettingsneedtobedonebeforestartingSystemAnalysisphase.
Abovestatedcustomizationisnotconsideredwithforceredefinitionofmigrationobjects.
YoucanachieveparalleldataselectionofanyfillheaderobjectwhichisbaseduponanyoffollowingTDMS
ERPfillheadersusingabovestatedsteps:

IdentificationofCompressedDataforFICODocuments(TD05X_FILL_ACCTHD)
IdentificationofAccountingDocuments(TD05X_FILL_BKPF)
IdentificationofOperatingConcernRelatedData(TD05X_FILL_CE)
IdentificationofControllingDocuments(TD05X_FILL_COTAB)
IdentificationofForeignTradeData(TD05X_FILL_EIKP)
IdentificationofPurchasingDocumentData(TD05X_FILL_EKKO)
IdentificationofObjectNumberswithStatus(TD05X_FILL_JEST)
IdentificationofMaterialLedgerDataforOrders(TD05X_FILL_VBSK)
IdentificationofRelevantSalesHistoryDocuments(TD05X_FILL_VBUK_2)

Manualwaytoparallelizedataselection:
Killalreadyrunningdataselectionjob(ifrunning)insendersystem.
Forceredefinemigrationobjectusingtroubleshooter.
ExecutereportDMC_GENERATE_ACPLAN_DELIMITERinthecentralsystem.Itgeneratesandexecutesa
reportinthesendersystemtocalculatethebasisoftheparallelization(thatmeans,itdetermineskeyfieldvalues
tobeusedasboundariesoftheselectionsappliedtothemultipleparallelrunningjobs).Notedownnameof
reportgenerated.
Gotosendersystem,opentcodesa38andpressF8.EnterPackagenumberandexecuteitinbackgroundmode
usingmenuoption(program>ExecuteinBackground):
Checkprogressofthisbatchjobinsm37ofsendersystem.Nameofbatchjobwouldbesameasthatofreport
programexecutedinpreviousstep

Oncethebatchjobinthesendersystemisfinished,reportDMC_CREATE_PRECALC_ACP_W_DELIMcanbe
startedinthecentralsystemtoretrievetheresults.
Afterthat,changenumberofmaximumjobsformigrationobjectusingtroubleshooterChangeTechnical
Settingsandstartdataselectionactivityfromprocesstree.

Note:
Technically,anyobjectcanbeconfiguredforparallelizationbutnumberoftablesinvolvedinmigrationobjectput
significantloadondatabaseserverofsendersystem.
Anynumberofbackgroundjobscanbecreatedforselectionofamigrationobjectbutitisrecommendedtouse
max20batchjobsforselectionofoneobject.

2.2ReadingType7:
ThisisnewreadingtypewhichislaunchedinnewDMIS2011(TDMS4.0).Thisoptionshouldbeusedif
thereareverymanychildtablerecordsforeachheadertablerecordinotherwordsitshouldbeusedif1:N
relationshipisthereinbetweenheaderandchildtablerecordwithNroughly>10onanaveragebase.

Prerequisite:
Thisisapplicableforstructuredmigrationobjects.
Thisisapplicableforobjectswhichhaveonetoonehierarchy(oneheaderandonechildtable).

Implementation:
UsetroubleshooterChangeReadBehaviorspresentundernode'ChangeSettingsofMigrationObjectsfor
dataselectionactivityfromwebuitochangereadingtypeto7.
ThereisonemorewaytochangethereadingtypeinviewV_CNVMBTCOBJforamigrationobjectandforce
redefinethemigrationobjectusingtroubleshooter.

Ifselectionisalreadyrunningandyouhaveobservedbadperformance,youcankillbatchjobinsender
systemandusetroubleshootertochangereadingtypeto7andforceredefineobject.
Oncethisisdone,restartdataselectionactivity.

Note:
YouneedtoforceredefineobjectifyouneedtochangereadingtypeonceGenerationofRuntimeObjects
activityisalreadyfinished.Itisrequiredtogeneratenewruntimeobjectsasperreadingtype7.
Incaseofmanyheadertablerecordsforthecurrentpackage,thismightcauseamemoryoverflowshortdump,
unlessyoualsospecifyaparallelizationwhichwouldsubdividethesetofheadertablerecordsintomanageable
subsets.

2.3DatabaseHints:
Incaseapplicationtablehaslargenumberofrecords,wecanusedatabasehintstousemultipleparallel
databaseprocesses.Incaseofreadingtype5,wearemakingfulltablescan.Wecanusemultipleparallel
databaseprocessestoperformfulltablescanonapplicationtable.

Implementation:
MaintainentryintableDMC_PERF_OPTIONSforparticularmasstransferID(MT_ID)andmigrationobject
(MIGR_OBJ_ALIAS)byprovidingfollowinginformation:

PACKAGE_SIZE(PackageSizeforFetchstatement)
Thiscustomizationisforpackagesizeusedinfetchstatement.
Wehavethefollowingdefaultvaluesforpackagesizeincurrentsolution:
IfnothingisspecifiedinthistableDMC_PERF_OPTIONS,thedefaultissetto1000records.
IncaseofFORALLENTRIESbeingactivated,wetakeonly100,tomakesurethattheFAEselectwill
notgivetoomanyrecords
Ifthislogicisconsideredasnotoptimal,customercanspecifyanyvalueinpackage_sizeforparticular
migrationobject.

FOR_ALL_ENTRIES(ForallEntrieslogic)
FOR_ALL_ENTRIESwillmakesurethatforallheadertablerecordswhichhavebeenselectedinaFETCH
statement,wedoonesingleSELECTwithFORALLENTRIEStogetallrelatedchildtablerecords.
Thiscanbehelpfultoimprovetheperformance,providedthatweknowthatwewillneverhavealargenumber
ofchildtablerecordsperofheadertablerecord.Otherwise,adataportionmightbecometoolarge.
WeareusuallymoreefficientifmorerecordsperDBoperationareconsidered,50mightbeagoodchoice.
ThisiscontrolledbytheMAX_IN_BLOCKfield.
FOR_ALL_ENTRIESandMAX_IN_BLOCKareinterrelated,specifyingavalue(possibly50thatmightbea
goodchoice)forMAX_IN_BLOCKisonlymeaningfulifFOR_ALL_ENTRIESissettoX/true.

PARALLEL_HINT(ParallelHints)
Customercanmentionparallelhinttobeusedinanyselectquery.ItisapplicableonlyfororacleandDB6.

2.4IncludeTechnique:
Wecanchangetheselectionlogicofstructuredmigrationobjectsbyreducingnumberofselectquerieson
headertable.Wecanachievethisindifferentwaysaspernumberofrecordsinheadertable.
Thistechniqueisreplacedbyreadingtype7andparallelizationofdataselection.Itshouldbeusedonlyin
limitedcases.
WehaveprovideddifferentincludedprogramswithnamingconventionofDMC_INCL_ACS*XXX*(XXX
nameofmigrationobject).

Prerequisite:
Thisisapplicabletostructuredmigrationobjectsonly.
Structuredmigrationobjectshouldbesplitinsuchawaythatitshouldhaveoneheaderandonechildtableonly.

Categories:SAPTDMSDataScrambling
4036Views
Topics:performanceTags:improvement,data,transfer,tdms,selection,tdms4

AverageUserRating
(1rating)

Share

Tweet

5 Like

7Comments
SyedHussainDec6,20137:30AM

Hii,

Nicedocumentswithwellarticulation.

Thanksforposting.

Regards
Syed
Like(0)

DongLiMar20,201410:08PM

Thanksforyourdocument.

ItisthebestarticleforTDMSperformancewhichIhaveread.Theinformationisveryhelpful.

Dong
Like(0)

FarheenHussainMar21,20147:53AM(inresponsetoDongLi)

ThankyouDong

.
Like(0)

DongLiJun27,201411:35PM

CanyouexplainmoreaboutinputingviewV_CNVMBTPRECALC?Howtofindcorresponding
migrationobjectforCNVTDMS_05_CEorCNVTDMS_05_COTAB?

Thanks
Dong
Like(0)

AmitSharmaJun29,20146:48AM(inresponsetoDongLi)

sure,hereyougo:
1.Tofindmigrationobjectnameofanytable:
CheckcontentoftablecnvmbttablesincontrolsystemofyourTDMSlandscapeviatcode
se16.Entermigrationpackagenumberinfieldpackageandnameoftable(inyourcase
CNVTDMS_05_CE)infieldtabname.pressF8.
Youwouldgetnumberofrows.Youwouldgetnameofmigrationobjectincolumn
convobject.
ForFillheaderbasedtables,youwouldseemultiplemigrationobjects.

2.ChecknumberofentriesinheadertableCNVTDMS_05_CEinsendersystemforgiven
migrationpackageinsendersystem.Incaseyouaregoingtostartnewone,checkcontent
forhistoricalpackages.
Letussupposeitis20millionandyouwanttostart4batchjobsforselectionofany
migrationobjectdependentonthistable.Youneedtomaintaindelimitervalueas5million
inv_cnvmbtprecalcforgivenheadertableandmigratrionobject.

Incaseyouwanttoenableparallelizationonlyforsomemigrationobjectsoutofallwhich
aredependentononeTDMSheadertable.Youneedtomaintainentryonlyforthose
objects.

Ihopeitwouldhelpyou.

Regards,AmitSharma
Like(0)

DongLiJul1,20142:36AM(inresponsetoAmitSharma)

Thanksforyourdetailedexplainations.Itisveryhelpfulforme.


Dong
Like(0)

AlanTibbottsOct18,20149:39PM

HiAmit,

IamhavingtroubledeterminingavalueforNUMRECtoinsertwhilemaintaining
CNVMBTPRECALC.

Yousuggestabovethatwecheckthecontentofhistoricalpackages.Wedohavesomeofthembut
theyaretrainwrecksasthecustomerhaseitherabandonedafterseveralfrustratingdays,ormore
commonlytheselectionhasabendedwithanORA01555.So,thehistoricaldatawehaveis
unreliableatbest.

DoyouhaveanotherwayofdeterminingavalueforNUMREC?

Regards,
Alan
Like(0)

SiteIndex
Privacy

ContactUs
TermsofUse

SAPHelpPortal
LegalDisclosure

Copyright

FollowSCN