18/03/2016

BigDataTools(LikeHadoopFramework)Integrati...|SCN

GettingStarted Newsletters

Hi,Guest

LogOn

JoinUs

Store

SearchtheCommunity

Products

Services&Support

AboutSCN

Downloads

Industries

Training&Education

Partnership

DeveloperCenter

Activity

LinesofBusiness

UniversityAlliances

Events&Webinars

Innovation

Browse

BigDataTools(LikeHadoopFramework)
IntegrationwithSAP

Communications

Actions

Version1

createdbyprabhithprabhakaranon28Feb201614:38,lastmodifiedbyprabhithprabhakaranon28Feb201615:06

Share

Tweet

Like

HelloAll,

Hadoop,Bigdata,SAPHANAareallsomeofthebuzzwordsintheDatamanagement/Enterprisedataware
housingspace.
SAPhasbeenworkingtoensurethatthereisagooddecentintegrationbetweenSAPAnalyticaltoolsandthe
bigdataframeworkslikeHadoop.
ForaPOC,wearetryingtoleveragevariousintegrationoptionsbetweenSAPandHadoopandthroughthis
document,Iwouldliketosharewithyoutheintegrationoptionsthatwehaveseentillnow.
Our Developments and configurations are still in very early stage. So, mostly I would be taking you through
thetheoreticalpartonly.Theactuallessonslearned,therealpainpoints,thechallenges,thelimitationsall
willbeaddedtothedocumentwhereverpossibleatlaterstages.

Wouldcertainlyliketohearfromyouaswell,incaseifyoumanagetofindmoreintegrationoptions.
I am lazy person and hence would like to loop in various links in between, which explains more on that
particulartopic.

So,letsseeindetail
1)UsingBODS(BusinessObjectsDataservices)

SAPHANAAcademyUsingDataServicestoimportdatafromaHADOOPsystem
https://www.youtube.com/watch?v=ls_MGp8R7Yk
TheaboveHANAAcademyvideoexplainstheconnectivityindetail.
ToSummarize:
InBODS,wehaveformatnamedHDFSFiles.

WejusthavetogivethenamenodehostandnamenodePortdetailsinitiallyandfurtherwehavetoprovide
theRootdirectoryandFilename.

http://scn.sap.com/docs/DOC70943

1/5

18/03/2016

BigDataTools(LikeHadoopFramework)Integrati...|SCN

NameNode>TheNameNodeisthecenterpieceofanHDFSfilesystem.Itkeepsthedirectorytreeofall
filesinthefilesystem,andtrackswhereacrosstheclusterthefiledataiskept.Itdoesnotstorethedataof
thesefilesitself.

WehavesomePigscriptingrelatedoptionsaswell.

Pig is a high level scripting language that is used withApache Hadoop. Pig enables data workers to write
complex data transformations without knowing Java. Pigs simple SQLlike scripting language is called Pig
Latin,andappealstodevelopersalreadyfamiliarwithscriptinglanguagesandSQL.

SampleUsecase:
InBODS,wecanhavetobasicallycreateaproject>CreateaJob>CreateaDataflow>DraginHDFS
fileasthesource>AddaQuerytransformation>CreateaHANAdatastoreasthetarget.

2)SAPVORA

SAPHANAVoraisanewinmemoryqueryenginewhichleveragesandextendstheApacheSparkexecution
frameworktoprovideenrichedinteractiveanalyticsonHadoop.

JohnApplebyhasgivenalotofinformationinthefollowingblog:
http://www.bluefinsolutions.com/blogs/johnappleby/september2015/thesaphanavorafaq

ThefollowingarethekeyfeaturesofVORA.
SAPHANAVoraincludesauniquesetoffeaturesandcapabilities:
AninmemoryqueryenginethatrunsonApacheSparkexecutionframework
CompiledqueriesforacceleratedprocessingacrossHadoopDistributedFileSystem(HDFS)nodes
EnhancedSparkSQLsemantictoincludehierarchiestoenableOLAPanddrilldownanalysis
Enhanced mashup application programming interface (API) for easier access to enterprise application data
formachinelearningworkloads
SupportforallHadoopdistributions
Anopendevelopmentinterface.

LotofVORAtopicscanbeseenfromtheHANAAcademyvideos:
https://www.youtube.com/playlist?list=PLkzo92owKnVyyNRkogvGloXYcDtLqeByX

Moredetailscanbefoundinthisdocument:
http://go.sap.com/docs/download/2015/08/aebfe2773d7c001082c7eda71af511fa.pdf

3)UniverseIDT>ConnectiontoHadoopJDBCDrivers

http://scn.sap.com/docs/DOC70943

2/5

18/03/2016

BigDataTools(LikeHadoopFramework)Integrati...|SCN

Followingarethreeofthewonderfuldocuments(thoughalittleold)thatexplainsthisintegrationindetail
http://docplayer.net/12952010Integratingsapbusinessobjectswithhadoopusingamultinodehadoop
cluster.html
http://hortonworks.com/wpcontent/uploads/2014/01/Creating_a_universe_on_Hadoop_Hive_Jan_2014.pdf
http://visualbi.com/blogs/sapbusinessobjects/bigdataintegratingsapbusinessobjectswithhadoop/

InthefollowingWiki,JacquelineRahn,hasquiteextensivelyexplainedtheconnectionofHadoopHivewith
IDT
http://wiki.scn.sap.com/wiki/display/BOBJ/Apache+Hadoop+Hive+Connection+With+Information+Design+Tool

Obviously,ifweareabletoreachuptouniverselevel,thenwecanfurthertakethesametothevariousBO
reportingtools/dashboards.

4)HadoopconnectivityusingSDA
SDAisanewmethodofSAPHANAforaccessingthedatastoredinremotedatasource.

*HerewecanseeanadapterwiththenameHADOOP(ODBC)

Leohasexplainedthedetailsinthefollowingblog:
http://scn.sap.com/community/developercenter/hana/blog/2014/06/02/saphanasmartdataaccess3how
toaccesshadoopdatathroughhivewithsda

DebajithasexplainedindetailontheSDAaccessusingHive/Hadoopinthefollowingdocument:
http://www.slideshare.net/DebajitBanerjee/hadoopintegrationwithsaphan

http://scn.sap.com/docs/DOC70943

3/5

18/03/2016

BigDataTools(LikeHadoopFramework)Integrati...|SCN

LotofHadoop/Hive/Spark/SDAtopicscanbeseenfromtheHANAAcademyvideos:
https://www.youtube.com/playlist?list=PLkzo92owKnVx_X9Qpjonm3FCmo41Fkzm

5)HadoopconnectivityusingLumira
Pleasefindsomeusefullinkshere:
http://scn.sap.com/community/lumira/blog/2013/11/18/analyzingbigdatawithhortonworkshadoopandsap
lumirav113

ThefollowingdocumentshowsustheconnectionusingOpenwithSQLMethod.
http://scn.sap.com/community/lumira/blog/2015/03/20/connectsaplumiratotheclouderausingapache
hadoophive

ThesedayswecanobserveadirectconnectivitytoHadoop:

This is a Collaborative document. My humble request to all of you is to add more

points/options to this document. Let us all work together and make this a very
useful repository which talks in and out of Bigdata Hadoop like framework
integrationwithourSAPTools.
Onceagainthanksalotforreadingmydocument.

Regards,
Prabhith

245Views
Products:sap_lumiraTags:hana,bigdata,universe,bods,hadoop,sda,hive,big.data,bigdata,spark,bigdataanalytics,hiveql,
vora

AverageUserRating
(3ratings)

Share

Tweet

Like

1Comment
LethikaK01Mar201602:22

LikeAlways,averyinterestingread.

http://scn.sap.com/docs/DOC70943

4/5

18/03/2016

BigDataTools(LikeHadoopFramework)Integrati...|SCN
ThisgiveusthebasicstoexploremoreontheSAPBigdataintegration.
Thanksaton...

Regards,
Lethika..
Like(0)

SiteIndex
Privacy

ContactUs
TermsofUse

http://scn.sap.com/docs/DOC70943

SAPHelpPortal
LegalDisclosure

Copyright

FollowSCN

5/5