SAP BW Extraction

Table of Contents
1. Extraction................................................................................................................................................. 8
1.1Introduction ......................................................................................................................................... 8
1.2 Step-by-step control flow for a successful data extraction with SAP BW: .......................................12
2. Data Extraction from SAP Source Systems.......................................................................................... 12
2.1 Introduction ...................................................................................................................................... 12
2.1.1 Process ....................................................................................................................................... 13
2.1.2 Plug-in for R/3 Systems..............................................................................................................14
2.2 Transfer Method - PSA and IDoc....................................................................................................... 14
2.2.1 Introduction ...............................................................................................................................14
2.2.2 Persistent Staging Area (PSA)..................................................................................................... 15
2.2.2.1 Definition ............................................................................................................................ 15
2.2.2.2 Use ...................................................................................................................................... 15
2.2.3 IDoc’s..........................................................................................................................................15
2.2.3.1 Definition ............................................................................................................................ 15
2.2.3.2 Example: ............................................................................................................................. 15
2.2.4 Two Methods to transfer data...................................................................................................15
2.2.4.1 Differences and advantages:...............................................................................................16
2.2.4.1.1 PSA ...............................................................................................................................16
2.2.4.2 ALE (data IDoc)................................................................................................................ 16
2.3 Data Source....................................................................................................................................... 16
2.3.1 Assigning DataSources to InfoSources and Fields to InfoObjects.............................................. 17
2.3.2 Maintaining DataSources...........................................................................................................17
2.3.3 Transferring Business Content DataSources into Active Version .............................................. 18
2.3.4 Extraction Structure ...................................................................................................................18
2.3.5 Transfer Structure ......................................................................................................................18
2.3.6 Replication of DataSources ........................................................................................................19
2.3.6.1 Replication of the Entire Metadata ....................................................................................19
 2.3.6.2 Replication of the Application Component Hierarchy of a Source System ........................ 19
2.3.6.3 Replication of the Metadata ...............................................................................................19
2.3.6.4 Replication of a DataSource of a Source System ................................................................19
2.4 Data Extraction Logistics ...................................................................................................................20
2.4.1 Data extraction Illustration ........................................................................................................20
2.4.1.1 Full Load:............................................................................................................................. 20
2.4.1.2 Delta Load: .......................................................................................................................... 22
2.5 LO Cockpit Functions......................................................................................................................... 23
2.5.1 Maintain Extract Structures ....................................................................................................... 23
2.5.2 Maintain Data Sources............................................................................................................... 23
2.5.3 Activating update.......................................................................................................................24
2.5.4 Controlling update .....................................................................................................................24
2.5.5 Setup Tables...............................................................................................................................24
2.5.6 Serialized V3...............................................................................................................................24
2.5.7 Queued Delta (the third update method).................................................................................. 25
2.5.8 Direct Delta ( 2nd delta update method in our list) .................................................................... 25
2.5.9 Unserialized V3: (The last one) ..................................................................................................25
2.6 LO Data Sources Data Flow in R/3 :................................................................................................... 25
2.6.1 Filling up the Appended Structure.............................................................................................30
2.6.2 Regenerate & Check the Customized Objects ...........................................................................34
2.7 Structure of Delta Method for LO Cockpit Data Sources..................................................................36
2.7.1 Delta Management in extraction...............................................................................................36
2.7.2 Step-by-Step Maintenance ........................................................................................................37
2.8 Delta Method ....................................................................................................................................46
2.8.1 Master Data ...............................................................................................................................46
2.8.2 TRANSACTIONAL DATA ..............................................................................................................47
2.8.3 Delta Process..............................................................................................................................48
2.9 Delta Method Properties .................................................................................................................. 49
2.9.1 Delta Initialization ......................................................................................................................49
2.9. 2 Delta Extraction.........................................................................................................................49
 2.9.3 Update Modes ........................................................................................................................... 50
2.9.3.1 V1 Update ........................................................................................................................... 50
2.9.3.2 V2 Update ........................................................................................................................... 51
2.9.3.3 V3 Update ........................................................................................................................... 51
2.10 Delta Queue Functions....................................................................................................................51
2.10.1 Direct Delta (V1 update) ..........................................................................................................52
2.10.2 Queued delta (V1 + V3 updates)..............................................................................................53
2.10.2.1 Benefits ............................................................................................................................. 53
2.10.2.2 Limits.................................................................................................................................54
2.10.3 Un-serialized V3 Update (V1/V2 + V3 Updates) ...................................................................... 54
2.11 Generic extraction...........................................................................................................................55
2.11.1Create Generic extraction [Master data]..................................................................................56
2.12 Generic Data Types .........................................................................................................................59
2.12.1 Master Data ............................................................................................................................. 59
2.12.1.1. Texts.................................................................................................................................59
2.12.1.2. Attributes .........................................................................................................................59
2.12.1.3. Hierarchies .......................................................................................................................59
2.12.2 Functions..................................................................................................................................59
2.12.2.1 Time-dependent Attributes ..............................................................................................59
2.12.2.2 Time-dependent Texts ...................................................................................................... 59
2.12.2.3 Time-dependent Texts and Attributes.............................................................................. 60
2.12.2.4 Language-dependent Texts...............................................................................................60
2.12.3 Transactional data....................................................................................................................60
2.13 Generic Data sources ......................................................................................................................61
2.13.1 Extraction Structure ................................................................................................................. 62
2.13.2 Editing the DataSource in the Source System..........................................................................62
2.13.3 Replication of DataSources...................................................................................................... 62
2.13.3.1 Replication Process Flow ..................................................................................................62
2.13.3.2 Deleting DataSources during Replication .........................................................................63
2.13.3.3 Automatic Replication during Data Request.....................................................................63
 2.14 Enhancing Business Content...........................................................................................................63
3. Extraction with Flat Files......................................................................................................................69
3.1 Data from Flat Files (7.0)...................................................................................................................69
3.2 Data from Flat Files (3.x) ...................................................................................................................70
3.3 Extracting Transaction and Master Data using Flat Files ..................................................................70
3.4 Data Types that can be extracted using Flat Files.............................................................................86
3.4.1 Basic Steps of Data Flow (ETL process): .....................................................................................87
3.4.2 Step-by-Step to upload Master Data from Flat File to InfoObjects ...........................................87
4. DB Connect...........................................................................................................................................106
4.1 Introduction ....................................................................................................................................106
4.2 Loading data from SAP Supporting DBMS into BI...........................................................................107
4.2.1 Process Description.................................................................................................................. 107
5. Universal Data Integration ................................................................................................................. 130
5.1 Introduction ....................................................................................................................................130
5.2 Process Flow....................................................................................................................................130
5.3 Creating UD source system.............................................................................................................130
5.4 Creating a DataSource for UD Connect...........................................................................................131
5.5 Using Relational UD Connect Sources (JDBC) ................................................................................. 133
5.5.1 Aggregated Reading and Quantity Restriction ........................................................................133
5.5.2 Use of Multiple Database Objects as UD Connect Source Object ...........................................133
5.6 BI JDBC Connector...........................................................................................................................133
5.6.1 Deploy the user data source’s JDBC driver to the server: .......................................................134
5.6.2 Configuring BI Java Connector ...............................................................................................134
5.6.2.1 Testing the Connections................................................................................................135
5.6.2.2 JNDI Names.................................................................................................................. 135
5.6.2.3 Cloning the Connections...............................................................................................135
5.6.3 Connector Properties................................................................................................................ 135
5.7 BI XMLA Connector .........................................................................................................................137
5.7.1 Using InfoObjects with UD Connect......................................................................................... 138
5.7.2 Using SAP Namespace for Generated Objects.........................................................................139
6. XML Integration...................................................................................................................................140
6.1 Introduction ....................................................................................................................................140
6.2 Benefits of XML Integration ............................................................................................................140
6.2.1 End-to-End Web Business Processes .......................................................................................140
6.2.2 Open Business Document Exchange over the Internet ........................................................... 141
6.2.3 XML Solutions for SAP services ................................................................................................141
6.3 Business Integration with XML ....................................................................................................... 141
6.3.1 Incorporating XML Standards ..................................................................................................142
6.3.2 SAP’s Internet Business Framework ........................................................................................142
6.3.3 SAP applications with XML....................................................................................................... 143
6.3.4 Factors leading to emergence of XML-enabled SAP solutions ................................................ 144
6.3.4.1 Changing Business Standards and their adoption ............................................................144
6.3.4.2 Internet Security Standards ..............................................................................................144
6.4 Web-based business solutions........................................................................................................144
6.4.1 Components of Business Connector........................................................................................144
6.5 How to Customize Business Connector (BC)...................................................................................145
6.5.1 Add New Users to BC ............................................................................................................... 145
6.5.2 Add SAP Systems......................................................................................................................145
6.5.3 Add Router Tables....................................................................................................................146
6.5.4 Access functionality in the Business Connector....................................................................... 146
7. Data Mart Interface ............................................................................................................................. 147
7.1 Introduction ....................................................................................................................................147
7.2 Special Features ..............................................................................................................................147
7.3 Data Mart Interface in the Myself System......................................................................................148
7.4 Data Mart Interface between Several Systems .............................................................................. 148
7.4.1 Architectures............................................................................................................................ 149
7.4.1.1 Replicating Architecture....................................................................................................149
7.4.1.2 Aggregating Architecture..................................................................................................150
7.4.2 Process Flow.............................................................................................................................151
7.4.2.1 In the Source BI................................................................................................................. 151
 7.4.2.2 In the Target BI.................................................................................................................. 151
7.4.3 Generating Export DataSources for InfoProviders...................................................................152
7.4.4 Generating Master Data Export DataSources..........................................................................152
7.4.5 Transactional Data Transfer Using the Data Mart Interface....................................................153
7.4.5.1 Delta Process.....................................................................................................................153
7.4.5.2 Restriction.........................................................................................................................153
7.4.6 Transferring Texts and Hierarchies for the Data Mart Interface ............................................. 153
7. Virtual InfoCubes.................................................................................................................................154
7.1 Introduction ....................................................................................................................................154
7.2 Create Virtual Infocube...................................................................................................................154
7.3 Different Types................................................................................................................................154
7.3.1 SAP RemoteCube .....................................................................................................................155
7.3.1.1 Creating a SAP RemoteCube.............................................................................................155
7.3.1.2 Structure ........................................................................................................................... 156
7.3.1.3 Integration ........................................................................................................................156
7.3.2 Remote Cube............................................................................................................................ 156
7.3.2.1 Structure ........................................................................................................................... 157
7.3.2.2 Integration ........................................................................................................................157
7.3.3 Virtual InfoCubes with Services ...............................................................................................158
7.3.3.1 Structure ........................................................................................................................... 158
7.3.3.2 Dependencies....................................................................................................................159
7.3.3.2.1 Description of the interfaces for user-defined function modules .............................160
7.3.3.2.2 Additional parameters for variant 2 for transferring hierarchy restrictions ............. 161
7.3.3.3 Method for determining the correct variant for the interface.........................................161
1. Extraction

1.1Introduction

Extraction programs that read data from extract structures and send it, in
the required format, to the Business Information Warehouse also belong to the data staging
mechanisms in the SAP R/3 system as well as the SAP Strategic Initiative products such as
APO, CRM and SEM. The IDOC structures or TRFC data record structures (if the user
chooses to use the PSA - Persistent Staging Area) that are generated from the transfer
structures for the Business Information Warehouse on the source system side are used for
this. These extraction tools are implemented on the source system side during
implementation and support various releases. In non-SAP applications, similar extraction
programs can be implemented with the help of third party providers. These then collect the
requested data and send it in the required transfer format using BAPIs to the SAP Business
Information Warehouse.
 The OLTP extraction tables form the basis of a DataSource on an R/3
OLTP system. The structure is written in the OLTP using the data elements that describe
the available data, usually from a table view. For an R/3 OLTP source system, the
‘DataSource Replication’ step is provided to duplicate the DataSource is replicated with its
relevant properties in BW. Once there the user can assign it to an InfoSource. The user can
request the Metadata for a DataSource, the Metadata for an application component, or all
the Metadata of a source system: To replicate the Metadata of a DataSource, choose Source
System Tree ® The user Source System à DataSource Overview à The user Application
Components à The user DataSource à Context Menu (right mouse click) à Replicate
DataSources in the BW Administrator Workbench To replicate the Metadata from a source
system into BW for an application component, choose Source System Tree ® The user
Source System ® DataSource Overview ® The user Application Components ® Context
Menu (right mouse click) à Replicate DataSources in the BW Administrator Workbench.
To update all the Metadata of a source system, choose Source System Tree à The user
Source System à Context Menu (right mouse click) à Replicate DataSources in the BW
Administrator Workbench
 All application data must be described in SAP BW using meta data. The
InfoObjects used for this are not just transaction and master data but also relationship sets
such as attributes or hierarchies for master data.

The SAP BW extractors carry out a number of functions in the SAP

OLTP, in order to guarantee smooth communication between SAP OLTP and SAP BW.
These tasks must be carried out by external software in non-SAP Systems that are set up in
the BAPI interfaces of the SAP BW. Various businesses have already been certified as
official third party providers in this field. A complete list can be found on the SAPnet BW
homepage.

Virtually any source of data can be extracted for use in the Business
Information Warehouse
  Two options for data transfer are possible

 TRFC (chosen by selecting PSA in the transfer structure)

TRFCs are faster than ALE but there is a 255 field and a 1962 byte limit
imposed. The transfer is asynchronous and runs in the background.
 ALE (IDOC) Here there is a 1000 byte limit imposed. The load process takes
place in two steps and tracing of the status of each step is provided.

 Benefits of TRFC

 Improved performance when loading data.

 Overcoming the 1000 byte limit (now a maximum of 255 fields and 1962
bytes).
 Possibility to synchronize InfoSources.

API to access the data stored in the ODS (read and update).
 1.2 Step-by-step control flow for a successful data extraction with SAP
BW:

1. An InfoPackage is scheduled for execution at a specific point of time or for a certain

system- or user-defined event.

2. Once the defined point of time is reached, the SAP BW system starts a batch job that
sends a request IDoc to the SAP source system.

3. The request IDoc arrives in the source system and is processed by the IDoc dispatcher,
which calls the BI Service API to process the request.

4. The BI Service API checks the request for technical consistency. Possible error
conditions include specification of DataSources unavailable in the source system and
changes in the DataSource setup or the extraction process that have not yet been
replicated to the SAP BW system.

5. The BI Service API calls the extractor in initialization mode to allow for extractor-
specific initializations before actually starting the extraction process. The generic
extractor, for example, opens an SQL cursor based on the specified DataSource and
selection criteria.

6. The BI Service API calls the extractor in extraction mode. One data package per call is
returned to the BI Service API, and customer exits are called for possible
enhancements. The extractor takes care of splitting the complete result set into data
packages according to the IDoc control parameters. The BI Service API continues to
call the extractor until no more data can be fetched.

7. The BI Service API finally sends a final status IDoc notifying the target system that
request processing has finished (successfully or with errors specified in the status
IDoc).

2. Data Extraction from SAP Source Systems

2.1 Introduction

Extractors are one of the data retrieval mechanisms in the SAP source
system. An extractor can fill the extract structure of a DataSource with the data from SAP
source system datasets.
 In a metadata upload, the DataSource, including its relevant properties,
is replicated in the BW. Once there the user can assign it to an InfoSource. The DataSource
fields are made available to be assigned to BW InfoObjects.

After specifying the data flow in the BW Administrator Workbench by

maintaining the transfer rules, the user can schedule an InfoPackage in the Scheduler. The
data loading process is triggered to the source system by a request IDoc.

2.1.1 Process

There are application-specific extractors, each of which are hard-coded

for the DataSource that was delivered with BW Business Content, and which fill the extract
structure of the DataSource.

In addition, there are generic extractors, with which the user can
extract more data from the SAP source system and transfer it into BW. Only when the user
calls up the generic extractor by naming the DataSource does it know which data is to be
extracted, and from which tables it should read it from and in which structure. This is how
it fills different extract structures and DataSources.

The user can run generic data extraction in the R/3 source system
application areas such as LIS, CO-PA, FI-SL and HR. This is how LIS, for example, uses
generic extraction to read info structures. DataSources are generated on the basis of these
(individually) defined info structures. We speak of customer-defined DataSources with
generic data extraction from applications.

Regardless of application, the user can generically extract master data

attributes or -texts, or transaction data from all transparent tables, database views or SAP
query functional areas or using the function module. The user can generate user-specific
DataSources here. In this case, we speak of generic DataSources.

The DataSource data for these types are read generically and
transferred into the BW. This is how generic extractors allow the extraction of data that
cannot be made available within the framework of Business Content.

The user can find further information in the implementation guide to

data extraction from SAP source systems. The user get there by choosing The user Source
System  Context Menu (right mouse click)  Customizing Extractors in the BW
Administrator Workbench – Modeling.

2.1.2 Plug-in for R/3 Systems

BW-specific source system functions, extractors and DataSources

are delivered by so-called plug-ins.Communication between the R/3 source system and the
SAP Business Information Warehouse is only possible if the appropriate plug-in is installed
in the source system.

2.2 Transfer Method - PSA and IDoc

2.2.1 Introduction

This information is taken from sap.help.com and some other sources

as well and rearranged to understand this concept much better. If time permits, we will
discuss this in the class in details.
2.2.2 Persistent Staging Area (PSA)

2.2.2.1 Definition

The Persistent Staging Area (PSA) is the initial storage area for
requested transaction data, master data attributes, and texts from various source systems
within the Business Information Warehouse.

2.2.2.2 Use

The requested data is stored in transparent, relational database tables. It

is stored in the form of the transfer structure and is not modified in any way, which means
that if the data contained errors in the source system, it may still contain errors. When the
user load flat files, the data does not remain completely unchanged, since it may be
modified by conversion routines (for example, the date format 31.12.1999 might be
converted to 19991231 in order to ensure the uniformity of the data). The user can check
the quality of the requests, their usefulness, the sequence in which they are arranged, and
how complete they are.

2.2.3 IDoc’s

2.2.3.1 Definition

The IDoc interface exchanges business data with an external system. The
IDoc interface consists of the definition of a data structure, along with processing logic for
this data structure. The business data is saved in IDoc format in the IDoc Interface and is
forwarded as IDocs. If an error occurs, exception handling is triggered using SAP tasks.
The agents who are responsible for these tasks and have the relevant authorizations are
defined in the IDoc Interface. Standard SAP format for electronic data interchange between
systems (Intermediate Document).Different message types (for example, delivery
confirmations or purchase orders) normally represent the different specific formats, known
as IDoc types. Multiple message types with related content can be assigned to one IDoc
type.

2.2.3.2 Example:

The IDoc type ORDERS01 transfers the logical message types ORDERS
(purchase order) and ORDRSP (order confirmation).Among other areas, IDocs are used in
both Electronic Data Interchange (EDI) and for data distribution in a system group (ALE).

2.2.4 Two Methods to transfer data

Basically, there are two transfer methods for SAP systems:

  With the IDoc method, IDoc interface technology is used to pack the data
into IDoc containers.
 With the PSA transfer method, IDoc containers are not used to send the
data. Instead, the data is transferred directly in the form of a transfer structure.
Information is sent from the source system (no data) through the Idoc interface (info
IDocs). This information can be, for example, the number of data records extracted or
information on the monitor.

2.2.4.1 Differences and advantages:

2.2.4.1.1 PSA

1. Data record length Max. 1962 bytes.

2. Number of fields per data record: Restricted to 255
3. Uses TRFC as transfer log.
4. Advantage: Improved performance since larger data packages can be transported. Error
handling is possible.
5. More common technology since it brings with it a better load performance and gives
the user the option of using the PSA as an inbound data store (For Master and
Transaction data).
2.2.4.2 ALE (data IDoc)

1. Data record length Max.1000 bytes.

2. Uses TRFC as transfer log
3. Uses Info-IDocs: Uses info and data IDocs.
4. Advantage: More detailed log through control record and status record for data IDoc.
5. Use with hierarchies.

The user will not be able to view the data in IDoc's while transferring the data. The most
advantageous thing about PSA is that we can see and do any editing if there is any error in
the records which means that we are able to view the data. That is not the case with IDoc’s.

2.3 Data Source

Data that logically belongs together is stored in the source system in the
form of DataSources. A DataSource consists of a quantity of fields that are offered for data
transfer into BW. The DataSource is technically based on the fields of the extraction
structure. By defining a DataSource, these fields can be enhanced as well as hidden (or
filtered) for the data transfer. It also describes the properties of the extractor belonging to it,
as regards the data transfer into BW. During a Metadata upload, the properties of the
DataSource relevant to BW are replicated in BW.

There are four types of DataSource:

 DataSources for transaction data

 DataSources for master data

These can be:

1. DataSources for attributes
2. DataSources for texts
3. DataSources for hierarchies

DataSources are used for extracting data from a source system and for transferring data into
the BW. DataSources make the source system data available on request to the BW in the
form of the (if necessary, filtered and enhanced) extraction structure. Data is transferred
from the source system into the SAP Business Information Warehouse in the Transfer
Structure. In the transfer rules maintenance, the user determines how the fields of the
transfer structure are transferred into the InfoObjects of the Communication Structure. The
user assign DataSources to InfoSources and fields to InfoObjects in the transfer rules
maintenance.

2.3.1 Assigning DataSources to InfoSources and Fields to InfoObjects

In the DataSource overview for a source system in the Administrator

Workbench – Modeling, there is also the additional option of assigning an unmapped
DataSource to an InfoSource. To do this, using the context menu (right mouse button) of a
DataSource, choose Assign InfoSource. If the user use this assignment option, the user can

1. Choose an InfoSource from a list containing InfoSources sorted according to the

agreement of their technical names
2. Create a new InfoSource

Assign several DataSources to one InfoSource, if the user wants to gather

data from different sources into a single InfoSource. This is used, for example, if data from
different IBUs that logically belongs together is grouped together in BW. The fields for a
DataSource are assigned to InfoObjects in BW. This assignment takes place in the same
way in the transfer rules maintenance.

2.3.2 Maintaining DataSources

Source system DataSources are processed in the customizing for

extractors. The user get to customizing via the context menu (right mouse button) for the
relevant source system in the source system tree of the BW Administrator Workbench -
Modeling. Or the user can go directly to the DataSource maintenance screen by choosing
Maintaining DataSource in Source System from the context menu of the source system
DataSource overview.

2.3.3 Transferring Business Content DataSources into Active Version

Business Content DataSources of a source system are only available to

the user in BW for transferring data, if the user have transferred these in their active
versions in the source system and then carried out a Metadata upload. If the user want to
transfer data from a source system into a BW using a Business Content DataSource, then
the user have to first transfer the data from the D version into the active version (A
version). With a Metadata upload, the active version of the DataSource is finally replicated
in BW.

In order to transfer and activate a DataSource delivered by SAP with

Business Content, select the user source system in the source system tree of the BW
Administrator Workbench and select Customizing ExtractorsBusiness Information
Warehouse Business Content DataSources/ Activating SAP Business Content Transfer
Business Content DataSources using the context menu (right mouse button).

2.3.4 Extraction Structure

In the extraction structure, data from a DataSource is staged in the

source system. The extraction structure contains the amount of fields that are offered by an
extractor in the source system for the data loading process.

The user can edit DataSource extraction structures in the source

system. In particular, the user can determine the DataSource fields in which the user hide
extraction structure fields from the transfer. This means filtering the extraction structure
and/or enhances the DataSource for fields, meaning completing the extraction structure. To
do this, in the BW Administrator Workbench choose Goto Modeling Source Systems
 the user Source System  Context Menu (right mouse click) Customizing Extractors
 Subsequent Processing of DataSources.

2.3.5 Transfer Structure

The transfer structure is the structure in which the data is transported

from the source system into the SAP Business Information Warehouse. The transfer
structure provides the BW with all the source system information available for a business
process. An InfoSource in BW requires at least one DataSource for data extraction. In a
SAP source system, DataSource data that logically belongs together is staged in flat
structure of the extract structure. In the source system, the user has the option of filtering
and enhancing the extract structure in order to determine the DataSource fields.
 In the transfer structure maintenance screen, the user specify the
DataSource fields that the user want to transfer into the BW. When the user activate the
transfer rules in BW, a transfer structure identical to the one in BW is created in the source
system from the DataSource fields. The data is transferred 1:1 from the transfer structure of
the source system into the BW transfer structure. From here it is transferred, using the
transfer rules, into the BW communication structure. A transfer structure always refers to a
DataSource in a source system and an InfoSource in a BW.It is a selection of DataSource
fields from a source system.

2.3.6 Replication of DataSources

2.3.6.1 Replication of the Entire Metadata

(Application Component Hierarchy and DataSources) of a Source System

 Choose Replicate DataSources in the Data Warehousing Workbench in the

source system tree through the source system context menu. or
 Choose Replicate DataSources in the Data Warehousing Workbench in the
DataSource tree through the root node context menu.

2.3.6.2 Replication of the Application Component Hierarchy of a Source System

Choose Replicate Tree Metadata in the Data Warehousing Workbench

in the DataSource tree through the root node context menu.

2.3.6.3 Replication of the Metadata

(DataSources and Possibly Application Components) of an Application Component

Choose Replicate Metadata in the Data Warehousing Workbench in

the DataSource tree through an application component context menu.

2.3.6.4 Replication of a DataSource of a Source System

 Choose Replicate Metadata in the Data Warehousing Workbench in the

DataSource tree through a DataSource context menu. or
 In the initial screen of the DataSource repository (transaction RSDS),
select the source system and the DataSource and then choose DataSource  Replicate
DataSource.

Using this function, the user can also replicate an individual DataSource that so far did not
exist in the BI system. This is not possible in the view for the DataSource tree since a
DataSource that has not been replicated so far will not be displayed.
2.4 Data Extraction Logistics

New technique to extract logistics information and consists of a series

of a standard extract structures (that is, from a more BW perspective, standard
datasources), delivered in the business content. Data Extraction is the process of loading
data from OLTP to OLAP (BW/BI).

2.4.1 Data extraction Illustration

Data can be extracted in two modes

1. Full Load – Entire data which available at source is loaded to BW/BI

2. Delta load - Only the new/changed/deleted data is loaded.
2.4.1.1 Full Load:

Document posting means creating a transaction, writing into the

application/transaction tables. So whenever sales order is created ( document posted), it
transaction is written into the database tables/ application tables/transaction tables (Ex.
EKPO, EKKO, VBAK, VBAP).Whenever the user are doing a full load, setup tables are
used.
2.4.1.2 Delta Load:

Various types of update methods are discussed below.

Serialized V3

1. Queued Delta
2. Direct Delta
3. Unserialized V3

Before that we need to understand V1, V2, V3 updates. These are

different work processes on the application server that makes the update LUW from the
running program and execute it. These are separated to optimize the transaction processing
capabilities.

These are different work processes on the application server that

makes the update LUW from the running program and execute it. These are separated to
optimize the transaction processing capabilities.
2.5 LO Cockpit Functions
2.5.1 Maintain Extract Structures

Here the user can add additional fields from the communication
structures available to the extract structure.

2.5.2 Maintain Data Sources

In the Data source maintenance screen, the user can customize the
data source by using the following fields: field name, short text, selection, hide field,
inversion or cancellation field or reverse posting, and field only known in customer exit.
2.5.3 Activating update

By Setting as active, data is written into extract structures both online

as well as during completion of setup tables or restructure table or LO initialization tables.
Depending on the update mode a job has to be scheduled with which the updated data is
transferred in the background into the central delta management (Delta Queue).

2.5.4 Controlling update

These talks about the delta update mode the user are using and how do
the user control the data load based on the volume of data. LO Cockpit supports 4 types of
update modes (delta modes, which we have already discussed): Serialized V3 update,
Direct Delta, Queued Delta, Unserialized V3 update.

2.5.5 Setup Tables

Access to application tables are not permitted, hence setup tables are
there to collect the required data from the application tables. When a load fails, the user can
re-run the load to pull the data from setup tables. Data will be there in setup tables. Setup
tables are used to Initialize delta loads and for full load. Its part of LO Extraction scenario.
With this option, the user avoid pulling from R/3 directly as we need to bring field values
from multiple tables. The user can see the data in the setup tables. Setup table name wiil be
extract structure name followed by SETUP. Set up table names starts with 'MC' followed
by application component '01'/'02' etc and then last digits of the datasource name and then
followed by SETUP Also we can say the communication structure (R/3 side, the user can
check it in LBWE also) name followed by 'setup'.

The setup tables are the base tables for the Datasource used for Full
upload. So if the user are going for only full upload full update is possible in LO extractors.
Full update is possible in LO extractors. In the full update whatever data is present in the
setup tables(from the last done in it) is sent to BW.But setup tables do not receive the delta
data from the deltas done after the init.So if users full update should get ALL data from the
source system will need to delete and re-fill setup tables.

2.5.6 Serialized V3

Take an example of the same PO item changing many times in quick

succession. V1 (with enqueue mechanism) ensures that the OLTP tables are updated
consistently. Update table gets these update records which may or may not end up in
correct sequence (as there is no locking) when it reaches BW. 'Serialized V3' was to ensure
this correct sequence of update records going from update tables to delta queue (and then to
BW).Since update table records have the timestamp, when the V3 job runs, it can sequence
these records correctly and thus achieve 'serialization'.
2.5.7 Queued Delta (the third update method)

With queued delta update mode, the extraction data (for the relevant
application) is written in an extraction queue (instead of in the update data as in V3) and
can be transferred to the BW delta queues by an update collective run, as previously
executed during the V3 update. After activating this method, up to 10000 document
delta/changes to one LUW are cumulated per datasource in the BW delta queues.
If the user use this method, it will be necessary to schedule a job to regularly transfer the
data to the delta queues As always, the simplest way to perform scheduling is via the "Job
control" function in LBWE.SAP recommends to schedule this job hourly during normal
operation after successful delta initialization, but there is no fixed rule: it depends from
peculiarity of every specific situation (business volume, reporting needs and so on).

2.5.8 Direct Delta ( 2nd delta update method in our list)

With this update mode, Each document posting is directly transferred

into the BW delta queue Each document posting with delta extraction leads to exactly one
LUW in the respective BW delta queues Just to remember that ‘LUW’ stands for Logical
Unit of Work and it can be considered as an inseparable sequence of database operations
that ends with a database commit (or a roll-back if an error occurs).

2.5.9 Unserialized V3: (The last one)

With this update mode, that we can consider as the serializer’s brother,
the extraction data continues to be written to the update tables using a V3 update module
and then is read and processed by a collective update run (through LBWE).But, as the
name of this method suggests, the V3 unserialized delta disowns the main characteristic
of his brother: data is read in the update collective run without taking the sequence into
account and then transferred to the BW delta queues.Issues:Only suitable for data target
design for which correct sequence of changes is not important e.g. Material Movements V2
update has to be successful.

2.6 LO Data Sources Data Flow in R/3 :

DataSources reside in the source system so If I need to customize

DataSource, I need to go to source system, one can directly login to Source System ( R/3 in
our case) or From BW also can remotely login to Source System.

1. Logon to BW
 a. Go to Administrator Workbench (RSA1).
b. Go to Source systems.
c. Choose R/3 Source System (where the user DataSource resides) - Right Click &
go to “Customizing for Extractors”.

2. It shall take the user to Source System ( R/3)

3. Choose “Edit DataSources and Application Component Hierarchy”

4. Go to “0CUSTOMER_ATTR” Customer Number DataSource ( or whatever

DataSource the user decide to customize)

5. Click on DataSource
6. Scroll down to reach the appended structure
7. Double Click on appended structure – ZBIW_KNA1_S1 (or the user chosen one) &
add the fields ( which the user wish to add to DataSource)

8. Check, save & activate - It will append this to Extract Structure of DataSource.
9. Go Back
10. Click on 0CUSTOMER_ATTR (DataSource)

11. Now Click on Extract Structure – BIW_KNA1_S (in my case), check out the
appended field below append structure.
2.6.1 Filling up the Appended Structure

1. Post appending the structure, we need to fill this Append with data – For this

a. LOGON to Source System (R/3)

b. Go to T Code – CMOD
c. Choose the project – Which the user is using for Master Data Enhancement.
Project “ZSDBWNEW” in my case. ( the user need to create project in case the
user are doing enhancement on this BW system for the first time)
d. Click on Enhancement Assignments
e. Click on Change


2. Press Continue – In case it prompts for some confirmation like “Enhancement

project already active” etc

3. Choose the following -

a. Enhancement – RSAP0001
b. Click on Components.

4. It will take the user to –

5. Choose”EXIT_SAPLRSAP_002” & Click on this.

6. Choose”EXIT_SAPLRSAP_002” & Click on this.

7. We need to write ABAP Code to fill up Appended Fields in “0Customer_Attr”
through the Include mentioned here i.e. include zxrsau02.
8. Double Clicking on this will take to ABAP Editor to put in the code.
9. Go to Change Mode.
10. This Code is to be inserted before “endcase” of already written code.
11. Make Sure Tables KNVV, BIW_KNVV_S, KNKK, KNA1, KNVP (these are table
from which I need to extract values of appended fields) are declared at the start of
the code.
12. Do the “Check” before Save, if no errors are found the Save & Activate”.
13. Go Back.
 14. Do the Check again & if no errors then Activate.
15. Go Back and then Activate. (Keep activating while going back on every step).

16. Come out.

2.6.2 Regenerate & Check the Customized Objects

1. Now Final Step – We need to regenerate the DataSource & make sure Newly Added
Attributes (Starting with ZZ) are not HIDE.
2. Logon to BW System – Go to Administrator Workbench (RSA1) – Go to Source
systems – Choose Source System (R/3 in our case) - Right Click & go to
“Customizing for Extractors”.
3. Choose “Edit DataSources and Application Component Hierarchy”
4. Go to “0CUSTOMER_ATTR” Customer Number DataSource & Click on CHANGE.
Scroll Down.
5. Keep clicking on Right Sign if it prompts for any confirmation.
6. Scroll down & Go to Fields starting with ZZ & make sure these are not HIDE
(remove the sign from HIDE check box).

7. Click on DataSource & Generate, as shown below

8. Go back. Next Steps are Checking Extractors & Loading data

2.7 Structure of Delta Method for LO Cockpit Data Sources

The delta InfoPackage is executed which extracts data from the delta
queue to the SAP BI system and the same is scheduled as a part of the process chain. The
data is extracted to the persistent staging area, which forms the first physical layer in BI
from where data is further staged into the DSO, which can be a part of a pass through layer
or an EDW layer. Note the negative values for the key figures for the before image record
in the PSA table. The same can be updated to a DSO, in overwrite and summation mode,
and an infocube.

When the data from the LO datasource is updated to a DSO with the setting „unique
record the before image is ignored.

Generic extractors of type (extraction method) F2 and delta process are AIE (After image
Via Extractor) will be using pull delta model 'F2': The data is extracted by means of a
function module that, in contrast to 'F1', occupies a simplified interface (see documentation
for data element ROFNAME_S).

Whenever we request delta data from BW, the data will pulled via delta queue and Delta
LUW’s will be saved in repeat delta table and repeat delta LUW’s only will be visible in
RSA7, But for the normal F1 type extractors both Delta and Repeat delta LUW’s will be
visible in RSA7

In the below screen shot Total =2 will refer number LUW’s in repeat delta table

2.7.1 Delta Management in extraction

The LO cockpit configuration screen (LBWE) contains following key parameters:

  Maintenance of extract structures
 Maintaining InfoSources
 Activating the update
 Job control
 Update Mode

a. Serialized V3 Update The document data is collected in the order it was

created and transferred into the BW as a batch job.
b. Direct Delta The extraction data is transferred directly from document
postings into the BW delta queue. The transfer sequence is the same as the
order in which the data was created.
c. Queued Delta The extraction data from document postings is collected in
an extraction queue, from which a periodic collective run is used to
transfer the data into the BW delta queue. The transfer sequence is the
same as the order in which the data was created.
d. Unserialized V3 Update This method is opposite to the Serialized V3
Update. The BW delta queue does not have to be the same as the order in
which it was posted.
2.7.2 Step-by-Step Maintenance
We need to first log into the R/3 System and call Transaction SBIW
(Display IMG).Here navigate as per the screenshot below and Delete the data from the
setup tables. There are two setup tables in R/3, which exist and get filled with the logistic
data before it’s extracted into BW.
Click on the clock symbol to delete the existing contents from the setup table if any so
that its free and we can go ahead and make or configure our Logistic Datasources. Select
the Application component and execute for deletion of data. Click YES.

Select the Application component and execute for deletion of data.

Click YES on the popup.

An information message will be displayed in the message box.

Call Transaction LBWE and navigate as per the below screenshot:

Click on the maintain Extract structure to maintain the fields

An information message will be displayed

In the above screenshot: Click on the Update Overview text to reach the following
screen. This will take the user to SM13 for any relative table updates and Execute.
Now go back to previous screen and click on BW Maintenance Delta Queue.

This will take the user to RSA7 transaction to view the delta queues if any
Click back to go reach this pop-up.

Click on Run and It will Prompt for confirming the entries in the Extract structure.
Assign a request so that it generates extract structure successfully.

Now on the Main LBWE screen, the user can see RED status before the datasource.
Now click on the Datasource as below.

Assign a Request to have the Datasource screen where properties related to fields can be
modified.
 As the user assign this and come back the user will see a change in the status color as
YELLOW.

Now go to the BW System and Replicate the related Datasource from the Exact Source
system.
Now go back to the R/3 System and Click on the ACTIVE parameter under the Job
Control, assign a request

Now the user will see that the status color will turn as GREEN and then the user can
assign the update mode as well.
Now in the BW system create transformations from the datasource to the Infoprovider.
Create an infopackage and DTP to load the data.

2.8 Delta Method

With this process the new information that is generated daily in the source system must be
sent to BW. The principal reason for this is the information volume that is imported to BW
every day.
This information is stored en the delta queue (RSA7 Transaction). It is
taken from here when BW asks for information to the source system. In the Delta Queue
will only be the information that has not been sent to BW and the last request.
The Data sources that should be used are:
2.8.1 Master Data
0ARTICLE_ATTR
0ARTICLE_TEXT
0CUSTOMER_ATTR
0CUSTOMER_TEXT
0PLANT_ATTR
0PLANT_TEXT
0MATL_GROUP_TEXT
0MAT_VEND_ATTR
 0VENDOR_ATTR
0VENDOR_TEXT
0SALESORG_ATTR
0SALESORG_TEXT
0SALES_DIST_TEXT
0SALES_GRP_TEXT
0SALES_OFF_TEXT
0SALESORG_ATTR
0SALESORG_TEXT
0SALES_DIST_TEXT
0SALES_GRP_TEXT
0SALES_OFF_TEXT

2.8.2 TRANSACTIONAL DATA

2LIS_02_SCL
2LIS_03_BF
2LIS_03_BX
2LIS_03_UM
2LIS_13_VDITM
2LIS_40_REVAL
In the screen Installation of Data Source from Business Content, the user must mark the
data source that the user wants to activate and select Activate Data Source.
 The delta process is a feature of the extractor and specifies how data is to be transferred. As
a DataSource attribute, it specifies how the DataSource data is passed on to the data target.
From this the user can derive, for example, for which data a DataSource is suited, and how
the update and serialization are to be carried out.
2.8.3 Delta Process
 Forming deltas with after, before and reverse images that are updated directly in the delta
queue; an after image shows the status after the change, a before image the status before
the change with a negative sign and the reverse image also shows the negative sign next
to the record while indicating it for deletion. This serializes the delta packets. The delta
process controls whether adding or overwriting is permitted. In this case, adding and
overwriting are permitted. This process supports an update in an ODS object as well as in
an InfoCube. (technical name of the delta process in the system: ABR)
 The extractor delivers additive deltas that are serialized by request. This serialization is
necessary since the extractor within a request delivers each key once, and otherwise
changes in the non-key fields are not copied over correctly. It only supports the addition
of fields. It supports an update in an ODS object as well as in an InfoCube. This delta
process is used by LIS DataSources. (technical name of the delta process in the system:
ADD)
 Forming deltas with after image, which are updated directly in the delta queue. This
serializes data by packet since the same key can be copied more than once within a
request. It does not support the direct update of data in an InfoCube. An ODS object must
always be in operation when the user update data in an InfoCube. For numeric key
figures, for example, this process only supports overwriting and not adding, otherwise
 incorrect results would come about. It is used in FI-AP/AR for transferring line items,
while the variation of the process, where extractor can also send records with the deletion
flag, is used in capacity in BBP. (technical name of the delta process in the system:
AIM/AIMD)

2.9 Delta Method Properties

2.9.1 Delta Initialization

In contrast with other business content and generic data sources, the
LO datasources use the concept of set up tables to carry out the initial data extraction
process. The data extractors for HR, FI etc. extract data by directly accessing the
application tables, but in case of LO extractors they do not access the application tables
directly. The presence of restructuring/set up tables prevents the BI extractors directly
access the frequently updated large logistics application tables and are only used for
initialization of data to BI. For loading data first time into the BI system, the set up tables
have to be filled. The restructuring/set up tables are cluster tables that hold the respective
application data, and the BI system extracts the data as a onetime activity for the initial data
load, and the data can be deleted from the set up tables after successful data extraction into
BI to avoid redundant storage.
The setup tables in SAP have the naming convention, <Extraction
structure>SETUP and the compressed data from application tables stored here can be
viewed through SE11. Thus the datasource 2LIS_11_VAITM having extract structure
MC11VA0ITM has the set up table MC11VA0ITMSETUP. A job is executed to fill the set
up tables, and the init InfoPackage extracts the initial data into BI.

2.9. 2 Delta Extraction

Once the initialization of the logistics transaction data datasource is
successfully carried out, all subsequent new and changed records are extracted to the BI
system using the delta mechanism supported by the datasource. The LO datasources
support ABR delta mechanism which is both DSO and InfoCube compatible. The ABR
delta creates delta with after, before and reverse images that are updated directly to the
delta queue, which gets automatically generated after successful delta initialization.

The after image provides status after change, a before image gives
status before the change with a minus sign and a reverse image sends the record with a
minus sign for the deleted records. The serialization plays an important role if the delta
records has to be updated into a DSO in overwrite mode. For e.g. in the sales document
1000, if the quantity of ordered material is changed to 14 from 10, then the data gets
extracted as shown in the table,
The type of delta provided by the LO datasources is a push delta, i.e. the delta data records
from the respective application are pushed to the delta queue before they are extracted to BI
as part of the delta update. The fact whether a delta is generated for a document change is
determined by the LO application. It is a very important aspect for the logistic datasources
as the very program that updates the application tables for a transaction triggers/pushes the
data for information systems, by means of an update type, which can be a V1 or a V2
update.
2.9.3 Update Modes
Before elaborating on the delta methods available for LO datasources it is necessary to
understand the various update modes available for the logistics applications within the SAP
ECC 6.0 system.
The following three update methods are available;
a) V1 Update
b) V2 Update
c) V3 Update
While carrying out a transaction, for e.g. the creation of a sales order, the user enters data
and saves the transaction. The data entered by the user from a logistics application
perspective is directly used for creating the orders, having an integrated controlling aspect,
and also indirectly forms a part of the information for management information reporting.
The data entered by the user is used by the logistic application for achieving both the above
aspects, but the former, i.e. the creation of the order takes a higher priority than result
calculations triggered by the entry. The latter is often termed as statistical updates.
The SAP system treats both these events generated by the creation of order with different
priorities by using two different update modes for achieving the same, the V1 update and
the V2 update, with the former being a time critical activity. Apart from these two update
modes SAP also supports a collective run, called the V3 update, which carries out updates
in the background. The update modes are separately discussed below.
2.9.3.1 V1 Update
A V1 update is carried out for critical or primary changes and these
affect objects that has a controlling function in the SAP System, for example the creation of
an sales order (VA01) in the system. These updates are time critical and are synchronous
updates. With V1 updates, the program that outputs the statement COMMIT WORK AND
WAIT which waits until the update work process outputs the status of the update. The
program then responds to errors separately.

The V1 updates are processed sequentially in a single update work process and they
belong to the same database LUW. These updates are executed under the SAP locks of the
transaction that creates the update there by ensuring consistency of data, preventing
simultaneous updates. The most important aspect is that the V1 synchronous updates can
never be processed a second time. During the creation of an order the V1 update writes data
into the application tables and the order gets processed. The V1 updates are carried out as a
priority in contrast to V2 updates, though the V2 updates are usually also processed straight
away.
2.9.3.2 V2 Update
A V2 update, in contrast with V1 is executed for less critical
secondary changes and are pure statistical updates resulting from the transaction. They are
carried out in a separate LUW and not under the locks of the transaction that creates them.
They are often executed in the work process specified for V2 updates. If this is not the case,
the V2 components are processed by a V1 update process but the V1 updates must be
processed before the V2 update. They are asynchronous in nature.
2.9.3.3 V3 Update
Apart from the above mentioned V1 and V2 updates, the SAP
system also has another update method called the V3 update which consists of collective
run function modules. Compared to the V1 and V2 updates, the V3 update is a batch
asynchronous update, which is carried out when a report (RSM13005) starts the update (in
background mode). The V3 update does not happen automatically unlike the V1 and V2
updates.
All function module calls are then collected, aggregated and updated together and are
handled in the same way as V2 update modules. If one of the function modules increments
a statistical entry by one, this is called up 10 times during the course of the transaction.
Implementing the same as a V2 update runs 10 times after the V1 for the same has been
completed; i.e. the database is updated 10 times. But when executed as a V3 update, the
update can be executed at any time in one single operation with the same being carried out
in one database operation at a later point in time. This largely reduces the load on the
system.

2.10 Delta Queue Functions

The LO datasource implements its delta functionality using the above
update methods either individually or as a combination of them. SAP provides different
mechanisms for pushing the data into the delta queue and is called update modes.
The different update modes available with LO datasources are;
a. Direct Delta
 b. Queued Delta
c. Un-serialized V3 Delta
The following section discusses in detail the concept behind the three different delta update
modes, the scenarios in which they are used, their advantages and disadvantages. The
figure below shows the update modes for delta mechanism from the LO Cockpit.

2.10.1 Direct Delta (V1 update)

A direct delta updates the changed document data directly as an LUW to

the respective delta queues. A logistics transaction posting leads to an entry in the
application tables and the delta records are posted directly to the delta queue using the V1
update. The data available in the delta queue is then extracted periodically to the BI system.

Advantage of direct delta

a. Writing to the delta queue within the V1 posting process ensures serialization
by document.
b. Recommended for customers with fewer documents.
c. Extraction is independent of V2 updating.
d. No additional monitoring of update data or extraction queue required.

Disadvantage of direct delta

a. Not suitable for scenarios with high number of document changes.
b. Setup and delta initialization required before document postings are resumed.
C.V1 is more heavily burdened.

When using this update mode, no document postings should be carried out during delta
initialization in the concerned logistics application from the start of the recompilation run
in the OLTP until all delta init requests have been successfully updated successfully in BW.
The data from documents posted is completely lost if documents are posted during the re-
initialization process.
2.10.2 Queued delta (V1 + V3 updates)

In the queued delta update mode the logistic application pushes the
data from the concerned transaction into an extraction queue by means of the V1 update.
The data is collected in the extraction queue and a scheduled background job transfers the
data in the extraction queue to the delta queue, in a similar manner to the V3 update, with
an update collection run. Depending on the concerned application, up to 10,000 delta
extractions of documents can be aggregated in an LUW in the delta queue for a datasource.

The data pushed by the logistic application can be viewed in the logistics queue overview
function in the SAP ECC 6.0 system (transaction LBWQ). SAP recommends the queued
delta process for customers with a high amount of documents with the collection job for
extraction from extraction queue to be scheduled on an hourly basis.

2.10.2.1 Benefits

 When the user need to perform a delta initialization in the OLTP, thanks to the logic
of this method, the document postings (relevant for the involved application) can be
opened again as soon as the execution of the recompilation run (or runs, if several
and running in parallel) ends, that is when setup tables are filled, and a delta init
request is posted in BW, because the system is able to collect new document data
 during the delta init uploading too (with a deeply felt recommendation: remember
to avoid update collective run before all delta init requests have been successfully
updated in the user BW!).
 By writing in the extraction queue within the V1 update process (that is more
burdened than by using V3), the serialization is ensured by using the enqueue
concept, but collective run clearly performs better than the serialized V3 and
especially slowing-down due to documents posted in multiple languages does not
apply in this method.
 On the contrary of direct delta, this process is especially recommended for
customers with a high occurrence of documents (more than 10,000 document
changes - creation, change or deletion - performed each day for the application in
question.
 Extraction is independent of V2 update.
 In contrast to the V3 collective run an event handling is possible here, because a
definite end for the collective run is identifiable: in fact, when the collective run for
an application ends, an event (&MCEX_nn, where nn is the number of the
application) is automatically triggered and, thus, it can be used to start a subsequent
job.
2.10.2.2 Limits

 V1 is more heavily burdened compared to V3.

 Administrative overhead of extraction queue.

The job uses the report RMBWV311 for collection run and the function module will have
the naming convention MCEX_UPDATE_<Application>, MCEX_UPDATE_11 for sales
orders. In the initialization process, the collection of new document data during the delta
initialization request can reduce the downtime on the restructuring run. The entire
extraction process is independent of the V2 update process.
2.10.3 Un-serialized V3 Update (V1/V2 + V3 Updates)
 In this mode of delta update the concerned logistic application writes
data to update tables which further transfers data to the delta queue by means of a
collection run call V3 update. Once the data is updated to the update tables by the logistic
applications, it is retained there until the data is read and processed by a collective update
run, a scheduled background job, the V3 update job, which updates all the entries in the
update tables to the delta queue.

As the name suggests the update is un-serialized, i.e. this mode of update does not ensure
serialization of documents posted to the delta queue. This means that the entries in the delta
queue need not correspond to the actual sequence of updates that might have happened in
the logistic application. This is important if the data from the datasource is further updated
to a DSO in overwrite mode as the last entry would overwrite the previous entries resulting
in erroneous data. An un-serialized delta update when used should always update data
either to an infocube or to a DSO with key figures in summation mode. It is also advised if
the un-serialized V3 update can be avoided to documents subjected to a large number of
changes when it is necessary to track changes.

2.11 Generic extraction

Generic R/3 data extraction allows us to extract virtually any R/3 data.
Generic data extraction is a function in Business Content that supports the creation of
DataSources based on database views or InfoSet queries. InfoSet is similar to a view but
allows outer joins between tables. The new generic delta service supports delta extractors
on monotonic ‘delta attributes‘like Timestamp, Calendar day, Numeric pointer (e.g.
document number, counter) – must be strictly monotonic increasing with time. Only one
attribute can be defined as the delta attribute. For extracting data from the VBAK table, the
Logistics Extraction Cockpit is the recommended method.
2.11.1Create Generic extraction [Master data]

1. Under Transaction SBIW – This step gives the user the option of creating and
maintaining generic Data Sources for transaction data, master data attributes or
texts from any kind of transparent tables, database views or SAP query functional
areas or via a function module, regardless of application. This enables the user to
use the generic extraction of data.

2. Create a Generic Data Source

a) Select the Data Source type and assign a technical name to it.
b) Choose Create
The screen for creating a generic Data Source appears

3.
a) Choose an application .Component to which the data source is to be assigned.
b) Enter the descriptive texts. The user can choose these freely.
c) Choose Generic Delta
4. Specify the delta-specific field and the type for this field. Maintain the
settings for the generic delta: Specify a safety interval. Safety interval should be set
so that no document is missed – even if it was not stored in the DB table when the
extraction took place.

5. Select Delta type: New status for changed records (I.e. after-image); This can be
used with Data target ODS (AIE).Additive Delta (I.e. aggregated data records)
(ADD) Then choose Save.

6. After step 4, the screen of step 3 comes back. Now choose Save again.
 This will generate the data source. After generating the data source, the user will
see the Delta Update flag selected. In systems as of basis release 4.0B,the user can
display the current value for the delta-relevant field in the delta queue.

7. Choose Save again

Delta Attributes can be monitored in delta queue (RSA7). Also note LUW count
does not equal to the changes records in the source table. Most of the time it will be
ZERO. Delta is enabled by data selection logic

LUW count can also have value 1.Whenever delta is extracted, the extracted data is
stored in the delta queue tables to serve as a fallback, when an error occurs during
 the update of the BW system. The user will see a '1' in this field (the extract counts
as one LUW) and are even able to be displayed in a detail screen.

2.12 Generic Data Types

2.12.1 Master Data

In SAP BW, three different types of master data can be differentiated

in InfoObjects.
2.12.1.1. Texts
Texts are used to describe a master record. In SAP Business
Information Warehouse (SAP BW), up to three texts can be maintained for each master
record. These texts can consist of the following: one short text, one medium text, and one
long text. An example of a master data text is the name of the supplier that goes with the
supplier number.
2.12.1.2. Attributes
Master data attributes are fields that are used to provide a more
detailed description of master data elements. These attributes are used to display additional
information so results can be better understood. An attribute table can be used by several
InfoCubes. This ensures a higher level of transparency for the user and a more
comprehensive consistency. An example of a master data attribute is the country of the
supplier that goes with the supplier number.

2.12.1.3. Hierarchies
Hierarchies can be used in the analysis to describe alternative views of
the data. A hierarchy consists of a quantity of nodes that have a parent child relationship
with one another. The structures can be defined in a version-specific as well as a time-
dependent manner. An example of this is the cost center hierarchy.
2.12.2 Functions

2.12.2.1 Time-dependent Attributes

If the characteristic has at least 1 time-dependent attribute, a time

interval is specified for this attribute. As master data must exist between the period of
01.01.1000 and 12.31.1000 in the database, the gaps are filled automatically.
2.12.2.2 Time-dependent Texts
 If the user creates time-dependent texts, the text for the key date is
always displayed in the query.
2.12.2.3 Time-dependent Texts and Attributes

If the texts and the attributes are time-dependent, the time intervals do
not have to agree.
2.12.2.4 Language-dependent Texts

In the Characteristic InfoObject Maintenance, the user can determine

whether the texts are language-specific (for example, with product names: German Auto,
English car), or not (for example, customer names). Only the texts in the selected
language are displayed.

If they are language-dependent, the user have to upload all texts with a language indicator.

2.12.3 Transactional data

Transaction data includes origination data, monthly account data,

competitive data, and macroeconomic data. It is regularly updated and loaded into the
system. When loading transactional data, the system utilizes elements of the master data to
process the transactions.
Transaction data - Any business event in SAP R/3 leads to transaction
data being generated. SAP R/3 handles transaction data at the document level or at the
summary level. Document level transaction data is detailed data and consists of headers, line
items, and schedule lines. Summary transactional data is used in SAP R/3 for mostly
reporting purposes.

Generic extractors are of 3 types:

1. Based on table/view
2. Based on Infoset Query
3. Based on Function module
2.13 Generic Data sources

Defines as extract structure and transfer structure from source system

(like SAP R/3) and transfer structure from BW side. ES -> What data to be extracted from
source system TS -> What data to be extracted from source system to BW. TS is a group of
fields which indicated how the data is coming from the source system BW Data that
logically belongs together is stored in the source system in the form of DataSources.A
DataSource consists of a quantity of fields that are offered for data transfer into BI. The
DataSource is technically based on the fields of the structure. By defining a dataSource,
these fields can be enhanced as well as hidden (or filtered) for the data transfer.
Additionally, the DataSource describes the properties of the associated extractor with
regard to data transfer to BI. Upon replication, the BI-relevant properties of the DataSource
are made known in BIDataSources are used for extracting data from an SAP source system
and for transferring data into BI. DataSources make the source system data available to BI
on request in the form of the (if necessary, filtered and enhanced) extraction structure.
In the DataSource maintenance in BI, the user determines which fields from the
DataSource are actually transferred. Data is transferred in the input layer of BI, the
Persistent Staging Area (PSA). In the transformation, the user determines what the
assignment of fields from the DataSource to InfoObjects from BI should look like. Data
transfer processes facilitate the further distribution of the data from the PSA to other
targets. The rules that the user set in the transformation apply here.
2.13.1 Extraction Structure

In the extraction structure, data from a DataSource is staged in the

source system. It contains the amount of fields that are offered by an extractor in the source
system for the data loading process. The user can edit DataSource extraction structures in
the source system. In particular, the user can determine the DataSource fields in which the
user hide extraction structure fields from the transfer. This means filtering the extraction
structure and/or enhancing the DataSource for fields, meaning completing the extraction
structure. In transaction SBIW in the source system, choose Business Information
Warehouse Subsequent Processing of DataSources.
2.13.2 Editing the DataSource in the Source System

The user can edit DataSources in the source system, using transaction
SBIW.
2.13.3 Replication of DataSources

In the SAP source system, the DataSource is the BI-relevant

metaobject that makes source data available in a flat structure for data transfer into BI. In
the source system, a DataSource can have the SAP delivery version (D version: Object type
R3TR OSOD) or the active version (A version: Object type R3TR OSOA).
The metadata from the SAP source systems is not dependent on the BI metadata. There is
no implicit assignment of objects with the same names. In the source system, information is
only retained if it is required for data extraction. Replication allows the user to make the
relevant metadata known in BI so that data can be read more quickly. The assignment of
source system objects to BI objects takes place exclusively and centrally in BI.
2.13.3.1 Replication Process Flow

In the first step, the D versions are replicated. Here, only the
DataSource header tables of BI Content DataSources are saved in BI as the D version.
Replicating the header tables is a prerequisite for collecting and activating BI Content
In the second step, the A versions are replicated. DataSources (R3TR RSDS) are saved in
the M version in BI with all relevant metadata. In this way, the user avoid generating too
many DDIC objects unnecessarily as long as the DataSource is not yet being used – that is,
 as long as a transformation does not yet exist for the DataSource.3.x DataSources (R3TR
ISFS) are saved in BI in the A version with all the relevant metadata.

2.13.3.2 Deleting DataSources during Replication

DataSources are only deleted during replication if the user performs

replication for an entire source system or for a particular DataSource. When the user
replicates DataSources for a particular application component, the system does not delete
any DataSources because they may have been assigned to another application component
in the meantime.
If, during replication, the system determines that the D version of a DataSource in the
source system or the associated BI Content (shadow objects of DataSource R3TR SHDS or
shadow objects of mapping R3TR SHMP) is not or no longer available in BI, the system
automatically deletes the D version in BI.
2.13.3.3 Automatic Replication during Data Request

The user can use a setting in the InfoPackage maintenance under

Extras  Synchronize Metadata to define that, whenever there is a data request, automatic
synchronization of the metadata in BI with the metadata in the source system takes place.
If this indicator is set, the DataSource is automatically replicated from the BI upon each
data request – that is, if the DataSource has changed in the source system.

2.14 Enhancing Business Content

SAP provides business contents, which are used in BW for extracting,

reporting and analysis. Steps involved in using the business content and generating reports.
1.Log on to the SAP R/3 server.
2.Install the Business content. The Transaction code (Tcode) is RSA5.
 3.Checking if the Business content is ready for use. Tcode RSA6.
4. Enhancing the structure. Here I am using BOM (Bill of Materials) from the table
VBAP from Sales and Distribution (SD).

5.Activate the enhanced structure.

6.Enhance the field BOM in to the structure, Hence it should be mapped to 0_FI_GL_4
from VBAP when the condition is satisfied. User exit is written for this purpose.

7.Include the following code in the user exit.

TABLES: AUFK.
TABLES: VBAP.
DATA: Temp_sorder like DTFIGL_4.
DATA: Temp_ZZSEQUENCNUMBER like DTFIGL_4-ZZSEQUENCENUMBER.
DATA: Temp_KDAUF like DTFIGL_4-KDAUF.
DATA: Temp_ZZORDERCAT like DTFIGL_4-ZZORDERCAT.
DATA: Temp_ZZBOM like DTFIGL_4-ZZBOM.
Case i_datasource.
When '0FI_GL_4'.
Loop at C_t_data into Temp_sorder.
Select single SEQNR from AUFK into Temp_ZZSEQUENCENUMBER
Where AUFNR = Temp_sorder-AUFNR and BUKRS = Temp_sorder-BUKRS.
Select single KDAUF from AUFK into Temp_KDAUF where AUFNR =
Temp_sorder-AUFNR and BUKRS = Temp_sorder-BUKRS.
Select single AUTYP from AUFK into Temp_ZZORDERCAT where AUFNR =
Temp_sorder-AUFNR and BUKRS = Temp_sorder-BUKRS.
Temp_sorder-ZZSEQUENCENUMBER = Temp_ZZSEQUENCNUMBER.
Temp_sorder-KDAUF = Temp_KDAUF.
Temp_sorder-ZZORDERCAT = Temp_ZZORDERCAT.
Select single STLNR from VBAP into Temp_ZZBOM where GSBER =
Temp_sorder-GSBER and AUFNR = Temp_sorder-AUFNR.
Temp_sorder-ZZBOM = Temp_ZZBOM.
Modify C_t_data from Temp_sorder.
End loop.

8.Activate the user exit.

9.Business content has been installed and activated. A field Bill of Material (ZBOM)
from VBAP (SD) has been enhanced to FI_GL_4 structure.

10. Log on to BW server.

11. Create an InfoObject for the enhanced field ZZBOM.
12. Replicate the data from SAP R/3 to the BW server.

13. Assign Data source. Map the fields manually since ZZBOM is an enhancement, it
should be mapped to the InfoObject created by us.
14. Create ODS.

15. Assign the objects corresponding to the Key fields and Data fields.
16. Create Update rules for the ODS.

17. Create InfoSource and InfoPackage.The data is extracted form the SAP R/3 while
we schedule the load from Info Package. The Data is then loaded into the
ODS which is the data target for the InfoPackage.

18. Schedule the load.

19. The data is loaded in to the ODS which could be monitored. The data load is
successful.
20. Create InfoCube.

21. Assign the corresponding characteristics, Time Characteristics and Key figures.
22. Create relevant Dimensions to the cube.
23. Activate the InfoCube and create update rule in order to load the data from the
ODS.
24. Update ODS and the InfoCube is successfully loaded. Manage the InfoCube to view
the available data.

25. Data available in the InfoCube.

26. As per the requirement the General Ledger InfoCube is created and ready for
Reporting. Create a new query using a query designer.
27. Query designer for 0_FI_GL_4.
28. General Ledger Balance Sheet for the period 1995, 1996 and 1997.
The SAP Business Information Warehouse allows the user to analyze data from operative
SAP applications as well as all other business applications and external data sources such
as databases, online services and the Internet.
The SAP Business Information Warehouse enables Online Analytical Processing (OLAP),
which processes information from large amounts of operative and historical data. OLAP
technology enables multi-dimensional analyses from various business perspectives. The
Business Information Warehouse Server for core areas and processes, pre-configured with
Business Content, ensures the user to look at information within the entire enterprise. In
selected roles in a company, Business Content offers the information that employees need
to carry out their tasks. As well as roles, Business Content contains other pre-configured
objects such as InfoCubes, queries, key figures, and characteristics, which make BW
implementation easier.

3. Extraction with Flat Files

3.1 Data from Flat Files (7.0)

BI supports the transfer of data from flat files, files in ASCII format
(American Standard Code for Information Interchange) or CSV format (Comma Separated
Value). For example, if budget planning for a company’s branch offices is done in
Microsoft Excel, this planning data can be loaded into BI so that a plan-actual comparison
can be performed. The data for the flat file can be transferred to BI from a workstation or
from an application server.
1. The user defines a file source system.
2. The user creates a DataSource in BI, defining the metadata for the user file in BI.
3. The user creates an InfoPackage that includes the parameters for data transfer to the
PSA.

3.2 Data from Flat Files (3.x)

The metadata update takes place in DataSource maintenance of

BI.Definition and updating of metadata, that is, the DataSource is done manually for flat
files in SAP BW. The user can find more information about this, as well as about creating
InfoSources for flat files under:

 Flexibly Updating Data from Flat Files

 Updating Master Data from a Flat File
 Uploading Hierarchies from Flat Files

The structure of the flat file and the metadata (transfer structure of the DataSource) defined
in SAP BW have to correspond to one another to enable correct data transfer. Make
especially sure that the sequence of the InfoObjects corresponds to the sequence of the
columns in the flat file.
The transfer of data to SAP BW takes place via a file interface. Determine the parameters
for data transfer in an InfoPackage and schedule the data request. The user can find more
information under Maintaining InfoPackages Procedure for Flat Files.
For flat files, delta transfer in the case of flexible updating is supported. The user can
establish if and which delta processes are supported during maintenance of the transfer
structure. With additive deltas, the extracted data is added in BW. DataSources with this
delta process type can supply both ODS objects and InfoCubes with data. During transfer
of the new status for modified records, the values are overwritten in BW. DataSources with
this delta process type can write the data into ODS objects and master data tables.

3.3 Extracting Transaction and Master Data using Flat Files

Go to T-CODE SE11 and select the radio button TRANSACTION DATA and give the
some technical name and click on CREATE button.
Give APLICATION COMPONENT, TABLE NAME and give Descriptions. Then click on
SAVE button.
Select the fields like master data otherwise click on hide. For transaction data KEY
FIGURES and REFERENCE are compulsory. So select click on SAVE button. Some key
figures along with the reference values. And then

-> Go to BW side and Replicate the user datasource by selecting the application
component, which we already assigned in R/3 side, in the source system tab.
-> Click on Assign infosource by the context menu of the user replicated datasource

-> Create the infosource as flexible updated

-> Assign the transfer rules by selecting necessary infoobject related to the object, which
we extracted by clicking F4 for each and every infoobject. And then click on Activate.
-> Go to info provider and create infocube.
-> After the creating of infocube, create update rules for that infocube.
-> Go to infosource and select the user datasource and create infopackage.
3.4 Data Types that can be extracted using Flat Files

Flat files are data files that contain records with no structured
relationships. Additional knowledge is required to interpret these files such as the file
format properties. Modern database management systems used a more structured approach
to file management (such as one defined by the Structured Query Language) and therefore
have more complex storage arrangements.
Many database management systems offer the option to export data to comma delimited
file. This type of file contains no inherent information about the data and interpretation
requires additional knowledge. For this reason, this type of file can be referred to as a flat
file.
FOR Example .csv is comma separated flat file. , .txt , .lis , .lst..
3.4.1 Basic Steps of Data Flow (ETL process):

1. Determine the Source System (Flat File/SAP R3/WebServices etc)

2. Establish an interface w.r.t the Source system used
3. Create a Data Source – which has a structure similar to the Source
structure. Consider two factors: Name of file and Structure same as
Source.
4. Create Info Provider to transfer data from Data Source to PSA (Persistent
staging Area)
5. Create a Target (Info Provider : InfoObjects/Info Cubes/DSO etc)
6. Create a Transformation rule to map Source & Target fields.
7. Create DTP (Data Transfer Process) to transport data stored within PSA ,
to the final Data Target.
3.4.2 Step-by-Step to upload Master Data from Flat File to InfoObjects

Transaction Code – RSA1 : Data Warehouse Workbench

Source/Input: Create a Flat File of type CSV, with the following structure:
Customer Language Land City Name
260 EN US SEATTLE JOHN
261 DE DE HAMBURG SAM
262 DE DE WALDORF MICHEAL
Right click on the InfoProvider column header. Select “Create InfoArea”
Enter the details – InfoArea name and Description

The InfoArea – IA for Customer appears on the InfoProvider list.

Right click on the created InfoArea and create InfoObject Catalog
Enter a suitable Name and Description for InfoObject Catalog.
Select ‘Char.’ or ‘Key Figure’ based on the type of data (Master data/ Trans. Data) needs to
be created.
In this case, we are creating Master data. Hence, we need to create Characteristics
InfoObject.
Select ‘Char.’
Click on Create.
The InfoObject Catalog appears right under the InfoArea created earlier.
Now, right click on the InfoObject Catalog and create Info Object.
 In the pop-up for InfoObject enter Characteristic name and description.
The first Characteristic InfoObject created can be treated as a key value in the Master table
to be created.
On the right window, the properties of the Characteristic ‘CUSTNO’ are visible.
 In the General Tab, enter the Data Type and Length.
Note: the ‘Attribute only’ field is not checked. This means that this Characteristic is a
primary key field in the Master Data table.
If the box is checked, then it becomes an attribute of this Characteristic Key field or just
another normal field in the Master data Table.
 In the Business Explorer tab, no changes to be done.
Save + Activate => P tables gets created
 In the Master data/texts tab,
 Check “With Master data”, to create a Master Data table, having
naming convention /BIC/P<characteristic name>
 Check “With Texts”, to create a corresponding Text table for this
Master Data table with two default fields – Language & Description.
Select ‘short/medium/long texts’, also ‘texts language dependent’.
 Select “Char is InfoProvider” (as our InfoObject is our target in this
scenario).
 Enter the InfoArea in the box below.

 In the Attribute tab, enter the fields (non-key fields) that need to be maintained
within the Master Data table.
Ex: Customer Id – primary key field
Land
City
On entry of each field, hit [Enter] and in the pop-up box, select the 1st option – “Create
Attribute as Characteristic”.

Enter the Data type and the Length. [Enter]

Repeat the same for other fields too.
[Activate ] - Activate all dependent InfoObjects.
Create File Interface: Source Systems  File  Create <New File System>
Or Use the existing file source system

Data Source creation/selection:

On the Navigator tab, select Data Source. Click on the ‘[X]’ button to choose the Source
System.
Choose the Source System  File  <Flat File source system name>

Right click on the header and Create Application Component.

Enter the name and description for Application Component.

Ex: Name: APC_CUSTOMER
Click ok.
The technical name of Application component APC_CUSTOMER now gets saved as =>
ZAPC_CUSTOMER
Right click on the Application component and create DataSource
Enter a suitable DataSource name : DS_Cust
Source System : Flatfile
Select Data Type of DataSource: Master Data attributes  as we are creating master data
table.
Note:
Select Master Data Attributes, while the user are uploading Master Data from file (DS1)
Select Master Data Text, while the user are uploading Text Descriptions for the Master
Data records (DS2)
In the right Window, the user can view the Data Source properties.
 General Info tab – provides information on the Data Source created
 Extraction tab – provides details of file to be extracted.
In a Data source, we need to mention 2 details:
 Name of Flat File (ASCII or CSV) to be uploaded
 Structure (Lathe user) of the input file, mentioning the fields.
Therefore, make the following changes:
 Select name of file using F4, search for the file on the user local desktop
 Header rows to be ignored = ‘1’ (if file has a header row as column titles)
= ‘0’ (if file has no header row)
 Select Data Format as – “Separated with Separator (for ex CSV)”.
Enter the Data Separator as ‘ , ‘.

 Proposal tab – provides a quick view of the data in the flat file to be uploaded
Click on the ‘Load Example’ tab. The details can be viewed in the below pane.
  Fields tab – provides details of all fields used to create a data structure, to map with
the fields from Source file.
 In the Template Info field, enter the field names (characteristics/attributes).
 [Enter] or [Save]
 [Copy] : all properties of the characteristics ( ex: data types, length etc) are
copied here.

Note: the fields 0LANGU and 0TXTSH are the standard SAP defined characteristics
InfoObjects.
[Save]
[Activate]

 Preview tab – gives a preview of data on flat file loaded onto a structure.
 Click on “Read Preview Data”
Info Package: An InfoPackage helps to transport data from the Data Source structures into
the PSA tables. (This is similar to transfer of data from Work areas to Internal Tables in
R/3 ABAP).
Rt. Click on the Data Source and Create InfoPackage.

Enter the InfoPackage name : IP_Cust

Enter a suitable Description
Select the corresponding DataSource from the list below and [Save].

On the right window, we have the InfoPackage properties:

 Data Selection tab – to initiate loading of data into the infopackage
 Select “Start Data Load Immediately”
 Click on [Start].
 Then click on the [Monitor]button – next to “Process Chain Mgmt”
Or [ F6].
In the next screen, we can view the status of the data loading to PSA.
Click on the [PSA Maintenance] or [Ctrl + F8] to maintain the PSA table with the flat file
values.

In the pop-up, defining the number of records per request, click OK.

This leads to the PSA display screen, with the table and the respective data in them.
Click on [Back] and come back to the DataSource screen.
Transformation: This is a rule that is defined, for mapping the source fields in the
DataSource to the final Target (basically, the Info Providers from where BI extracts the
reports).
In this case, our target is the InfoObject (Master data table), which has been already
created.
Note: Normally, the Transformation can be created on the Data Source or the Target. Here,
we are creating a Transformation on the DataSource.
Right click on DataSource  Create Transformation

In the pop-up,
 For the Target of Transformation,
 Enter Object Type: InfoObject (this is our target in this scenario)
 Enter subtype of Object: Attributes (as we’re considering Master data only.
Not transactional data)
 Enter the name of the InfoObject we created.
 For the Source of Transformation,
 Enter Object Type: DataSource (this is our source structure)
 Enter name of Data Source used
 Enter the Source System used.
Click on OK
Note:
While uploading Master Data, select Sub-type of Object : ‘Attributes’ (IP1 for DS1)
While uploading Text desc. For this Master Data, select Subtype of Object : ‘Texts’ (IP2
for DS2)
In the next screen pop-up, we get the no. of proposals (rules) generated to map source
fields with target fields. Click on Ok.

On the next window we can see a graphical representation of the mapping of Source fields
to Target fields.
Save and Activate, if the mapping if done correctly.

Data Transfer Process (DTP):

The DTP is a process used to transfer data from the PSA tables, into the data Targets, based
on the Transformation (mapping) rules.
DataSource  right click  Create Data Transfer Process.
 In the pop-up,
 The DTP name is described as : < DataSrc/SourceSys -> InfoObject >
 DTP type : standard
 Target of DTP : Object Type : InfoObject (in this case)
Subtype of Object : Attribute
Name : < name of InfoObject >
 Source of DTP: Object Type : Data Source
DataSource : <name>
Source System: Flat file
Click on OK.

In the DTP properties window,

 Extraction tab,
 Select Extraction Mode as ‘Full’.
[Save]
[Activate]
 Update tab – no changes to be made
 Execute tab,
 Click on [Execute] button
Click on [Refresh] until the icons in yellow, turn Green.
In case a Red icon appears, we need to track the error and rectify it.
Go Back.
To view the contents of the Target (Info Object), there are 2 ways:
1. Go to the InfoObject – CUSTNO  MasterData/Texts tab  Dbl click on the
Master data table created : /BIC/P<Char name>  Table display  Execute

On Transformation/DTP  Attribute  rt.click  Manage  rt windw Contents tab 

[Contents]  F8.

Text table Data Display is Language Dependent:

For Lang = ‘EN’, display corresponding Short Desc.

For Lang = ‘DE’, display corresponding Short Desc.

Observations:
In the above Master data table display,
The fields – Customer Name, Customer Land, and Customer City: have been created and
loaded with data as per the file.
These 3 fields have been created as a result of checking the “With Master data” for the
Characteristic. The contents of the Master data are stored in the newly created transparent
table /BIC/P<char. Name> Ex:
The fields – Language, Description : have been created and loaded with data as per the file.
These 2 fields have been created by default, as a result of checking the “With Text data”
for the Characteristic. The contents of the Text data are stored in the newly created
transparent table /BIC/T<char. Name> Ex:

4. DB Connect

4.1 Introduction

The DB Connect enhancements to database interface allow the user

to transfer data straight into BI from the database tables or views of external
applications. The user can use tables and views in database management systems
that are supported by SAP to transfer data. The user use Data Sources to make the
data known to BI. The data is processed in BI in the same way as data from all other
sources.
SAP DB Connect only supports certain Database Management systems (DBMS). The
following are the list of DBMS

 Max DB [Previously SAP DB]

 Informix
 Microsoft SQL Server
 Oracle
 IBM DB2/390, IBM DB2/400, IBM DB2 UDB

There are 2 types of classification. One is the BI DBMS & the other is source DBMS. The
main thing which is, both these DBMS are supported on their respective operating system
versions, only if SAP has released a DBSL. If not, they don’t meet the requirements &
hence can’t perform DB Connect. In this process we use a Data source, to make the data
available to BI & transfer the data to the respective Info providers defined in BI system.
Further, using the usual data accusation process we transfer data from DBs to BI system.
Using this SAP provides options for extracting data from external systems, in addition to
extracting data using standard connection; the user can extract data from tables/views in
database management systems (DBMS)

4.2 Loading data from SAP Supporting DBMS into BI

Steps are as follows:-

1. Connecting a database to Source system -- Direct access to external DB

2. Using Data source, the structure for table/view must be known to BI.

4.2.1 Process Description

Go to RSA1 à Source Systems à DB Connect à Create

Now, create the source system using

1. Logical System Name à MSSQL

2. Source System Name à MS SQL DB Connect
3. Type & Release

Now, Under DB Connect, we can see the name of our Source System (MS SQL DB
Connect)

The logical DB Connect name is MSSQL

In Data sources we need to create an Application Component area to continue with
the export

Goto RSA1  Data sources  Create Application Component

After creating an Application Component Area called “ac_test_check”, we now

have to create a Data source in the component area. So right click the Application
component area à Create Data source (as in below figure).
The Data source name here is “ds_ac_tech”

The Source System here is the defined “MSSQL”

The type of data type data source that we have here is “Master Data Attributes”

The below screen shot describes how to perform extraction or loading using a
Table/View. As the standard adapter is “Database Table” (by default), we can
specify the Table/View here
Now, choose the data source from the DB Object Names.

Now, we have selected the “EMPLOYEES” as the Table/View.

Or we can choose the Table/View à “REGION”

We have 2 database fields

 Region ID
 Region Description

Now that the Data source has to be activated before it is loaded, we “ACTIVATE” it
once.
After activation, the data records (4) are displayed. Eastern, Western, Northern &
Southern

Right click the Info package DS_AC_TEST à Create Info package

We now create an Info package called “IP_DS_AC_TECH”, with Source system as

MSSQL
Once done we perform a schedule on the Info package à “Start”

Now, we need to create an Info Area to create an Info provider (like Info Cube)
After creating the info cube we check for the data in the PSA by “Manage the PSA”

This can be also done using the Key controls (Ctrl + Shift + F6)
he number of records displayed: 4 No’s

Using the PSA Maintenance, we can view the following factors

1. Status
2. Data Packet
3. Data records
4. REGION ID
5. REGION Description

The Table/View “CUSTOMERS” is now chosen for Extraction. In the next tab we
have “PROPOSAL”, which describes all the Database fields, and we have to specify
the Data source fields, types & length.
Now, we create an Info package à IP_TEST_CUST

Now, go to RSA1 à Info objects à Info object (Test) à Create Info Object Catalog
Now, we can preview the Region ID & Region Description.

We now create 2 Info objects & pass the Region ID & Region Description to the 2
objects.

1. Region Description à Region2 (Region)

2. Region Description à reg_id (Region ids)
Now, these are the 2 variables created under the Info object “test2”

 Region (REGION2)
 Region ids (REG_ID)
We create Characteristic as Info Provider for the Master Data loading in the “Info
Provider” section  Insert Characteristic as Info Provider

Now, we create a transformation using “Create Transformation” on the Region ids

(Attributes)
We now choose the Source System after this à MSSQL  MS SQL DB CONNECT

After checking the Transformation mappings on the Region ID, we now perform a
DTP Creation on the same Region ID (Attribute)

We choose the Target system (default) as Info object  Region ID  REG_ID &
the Source Type as Data source with the Source System  MSSQL
After this step, we proceed with creating an Info package  IP_DS_TEDDY which
has the source system as MSSQL. Further we start the scheduling of the Info
package. Once the info package has been triggered we can go to “Maintain PSA” &
monitor the status of data in PSA
Further, we EXECUTE the DTP. And, we can monitor transfer of data from PSA 
Info cube
5. Universal Data Integration

5.1 Introduction

UD Connect (Universal Data Connect) uses Application Server J2EE connectivity to

enable reporting and analysis of relational SAP and non-SAP data. To connect to data
sources, UD Connect can use the JCA-compatible (J2EE Connector Architecture) BI Java
Connector. Firstly, the user can extract the data, load it into BI and store it there physically.
Secondly, provided that the conditions for this are met, the user can read the data directly in
the source using a VirtualProvider.

UD Connect Source

The UD Connect Sources is the instances that can be addressed as data sources using the
BI JDBC Connector.

UD Connect Source Object

UD Connect source objects are relational data store tables in the UD
Connect source.
Source Object Element
Source object elements are the components of UD Connect source
objects – fields in the tables.

5.2 Process Flow

1. Create the connection to the data source with the user relational or multi-dimensional
source objects (relational database management system with tables and views) on the
J2EE Engine.
2. Create RFC destinations on the J2EE Engine and in BI to enable communication
between the J2EE Engine and BI. In the Implementation Guide for SAP NetWeaver
→Business Intelligence →UDI Settings by Purpose →UD Connect Settings.
3. Model the InfoObjects required in accordance with the source object elements in BI.
4. Define a DataSource in BI.

5.3 Creating UD source system

1. In the source system tree in Data Warehousing Workbench, choose Create in the
2. Context menu for the UD Connect folder.
3. Select the required RFC Destination for the J2EE Engine.
4. Specify a logical system name.
5. Select JDBC as the connector type.
6. Select the name of the connector.
7. Specify the name of the source system if it has not already been derived from the
8. Logical system name.
9. Choose Continue.

5.4 Creating a DataSource for UD Connect

To transfer data from UD Connect sources to BI, the metadata
(information about the source object and source object elements) must be create in BI in the
form of a DataSource.

1. Select the application component where the user want to create the DataSource and
choose Create DataSource.
2. On the next screen, enter a technical name for the DataSource, select the type of
DataSource and choose Copy. The DataSource maintenance screen appears.
3. Select the General tab.

a) Enter descriptions for the DataSource (short, medium, long).

b) If required, specify whether the DataSource is initial non-cumulative and might
produce duplicate data records in one request.

4. Select the Extraction tab.

a) Define the delta process for the DataSource.
b) Specify whether the user want the DataSource to support direct access to data.
c) UD Connect does not support real-time data acquisition.
d) The system displays Universal Data Connect (Binary Transfer) as the adapter
for the DataSource. Choose Properties if the user want to display the general
adapter properties.
e) Select the UD Connect source object.

A connection to the UD Connect source is established. All source objects available in the
selected UD Connect source can be selected using input help.

5. Select the Proposal tab.

The system displays the elements of the source object (for JDBC it is these fields) and
creates a mapping proposal for the DataSource fields. The mapping proposal is based
on the similarity of the names of the source object element and DataSource field and the
compatibility of the respective data types. Note that source object elements can have a
maximum of 90 characters. Both upper and lower case are supported.

a) Check the mapping and change the proposed mapping as required. Assign the
non-assigned source object elements to free DataSource fields. The user cannot
 map elements to fields if the types are incompatible. If this happens, the system
displays an error message.
b) Choose Copy to Field List to select the fields that the user want to transfer to
the field list for the DataSource. All fields are selected by default.

6. Define the Fields tab.

Here, the user can edit the fields that the user transferred to the field list of the DataSource
from the Proposal tab. If the system detects changes between the proposal and the field list
when switch from the Proposal tab to the Fields tab, a dialog box is displayed where the
user can specify whether the user want to copy changes from the proposal to the field list.

a) Under Transfer, specify the decision-relevant DataSource fields that the user
wants to be available for extraction and transferred to BI.
b) If required, change the values for the key fields of the source. These fields are
generated as a secondary index in the PSA. This is important in ensuring good
performance for data transfer process selections, in particular with semantic
grouping
c) If required, change the data type for a field.
d) Specify whether the source provides the data in the internal or external format.
e) If the user chooses an External Format, ensure that the output length of the
field (external length) is correct. Change the entries if required.
f) If required, specify a conversion routine that converts data from an external
format to an internal format.
g) Select the fields that the user wants to be able to set selection criteria for when
scheduling a data request using an InfoPackage. Data for this type of field is
transferred in accordance with the selection criteria specified in the
InfoPackage.
h) Choose the selection options (such as EQ, BT) that the user wants to be
available for selection in the InfoPackage.
i) Under Field Type, specify whether the data to be selected is language-
dependent or time-dependent, as required.

If the user did not transfer the field list from a proposal, the user can define the fields of the
DataSource directly. Choose Insert Row and enter a field name. The user can specify
InfoObjects in order to define the DataSource fields. Under Template InfoObject, specify
InfoObjects for the fields of the DataSource. This allows the user to transfer the technical
properties of the InfoObjects to the DataSource field.

Entering InfoObjects here does not equate to assigning them to DataSource fields.
Assignments are made in the transformation. When the user define the transformation, the
system proposes the InfoObjects the user entered here as InfoObjects that the user might
want to assign to a field
7. Check, save and activate the DataSource
8. Select the Preview tab.

If the user selects Read Preview Data, the number of data records the user specified in the
user field selection is displayed in a preview. This function allows the user to check
whether the data formats and data are correct.

5.5 Using Relational UD Connect Sources (JDBC)

5.5.1 Aggregated Reading and Quantity Restriction

In order to keep the data mass that is generated during UD Connect access to a JDBC data
source as small as possible, each select statement generated by the JDBC adapter receives
a group by clause that uses all recognized characteristics. The recognized key figures are
aggregated. What is recognized as a key figure or characteristic and which methods are
used for aggregation depends on the properties of the associated InfoObjects modeled in
SAP BW for this access.
The amount of extracted data is not restricted. To prevent exceeding the storage limitations
of the J2EE server, packages with around 6,000 records are transferred to the calling ABAP
module.

5.5.2 Use of Multiple Database Objects as UD Connect Source Object

Currently only one database object (table, view) can be used for a UD Connect Source. The
JDBC scenario does not support joins. However, if multiple objects are used in the form of
a join, a database view should be created that provides this join and this object is to be used
as a UD Connect source object. The view offers more benefits:
 The database user selected from SAP BW for access is only permitted to access
these objects.
 Using the view, the user can run type conversions that cannot be made by the
adapter (generation of the ABAP data type DATS, TIMS etc.)

5.6 BI JDBC Connector

Sun's JDBC (Java Database Connectivity) is the standard Java API for Relational
Database Management Systems (RDBMS). The BI JDBC Connector allows the user to
connect
applications built with the BI Java SDK to over 170 JDBC drivers, supporting data sources
such as Teradata, Oracle, Microsoft SQL Server, Microsoft Access, DB2, Microsoft Excel,
and text files such as CSV. This connector is fully compliant with the J2EE Connector
Architecture (JCA).
The user can also use the BI JDBC Connector to make these data sources available in SAP
BI systems using UD Connect. The user can also create systems in the portal that are based
on this connector.
The connector adds the following functionality to existing JDBC drivers:
• Standardized connection management that is integrated into user management in the
portal
• A standardized metadata service, provided by the implementation of JMI capabilities
based on CWM
• A query model independent of the SQL dialect in the underlying data source
The JDBC Connector implements the BI Java SDK's IBIRelational interface.
5.6.1 Deploy the user data source’s JDBC driver to the server:

1. Start the Visual Administrator.

2. On the Cluster tab, select Server x → Services → JDBC Connector.
3. In the right frame, select the Drivers node on the Runtime tab.
4. From the icon bar, choose Create New Driver or Data source.
5. In the DB Driver field in the Add Driver dialog box, enter a name for the user
JDBC driver.
6. Navigate to the user JDBC driver's JAR file and select it.
7. To select additional JAR files, select Yes when prompted, and when finished,
8. Select No.

5.6.2 Configuring BI Java Connector

To prepare a data source for use with the BI Java SDK or with UD
Connect, the user first need to configure the properties in BI Java Connector used to
connect to the data source. The user does this in SAP NetWeaver Application Server’s
Visual Administrator.

In the service Connector Container, configure a reference to the JDBC driver of the user
data source. This can be done by performing the following steps:

1. Select the BI JDBC Connector in the Connectors tree.

2. Choose the Resource Adapter tab.
3. In the Loader Reference box, choose Add to add a reference to the user JDBC
driver.
4. Enter library :<jdbc driver name> and choose OK.
5. The <jdbc driver name> is the name the user entered for the user driver when the
user loaded it (see Prerequisites in BI JDBC Connector).
6. Save the settings.
5.6.2.1 Testing the Connections

After the user has configured the BI Java Connector, the user can
perform a rough installation check by displaying the page for the connector in the user
server. Perform the tests for the connector by visiting the URLs

5.6.2.2 JNDI Names

When creating applications with the BI Java SDK, refer to a connector by its
JNDI name: The BI JDBC Connector has the JNDI name SDK_JDBC.

5.6.2.3 Cloning the Connections

The user can clone an existing connection by using the Clone button in the
toolbar.

5.6.3 Connector Properties

Refer to the table below for the required and optional properties to
configure for the user connector: BI JDBC Connector Properties
5.7 BI XMLA Connector

Microsoft's XMLA (XML for Analysis) facilitates Web services-based, platform-

independent access to OLAP providers. The BI XMLA Connector enables the exchange of
analytical data between a client application and a data provider working over the Web,
using SOAP based XML communication API.

The XMLA Connector sends commands to an XMLAcompliant OLAP data source in order
to retrieve the schema rowsets and obtain a result set. The BI XMLA Connector allows the
user to connect applications built with the BI Java SDK to data sources such as Microsoft
Analysis Services, Hyperion, MicroStrategy, MIS, and BW 3.x. This connector is fully
compliant with the J2EE Connector Architecture (JCA).The user can also use the BI
XMLA Connector to make these data sources available in SAP BI Systems via UD
Connect, or the user can create systems in the portal based on this connector. The BI
XMLA Connector implements the BI Java SDK's IBIOlap interface.
5.7.1 Using InfoObjects with UD Connect
 When modeling InfoObjects in BI, note that the InfoObjects have to
correspond to the source object elements with regard to the type description and length
description. For more information about data type compatibility,
.
The following restrictions apply when using InfoObjects:
• Alpha conversion is not supported
• The use of conversion routines is not supported
• Upper and lower case must be enabled
These InfoObject settings are checked when they are generated.
5.7.2 Using SAP Namespace for Generated Objects

The program-technical objects that are generated during generation of

a DataSource for UD connect can be created in transportable or local format. Transportable
means that the generated objects can be transferred to another SAP BW system using the
correction and transport system. The transportability of an object depends on, among other
things, in which namespace it is created.

The delivery status allows for the generation of transportable objects

in the SAP namespace. If this appears to be too laborious (see the dependencies listed
below), there is also the option of switching to generation of local objects. To do this, the
user run the RSSDK_LOCALIZE_OBJECTS report in the ABAP editor (transaction: SE
38). Then the system switches to local generation. The objects generated afterward are not
transportable. If the report is executed again, the generation is changed back to
transportable. The status of already generated objects does not change. All new objects are
created as transportable. If the user need to work with transportable objects, the user should
be aware of the following dependencies:

 System changeability

These objects can only be generated in systems whose system

changeability permits this. In general, these are development systems, because productive
systems block system changeability for security reasons.• If a classic SAP system
landscape of this type exists, then the objects are created in the development system and
assigned to package RSSDK_EXT. This package is especially designated for these objects.
The objects are also added to a transport request that the user create or that already exists.
After the transport request is finished, it is used to transfer the infrastructure into the
productive environment.

 Key

Because the generated objects are ABAP development objects, the user
must be authorized as a developer. A developer key must be procured and entered.
Generation requires the customer-specific installation number and can be generated online.
The system administrator knows this procedure and should be included in the procurement.
The key has to be procured and entered exactly once per user and system. Because the
generated objects were created in the SAP namespace, an object key is required.

Like the developer key, this is customer specific and can also be procured online. The key
is to be entered exactly once per object and system. Afterwards, the object is released for
further changes as well. Further efforts are not required if there are repeated changes to the
field list or similar.

6. XML Integration

6.1 Introduction

SAP’s Business Connector (SAP BC) is a business integration

tool to allow SAP Customers the ability to communicate with other SAP customers or SAP
marketplaces. The Business Connector allows integration with R/3 via open and non-
proprietary technology. This middleware component uses the Internet as communication
platform and XML/HTML as data format thus seamlessly integrating different IT
architectures with R/3.

SAP BC integrates RFC server and the client and provides an

XML layer over R/3 functionality. That is, it comes with an XML automation that converts
SAP’s RFC format into XML and supports both synchronous RFC and asynchronous RFC
protocols such that there is no requirement of SAP R/3 automation at the receiving end.
Also, SAP BC has a built in integration support for SAP’s specification of IDOC-XML and
RFC-XML standards. Hence, whenever dealing with the messages conforming to the same
standards it supports the integration.

6.2 Benefits of XML Integration

 End-to-End Web Business Processes

 Open Business Document Exchange over the Internet
 XML Solutions for SAP services

6.2.1 End-to-End Web Business Processes

Internet bridges the gap between different businesses, systems

and users and facilitates them do business via web. SAP BC makes this communication
easier by its XML conversions. Through its integration with XML it enables the exchange
of structured business documents over the Internet by providing common standard for
different applications and IT systems to communicate and exchange business data.
6.2.2 Open Business Document Exchange over the Internet

SAP Business Connector uses hypertext transfer protocol (HTTP) to

exchange XML-based documents over the Internet. This makes the process of data transfer
a lot easier by providing a common structure for all business documents. It also ensures
secure exchange of documents with the help of its Security Socket Layer (SSL) technology
and allows implementation of maps, branches and loops without any coding by using
developer tool. In addition, SAP BC provides openness and flexibility to comply with
emerging business semantics that continuously keep on changing.

6.2.3 XML Solutions for SAP services

SAP BC makes all mySAP.com solutions accessible via XML based

business documents. It supports all major existing interfaces that are provided by SAP with
the help of XML-based Interface Repository (IFR) and empowers SAP customers to
benefit from SAP functionality over the Internet. This IFR gives the option for
downloading XML schemas for operational use and provides a uniform XML interface
representation despite different implementation technologies such as RFC, IDOC and
BAPI. Also with SAP BC, XML messages are translated into corresponding SAP internal
call whenever required and converted back into XML format when received from SAP
system enhancing the existing programming model for distributed applications formed by
ALE. In addition, SAP BC extends business scenarios across firewalls enabling secure flow
of business documents without requiring changes to establish security infrastructures.

6.3 Business Integration with XML

Business processes are increasingly characterized by structures

operating between companies. Companies no longer act in isolation: instead, they are
integrated into geographically distributed production networks, pursuing the production
process in cooperation with other firms. This means that exchanging data quickly and
securely between applications and systems is an increasingly important requirement.

XML acts as a uniform standard for exchanging business data, through

which heterogeneous applications can communicate with one another over uniform
interfaces and in a language, which everyone involved, can understand. With XML, simple
and complex structures can be presented at any data level and for any category of data.

Non-SAP applications can now be used by SAP customers owing to

XML compliance. For example, using a company’s services in the cyber marketspace
allows the customer data to be received and directly stored on vendor system as both are
using data formatted in XML. These third-party systems using the XML standard data
format are increasing rapidly. A customer using SAP can use a range of add-on products
and services to their existing applications.
 XML integration is also easier than using proprietary communication
formats like SAP's BAPIs and RFCs. This has found widespread customer acceptance as it
reduces integration and maintenance cost of interface integration during the
implementation of non-SAP systems. Furthermore, customers can now exchange data with
XML standard by using the Internet infrastructure through a much-more user-friendly Web
Browser.

An XML environment of data exchange over the Internet via security

protocols such as HTTP, HTTPS and FTP fully supports collaborative business scenarios,
increasingly common in an integrated world. XML formatting and message handling with
the help of the SAP Business Connector allows customers to use an industry-wide accepted
format for data exchange between the SAP system and partner systems including
historically grown proprietary systems.

The ability to integrate and analyze business data across applications,

structured and unstructured information, and heterogeneous systems extends the traditional
business environment and provides users with a complete view of the business.

6.3.1 Incorporating XML Standards

Various XML standards are supported by SAP. It presents data

according to an SAP specification either in IDoc-XML or BAPI-XML, the SAP XML
extensions for IDocs and BAPIs. It uses preprepared representations of the SAP interfaces
to XML Commerce Business Language (xCBL) messages to facilitate communication with
the MarketSet marketplaces.

Messages are stored in a generic envelope. This envelope contains

metadata that controls, among other things, the routing of the messages. SAP supports two
different standards for these envelopes - Microsoft BizTalk and a format similar to Simple
Object Access Protocol (SOAP). Packaging SAP BAPIs in a standard envelope offers
several advantages, including direct processing of messages by external applications and a
uniform system of error handling.

6.3.2 SAP’s Internet Business Framework

SAP's Internet Business Framework (IBF) attempts to address business

collaboration issues by enabling integration with Internet technologies at the user,
component, and business-process levels:

 User integration is achieved by providing a single point of Web-based access to the

workplace (that is, local and company wide systems) and the marketplace (that is,
systems across a number of companies).
  Component integration consists of the integration of Internet technologies at the
front-end and application-server levels with the aid of HTML, HTTP, and XML
messages.
 Business process integration across company boundaries is accomplished through
the exchange of transactions between companies based on open Internet standards.
6.3.3 SAP applications with XML

SAP BC provides an add-on XML layer with R/3 functions to ensure

compatibility of non-SAP applications with R/3 internal data structures or protocols. SAP
Business Connector can help achieve seamless B2B integration between businesses
through the integration framework.

For example, there can be a real-time integration between supplier

inventories and an enterprise’s SAP system; or a multi-vendor product, price and
availability information and a customer’s purchasing application. The SAP proprietary
RFC format is converted to XML (or HTML) so that no SAP software is needed on the
other end of the communication line and developing applications does not require SAP R/3
knowledge. SAP BC has built in support for SAP's specification of IDoc-XML and RFC-
XML. Whenever the user deal with messages conforming to these standards, integration is
supported out of the box. For cases where other XML formats are required the user can
create maps with a graphical tool or insert the user own mapping logic.

XML Based communication over the Internet is achieved through SAP's Business
Connector

Figure 1
6.3.4 Factors leading to emergence of XML-enabled SAP solutions

6.3.4.1 Changing Business Standards and their adoption

Mapping of SAP business documents and the XML-based business

documents can be easily done with SAP Business Connector. The flexible architecture of
the SAP Business Connector makes it easy to add specific schemas and interfaces To
comply with business documents standards, the Business Connector provides for automated
generation of server and client side codes thus solving the interconnectivity problem by
enabling uniform transactions among customers.

A business model based on these standards allows companies to move

towards becoming make-to-order businesses replete with all the marketing, sales,
distribution, manufacturing and other logistic-driven operational cost savings.

6.3.4.2 Internet Security Standards

The effectiveness of E-commerce is premised on a secure exchange of

information. The SAP business connector provides security essential to an online business
transaction. Business partners can be authenticated and business documents can be securely
exchanged. The SAP Business Connector supports the well-established standard encryption
technology Secure Socket Layer (SSL) for secure document exchange. Digital signature
will ensure authentication of crucial data.

SAP communication is achieved through Idocs, BAPIs, and RFCs.

These documents must be made compliant with Internet. This conversion is achieved using
the Business Connector. Business Connector converts SAP documents into XML so that
they can be exchanged using Internet protocols.

6.4 Web-based business solutions

The SAP Business Connector allows businesses to effectively use intra

and inter-enterprise information. For example-companies can use the SAP Business
Connector to retrieve catalog information from a suppliers’ Web site and to integrate the
information with internal processes - in real time.

6.4.1 Components of Business Connector

A business connector constitutes a Server and an Integrator. The

Integrator can be used to add any additional functionality to the Server.

SAP Business connector in association with a third party XML

enabled software product can only be used for collaborative business-to-business scenarios
such as transmitting catalog information, managing purchase orders and availability to
promise checks, acknowledging purchase orders, and handling invoices.

The requirements for XML interface certification for SAP's complementary software
include:

 Use of HTTP and HTTPS protocols with the SAP Business Connector
 Customization for a specific Business Application
 Sending and receiving the communication objects (i.e. Idocs, BAPIs or RFCs)

6.5 How to Customize Business Connector (BC)

1. Start the Business Connector by calling <BC_dir>/bin/server.bat. If the user wants to

have a debug output, enter server.bat –debug <debuglevel> –log <filename>. Debuglevel
cannot be > 10.

2. Open the Business Connector administration screen into a web browser window. Enter
the user Username and corresponding password.

6.5.1 Add New Users to BC

3. If the user want to transmit data from an SAP system to the BC, the user need to have
the same user in both systems. For creating users in the BC click Security > Users and
Groups > Add and Remove Users.

4. Enter the desired SAP User, assign the corresponding Password and click Create
Users. This creates a User. Mark the just created User in the Groups box section and
make sure that the ‘Select Group’ ="Administrators". Now, add the User into the
Administrators group by clicking (below the right selection box). Click Save Changes to
save the settings.

6.5.2 Add SAP Systems

5. All the proposed SAP system(s) should be added within the Business Connector. To
achieve this, click Adapters > SAP which opens a new window. In the new window click
SAP > SAP Servers > Add SAP Server.

6. Enter the necessary information for the SAP server (System, Login Defaults, Server
Logon, and Load Balancing. "Save" (as illustrated in screen 1).
6.5.3 Add Router Tables

7. All incoming calls to Business Connector are scrutinized and routing information is
extracted about the ‘Sender’, ‘Receiver’ and ‘Msg Type’. Using the Rules it finds the
recipient and the format that should be used to send the call to this particular recipient.
Clicking Adapters > Routing opens a new window. Enter information like ‘Sender’,
‘Receiver’ and ‘MsgType’. With "Add Rule" the rule is created on other details like
"Transport" and "Transport Parameters" must be provided.

8. After entering all the details, click Save and enable the rule by clicking No under the
"Enabled?" column.

6.5.4 Access functionality in the Business Connector

9. Post a document containing the XML format of the Idoc or BAPI/RFC-call to the
Business Connector service.

For example: Use the following statements to post a document to the /sap/InboundIdoc
service of the Business Connector.

The user cannot use the SAP Business Connector in a web application (e-Commerce,
Procurement, etc), but can use it to facilitate a business-to-business transaction in an EDI-
like manner. For example: The user can send an XML document to the user vendor, and
they send the user an XML packet back.
7. Data Mart Interface

7.1 Introduction

The data mart interface makes it possible to update data from one
InfoProvider to another. Data exchange of multiple BI Systems: the data-delivering system
is referred to as the Source B; the data-receiving system as the Target BI. The individual BI
systems arranged in this way are called data marts. The InfoProviders of the source BI are
used as sources of data here. Data exchange between BI systems and other SAP systems.

Data marts can be used in different ways:

 They save a subset of the data of a Data Warehouse in another database, possibly
even in a different location.
 They are smaller units of a Data Warehouse.
 They are stored as intentionally redundant segments of the (logical, global) overall
system (Data Warehouse).

A BI system defines itself as the source system for another BI system by:

 Providing metadata
 Providing transaction data and master data

An export DataSource is needed to transfer data from a source BI into a target BI. Export
DataSources for InfoCubes and DataStore objects contain all the characteristics and key
figures of the InfoProvider. Export DataSources for master data contain the metadata for all
attributes, texts, and hierarchies for an InfoObject.

7.2 Special Features

 Changes to the metadata of the source system can only be added to the export
DataSources by regenerating the export DataSources.
 The Delete function is not supported at this time.
 The user can only generate an export DataSource from an InfoCube if:
 The InfoCube is activated
 The name of the InfoCube is at least one character shorter than the
maximum length of a name, since the DataSource name is made up of the
InfoCube name and a prefix.
7.3 Data Mart Interface in the Myself System

The data mart interface in the Myself System is used to connect the
BW System to it.. This means the user can update data from data targets into other data
targets within the system. The user can import InfoCube data by InfoSource into BW again
and, thereby, fill another InfoCube. The user can carry out a data clean--up, for example,
using transfer rules and update rules.

The user can update data directly from ODS objects into other data
targets, since the ODS data can be used directly as a DataSource in the same system.

Updating ODS data into another ODS object

object makes it possible to track
the status of orders, that is, to see which orders are open, delivered in part etc. Data about
orders and deliveries are in two separate ODS objects. In order to trace the status in
reporting, update the data to another ODS object
o and merges the objects together.

7.4 Data Mart Interface between Several Systems

Data marts are found both in the maintenance and in the definition, the
same as in the SAP source system. Here too, the user can group together data from one or
more source
ource systems in a BI System, or continue to work in several BI Systems.

The data mart interface can be used between two BI Systems or between another SAP
system and a BI System:
  To enable large amounts of data to be processed
 To increase the respective speeds of data transfer, logging and analysis.
 To achieve improved concision and maintenance of the individual Data
Warehouses.
 To enable data separation relating to the task area, on the one hand, and to be able
to perform analyses of the state of the entire dataset on the other.
 To make it possible to have fewer complexities when constructing and
implementing a Data Warehouse.
 To construct hub and spoke scenarios in which a BI System stands in the middle
and the data from distributed systems runs together and is standardized.

7.4.1 Architectures

The functions can produce different architectures in a Data Warehouse landscape.

 Replication Architecture
 Aggregation Architecture

7.4.1.1 Replicating Architecture

If the user selects this architecture, the data for a BI server is available
as source data and can be updated in further target BI Systems.
7.4.1.2 Aggregating Architecture

With aggregating architecture, data is grouped together from two or

more BI servers, and is then available for further processing.
7.4.2 Process Flow

To define an existing BI system as a source system, access the

functions of the data mart we use the Data Mart Interface.

7.4.2.1 In the Source BI

Before a BI system can request data from another BI system, it must

have information about the structure of the data to be requested. To do this, the user has to
upload the metadata from the source BI into the target BI.

The user generates an export DataSource for the respective

InfoProvider in the source BI. This export DataSource includes an extraction structure,
which contains all the characteristics and key figures of the InfoProvider.

7.4.2.2 In the Target BI

 If required, define the source system. This is only necessary if the source system
has not yet been created in the target BI.
 Create the InfoProvider into which the data is to be loaded.
 Replicate the metadata of the user export DataSource from the user source BI into
the user target BI. Using the source system tree, the user can replicate all the
metadata of a source system, or only replicate the metadata of the user DataSource.
 Activate the DataSource.
 Create an InfoPackage at the user DataSource using the context menu. The user
source BI system is specified as the source by default.
 Using the context menu for the user DataSource, create a data transfer process with
the InfoProvider into which the data is to be loaded as the target. A default
transformation is created at the same time.
 The complete data flow is displayed in the InfoProvider tree under the user
InfoProvider.
  Schedule the InfoPackage and the data transfer process. We recommend that the
user always use process chains for loading processes.
7.4.3 Generating Export DataSources for InfoProviders

The export DataSource is needed to transfer data from a source BI

system into a target BI system. The user can use the selected InfoProvider as a DataSource
for another BI system.

1. In the Data Warehousing Workbench in the BI source system, choose Modeling

and select the InfoProvider tree.
2. Generate the export DataSource using the context menu of the user InfoProvider.
To do this, choose Additional Functions  Generate Export DataSource.

The technical name of the export DataSource is made up of the number 8 together with the
name of the InfoProvider. Technical name of an InfoCube: COPA.Technical name of the
export DataSource: 8COPA

7.4.4 Generating Master Data Export DataSources

The export DataSource is needed to transfer data from a source BI

system into a target BI system. The user can generate an export DataSource for master data,
and thus for individual InfoObjects. By doing this, all the metadata is created that is
required to extract master data (all attributes, texts, and hierarchies) from an InfoObject.

1. Select the InfoObject tree in the Data Warehousing Workbench in the source BI
system.
2. Generate the export DataSource from the context menu of the user InfoObject. To
do this, choose Additional Functions  Generate Export DataSource.

The technical name of the export DataSource is

8******M for attributes (M stands for master data)

8******T for texts
8******H for hierarchies.
(The asterisks (*) stand for the source InfoObject).

When the user create an InfoObject or a master data InfoSource in the source BI system,
the user must therefore make sure that the length of the technical name of each object is no
longer than 28 characters.
7.4.5 Transactional Data Transfer Using the Data Mart Interface

The data transfer is the same as the data transfer in an SAP system.
The system reads the data, taking into account the specified dimension-specific selections,
from the fact tables of the delivering BI system.

7.4.5.1 Delta Process

Using the data mart interface, the user can transfer data by full upload as well as by delta
requests. A distinction is made between InfoCubes and DataStore objects.

 The InfoCube that is used as an export DataSource is first initialized, meaning that
the current status is transferred into the target BI system. When the next upload is
performed, only those requests are transferred that have come in since initialization.
Different target systems can also be filled like this. Only those requests are
transferred that have been rolled up successfully in the aggregates. If no aggregates
are used, only those requests are transferred that are set to Qualitative OK in the
InfoCube administration.
 For DataStore objects, the requests in the change log of the DataStore object are
used as the basis for determining the delta. Only the change log requests that have
arisen from reactivating the DataStore object data are transferred.
7.4.5.2 Restriction

It is possible to only make one selection for each target system for the
delta. The user first makes a selection using cost center 1 and load deltas for this selection.
Later on, the user also decides to load a delta for cost center 2 in parallel to the cost center
1 delta. The delta can only be fully requested for both cost centers, meaning that it is then
impossible to separately execute deltas for the different selections

7.4.6 Transferring Texts and Hierarchies for the Data Mart Interface

The transfer of master data is scheduled in the Scheduler. It

corresponds to the data transfer for an R/3 System. If the user wants to load texts or
hierarchies using the Scheduler, the user must first create them in the source system.

When the user load hierarchies, the user get to the available hierarchies
from the Hierarchy Selection tab page by using the pushbutton Available Hierarchies in
OLTP. Select the user hierarchy and schedule the loading.
7. Virtual InfoCubes

7.1 Introduction

They only store data virtually, not physically and are made available
to Reporting as Info-Providers. Virtual remote cube can be used to 'act like an infocube,
feel like an infocube, but is not an infocube', The user can create this virtual infoprovider,
having the same structure similar to infocube, with dimensions, characteristics and key
figures. But the actual generation of data for this cube, is, well depends on the user making.

The user can choose to create a virtual provider with a 'direct access' to
R/3. This means that upon viewing the data in this virtual provider, a remote function call
is called direct to R/3 to get the values on the fly. The user can also choose to get value
from a function module of the user calling, so the implementation details are really up to
the user.

7.2 Create Virtual Infocube

In BI7, just right click on any infoarea, and choose create 'virtual
provider'. For direct access, the user chooses Virtual Provider with 'direct access'. There's
an option to use the old R/3 infosource. What this means is that the user can create an
infosource with transfer rules and such, and when the user create the virtual infoprovider, it
will use the structure in the infosource automatically and the flow of data is automatically
link R/3 with the transfer rules and the virtual infoprovider. Note that no update rules exist
with this kind of setup.

But in Bi7, the user have a more flexible approach in that the user can
create a transformation to configure the 'transfer logic' of the user virtual infoprovider,
along with start routine, end routine or any other transformation technique visible with
using a transformation rule.

But using BI7 setup, the user need to create a sort of 'pseudo' DTP,
which doesn't actually do anything, meaning the user do not 'execute' it to start a data
transfer. After all is done, the user need to right click on the virtual infoprovider and choose
'Activate Direct Access'. If the user uses Infosource, go to the Infosource tab, and choose
the infosource. If the user is using BI7 setup, choose the DTP related to the transformation
and save it.

7.3 Different Types

A distinction is made between the following types:

  SAP RemoteCube
 RemoteCube
 Virtual InfoCube with Services
7.3.1 SAP RemoteCube

An SAP RemoteCube is a RemoteCube that allows the definition of queries with direct
access to transaction data in other SAP systems.

Use SAP RemoteCubes if:

 The user need very up-to-date

up data from an SAP source system
 The user only access a small amount of data from time to time
 Only a few users execute queries simultaneously on the database.

Do not use SAP RemoteCubes if:

 The user request a large amount of data in the first query navigation step, and no
appropriate aggregates are available in the source system
 A lot of users execute queries simultaneously
 The user frequently access the same data

7.3.1.1 Creating a SAP RemoteCube

The SAP RemoteCube can be used in reporting in cases where it does not differ from other
InfoProviders.

1. Choose the InfoSource tree or InfoProvider tree in the Administrator Workbench

under Modeling.
2. In the context menu, choose Create InfoCube.
3. Select SAP RemoteCube as InfoCube type and enter the user InfoSource.
4. The user can define if a unique source system is assigned to the InfoCube using
an indicator. Otherwise the user must select the source system
stem in the relevant query. In this
case the characteristic (0LOGSYS) is added to the InfoCube definition.
5. In the next screen the user can check the defined SAP RemoteCube and adjust it
if necessary before activation.
6. The user can then assign source systems to the SAP RemoteCube. To do this,
choose Assign Source System from the context menu in the user SAP RemoteCube. In the
dialog box that follows, choose one or several source systems and select Save
Assignments.. The source system assignments are local to the system and are not
transported.
7.3.1.2 Structure

SAP RemoteCubes are defined based on an InfoSource with flexible

updating. They copy the characteristics and key figures of the InfoSource. Master data and
hierarchies are not read directly in the source system. They are already replicated in BW
when the user executes a query. The transaction data is called during execution of a query
in the source system.

During this process, the selections are provided to the InfoObjects if

the transformation is only simple mapping of the InfoObject. If the user have specified a
constant in the transfer rules, the data is transferred only if this constant can be fulfilled.
With more complex transformations such as routines or formulas, the selections cannot be
transferred. It takes longer to read the data in the source system because the amount of data
is not limited. To prevent this user can create an inversion routine for every transfer
routine. Inversion is not possible with formulas, which is why SAP recommends that the
user use formulas instead of routines.

7.3.1.3 Integration

To be assigned to an SAP RemoteCube, a source system must meet the following

requirements:

 BW Service API functions (contained in the SAP R/3 plug-in) are installed.
 The Release status of the source system is at least 4.0B
 In BW, a source system ID has been created for the source system
 DataSources from the source system that are released for direct access are assigned
to the InfoSource of the SAP RemoteCube. There are active transfer rules for these
combinations.
7.3.2 Remote Cube

A RemoteCube is an InfoCube whose transaction data is not managed

in the Business Information Warehouse but externally. Only the structure of the
RemoteCube is defined in BW. The data is read for reporting using a BAPI from another
system. Using a RemoteCube, the user can carry out reporting using data in external
systems without having to physically store transaction data in BW. The user can, for
example, include an external system from market data providers using a RemoteCube. By
doing this, the user can reduce the administrative work on the BW side and also save
memory space.
7.3.2.1 Structure

When reporting using a RemoteCube, the Data Manager, instead of using a Basic Cube
filled with data, calls the RemoteCube BAPI and transfers the parameters.

 Selection
 Characteristics
 Key figures

As a result, the external system transfers the requested data to the OLAP Processor.

7.3.2.2 Integration

To report using a RemoteCube the user has to carry out the following steps:

1. In BW, create a source system for the external system that the user wants to use.
2. Define the required InfoObjects.
3. Load the master data:
 Create a master data InfoSource for each characteristic
 Load texts and attributes
4. Define the RemoteCube
5. Define the queries based on the RemoteCube
7.3.3 Virtual InfoCubes with Services

A virtual InfoCube with services is an InfoCube that does not

physically store its own data in BW. The data source is a user-defined function module.
The user has a number of options for defining the properties of the data source more
precisely. Depending on these properties, the data manager provides services to convert the
parameters and data.

The user uses a virtual InfoCube with services if the user wants to
display data from non-BW data sources in BW without having to copy the data set into the
BW structures. The data can be local or remote. The user can also use the user own
calculations to change the data before it is passed to the OLAP processor.

This function is used primarily in the SAP Strategic Enterprise

Management (SEM) application.

In comparison to the RemoteCube, the virtual InfoCube with services

is more generic. It offers more flexibility, but also requires more implementation effort.

7.3.3.1 Structure

When the user creates an InfoCube the user can specify the type. If the
user chooses Virtual InfoCube with Services as the type for the user InfoCube, an extra
Detail pushbutton appears on the interface. This pushbutton opens an additional dialog box,
in which the user defines the services.

1. Enter the name of the function module that the user wants to use as the data
source for the virtual InfoCube. There are different default variants for the
interface of this function module. One method for defining the correct variant,
together with the description of the interfaces, is given at the end of this
documentation.
2. The next step is to select options for converting/simplifying the selection
conditions. The user does this by selecting the Convert Restrictions option. These
conversions only change the transfer table in the user-defined function module.
The result of the query is not changed because the restrictions that are not
processed by the function module are checked later in the OLAP processor.
Options:
 No restrictions: if this option is selected, any restrictions are passed to the
InfoCube.
  Only global restrictions: If this option is selected, only global restrictions
(FEMS = 0) are passed to the function module. Other restrictions (FEMS > 0)
that are created, for example, by setting restrictions on columns in queries, are
deleted.
 Simplify selections: Currently this option is not yet implemented.
 Expand hierarchy restrictions: If this option is selected, restrictions on
hierarchy nodes are converted into the corresponding restrictions on the
characteristic value.
3. Pack RFC: This option packs the parameter tables in BAPI format before the
function module is called and unpacks the data table that is returned by the
function module after the call is performed. Since this option is only useful in
conjunction with a remote function call, the user have to define a logical system
that is used to determine the target system for the remote function call, if the user
select this option.
4. SID support: If the data source of the function module can process SIDs, the user
Should select this option.

If this is not possible, the characteristic values are read from the data source and
the data manager determines the SIDs dynamically. In this case, wherever
possible, restrictions that are applied to SID values are converted automatically
into the corresponding restrictions for the characteristic values.

5. With navigation attributes: If this option is selected, navigation attributes and

restrictions applied to navigation attributes are passed to the function module.
If this option is not selected, the navigation attributes are read in the data manager
once the user-defined function module has been executed. In this case, in the
query, the user need to have selected the characteristics that correspond to these
attributes. Restrictions applied to the navigation attributes are not passed to the
function module in this case.
6. Internal format (key figures): In SAP systems a separate format is often used to
display currency key figures. The value in this internal format is different from
the correct value in that the decimal places are shifted. The user use the currency
tables to determine the correct value for this internal representation.
If this option is selected, the OLAP processor incorporates this conversion during
the calculation.

7.3.3.2 Dependencies

If the user uses a remote function call, SID support must be switched
off and the hierarchy restrictions must be expanded.
7.3.3.2.1 Description of the interfaces for user-defined
user function modules

Variant 1:

Variant 2:
7.3.3.2.2 Additional parameters for variant 2 for transferring hierarchy restrictions

With hierarchy restrictions, an entry for the 'COMPOP' = 'HI' (for

hierarchy) field is created at the appropriate place in table I_T_RANGE (for FEMS 0) or
I_TX_RANGETAB (for FEMS > 0), and the 'LOW' field contains a number that can be
used to read the corresponding hierarchy restriction from table I_TSX_HIER, using field
'POSIT' . Table i_tsx_hier has the following type:

Variant 3:

SAP advises against using this interface.

The interface is intended for internal use
use only and only half of it is given here.
Note that SAP may change the structures used in the interface.

7.3.3.3 Method for determining the correct variant for the interface
The following list describes the procedure for determining the correct
interface for the user-defined
defined function module. Go through the list from top to the bottom.
The first appropriate case is the variant that the user should use:

If Pack RFC is activated: Variant 1

If SID Support is deactivated: Variant 2