Document: Oracle Data Integrator 11g (11.1.

1) Chapter--VIII Designing Interfaces Description:

(This document describes the basic concepts of the ODI

interface, the mapping, joins and knowledge Modules)

History:
Version Description Change Author Publish Date

0.1

Initial Draft

Gourav Atalkar

19-Jan-2011

Reviewed By: Amit Sharma

Business Intelligence Solution Providers

Designing Interfaces

1) Multiple-Source Datastores
Populating a target datastore from multiple-source datastores has a wide range of uses. There should be a relationship between the tables, which is then expressed as a join. If any source datastores are unjoined when you try to execute an interface, ODI displays an error. Joins are made in two ways: If references such as foreign keys are in the source datastores, these are automatically converted into joins in the interface. Alternatively, you can create joins manually by using the drag-and-drop functionality in the diagram. This is necessary for datastores with no references defined.

i)

Creating a Join Manually

To manually create a join, you drag a column from one datastore onto a column in another datastore. ODI creates a simple join, linking rows where the values in these two columns are identical. To express more complex relationships, you modify the join expression manually. Thus, you can incorporate database functions such as arithmetic or string-handling functions as more advanced ODI functionality.

Business Intelligence Solution Providers

Designing Interfaces

The best practice is to use the Expression Editor to avoid making errors in code.

After creating the join expression, you should have its syntax checked by the DBMS, if possible. To do this, click the Check Expression button next to the join expression box.

Business Intelligence Solution Providers

Designing Interfaces

ii)

Advanced Joins

Simple joins, as seen earlier, feature two similar datastores from the same model, and thus share the same technology. However, you can create joins between datastores in different models (for example, two database tables in different schemas), and even across two servers that use different technologies. In these cases, ODI migrates the data to a staging area and performs the join there. Similarly, you can have joins between three or more tables. To do this, drag column names from other datastores in the diagram onto the join expression box. iii) Types of Joins

The four types of joins are cross, inner, outer, and natural. A cross-join retrieves all possible row combinations between the two tables, without any filtering. In an inner join, records from two tables are combined and added to a querys results only if the values of the joined fields meet the specified criteria. An outer join returns all the rows from the joined tables whether or not there is a matching row between them. In the three types of outer joinsleft, right, and fullthe type indicates the source of the main data.

Business Intelligence Solution Providers

Designing Interfaces

When you use a left outer join to combine two tables, all the rows from the left table are included in the results. A right outer join is the same as a left outer join, but in the other way. Rows from the left table can be included more than once. A full outer join retrieves all the rows from both joined tables. It returns all paired rows where the join condition is true, and the unpaired rows from each table concatenated with NULL rows from the other table. Effectively, it returns any rows that would be returned by either a left or right outer join. iv) Setting Up a Join

The SQL join expression expresses the join, written in SQL. A valid SQL expression involving qualified column names from at least two different datastores is acceptable. Note that the word JOIN is never used. You can also launch the Expression Editor, which helps you create more complex joins and avoid mistyping names. The join type check boxes enable you to choose between a left, right, natural, or cross join. After you click the Validate Expression button, when available, the expression is sent to the DBMS for syntax checking. To avoid unpleasant surprises later, perform the syntax checking when you finish a join. There are also some more advanced options. For example, the Execution location determines whether the joins are executed on the source or staging area server. This choice can be important for performance or compatibility reasons. v) Lookups

A wizard is available in the interface editor to create lookups by using a source as the driving table and a lookup datastore or interface. These lookups now appear as compact graphical objects in the interface sources diagram. Make sure that your chosen technology supports lookups. Three possible levels of Lookup Support: None | Join | Select Two ways to create lookups from an interface: File > New to open the Gallery and the Table Lookup Wizard Click the Lookup Wizard button.

Business Intelligence Solution Providers

Designing Interfaces

2 1

The lookup wizard has two steps. In the first step, you simply select your driving and lookup tables. In the second step, you define the lookup condition, and you can specify several options. Lookup Limitations: Limitations Left-outer-join lookups will be available only on sources or staging technologies that support them. Expression-select lookups will be available only on sources or staging technologies that support them.

Business Intelligence Solution Providers

Designing Interfaces

vi)

Filtering Data

Creating filters: Filters defined on the datastores in their models are automatically copied into the diagram. You can manually add further filters in the diagram. Filters are used in interfaces to reduce the amount of data retrieved from the source datastores. Filters are not constraints. Non matching data is not erroneous. Defining a Filter Manually 1. Drag a column from one data source onto the diagram. A funnel representing the filter appears in the diagram. In the filter expression field, the column name appears. 2. Modify the filter expression to implement your filtering rule. You can call the Expression Editor. 3. Validate the expression if possible. 4. Test the filter: Right-click and select Data.

You should also test the filter by right-clicking it and selecting Data. This shows you the data that matches the filter. You can also select Number of Rows to see the row count. You can do this only when the technology supports filters, and when the filter is on the source. The Execution location specifies the server on which the filtering is performed.

Business Intelligence Solution Providers

Designing Interfaces

vii)

Flow in ODI Interface: It refers to the passage of data between sources of data and integration targets. Flow is the path taken by data from the sources to the target in an ODI interface. The flow determines where and how data is extracted, transformed, and then integrated into the target.

What Defines the Flow? Each interface defines its flow with three factors. These are effectively the choices that you make about how data integration occurs: 1). The most fundamental and important choice is where you put the Staging area. This temporary workspace can be placed on the target server, on a source server, or on a third server, which is neither a source nor the target. 2). Second, you can select the location of individual mappings, filters, and joins. Moving a join to the Staging area, for example, causes all source data to be transferred to the staging area for the join. Moving it back to a source can mean that less data transfer is required. Similarly, you can mark all transformations as active or inactive. Making a transformation inactive also affects the flow of data. 3). You must select the knowledge modules that will generate the code to implement the integration. Different knowledge modules take advantage of different ways of moving data. Thus, knowledge modules define how data is loaded, transformed, and integrated.

Business Intelligence Solution Providers

Designing Interfaces

viii) Staging Area A Staging area is a separate, dedicated area in an RDBMS where ODI creates its temporary objects and executes some of your transformation rules. The Staging area is where most of the transformation and error checking is usually performed. After those tasks are performed, you just copy, or load, the data into the target server. The Staging area is a special area that you create when you set up a database in ODI. ODI creates temporary tables in the Staging area, and uses them to perform data transformation. You can place the Staging area on your source server, your target server, or another server altogether. However, the best place for the Staging area is usually on the target server. This gives you the greatest scope for data consistency checking and minimizes network traffic The Staging area may be located: On the Oracle target database (default) On a third RDBMS database server On the Sybase source database The Staging area cannot be placed on the file source data server. This data server is not an RDBMS You must select an RDBMS technology for the Staging area. Other data servers, such as files, message-oriented middleware (MOM), Lightweight Directory Access Protocol (LDAP), or online analytical processing (OLAP) cannot be used this way. If the target of your interface is a non relational technology, you must move the Staging area to another schema. Specify the Staging Area Click the interfaces Definition tab. To place the Staging area on the target, deselect the Staging Area Different from Target check box. Otherwise, select that option, then select the logical schema that will be used as the Staging area. Click the Flow tab to see the new flow.

Business Intelligence Solution Providers

Designing Interfaces

ix)

Selecting the Knowledge Module

If processing happens between two data servers, a data transfer KM is required. Before integration (Source > Staging Area) Requires a Loading Knowledge Module (LKM), which is always multi technology At integration (Staging Area > Target) Requires a multi technology Integration Knowledge Module (IKM) If processing happens within a data server, it is performed entirely by the server. A single-technology IKM is required. No data transfer is performed. Four possible arrangements:

If your staging area is on the target server, you need only a single-technology IKM. If the staging area is on the source server, you do not need an LKM at all, but you need a multiple-technology IKM to transfer the data to the server. Note that if you have more than one source on different data servers, you need multiple-technology LKMs to collect data from the other sources. Lastly, if the staging area, source, and target are all on the same data server, you do not need an LKM. A single-technology IKM is sufficient.

Business Intelligence Solution Providers

Designing Interfaces

10

Identifying IKMs and LKMs Naming conventions: LKM <source tech> to <target tech> [(<method>)] IKM <tech1> [to <tech2>] <strategy> [(<method>)] [To <tech2>] indicates a multiple-technology IKM. The technology name SQL stands for any SQL-enabled technology. It can be used for most technologies that support the SQL-89 syntax. <Method> is the physical means used to transfer the data. Database loader to perform transfers Database-specific method to merge data <Strategy> is the general technique used to integrate data into a target. Append or control append Incremental update Slowly changing dimensions
Specify an LKM

Business Intelligence Solution Providers

Designing Interfaces

11

Specify an IKM

Common KM Options Some of the typical options are as follows: INSERT and UPDATE are two options that apply to many IKMs. If you turn off INSERT, no rows will be added, but rows may still be updated, if applicable. Turning off UPDATE does the reverse. Typically, the Incremental Update IKMs have both options, whereas the Append IKMs have only the INSERT option. COMMIT causes ODI to generate COMMIT statements between inserts. Sometimes it is useful to turn this option off, to cause several steps to happen in the same transaction. One use of this is to delay the execution of triggers on Microsoft SQL Server. FLOW CONTROL specifies whether data is checked for correctness before integration. This option enforces the constraints that you defined in ODI, such as uniqueness, reference, and conditional constraints. Only single-technology IKMs have this option. STATIC CONTROL specifies whether data is checked for correctness after being loaded. This option performs almost the same checks as the flow control option, but on the data that has been integrated. TRUNCATE DELETE ALL and DELETE TEMPORARY OBJECTS enable you to specify your preferences for truncating or deleting target data and temporary objects.
Business Intelligence Solution Providers | Designing Interfaces 12