Documente Academic
Documente Profesional
Documente Cultură
O racle D ata Integrator perform s data transform ation/validation using below steps.
Extract
Load
Transfer
Traditional tools are used to do Extract, Transfer and Load i.e. ETL . B ut, O D I uses
ELT. The advantage of ELT is that data gets m oved from the Source system on an as
is basis, w ithout adding extra load to the Source system . H eavy-duty validation and
transform ation happens in the O D I server. This w ill enable O D I to handle large sets
of data. A s data grow s,it can be m ade scalable to m eet the grow th.
3.1
3.2
W hat is E LT ?
ELT m eans Extract data from source, Load data into staging area, and perform
Transform ation as required.
3.2.1 E xtract
In this step, the data gets extracted from one or m ore source system s running on different
operating system s and databases. The O perating system can be W indow s Server, U N
IX , Linux etc.,the database can be a SQ L server,O racle,Flat File,Excel spreadsheet etc.,
3.2.2 L oad
In this step, extracted data gets loaded into a data w arehouse. In general it gets
loaded into a staging area for further validations.
3.2.3 Transform
In this step, the extracted data gets validated to ensure that the dow nstream reporting
w il be accurate.D uring this step,surrogate keys get assigned to the records.
4 O D I Studio O verview
In this section you w ill learn to know the different com ponents of O racle D ata
Integrator (O D I).
4.1 D esigner
This is the m ost frequently used com ponent of O racle D ata Integrator used by D eveloper. D
eveloper uses this section to define Projects, M odels, ETL M appings, Variables, K now ledge
base,Load Plan etc.,W ork repository stores the D esigner m etadata.
4.1.1 Projects
This section of the O racle D ata Integrator enables D eveloper to create, Project
specific folders. A ll Variables and K now ledge m odules defined w ith a Project
are private to its Project only. A s soon as a Project folder gets created, it w ill
create the below sub-sections by default.
4.1.1.1 Packages
The O D I packages are used to group m ultiple O D I objects such as Variable, M apping,
and Procedures in a specific sequence of execution. U sing Packages, one can evaluate
True or False and take a different path based on the U sers need. Package is a diagram m
atic representation of Jobs.The w orkflow of Jobs gets defined by drag and drop scenarios.
4.1.1.2 M appings
M apping is an interface w hich consists of Source and Target w ith validation
logics built in.The core logic of ETL is defined in the m apping section.
4.1.1.3 R eusable M appings
R eusable m appings are sim ilar to regular m apping. In addition to m apping, R
eusable m apping can be used w ith a m apping. R eusable m apping has both
Input and O utput param eters.
4.1.1.4 Procedures
Procedure is a perform ing logic, w hich is not suitable for ETL M apping. Even though
Procedure can be used to validate and transform data, ETL m apping should be the first
choice to do data validation and transform ations. Procedure w ill support m ultiple
technologies such as O perating System com m and, FTP, JM S com m and etc., O D I
Procedures can be called w ith in an O D I Package. O D I Procedure can have m ore than one set
of com m ands. M ultiple com m ands are processed sequentially. A Procedure consists of
4.1.1.6 Sequences
Sequences enable users to autom atically generate sequence num bers w ith a
specific increm ental value. Sequence can be an O D I sequence or it can be
based on a database. The sequence can be used in ETL m apping.
4.1.1.7 U ser Functions
U ser functions are used to create Project/System specific custom functions. It
can be defined at a Project level or at a G lobal level. The O D I function can be
used in O D I M apping and O D I Procedures. U ser functions enable easy m
aintenance of com m only used functions.
4.1.1.8 K now ledge M odules
A K now ledge M odule is a set of generic code that perform s a specific task. K M uses O D I
specific syntax to reference variables.O racle provides several out of the box K M to enable
faster developm ent of ETL M appings. O nce defined in the K M , it can be used
in m ultiple ETL m appings.K M can handle m ultiple technologies.
4.1.1.9 M arkers
M arker is used to flag O D I objects. This w ill enable O D I objects to be
grouped. O racle provides 3 out of the box m arkers as show n below :
M odel stores the structure of Source and Target objects such as Table, Files. U nless it is
defined in the M odel, the objects cannot be used in ETL m apping. D atabase table
structure can be added to the M odel using reverse engineering. M odel does not store any
data. It stores the structure of the object only. U sing M odel, data can be queried. M odel
can be used to add additional constraints to the objects,other than defined in the database.
4.2 O perator
U se the O perator section is used to m onitor the ETL jobs. A fter executing a
scenario, a job can be m onitored thoroughly in the O perator section. O perator has
the flexibility of locating a Job by D ate,A gent,Session,Status,K eyw ord and U ser.
4.2.4 Scheduling
This section show s all scheduled Load Plans by A gent and all scheduled jobs.
4.3 Topology
Topology is part of a M aster repository. U sing Topology navigator, you can m anage
the sever connection,database connection,physical and logical connections etc.,
To im port all Table structures,select Standard check Table option and enter %
for M ask as show n below
C lick on R everse Engineer to extract the table structure from the database.
D rag and drop SA LES_R EP_D IM _S sequence to the editor as show n below :
C lick O K
C lick on JO IN
G o to Property W indow and click C onnector Point You
can review the IN PU T and O U TPU T connectors.
Save the
ELT R un
the ELT
A fter the above 2 steps,the screen w ill look like as show n below :
C lick on Test C
onnection
C lick Test
C lick O K
The connection tested successfully.N ow w e are done w ith connecting to the desktop.
R ight click
Pick N ew Physical Schem a
Set the directory to D :\ as show n below :
Save
You get the follow ing w
arning m essage.C lick O K
.W e w ill attach logical
schem a later.
13.5 M odel
G o to D esigner - > M odels
C lick on
C lick
on
R ight C lick
Pick N ew M odel
D efine a M odel as show n below :
R ight click
Pick N ew D atastore
D efine a data store as show n below :
C lick on A ttributes.
W e are going to define the fields in the file.
Save
The designer w ill look like as show n below :
C heck K ey
U ncheck U pdate
C lick on Physical
C lick on PA RT _D IM _TA R G ET
M ake the Integration know ledge M odule to IK M O racle Increm ental U pdate.
R un the m
apping.
19 O D I Variables
You can define variables and it can be populated using Select statem ents. These
variables can be used in Packages as runtim e variables.
G o to D esigner
R ight C lick on Variables
Pick N ew Variable
D efine Variable
N am e: FIR ST _VA R IA B LE
D ata Type: A lphanum eric
K eep H istory: A ll Values
G o to R efreshing
Select Schem a
Save
C lick R efresh button to run the SQ L