<Insert Picture Here>

OTM Data Management Using Python

Eric Rosenbloom (eric.rosenbloom@oracle.com)
Solutions Director, Consulting
August 2012
 OTM Data Management Using Python

• We show you how to:

• Use ClientUtil.py to export/import CSV
• Write a Python script that uses ClientUtil.py to automate an
OTM to OTM data replication process using CSV approach
• Use ClientUtil.py to export/import db.xml
• Use ClientUtil.py to push a GLogXML document to OTM
• Time permitting, touch briefly upon using Python to perform
XML data transformations
 Target Audience

• This talk is meant primarily for technically inclined

individuals responsible for OTM data management
 Background on Python

• Python is an interpreted scripting language that

comes pre-installed on your OTM application server
along with OTM specific utilities like ClientUtil.py
• If you have no prior experience with Python, suggest
doing the python tutorial at
http://docs.python.org/tutorial/
• Examples presented utilize Python that is pre-
installed on an OTM application server.
• However, they will work as well when installed on a laptop or
desktop
Diagram: ClientUtil.py in the middle
 CSV, db.xml, and GLogXML

• CSV = “Comma Separated Variable”. Standard file

format whereby fields are delimited by commas.
Supported by Microsoft Excel
• Db.xml is a “database-centric” xml format where there
is a close mapping between the xml and the
underlying table and columns
• GLogXML is described by the GLogXML.xsd schema.
Specifies XML representation for OTM business
objects like Item, Location, Release,
PlannedShipment, etc
 PYTHONPATH environment variable

• On your OTM application server, set PYTHONPATH

environment variable to include directory where OTM
Python utilities are installed:

export PYTHONPATH=$GLOG_HOME/utils/integration/python
Contents of utils/integration/python
 Running ClientUtil.py for help
• Run ClientUtil.py with no arguments to get help text:
 ClientUtil.py to CSV export
python ClientUtil.py
-command csvExport
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-tableName ACTIVITY
-whereClause "rownum < 2"
-localDir .
-localFileName myfile.csv
-excludePublic N
 Myfile.csv contents

ACTIVITY
ACTIVITY_GID,ACTIVITY_XID,ACTIVITY_NAME,DOMAIN_NAME,INSERT_USER,INSERT_D
ATE,UPDATE_USER,UPDATE_DATE
EXEC SQL ALTER SESSION SET NLS_DATE_FORMAT = 'YYYYMMDDHH24MISS'
"RECEIVE","RECEIVE","RECEIVING FREIGHT","PUBLIC","DBA.ADMIN","20110130185513",,
 ClientUtil.py to CSV export: Ex #2
python ClientUtil.py
-command csvExport
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-tableName LOCATION
-whereClause "1=1"
-localDir .
-localFileName location.csv
-excludePublic Y
 Export a subset of columns
• The csvExport command exports all columns in the
table
• If you want a subset of the columns in the table, use
the csvQuery command instead of the csvExport
command
 CSV query example
python ClientUtil.py
-command csvQuery
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-sqlQuery "SELECT location_gid FROM location"
-localDir .
-localFileName myfile.csv
 Myfile.csv contents
SELECT location_gid FROM location
LOCATION_GID
"PHILLY"
"NYC"
,
 CSV Import Example: data content

LOCATION
LOCATION_GID,LOCATION_XID,COUNTRY_CODE3
_GID,DOMAIN_NAME
GUEST.SIG2012,SIG2012,USA,GUEST
,
 CSV import example: command-line
python ClientUtil.py
-command csvImport
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-localDir .
-localFileName new_location.csv
-xvalidate Y
-csvUtilCommand ii
 CSV import example: xml output
<?xml version="1.0" encoding="UTF-8"?>
<CSVUtilServletHelper>
<CSVUtil>
<Command>ii</Command>
<DataDir>/home/oracle/otm62/temp/upload/</DataDir>
<DataFileName>Transmission1338186382454.csv</DataFileName>
<ExcludePublic>true</ExcludePublic>
<ProcessCSV>
<xvalidate>true</xvalidate>
<DatabaseGlobalName>CONSAPP06.US.ORACLE.COM</DatabaseGlobalName>
<TableName>LOCATION</TableName>
<ColumnList>LOCATION_GID,LOCATION_XID,COUNTRY_CODE3_GID,DOMAIN_NAME</ColumnList>
<sqlString>insert into LOCATION (
LOCATION_GID,LOCATION_XID,COUNTRY_CODE3_GID,DOMAIN_NAME) values (?,?,?,?)</sqlString>
<ProcessCount>1</ProcessCount>
<ErrorCount>0</ErrorCount>
<SkipCount>0</SkipCount>
</ProcessCSV>
</CSVUtil>
</CSVUtilServletHelper>
 Data Replication Script

• Now let’s write our own “copytabs.py” Python script to

use ClientUtil to facilitate replicating data from OTM
instance A to OTM instance B
copytabs.py: command line
copytabs.py: includes
copytabs.py: main
copytabs.py: Main class
copytabs method (all in list)
copytab method (copy one table)
 Rate Data Replication Script

• Now let’s write a shell script that uses copytabs.py to

copy rate data from one OTM instance to another
copyrates.sh
 ClientUtil.py to single table db.xml
export
python ClientUtil.py
-command xmlQuery
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-sqlQuery "select * from activity"
-rootName ACTIVITY
-localDir .
-localFileName activity.db.xml
single table db.xml export
 ClientUtil.py to multi-table db.xml
export
python ClientUtil.py
-command xmlQuery
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-sqlQuery "select location.*, (cursor (select
location_role_profile.* from location_role_profile
where location_role_profile.location_gid =
location.location_gid)) as location_role_profile from
location where domain_name = ‘GUEST’ "
-rootName LOCATION
-localDir .
-localFileName location.db.xml
multi-table db.xml export
 ClientUtil.py db.xml import

python ClientUtil.py
-command xmlImport
-hostname consapp07
-username GUEST.ADMIN
-password CHANGEME
-transactionCode IU
-localDir .
-localFileName location.db.xml
db.xml import
Sample “Location” GLogXML
document
 ClientUtil.py: Push GLogXML
document
python ClientUtil.py
-command sendXMLFileViaHttpPost
-url
http://consapp07/GC3/glog.integration.servlet.WMSer
vlet
-fileName Location.xml
Sample Command Line and Response
Login to OTM
Query Location Manager
 Python for XML Data Transformations

• Performing XML data transformations in Python is

relatively straight-forward
• Helper methods in commonTools.py
• Sample script SampleLocationTransform.py
SampleLocationFlatFile.loc
SampleLocationTransform.py
 Summary

• You have learned

• How to get started with Python
• About CSV, db.xml, and GLogXML
• About the various Python tools that come installed with OTM
• How to use ClientUtil.py to export/import CSV and db.xml
• Intro to writing your own Python scripts
• Use ClientUtil.py to
• export/import db.xml
• Push GLogXML to OTM
• Introduction to using Python for XML data transformations
 If you want to learn more…

• Reference the OTM Data Management Guide:

• http://download.oracle.com/docs/cd/E20111_01/otm/acrobat/
datamanagement.pdf
If you want to learn more…