External Tables in Oracle Database 11g

Thanks to Oracle:
http://download.oracle.com/otndocs/products/database/enterprise_edition/utilities/pdf/xtables_pr
eproc11g_1106.pdf

Examples Using the New PREPROCESSOR Access Parameter:

The following two examples demonstrate use of the preprocessor feature. In both examples,
input data in compressed format is preprocessed to uncompress it and then sent to the
ORACLE_LOADER access driver in uncompressed format. Example 1 specifies the
preprocessor program directly on the PREPROCESSOR clause. Example 2 specifies the
preprocessor program within a shell script, because the program uses additional arguments.
Example 1
This example requires you to take several distinct steps, including: supplying a preprocessor
program; creating a simple data file, directory objects, and an external table; and then querying
from that table to verify the data was successfully uncompressed.
Step 1 Supply a preprocessor program
From within the shell, use the following command to copy the preprocessor program (in this
case, zcat) to some other directory on your system.
% /bin/cp /bin/zcat /somedirectory/bin/zcat
Using the Preprocessor Feature with External Tables in Oracle Database 11g Release 2
3
Step 2 Create a simple data file in compressed format
In another directory, create a simple data file named foo.dat containing the words Hello World
and then use the gzip executable to compress the foo.dat file into a file named foo.dat.gz. Verify
that the compressed file has been created by issuing an ls -l command.
% echo Hello World > foo.dat
% gzip foo.dat
% ls -l foo.dat.gz
-rw-rw-r-- 1 oracle dba 40 Oct 2 15:10 foo.dat.gz
Step 3 Create directory objects and grant required privileges
Directory objects must be created for the directories that hold the preprocessor programs and
data files. For this example, the necessary privileges on those directories are granted to user
scott. The creation of directory objects and granting of privileges on them to only certain users is
necessary for security reasons (see BEST PRACTICES). In the following examples, replace the
name somedirectory with the name of the directory to which you copied the zcat program in
step 1. Replace the name somedirectory1 with the name of the directory in which you created
the foo.dat.gz file in step 2.
SQL> create or replace directory execdir as '/somedirectory/bin';
Directory created.
SQL> create or replace directory data_dir as '/somedirectory1';
Directory created.
SQL> grant read, execute on directory execdir to scott;

Grant succeeded.
SQL> grant read, write on directory data_dir to scott;
Grant succeeded.
Step 4 Create an external table named xtab
SQL> CONNECT scott/tiger
Connected.
SQL> CREATE TABLE xtab (COL1 varchar2(2000))
2 ORGANIZATION EXTERNAL (
3 TYPE ORACLE_LOADER
4 DEFAULT DIRECTORY data_dir
5 ACCESS PARAMETERS (
Using the Preprocessor Feature with External Tables in Oracle Database 11g Release 2
4
6 RECORDS DELIMITED BY NEWLINE
7 PREPROCESSOR execdir:'zcat'
8 FIELDS (COL1 char(2000)))
9 LOCATION ('foo.dat.gz'))
10 REJECT LIMIT UNLIMITED
11 PARALLEL 2;
Table created.
Step 5 Select from the external table to verify the data is uncompressed
SQL> SELECT * FROM xtab;
COL1
Hello World
1 row selected.
Example 2
The following example demonstrates using a shell script to uncompress the data. Shell scripts are
necessary when preprocessor programs require additional arguments. Note that /bin/zcat and
/bin/gunzip c are functionally equivalent.
Step 1 Create the preprocessor shell script
% echo /bin/gunzip c $1 > uncompress.sh
% chmod +x uncompress.sh
% cp uncompress.sh /somedirectory/bin/uncompress.sh
Note the following when creating a preprocessor shell script:
The full path name must be specified for system commands (for example, gunzip)
The data file listed in the external table LOCATION clause should be referred to by $1. (On
Windows systems, the LOCATION clause should be referred to by %1.)
On Windows systems, the first line in the .bat file must be the following:
@echo off
Otherwise, by default, Windows will echo the contents of the batch file (which will be treated
as input by the external table access driver).
Using the Preprocessor Feature with External Tables in Oracle Database 11g Release 2
5
Make sure the preprocessor shell script has EXECUTE permissions
Steps 2 and 3 remain the same as in Example 1.
Step 4 Create an external table named xtab

SQL> CONNECT scott/tiger

Connected.
SQL> drop table xtab;
Table dropped.
SQL> CREATE TABLE xtab (COL1 varchar2(2000))
2 ORGANIZATION EXTERNAL (
3 TYPE ORACLE_LOADER
4 DEFAULT DIRECTORY data_dir
5 ACCESS PARAMETERS (
6 RECORDS DELIMITED BY NEWLINE
7 PREPROCESSOR execdir:'uncompress.sh'
8 FIELDS (COL1 char(2000)))
9 LOCATION ('foo.dat.gz'))
10 REJECT LIMIT UNLIMITED
11 PARALLEL 2;
Table created.
Step 5 Select from the external table to verify the data is uncompressed
SQL> SELECT * FROM xtab;
COL1
Hello World
1 row selected.
Data Load Using External Tables
For more details:
http://orafaq.com/node/848
http://www.packtpub.com/article/external-tables-oracle-10g-11g-database-part1

External Table:

An External Table is basically a file that resides on the server side, as a regular flat
file or as a data pump formatted file. The External Table is not a table itself; it is an
external file with an Oracle format and its physical location. This feature first
appeared back in Oracle 9i Release 1 and it was intended as a way of enhancing the
ETL process by reading an external flat file as if it was a regular Oracle table. On its
initial release it was only possible to create read-only External Tables, but, starting
with 10git is possible to unload data to External Tables too.

Simple Example of Data loading using External Table:

ext_tab_dir --> Name of the Directory
xx_reports_access_rmg --> Name of the custom table
xx_access_external_tab --> Name of the external table
xx_reports_access.csv --> Name of the data file

CREATE OR REPLACE DIRECTORY ext_tab_dir AS

'/u02/DEV/apps/apps_st/appl/custom_top/12.0.0/bin'

CREATE TABLE son_access_external_tab

(user_name
VARCHAR2(100 BYTE),
poo_org_id
NUMBER,
project_owning_org VARCHAR2(240 BYTE),
hcc_org_id
NUMBER,
home_cost_center_org VARCHAR2(240 BYTE),
attribute1
VARCHAR2(100 BYTE),
attribute2
VARCHAR2(100 BYTE),
attribute3
VARCHAR2(100 BYTE),
attribute4
VARCHAR2(100 BYTE),
attribute5
VARCHAR2(100 BYTE),
attribute6
VARCHAR2(100 BYTE),
attribute7
VARCHAR2(300 BYTE),
attribute8
VARCHAR2(500 BYTE))
ORGANIZATION EXTERNAL
(TYPE oracle_loader
DEFAULT DIRECTORY ext_tab_dir
ACCESS PARAMETERS (RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
MISSING FIELD VALUES ARE NULL)
LOCATION ('xx_reports_access.csv'))
REJECT LIMIT UNLIMITED;

SELECT * FROM xx_access_external_tab

SELECT * FROM xx_reports_access_rmg

-- Simple Insert
INSERT INTO xx_reports_access_rmg
(user_name, poo_org_id, project_owning_org, hcc_org_id,
home_cost_center_org)
(SELECT user_name, poo_org_id, project_owning_org, hcc_org_id,
home_cost_center_org
FROM xx_access_external_tab)

-- Insert with your custom logic

INSERT INTO xx_reports_access_rmg
(user_name, poo_org_id, project_owning_org, hcc_org_id,
home_cost_center_org)
(SELECT user_name, LTRIM (RTRIM (poo_org_id)),
DECODE
(ASCII (SUBSTR (project_owning_org,
(LENGTH (project_owning_org)
),
LENGTH (project_owning_org)
)
),
13, SUBSTR (project_owning_org,
1,
LENGTH (project_owning_org) - 1
),
project_owning_org
) project_owning_org,
LTRIM (RTRIM (hcc_org_id)),
DECODE
(ASCII (SUBSTR (home_cost_center_org,
(LENGTH (home_cost_center_org)
),
LENGTH (home_cost_center_org)
)
),
13, SUBSTR (home_cost_center_org,
1,
LENGTH (home_cost_center_org) - 1
),
home_cost_center_org

) home_cost_center_org
FROM xx_access_external_tab)

SQL*Loader
SQL*Loader is a high-speed data loading utility that loads data from external files
into tables in an Oracle database. SQL*Loader accepts input data in a variety of
formats, can perform filtering, and can load data into multiple Oracle database
tables during the same load session.
SQL*Loader provides three methods for loading data: Conventional Path Load, Direct
Path Load, and External Table Load.

External Tables
A feature has been added to external tables that allows users to preprocess input
data before it is sent to the access driver. The ability to manipulate input data with a
preprocessor program results in additional loadable data formats, which greatly
enhances the flexibility and processing power of external tables.
The types of preprocessor programs that can be used are versatile, ranging from
system commands, user-generated binaries (for example, a C program), or usersupplied shell scripts. Because the user supplies the program to preprocess the
data, it can be tailored to meet the users specific needs. This means that the
number of loadable formats is restricted only by the ability to manipulate the
original data set.

FLASHBACK command
Undo the DROP command using FLASHBACK command
Example:
CREATE TABLE xx_drop_test (ID NUMBER,attribute1 VARCHAR2(10))
SELECT *
FROM xx_drop_test
SELECT *
FROM all_objects

WHERE object_name = 'XX_DROP_TEST' AND object_type = 'TABLE'

-- 1 row returned
DROP TABLE xx_drop_test
COMMIT;
SELECT *
FROM xx_drop_test
--Table or view does not exists
SELECT *
FROM all_objects
WHERE object_name = 'XX_DROP_TEST' AND object_type = 'TABLE'
-- No rows returned

-- To view all dropped tables

SELECT *
FROM user_recyclebin
WHERE original_name = 'XX_DROP_TEST' AND TYPE = 'TABLE'

--FLASHBACK TABLE <TABLE_NAME> TO BEFORE DROP

FLASHBACK TABLE xx_drop_test TO BEFORE DROP

--FLASHBACK TABLE <TABLE_NAME> TO BEFORE DROP RENAME TO

<NEW_TABLE_NAME>
FLASHBACK TABLE xx_drop_test TO BEFORE DROP RENAME TO
xx_drop_test
-- Ignore the message: ORA-27231: Message 27231 not found;
product=RDBMS; facility=ORA
-- To delete table permanantly, we cann't flashback it. We can't find in
user_recyclebin
--DROP TABLE <TABLE_NAME> PURGE
DROP TABLE xx_drop_test PURGE
Simple steps to do explain plan - Performance Tuning

For my reference:
1.

Set autotrace on

2.

Set autotiming on

3.

Explain plan for select * from pa_projects_all

4.

It will display elapsed time; to open the trace file write following command

5.

Select * from table(dbms_xplan.display);

6.

Trace file will be diplayed; Analyze the cost and the indexes used in table

7.

To do fine tuning we use HINTS (search in google Ex: /*+ all_rows*/ etc..

8.
9.

Use hints before columns (Ex: Explain plan for select /*+all_rows*/
project_id,segment1,creation_date from pa_projects_all)
Search in web : Index Hint, Bitmap Index, Optimization (Cost based, Rule Based)etc

10.Simple Example of Ref Cursor

11.
CREATE OR REPLACE PACKAGE test_ref
IS
TYPE ref_cur IS REF CURSOR;
PROCEDURE print_project;
END;

CREATE OR REPLACE PACKAGE BODY test_ref

IS
PROCEDURE print_project
IS
c_test
ref_cur;
l_project_id NUMBER;
l_pm
VARCHAR2 (100);
c_pm
ref_cur;
l_name
VARCHAR2 (200);

BEGIN
OPEN c_test -- We can open this ref cursor dynamically using any query here
FOR
SELECT project_id, NAME
FROM pa_projects_all
WHERE TRUNC(SYSDATE) BETWEEN TRUNC(start_date) AND
TRUNC(NVL(completion_date,SYSDATE))
AND project_id = 530;
FETCH c_test
INTO l_project_id, l_name;
DBMS_OUTPUT.put_line (' Project ID : ' || l_project_id);
DBMS_OUTPUT.put_line (' Project Name: ' || l_name);
OPEN c_pm -- We can open ref cursor at runtime dynamically using any query
FOR
SELECT papf.full_name
FROM pa_project_players ppp,
pa_project_role_types pprt,
per_all_people_f papf
WHERE project_id = l_project_id
AND papf.person_id = ppp.person_id
AND ppp.project_role_type = pprt.project_role_type
AND TRUNC(SYSDATE) BETWEEN TRUNC(papf.effective_start_date) AND
TRUNC(NVL(papf.effective_end_date,SYSDATE))
AND TRUNC(SYSDATE) BETWEEN TRUNC(ppp.start_date_active) AND
TRUNC(NVL(ppp.end_date_active,SYSDATE))
AND TRUNC(SYSDATE) BETWEEN TRUNC(ppp.start_date_active) AND
TRUNC(NVL(ppp.end_date_active,SYSDATE))
AND pprt.meaning LIKE 'Project Manager';
FETCH c_pm
INTO l_pm;
DBMS_OUTPUT.put_line (' Project Manager : ' || l_pm);
CLOSE c_pm;
CLOSE c_test;
END print_project;
END test_ref;