g:\prints\ebs_patching\12.1.3_ebs_patching.

txt
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Applying Application Patches to Oracle EBS 11i/R12 Applications
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++:

Step 1: Before applying a patch you must check whether the patch is applied or not:

Method 1:

For this we query the database:

Sqlplus apps/<apps_password>

Sql> select bug_id, bug_number from ad_bugs where bug_number='&num';

Sql> select patch_type, patch_name from ad_applied_patches where patch_name='&num';

Method 2:

Checked with OAM

a) Connect to OAM
b) Go to System Administrator --> OAM-->Dashboard --> Site map --> Maintenance -->
Applied patches
c) Enter patch id and press �Go�
d) See if patch was returned

Step 2: Download the patch:

Example:

www.support.oracle.com

Step 3: Unzip the patch:

Exapmple: Command:

Unzip < patch.zip>

Step 4: Before applying patch:(check invalid objects in DB) and Backup the invalids
object before applying the patch:

Example:

Sql> select name from v$database;

Sql>select owner,count(*) from dba_objects where status='INVALID' group by owner;

Backup the invalids with CTAS:

create table dba_objects_18apr2013 AS select * from dba_objects where

status='INVALID';

Note:
Send outage communication/put mail to business

Step 5: Stop all application services (If we don�t want to close application
services, we have hot patch option:
Note: If it is shared application file system you have to bring down admin tier
service i.e, concurrent node it depends on business requirement.

ps -fu applmgr|grep -i FND|wc -l

If processes not went down, do kill processes and proceed ahead

Step 6: Enable Maintenance mode

Example:

$ adadmin

5. change maintenance mode

To check maintenance mode enable or not use below query

select fnd_profile.value('APPS_MAINTENANCE_MODE') from dual;

If the status:

�MAINT� = MAINTENANCE MODE HAS BEEN ENABLED AND THE USERS WILL NOT BE ABLE TO
LOGIN.

�NORMAL� = MAINTENANCE MODE HAS BEEN DE-ACTIVATED AND THE USERS WILL BE ABLE TO
LOGIN.

Reasons For Enabling Maintenance Mode:

To ensure optimal performance and reduce downtime during patching sessions,

AutoPatch requires that you enable Maintenance mode when you apply a patch.
Enabling this feature shuts down the Workflow Business Events System and sets up
function security so that Oracle Applications functions are unavailable to users.
This provides a clear separation between normal runtime operation and system
downtime for patching.

Note:
Now fire adpatch from VNC server as its process will be created on the server.
You can also use putty session but this process will run at client side. Putty
session will be inactive after 30 mins. if your patch taking more than 30 mins
your session will gone. You have to start adpatch from the start. So it's better
practice to use VNC.

Step 7: Applying patch using �adpatch� (auto patch utility):

FIRE :adpatch
as adpatch will ask for some questions like logfile
name,system,apps,ORACLE_HOME,number of worker,driver file, once the patch is
applied successfully. Please check the logfile for errors and warnings.
Use adpatch by going to patch directory
Go to the patch top directory,where all driver and required ldt files are present
with the application file system owner,makes sure all files have read,write and
execute permission.

As described below:

Note:It is very important to review the readme.txt in unix vi editor before

applying applying patch and follow the instruction as given in the readme.txt file
and apply any pre-requistics patch if required. Autoconfig run also not required
unless it is specified after patch installation in readme.txt file. After this
answer the questions of autopatch. As autopatch finishes its tasks, it writes
timing information to the AD timing report for jobs running in parallel (if any )
and reminds you to run the log files for any errors.
If you don�t see the �autopatch is complete� message at the end of the Autopatch
log file, Autopatch did not complete successfully.
The most important step after autopatch completes is to check the log files for any
errors that may occurred during the patching process. Check the main Autopatch log
file first,then additional log files as necessary.
The default name of main autopatch log file is adpatch.log .The file is located in
$APPL_TOP/admin/<SID>/log

Step 8 : Please check whether patch is applied or not.

Step 9: Disable Maintenance mode again by using 'adadmin' utility:

Step 10: Start application services :

Step 11:After applying PATCH:

step a:Verify the patch is applied successfully:

SQL> select name from v$database;

NAME
---------
TESTDB

SQL> select bug_number,creation_date from ad_bugs where bug_number='5522470';

BUG_NUMBER CREATION_DATE
------------------------------ ---------------
5549427 12-FEB-13

Step b:Run cmclean.sql

Run cmclean.sql from application node by going to

$COMMON_TOP/admin/scipts/TESTDB_ebstest in 11i,where as in R12 goto
$ADMIN_SCRIPTS_HOME or $INST_TOP/admin/scripts
commit;
Note:We run cmclean after clonning also to make sure the node name is updated in
the FND_NODES Table correctly

SQL> select owner,count(*) from dba_objects where status='INVALID' group by owner;

OWNER COUNT(*)
------------------------------ ----------
SYS 1
TEST_USER1 2
TEST_USER2 5
APPS 22

Step c:Check the file versions got changed successfully after applying patch:

strings -a POXWARMB.pls|grep Header =>Can be used to check the file version.

Step d:Do the Health Check of Oracle EBS Application

Note :We can get HOME page URL by using below query:
SQL>Select Home_URL from icx_parameter;

Health check completed successfully by submitting active user Concurrent request

REQUEST ID 28758820.

Step e:Intimate end User

Make sure you intimate the end User or release your application to the end User

do sanity check and release instance for user access .

In sanity check you will submit below two requests:

Go to System adminstrator --->Request--->Run

1)Active User =====>Check View LOG/vIEW OUT

2)Active Responsiblity ===>Check View LOG/vIEW OUT

In Oracle, to apply database patch we use �opatch� ($ORACLE_HOME/OPatch) where as

in order to apply applications patch we use ad utility �adpatch� ($AD_TOP/bin).
Patch Zipped File Details: When you unzip a patch you will see the following
files:readme.txt - This file contain steps to apply the patch, list of
prerequisites should be there on apps Instance (If not apply that patch )
cXXXXXXX.drv c stand for copy driver file , this copies patch content to respective
patch location , driver is like bus driver which provides instruction on work
adpatch need to perform. dXXXXXXX.drv d stand for Database driver & contain content
related to database like creating packages, tables, adding column�. gXXXXXXX.drv
This contain files related to forms , reports, graphics or messages uXXXXXXX.drv
Sometime these three types of files are bundled together into single driver file
called Unified driver file

Maintenance Mode/ Hot Patch: Maintenance Mode is mode of operation introduced with
AD.I.2, in which the oracle application system is made accessible only for patching
activities. Greatly improves performance by minimizing downtime. If you wish to
apply patch without putting applications in maintenance mode (for small patches)
use options=hotpatch with adpatch.

g:\prints\ebs_patching\adop patching.txt
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Online Patching (ADOP) in Oracle EBS R12.2
===========================================:

Online patching is supported by the capability of storing multiple application

editions in the database, and the provision of a dual application tier file system.
At any given point in time, one of these file systems is designated as run (part of
the running system) and the other as patch (either being patched or awaiting the
start of the next patching cycle).

Important Points About ADOP

===========================:

- Adop utility is put under $APPL_TOP_NE/ad/bin. It is a wrapper script which calls

internally the perl script $AD_TOP/bin/adzdoptl.pl which does actual work of
applying the patch.

- The adzdoptl.pl drives the patching cycles, does the patching work and even
executes some java commands when necessary.
- Adop will automatically set its environment as required, but it is the user�s
responsibility to set the environment correctly for any other commands that may be
run. Set the run edition environment whenever executing commands that you intend to
affect the run edition.

- All the phases need to be completed and you can�t skip any of these. For example,
if you try to skip prepare phase, you may get error message like �Apply phase can
only be run while in a patching cycle, i.e. after prepare phase.�

- After an online patching cycle is started, you should not perform any
configuration changes in the run edition file system. Any that are made will not be
propagated and will therefore be lost after cutover is complete.

- You should not attempt to clone an Oracle E-Business Suite system while an online
patching cycle is in progress.

- The prepare, apply, and fs_clone phases all require at least 10GB of free disk
space. All other phases require 1GB of free space. A warning message will be
displayed if less than the needed amount is available.

- The directories where you extracted the patches applied in a given patching cycle
must be retained, in the same location and with the same contents, until the next
prepare phase completes. This is also a requirement for patches applied in a
hotpatch session.

- Maintenance Mode is not needed for online patching, and so Maintenance Mode is
not available in Oracle E-Business Suite Release 12.2.

For applying a patch in R12.2 you need to use adop and run through all below phases
in sequence mentioned below.

1) adop phase=prepare
2) adop phase=apply patches=<patch_number1>,<patch_number2>
workers=<number_of_worker>
3) adop phase=finalize workers=<number_of_worker> (called automatically)
4) adop phase=cutover workers=<number_of_worker>
5) adop phase=cleanup (called automatically)

OR

Running all phases in single command:

adop phase=prepare,apply,finalize,cutover,cleanup
patches=<patch_number1>,<patch_number2>

1) PREPARE PHASE DETAILS

-------------------------:

Used to start a new online patching cycle

How to execute:
A) Set the environment by executing (sourcing) the run file system environment
file:
$ source <EBS install base>/EBSapps.env run
B) Verify envirionment
You can confirm that the environment is properly set by examining the relevant
environment variables:
$ echo $FILE_EDITION
run
$ echo $TWO_TASK
dbSID
C) Download Patches

Download patches to be applied and place then in the $PATCH_TOP directory of your
system. This directory is pre-created by the install in the non-editioned file
system (fs_ne) and should not be changed.

Important: On a multi-node system with non-shared file systems, you must copy the
patch files to each separate $PATCH_TOP directory, so that the patch files are
available from the same location on all nodes.
D) Unzip the patch

$ unzip <patch>.zip
E) Run Prepare Command

Prepare the system for patching by running the following command to start a new
patching cycle:

$ adop phase=prepare
Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Gets patching filesystem ready to apply a patch.

-Checks that ETCC has been run against the database Oracle Home and that all
required patches are applied.
-synchronize filesystems - If any configuration changes have been made, this will
cause the prepare phase to initiate a fs_clone of the current run filesystem to the
patch filesystem.
-run autoconfig if necessary.
-check tablespaces for sufficient free space before starting.
-check APPS_FNDFS listener to make sure concurrent requests will launch.
-launch concurrent request - "Online Patching In Progress" (ADZDPATCH).
-synchronize snapshots between run and patch filesystems.
-Prepare the patch file system to ready for the patching
Create Patch Edition
Submit ADZDPATCH concurrent request to stop any conflict requests to submit from
patch edition.
-Create session ID in ad_adop _sessions table with prepare_status as 'Y' at the end
of prepare.
-Run $AD_TOP/patch/115/bin/txkADOPPreparePhaseSynchronize.pl to synchronize the
patches which have been applied to the run APPL_TOP, but not the patch APPL_TOP.
The script depends on the adop repository for patches that have been applied on the
run APPL_TOP but not the patch APPL_TOP.
-During patching phase, adop will use this tns entry to connect to database.

2) APPLY PHASE DETAILS

----------------------:

In the apply phase, adop applies the specified patches to the system. Patches are
applied to the patch edition of the database and file system.
How to execute:

Example:

$ adop phase=apply patches=1234,7891 workers=8, merge=yes

Enter the APPS password:

Enter the SYSTEM password:
Enter the WLSADMIN password:

Where 1234 and 7891 are the patch numbers

What it will do:

-Merging patches - ADOP determines correct options automatically. (if choosen merge
option)
-Patch repository location.
-ADOP log directories.
-How to run ADOP unattended by passing the required passwords into the ADOP
command.
-How to use adctrl with ADOP.
-Viewing the ADOP patch log file while ADOP is running.
-Apply - Actual patching activity
-Apply the specified patch to Patch File system.
-Update the session ID created by prepare with apply_status as 'P' at the end of
apply.
-Insert entry in ad_adop_session_patches table with bug_number as the patch number.

3) FINALIZE PHASE DETAILS

-------------------------:

The finalize phase will be executed while the application is still online. It is
used to perform any remaining processing that is needed to ensure the system is
ready for the fastest possible cutover.

Used to perform the final patching operations that can

How to execute:

$ adop phase=finalize

Enter the APPS password:

Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Compiles invalids objects in the patch edition.

-Determines DDL statements to execute at cutover.
-Validates that the system is ready for a cutover.
-Gathers statistics for performance.
-Finalize - To prepare DB and APPS for cutover

VERY IMPORTANT 1 : Up to this phase, you can run a special phase called abort,
which will undo the changes made so far in the patching cycle. After cutover is
complete, however, you cannot do this.

VERY IMPORTANT 2 : In an online patching cycle, the requisite JAR files are
initially stored in the $APPL_TOP/admin/<SID>/out directory, and then uploaded into
the database during the cutover phase. Therefore, the out directory must not be
deleted at least until cutover (next phase) is complete.

4) CUTOVER PHASE DETAILS

-------------------------:

Used to perform the transition to the patched environment. Shuts down application
tier services, makes the patch edition the new run edition, and then restarts
application tier services. This is the only phase the involves a brief downtime.

Important: No users should remain on the system during cutover, as there will be a
short downtime period while the application tier services are restarted. Also, any
third-party processes connected to the
old run edition of the database should be shut down, or they will be terminated
automatically.

How to execute:

$ adop phase=cutover

Enter the APPS password:

Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Once started, the abort phase can not be issued.

-Runs the finalize phase if it was missed.
-Cancels concurrent request - "Online Patching In Progress" (ADZDPATCH).
-Shuts down the internal concurrent manager.
-Shuts down the Nodemanager
-Stops any processes that may have been started on the patch filesystem.
-Stops the RUNNING applications tiers.
-Executes the stored cutover DDL statements prepared during the finalize phase.
-Loads jar files into the database.
-Synchronizes snapshots between run and patch filesystems.
-Broadcasts a message to local users that a cutover is taking place and to re-
source their working environment.
-After flipping filesystems for run and patch editions, it will automatically start
admin server and middle tier services.

Note:

1.When you run the cutover phase, you can nolonger issue an adop abort command.
Your changes are permemnant and you will need to restore you instance from backup
prior to your adop cycle.

2. (All users must re-source the environment) This broadcast message is only
recieved by unix users that are logged into the server. If anyone i scconnected
though sqlplus or other from their desktop, they will not see this broadcase
message.

5) CLEANUP PHASE DETAILS

-------------------------:

Important: If you fail to run the cleanup phase explicitly, it will be run
automatically on the next prepare cycle, but this will cause a delay in starting
your next online patching cycle.

This adop phase is used to remove obsolete code and data from old editions.

How to execute:

$ adop phase=cleanup

Enter the APPS password:

Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Various actions are performed during cleanup, including dropping (removing)

obsolete: Crossedition triggers, Seed data, Editioned code objects (covered
objects), Indexes, Columns, Editions.
-Three parameters used using cleanup_mode:
QUICK: cleanup performs removal of obsolete crossedition triggers and seed data.
STANDARD: cleanup performs removal of obsolete crossedition triggers and seed data,
drops (removes) obsolete editioned code objects (covered objects). (DEFAULT MODE)
FULL: cleanup performs maximum cleanup, which drops all obsolete code and data from
earlier editions
-Cleanup_mode=full: does the same as standard mode, and also drops obsolete columns
and old editions.
-If you execute adop phase=cleanup without specifying cleanup_mode=, then you get
the default mode.

THERE ARE TWO SPECIAL PHASES:

A) ABORT PHASE DETAILS

-----------------------:

Abort PHASE is conditional phase. This phase cannot be specified with any other
phase.

If for some reason either the prepare or apply phase failed or gave problems, you
can abort the patching cycle at either of these points by running a special phase
with the Command. The actions taken will be discarded (rollbacked).

IMPORTANT: This abort command is only available upto (but not including) the
cutover phase. After cutover, the system is running on the new edition, and abort
is no longer possible for that patching cycle.
How to execute:

The command to perform this operation is:

$ adop phase=abort

Enter the APPS password:

Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

� Confirms that there is an in-progress online patching cycle, so the abort call is
therefore valid.
� Checks for the existence of a patch edition and drops one if it exists.
� Cancels the ADZDPATCH concurrent program, if it is running.
� Deletes the rows inserted for the pending session ID from the ad_adop_sessions
and ad_adop_session_patches tables.

VERY IMPORTANT: After running abort, a full cleanup must be performed. The cleanup
command is: adop phase=cleanup cleanup_mode=full). This will remove any columns
that were added by the patch but are no longer needed because of the abort. If they
are not removed, they may cause problems in a later patching cycle. Alternatively,
you can run a combined command to abort the patching cycle and perform a full
cleanup:

$ adop phase=abort,cleanup cleanup_mode=full

If any attempt was made to apply patches to the patch edition, after abort you must
run the fs_clone phase (adop phase=fs_clone) to recreate the patch file system.

B) FS_CLONE PHASE DETAILS

-------------------------:

The fs_clone phase is a command (not related to adcfgclone.pl) that is used to

synchronize the patch file system with the run file system. The fs_clone phase
should only be run when mentioned as part of a specific documented procedure.

How to execute:

The fs_clone phase is run using the following command:

$ adop phase=fs_clone

Enter the APPS password:

Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do

-This is a seperate phase, so we have a new session id.

-It is a stand-alone command used for file system cloning.
-Standard cloning (using adcfgclone.pl) cannot be used to synchronize the run and
patch file systems.
-This special phase must be run from the run file system, and cannot be specified
with any other phase.
-It should be before the next prepare phase is run.
-It is an alternate to PREPARE phase

FS_CLONE Phase - alternate to PREPARE phase ?

-Create a new patch file system by cloning the run file system.
-This method is useful if the APPL_TOPs have become very unsynchronized (meaning
that there would be a large number of delta patches to apply).
-It is a heavyweight process, taking a backup of the entire current patch APPL_TOP
and then cloning the run APPL_TOP to create a new patch APPL_TOP.
-A total of least 75 GB of free disk space is required. Also, you will need at
least 25 GB of free space in your temporary directory (typically /tmp).
-As well as being resource-intensive, this method is very time-consuming and
requires several manual actions by the user.
-It should therefore be used only when necessary.
Note:

There are total 5 columns when ADOP patching is running, they are:

ADOP (C.Delta.7)
Session ID: 16
Node: mqmprod
Phase: fs_clone
Log:/u01/fs_ne/220121_0891.log

ADOP ON MULTI-NODE
===================:
In a multi-node environment, one application tier node will be designated as the
primary node. This is the node where the Admin Server is located, and will usually
also be the node that runs Oracle HTTP Server. All other application tier nodes are
designated as secondary nodes.

adop commands are invoked by a user on the primary node. Internally, adop uses
Secure Shell (ssh) to automatically execute required patching actions on all
secondary nodes. You must set up passwordless ssh connectivity from the primary
node to all secondary nodes.

If a node unexpectedly becomes inaccessible via ssh, it will be abandoned by adop,

and the appropriate further actions taken. Consider a scenario where the adop
phase=prepare command is run in a system with ten application tier nodes. The
command is successful on nine nodes, but fails on the tenth. In such a case, adop
will identify the services enabled on nodes 1-9. If they are sufficient for Oracle
E-Business Suite to continue to run normally, adop will mark node 10 as abandoned
and then proceed with its patching actions. If they are not sufficient, adop will
proceed no further.

ORACLE R12.2 ADOP LOGFILES and STATUS CHECK

============================================:

To debug Online Patching issues which utilize the adop (AD Online Patching) tool we
need to collect adop log files located in the <INSTALL BASE>/fs_ne/EBSapps/log/adop
directory
e.g. /u01/PROD/fs_ne/EBSapps/log/adop

Each cycle of adop creates a subdirectory corresponding to the patch session ID,
e.g.

/u01/PROD/fs_ne/EBSapps/log/adop/1
/u01/PROD/fs_ne/EBSapps/log/adop/2
etc..etc

When running adop the on-screen terminal output will mention which adop session ID
is in use.
e.g. /u01/PROD/fs_ne/EBSapps/log/adop/9/apply_20121011_024437

STEP 1: Obtain all of the log files within the session ID directory. It is best to
obtain a zip of the entire directory.

e.g. obtain a zip of /u01/PROD/fs_ne/EBSapps/log/adop/9

The session ID directory will contain:

a) A trace file for each phase;
e.g.
adop_20130316_085026.log
adop_20130316_091340.log
adop_20130316_210950.log

b) Logs grouped in phase directories

e.g.
prepare_20130316_085026
apply_20130316_091340
cutover_20130316_210950

You can see the timestamps match between each log directory and the respective
trace file.
prepare_20130316_085026 directory matches adop_20130316_085026.log
apply_20130316_091340 directory matches adop_20130316_091340.log
etc etc

The same goes for fs_clone activities. i.e. you will see something like this
fs_clone_20130319_233614 --> this is a directory
adop_20130319_233614.log

In almost all cases, when debugging an adop failure the patch log directory will
contain the information you need to determine root cause
e.g.
/u01/PROD/fs_ne/EBSapps/log/adop/2/cutover_20130316_210950/

In some cases the onscreen error will indicate which subroutine has failed. You
will likely find a sub directory containing log files which matches the failing
routine. This should be where you focus your attention
e.g.
/
u01/PROD/fs_ne/EBSapps/log/adop/2/cutover_20130316_210950/VIS_ufc/TXK_CTRL_forceshu
tdown_Sat_Mar_16_21_14_02_2013

STEP 2: Run the following SQL statements

This will show you the status for each adop phase along with its corresponding
session id. This is effectively a history of online patching in an environment.

a) select
ADOP_SESSION_ID,PREPARE_STATUS,APPLY_STATUS,FINALIZE_STATUS,CUTOVER_STATUS,CLEANUP_
STATUS,ABORT_STATUS,STATUS,ABANDON_FLAG,NODE_NAME from AD_ADOP_SESSIONS order by
ADOP_SESSION_ID;

Note:
Ydenotes that the phase is done
Ndenotes that the phase has not been completed
Xdenotes that the phase is not applicable
Rdenotes that the phase is running (in progress)
Fdenotes that the phase has failed
P(is applicable only to APPLY phase) denotes at least one patch is already
applied for the session id
C denotes that the status of this ADOP session has completed

Note: Numerical statuses are only relevant for the cutover phase

STEP 3: Check the current status of the adop cycle

Source the run filesystem environment file and run command
adop -status

usage
adop -status generates a summary report
adop -status <sessionID> generates a summary report for that session ID
adop -status -detail generates a detailed report

General Problems

A.1 General problems with Finalize phase and Abort command

For phase=finalize issues run the following command and attach the adzdshowlog.out
file generated:
sqlplus <apps_schema_name>/<apps_Schema_password> @$AD_TOP/sql/ADZDSHOWLOG.sql
Note: the contents of the table will be truncated every time cleanup/prepare phase
is run.
Older log information is stored in adzdshowlog.out

or

select * from ad_zd_logs order by log_sequence desc;

A.2 'Duplicate keys found' during Finalize

ERROR at line 1:
ORA-20001: Error: while calling ad_zd.finalize .ORA-01452: cannot CREATE UNIQUE
INDEX; duplicate keys found
ORA-06512: at line 8

Run the following:

sqlplus <apps_schema_name>/<apps_Schema_password> @$AD_TOP/sql/ADZDSHOWLOG.sql

The output will highlight the unique index which is failing. You can then use the
following sql to identify the duplicate key values

select <list of columns for which unique index creation failed>, count(*)
from <schema_name>.<table_name> group by <list of columns again> having count(*)>1
e.g.
select REPRESENTATION_CODE, TRX_NUMBER#2, ORG_ID, count(*) from
OKL.OKL_TRX_CONTRACTS_ALL group by REPRESENTATION_CODE, TRX_NUMBER#2, ORG_ID having
count(*)>1

A.3 Problems with Online Enablement

For failed attempts at Online Enablement i.e. failures of patch 13543062, gather
the following information:
1. Patch log
2. Worker logs
3. Output of
select * from ad_zd_logs order by log_sequence desc;

4. output from:
@$AD_TOP/sql/ADZDSHOWDDLS.sql

If enablement appears to be having performance issues or seems to have hanged you

can run the following script to determine if enablement is progressing or has
hanged:
select count(1) from ad_zd_logs;
Run the scripts every 5 or 10 minutes. If the count is increasing then enablement
is progressing.If enablement is progressing but very slowly ensure the DB
initialization parameters are set as per the DB 11.2.0.3 and 12.2 requirements.

A.4 To show which patches were applied in each ADOP_SESSION_ID (patching cycle)

select * from ad_adop_session_patches order by end_date desc;

or

set pagesize 200;

set linesize 160;
column adop_session_id format 999999999999;
column bug_number format a15;
column status format a15;
column applied_file_system_base format a23;
column patch_file_system_base format a23;
column adpatch_options format a15;
column node_name format a15;
column end_date format a15;
column clone_status format a15;

select ADOP_SESSION_ID, BUG_NUMBER, STATUS, APPLIED_FILE_SYSTEM_BASE,

PATCH_FILE_SYSTEM_BASE, ADPATCH_OPTIONS, NODE_NAME, END_DATE, CLONE_STATUS
from ad_adop_session_patches
order by end_date desc;

Note: STATUS
N - Not Applied In the current node but applied in other nodes
R - Patch Application is going on.
H - Patch failed in the middle. (Hard Failure)
F - Patch failed in the middle but user tried to skip some failures.
S - Patch Application succeeded after skipping the failed jobs.
Y - Patch Application succeeded.
C - Reserved for clone and config_clone. Indicates clone completed

A.5 Issues with FS_CLONE

a) select ADOP_SESSION_ID,BUG_NUMBER,CLONE_STATUS,STATUS,NODE_NAME from

AD_ADOP_SESSION_PATCHES order by ADOP_SESSION_ID;
b) sqlplus <apps_schema_name>/<apps_Schema_password> @$AD_TOP/sql/ADZDSHOWLOG.sql
c) zip of files in fs1/EBSapps/comn/clone/FMW/logs,
fs2/EBSapps/comn/clone/FMW/logs. These log files are also useful for clone issues
during the prepare phase. ( this captures cloning failures for WLS)
d) zip of directory $APPLRGF/TXK/ohsCloneLog ( this captures cloning failures for
OHS )

you can check for fsclone issues in Prepare phase error link

Other key things to consider in multi web node instances.

- SSH should be enabled to allow communication from the primary node to the
secondary nodes. Use txkRunSSHSetup.pl to enable SSH.

- If SSH is not enabled pay very special attention to the specific requirements for
running adop in this case (see the maintenance guide)

- When patching ensure you copy patches to the patch home directory of all
application tiers e.g. fs_ne/EBSapps/patch

- When debugging issues make sure to you review the adop logs on all application
tiers

Note:

Each cycle of ADOP creates a subdirectory corresponding to the patch session ID,
e.g.

/u01/PROD/fs_ne/EBSapps/log/adop/1
/u01/PROD/fs_ne/EBSapps/log/adop/2

Adop phases and parameters

===========================:

Adop phases

1) prepare - Starts a new patching cycle.

Usage: adop phase=prepare

2) Apply - Used to apply a patch to the patch file system (online mode)
Usage: adop phase= apply patches = <>

Optional parameters during apply phase

--> input file : adop accepts parameters in a input file

adop phase=apply input_file=

Input file can contain the following parameter:

workers=
patches=:.drv, :.drv ...
adop phase=apply input_file=input_file
patches
phase
patchtop
merge
defaultsfile
abandon
restart
workers
Note : Always specify the full path to the input file

--> restart -- used to resume a failed patch

adop phase=apply patches=<> restart=yes

--> abandon -- starts the failed patch from scratch

adop phase=apply patches=<> abandon=yes

--> apply_mode
adop phase=apply patches=<> apply_mode=downtime

Use apply_mode=downtime to apply the patch in downtime mode ( in this case,patch is

applied on run file system)

--> apply=(yes/no)
To run the patch test mode, specify apply = no

--> analytics
adop phase=apply analytics=yes
Specifying this option will cause adop to run the following scripts and generate
the associated output files (reports):

ADZDCMPED.sql - This script is used to display the differences between the run and
patch editions, including new and changed objects.
The output file location is: /u01/R122_EBS/fs_ne/EBSapps/log/adop////adzdcmped.out.

ADZDSHOWED.sql - This script is used to display the editions in the system.

The output file location is: /u01/R122_EBS/fs_ne/EBSapps/log/adop///adzdshowed.out.

ADZDSHOWOBJS.sql - This script is used to display the summary of editioned objects

per edition.
The output file location is:
/u01/R122_EBS/fs_ne/EBSapps/log/adop///adzdshowobjs.out

ADZDSHOWSM.sql - This script is used to display the status report for the seed data
manager.
The output file location is: /u01/R122_EBS/fs_ne/EBSapps/log/adop///adzdshowsm.out

3) Finalize : Performs any final steps required to make the system ready for
cutover..invalid objects are compiled in this phase

Usage: adop phase=finalize

finalize_mode=(full|quick)

4) Cutover : A new run file system is prepared from the existing patch file
system.
adop phase=cutover
Optional parameters during cutover phase:

-->mtrestart - With this parameter, cutover will complete without restarting the
application tier services
adop phase=cutover mtrestart=no

-->cm_wait - Can be used to specify how long to wait for existing concurrent
processes to finish running before shutting down the Internal Concurrent Manager.
By default, adop will wait indefinitely for in-progress concurrent requests to
finish.

5) CLEANUP
cleanup_mode=(full|standard|quick) [default: standard]

6) FS_CLONE : This phase syncs the patch file system with the run file system.
Note : Prepare phase internally runs fs_clone if it is not run in the previous
patching cycle

Optional parameters during fs_clone phase:

i) force - To start a failed fs_clone from scratch

adop phase=fs_clone force=yes [default: no]

ii ) Patch File System Backup Count ==> s_fs_backup_count [default: 0 : No backup

taken]
Denotes the number of backups of the patch file system that are to be preserved by
adop. The variable is used during the fs_clone phase, where the existing patch file
system is backed up before it is recreated from the run file system.

7) Abort - used to abort the current patching cylce.

abort can be run only before the cutover phase
adop phase=abort

Using adop, patches can be applied in both interactive mode and non interactive
mode
===================================================================================
==:

INTERACTIVE MODE:

To run the patch in interactive mode, do not mention the patch number..Only mention
the phase. This runs the patch in interactive mode and also the patch doesn't come
out in case of any failures.
It waits till the worker is fixed just like adpatch.

Ex: adop phase=apply

When asked for any command line arguments, mention the following
patchtop= driver= <> logfie=<> worker=<>

NON- INTERACTIVE MODE:

To apply the patch in non - interactive mode,provide the patch number in the
command line

Ex: adop phase=apply patches=<>

In case of any failure, the patch exists out.
To re-run the patch, after the issue gets fixed, use the following

adop phase=apply patches=<> restart=yes

To restart the patch from starting , use the below

adop phase=apply patches=<> abandon= yes

Applying patches in Downtime mode:

To apply patch in downtime mode, bring down all the services on run file system .
Prepare, finalize , cutover phases are not required when applying the patch in
downtime mode.
Ex; adop phase=apply apply_mode=downtime

Applying patch in downtime mode reduces the overall time to apply the patch.

adop phase=prepare -> copies the application code

adop phase=apply -> apply patches to PATCH Environment
adop phase=finalize -> makes ready the system for cutover
adop phase=cutover -> bounce the system and does filesystem switchover. fs2 becomes
RUN environment.
adop phase=cleanup -> remove obsolete objects.
adop phase=fs_clone -> synchronize filesystems

Note:- In case, the previous patch cycle did not complete its cleanup phas, The new
adop prepare phase will execute the clean-up phase of previous cycle, if needed,
and then proceed with preparing the new patch edition for new Online patching
cycle.

Note:- We do not need to execute the cutover immediately; Cutover can be delayed
until a convenient time in the future. In the meantime, We may also apply
additional patches if needed, but you will need to run the finalize phase again
after doing so.

Note:- Use full cleanup periodically or after major updates to restore the system
to optimal space usage.

Warning: full cleanup can take many hours and should only be done when there is no
immediate need to start a new patching cycle.

adop phase=cleanup
adop phase=cleanup cleanup_mode=full

g:\prints\ebs_patching\patching format.txt
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Patch Format classification
============================:

-Standalone/One-off patch/GDF patches

Resolves specific bug

-Minipack
Collection of one-off patches for specific module.

-Family pack
Collection of minipack for particular family group.

-Consolidated update (CU)

Includes recommended patches for all products, Updates apps to latest recommended
patch level.

-Maintenance pack
Collection of all minipacks for all products. Maintenance pack upgrades system to
new point release, such as 11.5.10

-what is the use of interoperability patch?

We apply this patch for the compatibility between OS and Oracle.

-What are consolidated seed, AD and TXK Delta Patches.

Without the consolidated Seed patches, you will receive seed errors applying future
patches. The AD/TXK patches required are patches which deliver updated AD
utilities for patching and maintenance and TXK provides mostly autoconfig templates
required for the applications.

R12

In R12 patches are grouped in to code lines and associated with Code levels.
Codeline includes all patches to maintain that point release.
Lets say 12.0 is the point release, it introduces codeline A. Similarly 12.1
introduces codeline B and so on.
Code level includes all bug fixes and codelevels for particular codeline. Code
levels are cumulative. All code levels created after the initial point release are
aggregated into cumulative release update packs (RUPs)
Base code level for 12.0 is A. Cumulated New bug fixes for products are released as
A.1.
A.2 includes A.1 & A.2; similarly A.3 includes A.1 to A.3 and so on.

Individual bug fix

Similar to one off in 11i

Product Family Release Update pack (RUP)

Similar to family pack in 11i.Aggregation of all one-off in particular codeline
specific to a family

Release Update Pack (RUP)

Cumulative of all Product Family RUPs.Changes version from 12.0 to 12.0.1,12.0.2
and so on.

Consolidated Upgrade
Similar to CU on 11i.Its available for upgrading R12 system from one point release
to another.

3. Patch naming convention:

11i
One-off patch
Patchnum e.g.: 8234812

Minipack
11. <PROD>.A e.g.: 11.AD.I

Family Pack :
11i. <PROD family>. letter e.g.: 11i.HR_PF.A

Maintenance pack:
11.5.x e.g. 11.5.9

R12

The patch naming convention has changed to R12.PROD.CL. #

Where PROD � Product (E.g. GL, OAM, AP, AR�)
CL �Code line (E.g. A, B, C�) R12.PROD.CL -> code level
# � Fix sets (E.g. 1,2.)
E.g.: R12.OAM.A.1
This is code level A with first set of fixes.
4. Downloaded patch Example:
11i R12
P123456_11i_linux.zip P123456_R12.PROD.A_R12_linux.zip
5.Patch driver files:
An 11i patch comes with c, d, g & unified driver format. R12 patches have unified
drivers.
6.Applying patch:
Use adpatch (Both 11i & R12)
Help � adpatch -help

1. What is a one-off patch?

Ans : An one-off patch is a small patch (of without any pre-requisites) to fix a
bug.

2. What is a mini pack?

Ans : A mini pack is one which will upgrade any product patchset level to next
level (like AD.H to AD.I).
3. What is Family pack?
Ans : A Family pack is one which will upgrade the patchset level of all the
products in that family to particular patchset level.

4. What is Maintenance pack?

Ans : A maintenance pack will upgrade applications from one version to another
(like 11.5.8 to 11.5.9).

5. What is a Rollup patch?

Ans : A rollup patch is one which will deliver bug fixes identified after the
release of any major application versions (like 11.5.10.2/12.1.0).

6. What is consolidated patch?

Ans: Consolidated patches will come into picture after upgrades from one version of
applications to another, all post upgrade patches will a consolidated and given as
consolidated patch.

7. How you apply a patch?

Ans : adpatch

8. How to find latest patchset level for module installed?

Ans : select APP_SHORT_NAME, max(PATCH_LEVEL) from AD_PATCH_DRIVER_MINIPKS GROUP BY
APP_SHORT_NAME;

9. How you will find whether a patch is applied/not?

Ans : Query ad_bugs.

10. What is the other table where you can query what are the patches applied?
Ans : ad_applied_patches.

11. How to find out which patch driver is applied (like c,d,g or u)?
Ans: Query ad_patch_drivers.

12. How to find out whether a language patch is applied for a particular patch?
Ans : Query ad_patch_driver_langs.

13. What is the difference between ad_bugs and ad_applied_patches?

Ans: A patch can deliver solution for more than one bug, so ad_applied_patches may
not give the perfect information as in case of ad_bugs.

14. What inputs you need to apply a patch other than driver name and etc?
Ans : apps and system passwords

15. What are the tables adpatch will create and when?
Ans : Adpatch will create FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS table when it
will apply d,g and u drivers.

16. What is a patch and name some different types of patches?

Ans : Patch is a program which fixes the bug.

There are 4 different types of patches

� One-off patch:
o This is the simplest type of patch. It is created to resolve a particular
problem.

� Mini pack Patch:

o It is a collection of one-off patches related to a particular module or product.
o Mini Pack version of module is denoted by Alphabetic characters.
� Family pack patch:
o This is a collection of mini pack patches in one family.
o Alphabetic characters denote the family pack version.

� Maintenance pack patch:

o This is a collection of family pack patches.
o Oracle Applications Release 11.5.10 is an example of Maintenance pack.
rollup patches,
consolidated patches.

17. What are the contents of patch and drivers present in patching?
Ans : Contents of patch:
� Readme files
� Driver files
� Metadata files
� Replacement files

Drivers present in patching:

� C (copy) driver - It copies all the files and links the executables.
� D (database) driver - It is responsible for running SQL scripts which updates the
database.
� G (generate) driver - It is responsible for generating forms, reports and message
files.
� U (unified) driver - It is a united driver containing all copy and database
actions.

18. What do you do if patch fails?

Ans :
� Evaluate log files to determine the cause of the error.
� Repair the cause of error.
� Now restart adpatch.
� Answer 'Yes' when adpatch asks if you want to continue the previous session.
� Adpatch starts from where it left off by skipping all completed steps.

19. How can I determine the effects a patch will have on my application system?
Ans :This can be done by patch wizard in the Oracle Applications manager.
� Checking total number of files in the patch and which are installed.
� Products that contain updated files.
� Total number of files introduced by the patch.
� Files on the target system changed by the patch.
� Files which depend on patched files.

20. How can you reduce the downtime when you have to apply multiple patches?
Ans :
� You can reduce the downtime by merging all the patches into one single patch
using admergepatch (admrgpch) tool.
� You can also apply multiple patches one by one by choosing options nocompiledb,
nocompilejsp, and nolink.
� Compilation of invalid objects, jsp�s and relinking can be skipped till the last
patch is applied.
� In the last patch you can compile all of them and then relink.
� You can also choose the max number of workers which your CPU permits.

1. What are the different types of patches?

Ans : oneoff, mini packs, family packs, maintanance packs, rollup pathches,
colsolidated patches.
2. What is a oneoff patch?

Ans : An oneoff patch is a small patch of (20-90K size) without any pre-req�s

3. What is a mini pack ?

Ans : A mini pack is one which will upgrade any product patchset level to next
level like AD.H to AD.I

4. What is Family pack ?

Ans : A Family pack is one which will upgade the patchset level of all the products
in that family to perticular patchsetlevel.

5. What is Maintanance pack ?

Ans : A maintanance pack will upgrade applications from one version to another like
12.1.2 to 12.1.3

6. What is a Rollup patch?

Ans : A rollup patch is one which will deliver bug fixes identified after the
release of any major application versions like 12.1.3

7. What is consilidated patch?

Ans: Consolidated patches will come into pictures after upgrades from one version
of applications to anoter, all post upgrade patches will a consolidated and given
as consolidated patch.

8. How u will find whether a patch is applied/not?

Ans : Query ad_bugs.

select substr(APPLICATION_SHORT_NAME,1,10) Product,substr(BUG_NUMBER,1,10) Patch#,

substr(ARU_RELEASE_NAME,1,10)
Version,last_update_date applied_date from applsys.ad_bugs where BUG_NUMBER=
to_char(�&bug_no�);

9. What is the other table where u can query what are the patches applied?

Ans : Ad_applied_patches

10. What is the difference between ad_bugs and ad_applied_patches?

Ans: A patch can deliver solution for more than one bug, so ad_applied_patches may
not give u the perfect information as in case of ad_bugs.

11. How u apply a patch?

Ans : using adpatch in R12.1 and using adop in R12.2

12. What inputs you need to apply a patch other than driver name and etc?

Ans : apps and system passwords

13. What are the table u r adpatch will create and when?

Ans : Adpatch will create FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS table when it

will apply d,g and u drivers

14. What is the significance of FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS table?

Ans: FND_INSTALL_PROCESSES table will store the worker information like what job is
assigned to which worker and its status. AD_DEFERRED_JOBS will come into picture
when some worker is failed, it will be moved to AD_DEFERRED_JOBS table, from where
again adpatch will take that job and try to resign, after doing this 3 times if
still that worker is failing, then adpatch will stop patching and throw the error
that perticular worker has failed. We need to trouble shoot and restrart the
worker.

15. If it is a multinode installation which driver we need to apply on which node?

Ans: c,d,g on concurrent node and c, g on web node. If it is u-driver we need to

apply on all nodes.

16.While applying a application patch is that necessary that u r database and

listener should be up?

Ans: Yes . why because adpatch will connect to database and update so many tables
etc�..

17. While applying a patch if that patch is failing because of a pre-req then how
you will apply that pre-req patch and resume with the current patch?

Ans: We need to take the backup of FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS

tables and restart directory at APPL_TOP/amdin/SID and then use adctrl to quit all
the workers.

mv $APPL_TOP/admin/<SID>/restart $APPL_TOP/admin/<SID>/restart.old
create table applsys.FND_INSTALL_PROCESSES_11510 as select * from
applsys.FND_INSTALL_PROCESSES
create table applsys.ad_deferred_jobs_11510 as select * from
applsys.ad_deferred_jobs

drop table applsys.FND_INSTALL_PROCESSES

drop table applsys.ad_deferred_jobs

Then apply the pre-req patch , after that rename u r restart directory to its
original name and create FND_INSTALL_PROCESSES and AD_DEFERRED_JOBS tables from the
bcakup tables. Start adpatch session and take the options want to continue previous
session.

18. What is adctrl?

Ans: Adctrl is one of the ad utilities, which is used to check the status of
workers and to manage the workers.

19. Can u name some of the menu options in adctrl?

Ans: Check the status of workers, tell manager that worker has quited, restart a
failed worker etc�.

20. How to skip a worker and why?

Ans: We can skip a worker using option 8 in adctrl which is hidden. We will go for
skipping a worker when we have executed the job which the worker is supposed to do.

How to restart and skip the job using adctrl

AD controller

21. How adpatch knows what are the pre-req�s for the patch which it is applying?

Ans: With every patch a file called b.ldt file will be delivered which contain the
pre-req information. adpatch load this into databse using FNDLOAD and check ,
whether those pre-req patches were applied or not.
22. What c-driver will do?

Ans: C-drive copies the files from patch unzipped directory to required location in
u r application file system. Before copying it will check the file version of the
existing file at the file system with the file version of the file in the patch. If
the patch file version is higher than what it is at file system level then only c-
driver will copy that files.

23. How adpatch will know the file versions of the patch delivered files?

Ans:With each patch a file with name f.ldt is delivered , which contain the file
versions of the files dilivered with the patch. Adpatch will use this file to
compare the file versions of files its delivering with the file on file system.

24. What is the adpatch log file location?

Ans : APPL_TOP/admin/SID/log

25. What is the worker log file name and its location?

Ans : adwork01,adwork02�� and location is APPL_TOP/admin/SID/log

26 How u will know what are the files the patch is going to change just my
unzipping the patch?

Ans:When u unzip a patch it will keep all the files related to a particular product
under that directory inside u r patch directory for example if the patch delivering
files related to FND product then it will create a sub directory under the patch
directory with the name FND in which it will put all related files to that product

27. What is the significance of backup directory under u r patch directory?

Ans:When we apply a patch it will keep the copy of the files which its going to
change in file system.

28. What are the different modes you can run your adpatch?

Ans :1.Interactive � default mode

2.Non interactive � Use defaults files to store prompt values

(adpatch defaultsfile= interactive=no)

3. Test � Without actually applying a patch just to check what doing.(adpatch

apply=no)
29. How you can enable maintainance mode without adadmin utility?

Ans we can use the below script to enable and disable maintainance mode
@$AD_TOP/patch/115/sql/adsetmmd.sql ENABLE
@$AD_TOP/patch/115/sql/adsetmmd.sql DISABLE

30. How you can analyze the impact of patch i.e what files will be replaced and
what new files will be added?

Ans we can find that using Patch wizard

System Administrator (Responsibility)
�> Oracle Application Manager
�> Patching and Utilitie
�> Site Map
�> Maintenance
�> Patch Wizard

31.How often should customers apply mini-packs, family packs, and maintenance
packs?
Ans You should keep your maintenance level up to date in order to:
Receive the latest fixes.
Reduce the number of file updates needed for emergency fixes.
Reduce the possibility of unfulfilled prerequisite patches when applying an
emergency fix.
Make it easier for Oracle Support and Oracle Development to assist you.
Keep core products such as AD (patches and maintenance fixes), FND (security and
technology stack updates), and HR (legislative updates) up to date.
32.Can I run multiple AutoPatch sessions at the same time?
Ans You cannot currently run multiple sessions simultaneously. However, patches can
be merged and can be applied in a single patching session.

33.Can we merge US and NLS patches together?

Ans: Merging US and NLS patches is fully-supported. When AD Merge Patch merges a US
and NLS patch, it alters the phasing of the NLS patch so that all NLS actions occur
after all US actions. AD supports merging US and NLS patches because it works and
may be easier for the customer. If the customer does not have strict downtime
requirements, it may be more convenient for the customer to merge the US and NLS
patches and apply them together. They can possibly reduce their downtime by
separating US and NLS (or by using one merged NLS patch per language), but this is
more work for the customer. We should let the customer decide which of the various
supported options best meets their needs.

34.What are deferred jobs and how are deferred jobs handled?
Ans The first time a job fails, the manager automatically defers it to the end of
the current phase and assigns a new job to the worker.

If the deferred job fails the second time it is run , the manager defers it again
only if the total runtime of the job is less than 10 minutes . If the deferred job
fails a third time (or if the job�s total runtime is not less than 10 minutes the
second time it is run) the job stays at FAILED status and the worker waits.

At this point, you must address the cause of the failure, and then restart the
worker, using AD Controller.