Presentation #36693

Oracle High Availability

Solutions in a Nutshell
Daniel T. Liu
Senior Technical Consultant
First American Real Estate Solutions

Date: Tuesday, September 10, 2003 @ 8:30 AM - 9:30 AM

Place: Moscone Room 104
 Agenda
• Introduction
• High Availability
Concepts
• High Availability
Challenges
• High Availability
Solutions

OracleWorld 2003, Paper #36693, 2

Daniel T. Liu, FARES
 Agenda
• Oracle Advanced
Replication
• Oracle Real
Application Cluster
(RAC)
• Oracle Data Guard

OracleWorld 2003, Paper #36693, 3

Daniel T. Liu, FARES
 Agenda
• Oracle Streams
• Choose the Right
High-availability
Solutions
• Summary
• Q&A

OracleWorld 2003, Paper #36693, 4

Daniel T. Liu, FARES
 Introduction
• Why do we need to provide system and
database High Availability?
• Planned Downtime
– Database backup/upgrade/patching
– Operating system upgrade/patching
– Hardware and Network maintenance
• Unplanned Downtime
– Corruptions
• Logical corruptions
• Physical corruptions
OracleWorld 2003, Paper #36693, 5
Daniel T. Liu, FARES
 Introduction
• Unplanned Downtime
– Human Errors
• Accidentally drops, truncates a table
• Accidentally delete, update rows in a table
• Accidentally delete a data file or drop a
tablespace
– Disasters
• War, terrorism
• Earthquake, flood, fire or hurricane
• No power for a long period
• Server crash, malfunction of hardware

OracleWorld 2003, Paper #36693, 6

Daniel T. Liu, FARES
 High Availability Concepts
• What is High Availability?
– In the old days
• Local time
• Monday thru Friday
• 8 am to 6 pm
– Current environment
• Global
• 365 x 24 x 7
• Minimal downtime (planned or unplanned)

OracleWorld 2003, Paper #36693, 7

Daniel T. Liu, FARES
 High Availability Concepts
• Understanding High Availability
– Computing environments configured to provide
nearly full-time availability are known as high
availability systems.
– When failures occur, the failover process moves
processing performed by the failed component
to the backup component.
– The more transparent that failover is to users,
the higher the availability of the system.

OracleWorld 2003, Paper #36693, 8

Daniel T. Liu, FARES
 High Availability Concepts
• Measuring Availability
– The mean time to recover (MTTR)
– The mean time between failures (MTBF)
– Total uptime in a year (%)
Minutes of 5 60 1440 2880
Downtime
Minutes of 525595 525540 524160 522720
Uptime
Minutes in a Year 525600 525600 525600 525600

Total Uptime in a 99.9990% 99.9886% 99.7260% 99.4521%

Year (%)

OracleWorld 2003, Paper #36693, 9

Daniel T. Liu, FARES
 High Availability Challenges
• Challenges for Database High Availability
– Database size is getting bigger and bigger
– Database Backup Time
• A DLT tape writes 6 MB/Second, or 21 GB/hour
• A 2 TB database with one tape driver will take 97
hours to backup
– Database Recovery Time
– Is the backup good?
– Trouble shooting time

OracleWorld 2003, Paper #36693, 10

Daniel T. Liu, FARES
 High Availability Solutions
• Traditional High Availability Solutions
– Backup and restore
• High Availability Solutions
– Oracle Advanced Replication
– Oracle Real Application Clusters (RAC)
– Oracle Data Guard (Standby Database)
– Oracle Streams

OracleWorld 2003, Paper #36693, 11

Daniel T. Liu, FARES
 High Availability Solutions
• High Availability Solution means :
– No Single Points-of –failure
– Hardware Redundancy
– Software Redundancy
– Data Redundancy
– Application Redundancy

– More .. $?
– Save Money
OracleWorld 2003, Paper #36693, 12
Daniel T. Liu, FARES
 Oracle Advanced Replication

• Advanced Replication Overview

• Replication Components
• Types of Replication Environments
• Administration Tools for Replication

OracleWorld 2003, Paper #36693, 13

Daniel T. Liu, FARES
Advanced Replication Overview

• Replication is the process of copying and

maintaining database objects, such as
tables, in multiple database .
• Changes applied at one site are captured
and stored locally before being forward
and applied at each of the remote
locations.

OracleWorld 2003, Paper #36693, 14

Daniel T. Liu, FARES
 Replication Components

• Replication Objects
– Table, Indexes, View
– Procedures, Packages, Functions, Triggers
– User-Defined Types
• Replication Groups
– A collection of replication objects that are logically
related.
– Master group
– Materialized view group
OracleWorld 2003, Paper #36693, 15
Daniel T. Liu, FARES
 Replication Components

• Replication Sites
– Master Sites
– Materialized View Sites

OracleWorld 2003, Paper #36693, 16

Daniel T. Liu, FARES
 Types of Replication

• Multimaster Replication
– Each master site operates as an equal peer.
– Provides complete replicas of each replicated
table at each of the master sites.
– Replicate changes for each transaction.
– Two types of multimaster replication
• Asynchronous
• Synchronous

OracleWorld 2003, Paper #36693, 17

Daniel T. Liu, FARES
 Asynchronous Data Replication
Source Database Destination Database

Replicated Replicated
Change
Table Table

Trigger

Deferred Remote Procedure

Transaction Procedure
Call
Queue

Synchronous Data Replication

Source Database Destination Database

Replicated Replicated
Change
Table Table

Remote Procedure Procedure

Trigger
Call

OracleWorld 2003, Paper #36693, 18

Daniel T. Liu, FARES
 Types of Replication

• Materialized View Replication

– Replicate subset of master table data
– Batch-oriented operation (refresh)
– 3 types of materialized views
• Read-Only
• Updateable
• Writeable

OracleWorld 2003, Paper #36693, 19

Daniel T. Liu, FARES
 Materialized View Replication

Application

Update Query

Master Materialized
Table Refresh View
(updatable) (read-only)
Update Update

Update Master Materialized Update

Table Refresh View
(updatable) (updatable)

Master Materialized
Table Refresh View
(updatable) (writeable)

OracleWorld 2003, Paper #36693, 20

Daniel T. Liu, FARES
Replication Administration Tools

• Oracle Enterprise Manager

– Replication Manager
• Oracle-Supplied PL/SQL packages
– DBMS_REPCAT
• Replication Catalog
– On every master sites
– Materialized view sites

OracleWorld 2003, Paper #36693, 21

Daniel T. Liu, FARES
Oracle Real Application Clusters
(RAC)
• Real Application Clusters Overview
• Real Application Clusters Architecture
• Real Application Clusters Components
• Cache Fusion
• Transparent Application Failover (TAF)

OracleWorld 2003, Paper #36693, 22

Daniel T. Liu, FARES
 Real Application Clusters
Overview
• Multiple instances against the same
database.
• Involves a cluster of nodes with access to
a set of shared disks through Cluster
Management Software (CMS).
• Oracle’s solution for system failures.
• Transparent Application Failover (TAF)
• Connection Load Balancing
OracleWorld 2003, Paper #36693, 23
Daniel T. Liu, FARES
Real Application Clusters Architecture
Option 1 Option 2 (RAC)

Instance A Instance B Instance A interconnect Instance B

DB1 DB3 DB1

OracleWorld 2003, Paper #36693, 24

Daniel T. Liu, FARES
 Real Application Clusters Architecture

SGA SGA
Global Cluster Global
Resource Resource
Directory
Interconnect Directory

LMON LMD LMS LMON LMD LMS

Instance A Instance B

Cluster Group Cluster Group

Services Services

Vendor Vendor
CMS CMS

Node A Node B

Local Disks Local Disks

Shared Disks
Oracle Software Oracle Software
Data Files Archived Logs
Archived Logs
Control Files
Online Redo Files

OracleWorld 2003, Paper #36693, 25

Daniel T. Liu, FARES
 Real Application Clusters
Components
• Shared Disk
• Vendor CMS
• Cluster Group Services
• Global Resource Directory
• RAC Background Process
– LMON (Global Cache Service Process)
– LMD (Global Enqueue Service Daemon)
– LMS (Global Enqueue Service Monitor)
OracleWorld 2003, Paper #36693, 26
Daniel T. Liu, FARES
 Cache Fusion

• Oracle’s Global Cache Management

Technology.
• It provides cache to cache transfers of
data blocks between instances in a
cluster.
• It eliminates forced disk writes.
• Dynamic resource re-mastering

OracleWorld 2003, Paper #36693, 27

Daniel T. Liu, FARES
Transparent Application Failover
(TAF)
• Little or no user downtime.
• Applications and users are automatically
and transparently reconnected to another
system.
• DML transactions are rolled back

OracleWorld 2003, Paper #36693, 28

Daniel T. Liu, FARES
 Oracle Data Guard

• Data Guard History

• Data Guard Components
• Data Guard Roles
• Data Guard Interfaces
• Data Guard Process Architecture
• Data Guard Protection Mode

OracleWorld 2003, Paper #36693, 29

Daniel T. Liu, FARES
 Data Guard History
• History of Standby Database
– Oracle7.3: First Release of Standby
Database
– Oracle8i: Automatic shipping and
application of redo logs
– Oracle9i Release 1: Protection mode
– Oracle9i Release 2: Logical standby
database

OracleWorld 2003, Paper #36693, 30

Daniel T. Liu, FARES
 Data Guard Components
• Primary Database
• Standby Database
– Physical Standby Database
– Logical Standby Database (9iR2 only)
• Log Transport Services
• Network Configuration
• Log Apply Services
• Data Guard Broker

OracleWorld 2003, Paper #36693, 31

Daniel T. Liu, FARES
 CLI GUI

Primary
Database Data Data Standby
Guard Guard Database
Broker Broker

Oracle Net
Online
Redo
Logs

Log Log
Transport Apply
Services Services
Local Remote
Archived Archived
Logs Logs

OracleWorld 2003, Paper #36693, 32

Daniel T. Liu, FARES
 Data Guard Roles
A database can operate in one of the two
mutually exclusive roles:
• Failover
– One of the standby databases takes the
primary database role
• Switchover
– In Oracle9i, primary and standby
database can continue to alternate roles

OracleWorld 2003, Paper #36693, 33

Daniel T. Liu, FARES
 Data Guard Interface
• SQL*Plus and SQL Statements
SQL> alter database commit to switchover
to physical standby;
• Data Guard Broker GUI
– Data Guard Manager
• Data Guard Broker Command-Line Interface
$ dgmgrl
DGMGRL for Solaris: Version 9.2.0.1.0 -
Production.
(c) Copyright 2002 Oracle Corporation. All
rights reserved.
Welcome to DGMGRL, type "help" for information.
DGMGRL>
OracleWorld 2003, Paper #36693, 34
Daniel T. Liu, FARES
 Process Architecture
• Physical/Logical Standby Processes
– LGWR (Log Writer) process
– ARCH (Archiver) process
– LNS (LGWR Network Server) process
– RFS (Remote File Server) process
– MRP (Managed Recovery) process

OracleWorld 2003, Paper #36693, 35

Daniel T. Liu, FARES
 Process Architecture
• Physical/Logical Standby Processes
– FAL (Fetch Archive Log) Client process
– FAL (Fetch Archive Log) Server process
– LSP (Logical Standby) process
– PX (Parallel Execution) process
– DMON (Data Guard Broker Monitor) process

OracleWorld 2003, Paper #36693, 36

Daniel T. Liu, FARES
Physical Standby Processes Architecture

Primary DMON DMON

Physical
Database
standby
Database
SYNC

LGWR LNS ASYNC RFS

Oracle Net
Standby
Online Redo
Redo Logs
Logs
FAL
Client/
ARCH ARCH MRP

Local
Remote
Archived
Archived
Logs
FAL Logs
Server

OracleWorld 2003, Paper #36693, 37

Daniel T. Liu, FARES
Logical Standby Processes Architecture

DMON DMON
Primary Logical
Database Standby
Database
SYNC

LGWR LNS ASYNC RFS

PX PX

Oracle Net
Online Applying
Redo Group
Logs
LSP0

ARCH

PX PX

Local Remote Mining

Archived Archived Group
Logs Logs

OracleWorld 2003, Paper #36693, 38

Daniel T. Liu, FARES
 Data Protection Mode
• Oracle9i Release 2 has three data protection modes
Mode Log Writing Network Disk Write Redo Log Supported on
Process Trans Mode Option Reception
Option

Maximum LGWR SYNC AFFIRM Standby redo Physical

Protection logs are standby
required databases
Maximum LGWR SYNC AFFIRM Standby redo Physical and
Availability logs logical standby
databases

Maximum LGWR or SYNC or NOAFFIRM Standby redo Physical and

Performance ARCH ASYNC logs logical standby
databases

OracleWorld 2003, Paper #36693, 39

Daniel T. Liu, FARES
 Oracle Streams

• Oracle Streams Overview

• Oracle Streams Process Architecture
• Oracle Streams Rules
• Administration Tools for Oracle Streams

OracleWorld 2003, Paper #36693, 40

Daniel T. Liu, FARES
 Oracle Streams Overview

• Oracle9i’s new replication product.

• Similar to logical standby database
• Changes are captured at source
Database.
• Propagate information within a database
or from one database to another.
• Using Message Queuing.
• Heterogeneous information sharing.
OracleWorld 2003, Paper #36693, 41
Daniel T. Liu, FARES
 Oracle Streams Process
Architecture
• Capture changes at a database.
• Enqueue events into a queue.
• Propagate events from one queue to
another.
• Dequeue events.
• Apply events at a database.

OracleWorld 2003, Paper #36693, 42

Daniel T. Liu, FARES
Oracle Streams Processes Architecture

Source Queue Target Queue

LCR LCR
LCR LCR
Source Target
User User
Database Message Message Database
Propagate
User User
Message Message
LCR LCR
LCR LCR
Redo .. ..
.. ..
Log
.. ..

Capture Enqueue Dequeue Apply

OracleWorld 2003, Paper #36693, 43

Daniel T. Liu, FARES
 Oracle Streams Rules

• Rules are used to control which information to

share and where to share it.
• Rules can be used during capture, propagate,
and apply processes.
• Rules can define in three level:
– Table
– Schema
– Global

OracleWorld 2003, Paper #36693, 44

Daniel T. Liu, FARES
Oracle Streams Administration
Tools
• Oracle-Supplied PL/SQL packages
– DBMS_STREAMS_ADM
– DBMS_CAPTURE_ADM
– DBMS_PROPAGATION_ADM
– DBMS_APPLY_ADM
• Streams Data Dictionary views
– DBA_APPLY
– V$STREAMS_CAPTURE
• Oracle Enterprise Manager

OracleWorld 2003, Paper #36693, 45

Daniel T. Liu, FARES
 Choose High-Availability
Solution
• Product Licensing
• Unsupported Datatype
• Feature Comparison

OracleWorld 2003, Paper #36693, 46

Daniel T. Liu, FARES
 Product Licensing
High Availability Product Enterprise Edition

Advanced Replication Included

Real Application Clusters Additional License Fee

Data Guard Included

Streams Included

OracleWorld 2003, Paper #36693, 47

Daniel T. Liu, FARES
 Unsupported Datatype for
Logical Standby and Streams
Supported Datatypes Unsupported Datatypes
CHAR, NCHAR NCLOB
VARCHAR2, NVARCHAR2 LONG
NUMBER LONG RAW
DATE
BFILE
CLOB,BLOB
ROWID
RAW
TIMESTAMP UROWID
TIMESTAMP WITH TIME ZONE User-defined types
TIMESTAAMP WITH LOCAL TIME ZONE Object types
INTERVAL YEAR TO MONTH - REFS
INTERVAL DAY TO SECOND - Varrays
- Nested tables

OracleWorld 2003, Paper #36693, 48

Daniel T. Liu, FARES
 Feature Comparison
Advanced RAC Physical Logical Streams
Replication Standby Standby

Entire
Database YES N/A YES YES YES
Replication
Schema
Replication YES N/A NO NO YES

Table
Replication YES N/A NO NO YES

OracleWorld 2003, Paper #36693, 49

Daniel T. Liu, FARES
 Feature Comparison
Advanced RAC Physical Logical Streams
Replication Standby Standby

DML
Replication YES N/A YES YES YES

DDL
Replication YES N/A YES YES YES

Instance
Redundant YES YES YES YES YES

Database
Redundant YES NO YES YES YES

OracleWorld 2003, Paper #36693, 50

Daniel T. Liu, FARES
 Feature Comparison
Advanced RAC Physical Logical Streams
Replication Standby Standby

Cluster
Management NO YES NO NO NO
Software
Failover Manual Failover Failover Manual
Mechanism TAF
Failover Switchover Switchover Failover

Load YES
Balancing YES YES YES YES
Partial
OracleWorld 2003, Paper #36693, 51
Daniel T. Liu, FARES
 Feature Comparison
Advanced RAC Physical Logical Streams
Replication Standby Standby

Change
Captured Local Local Remote Remote Local

Heterogeneous
Database YES NO NO NO YES
Support
Datatype
Support ALL ALL SOME SOME SOME

OracleWorld 2003, Paper #36693, 52

Daniel T. Liu, FARES
 Feature Comparison
Advanced RAC Physical Logical Streams
Replication Standby Standby

OS Platform Must
Must Must
between Can be be Can be
source and Different Same be be Different
Target Same Same

Oracle
Version Can Must Must Must Can
Between be be be be be
Source and Different Same Same Same Different
Target
OracleWorld 2003, Paper #36693, 53
Daniel T. Liu, FARES
 Summary
• High Availability Concept
• High Availability Options
– Advanced Replication
– Real Application Clusters
– Data Guard
– Oracle Streams
• High Availability Product Comparison

OracleWorld 2003, Paper #36693, 54

Daniel T. Liu, FARES
 References
Oracle9i, Data Guard Concepts and Administration. Release 1 (9.0.1);
Oracle9i, Data Guard Concepts and Administration. Release 2 (9.2);
Oracle9i, Data Guard Broker. Release 2 (9.2);
Oracle9i, Real Application Clusters, Concepts. Release 1 (9.0.1);
Oracle9i, Advanced Replication. Release 2 (9,2);
Oracle9i, Streams. Release 2 (9.2);
Oracle Metalink Support;
Top DBA Shell Scripts for Monitoring Database, Daniel T. Liu; DBAZine;

I would also like to acknowledge the assistance of Larry Barry, Ann

Collins, Archana Sharma and Husam Tomeh of FARES, and Larry
Carpenter, Joseph Meeks, Roger Peterson of Oracle Corporation.
OracleWorld 2003, Paper #36693, 55
Daniel T. Liu, FARES
Thanks For Coming !!
Daniel Liu Contact Information
Phone: (714) 701-3346
Email: dliu@firstam.com
Email: daniel_t_liu@yahoo.com

Company Web Site:

http://www.firstam.com

OracleWorld 2003, Paper #36693, 56

Daniel T. Liu, FARES