Bea Internals

PeopleTools Application Server &
BEA Tuxedo Internals

david.kurtz@go-faster.co.uk
Go-Faster Consultancy Ltd.
http://www.go-faster.co.uk
Who am I?
Independent Consultant
Performance Tuning
PeopleSoft / Oracle

23
Rules of Engagement
Interrupt me:
if you think I have got something wrong

if you have a question
if you cant hear me

23
BEA Tuxedo
Historical Overview
Tuxedo Internal Architecture
PeopleSoft 3-tier client implementation
How does PeopleSoft Internet Architecture (PIA) fit in
Configuration issues
Monitoring

23
Brief Historical Background

In the beginning
monolithic
Presentation
Business
Logic

Data
Access
(DBMS)
23
Client Server
Two-tier client-server
Presentation
Business
Logic
SQL
Raw Data
Data
Access
(DBMS)
LAN
Client
Server
23
3-Tier Client Server

1st Generation
GUI
+
Application
Logic
Data
Access
(DBMS)
Request
Response
LAN
Client
Business
Logic
Server
23
3-Tier Client Server

2nd Generation - Tuxedo
Presentation
Presentation
Logic
TUXEDO Clients
Business Logic
TUXEDO
Messaging
Service 1
Service 2
...
Service n
Data Management
SQL
DBMS 1
...
DBMS n
TUXEDO Servers
Database Servers
23
Where did Tuxedo come from?

Bell Labs - 1993
Transactions for
UniX
Extended for
Distributed
Operations

23
What is Tuxedo?
It is the knots on the ends of the string!

23
PeopleSoft 2-Tier Client
Presentation
Logic
Panel
Load
Panel
Save
Field
Change
etc...

23
Tuxedo Server
Tuxedo Client
Presentation
Logic
Panel
Load
Panel
Save
Field
Change
etc...
Server Side
Client Side

23

Client Connects Directly to the Database
1 database connection per window

inactivity timeout
DATABASE

23

Client Connects Directly to the Database
1 database connection per window

inactivity timeout
What happens in 3-Tier?

DATABASE

23
PS/Tuxedo Internal Structure

1st Application Server Process Started
Bulletin Board Liaison (BBL)

Reads PSTUXCFG configuration file
Shared Memory Segment
Bulletin Board (BB)
BB
BBL
PSTUXCFG

DATABASE
23

Workstation Listener (WSL)
Listens for connection request from clients on a specified port

Client initially contacts the WSL
WSL
BB
BBL
DATABASE

23

Workstation Listener (WSL)
Client needs to know where to find the WSL

Machine and port specified in Configuration Manager
WSL
BB
BBL

23

Workstation Handler (WSH)
WSL
handles traffic from client after connection

Client is assigned to a port on the WSH
WSH port range can be constrained to match a firewall
Additional WSH can be spawned if necessary
Compress Messages
WSH
BB
BBL
DATABASE

23
WSL
AUTHQ
PSAUTH
APPQ
PSAPPSRV
QCKQ
PSQCKSRV
QRYQ
PSQRYSRV
APIQ
PSAPISRV
SAMQ
PSSAMSRV
WSH
BB
BBL
DATABASE

23
How does it work?
WSHQ
WSL
AUTHQ
PSAUTH
APPQ
PSAPPSRV
QCKQ
PSQCKSRV
QRYQ
PSQRYSRV
APIQ
PSAPISRV
SAMQ
PSSAMSRV
WSH
BB
BBL
DATABASE

23
How does PIA work?

Java Client & PIA use Jolt
Java Class Library

Jolt Listener maps Java functions to Tuxedo Messages

23
Jolt Internal Structure

AUTHQ
PSAUTH
APPQ
PSAPPSRV
QCKQ
PSQCKSRV
QRYQ
PSQRYSRV
APIQ
PSAPISRV
SAMQ
PSSAMSRV
JSHQ
JSL
WSL
JSH
JREPSRV
WSH
JREPOSITORY
BB
BBL
DATABASE

23
New structure for PIA (Apache Webserver)

http 1.1
address+cookie
(stateless)
Sessionid =
xy1kvm8010
Cookie
Java
JVM
Servlet Engine
Servlet Container
Servlet
thread
IS stateful
Servlet
Webserver
HTTPD
Disk
(static
objects)
Servlet
Thread
Servlet
Thread
Servlet
Thread
Client
Client
mod_jserv
Port 8010
JSL

JSH
JSH
JSH
23
Tuxedo
Application
Server
What is the difference between the web servers?

Apache
Separate Servlet Engine

Separate encryption module (Raven)
Weblogic
Contains an integral servlet container

23
Progressive Connection Concentration

Typical picture
35,000 users
35000 users
1000 concurrent connections
1000 users active in the last n minutes

where n is the Java + JSH timeout
therefore need 1000 threads
but maybe only 4 JVMs
100 JSH
100 JSHs
assumes 10 clients per JSH
10-20 PSAPPSRV processes
1000 connections
4 JVMs
1000 Java Threads
each PSAPPSRV connects to database
10-20 PSAPPSRVs
1 database
1 database
23
Configuration
psappsrv.val
psadmin
(configuration
dialogue)
psappsrv.cfg
ubbgen
(PS Utility)
psappsrv.ubb
tmloadcf
(tuxedo
process)
PSTUXCFG
BBL
(domain
startup)
psappsrv.ubx
23
Configuration Files
psappsrv.cfg
contains configurable parameters
psappsrv.val
validation file for configurable values
psappsrv.ubx
configuration template for Tuxedo domain
psappsrv.ubb
configuration file for Tuxedo domain
PSTUXCFG
compiled configuration file

read by BBL at domain startup

23
psappsrv.ubx -v- psappsrv.ubb

psappsrv.ubx & psappsrv.cfg
PeopleSoft file
make changes to these files
difficult to read
psappsrv.ubb

Tuxedo file
never make changes to this file
easier to read
this file is compiled by tmloadcf
to generate domain
configuration file PSTUXCFG
variables fully resolved
23
psappsrv.ubb
Five (or Six) Sections
Resources
resource limits -> BB sizing
Machine
Resource limits. Environment

(Network
Connection between Machines)
Groups
Servers
Logical Split (BASE, APPSRV, JREPGRP, JSLGRP)

Number of servers, Command line
Services
Timeout, Load, Priority

23
Application Server Configuration Issue

Move quick PSAPPSRV services into a second server
(PSQCKSRV) (y/n)?
<=PT7.58 this question is misleading

services are additionally advertised on PSQCKSRV as well as
PSAPPSRV
so they are advertised on more than one queue
>=PT7.59 and PT8.1
services are de-advertised from PSAPPSRV
quick services not used in PIA, only IC* services
so not a problem

23
PeopleTools <=7.58
Quick Service
WSL
APPQ
PSAPPSRV
QCKQ
PSQCKSRV
WSH
BB
BBL
DATABASE

23
Inside the Bulletin Board

Without Load Balancing
(default)
Table of servers process where

each service is advertised
Read and populated as a stack
WSL looks for first free server
where request can be enqueued
This can cause problems
PSAPPSRV
PSQCKSRV
PSQCKSRV
PSQCKSRV
PSAPPSRV
PSAPPSRV
PSAPPSRV

23
Load Balancing
Load Balancing is about choosing the queue upon which to
place the message
by default all requests have the same load
50
50
50
APPQ
PSAPPSRV
50
50
50
50
50
QCKQ

50
50
PSQCKSRV
23
Load Balancing Trace

Add -r to application server command line
CLOPT="-r -e APPQ.stderr -A -- -C psappsrv.cfg -D GP81O81 -S

PSAPPSRV
-r enables trace to file stderr (not channel 2)

-e overrides the output filename
process trace file with Tuxedo utility txrpt

obtain average service time
make load proportional to average service time

23
Load Balancing Trace
Output from TXRPT
SVCNAME
18p-19p
Num/Avg
TOTALS
Num/Avg
--------------SqlRequest
MgrClear
RamList
SamGetParmsSvc
--------------TOTALS
-------16/0.10
5/0.04
1/0.43
1/0.50
------23/0.12
------16/0.10
5/0.04
1/0.43
1/0.50
------23/0.12

23
Load Balancing
load = execution time (seconds) x 100
78
386
135
APPQ
PSAPPSRV
599
50
6
30
63
QCKQ
7
PSQCKSRV
116

23
Multiple Queues on Busy System

You can have many server processes on a single queue
BEA recommend no more that 10 server processes per queue
If you have queue contention it is likely to impede the
WSH/JSH processes enqueuing requests. You are unlikely to

observe queuing with PQ.
So, if you want more than 10 PSAPPSRV processes you should

use two APPQ queues
In, which case you should to enable load balancing and you
must calculate the load.
Less significant with PIA because services different.

23
WSL
APPQ1
PSAPPSRV
APPQ2
PSAPPSRV
WSH
BB
BBL
DATABASE

23

psadmin will not correctly calculate
MAXACCESSERS
MAXSERVICES
tmadmin->bbs
will report maximum number of services

count of all services advertised on each server process
and if you have multiple work station listeners
MAXWSCLIENTS

23
Service Priority
Priority is about queue jumping
servers de-queue the request

9 out of 10 de-queued on the basis of priority
1 out 10 is taken from the front of the queue
80
50
50
50
QCKQ

50
50
PSQCKSRV
23
Service Priority
All PeopleSoft Services have same priority
specified in SERVICES section in psappsrv.ubb

therefore there is no queue jumping
no reason to believe that it would be advantageous

23
Tuning Tip
Financials
On-line journal edit & post - remote call

Service initiates a Cobol process
Process can take minutes
RemoteCall services can block APPQ
Move RemoteCall service to a separate queue

23
Multi-Node Domain
1 domain on many nodes
DBBL
WSL
WSH
BB
APPQ
PSAPPSRV
BBL
BRIDGE
TLISTEN
DATABASE
BRIDGE
TLISTEN
BBL
WSL
WSH
BB
APPQ
PSAPPSRV
23
Multi-Node Domain
Distinct Bulletin Board Liaison process
Master Node
TLISTEN process
Listens for incoming requests
BRIDGE
Sends & Receives messages between nodes

in a load balancing environment
configurable bridge load add to remote queues
limiting number of messages sent across the bridge

23
Multi-Node Domain
DO ATTEMPT THIS WITHOUT BEA CONSULTANCY
At UBS
PT 7.5x
susceptible to network errors
integrated into HP service guard
ultimately this was removed from production because of
transient network errors between the nodes causing the domain
to partition

23
So what is going on?

tmadmin
Tuxedo command line interface

also available within psadmin

23
tmadmin commands
pq - printqueues
amount of work queued and waiting

not always accurate

23
tmadmin commands
psr - printservers
reports activity of individual server processes

23
tmadmin commands
psc - printservices
each service on each server

23
tmadmin commands
Lots of other tmadmin commands
documented in the Tuxedo documentation
Use them to
dynamically reconfigure the domain

dynamically resize the domain
start/shutdown extra servers
monitoring scripts
tmadmin -r for read only mode

23
Java Administrative Applet

23
Conclusion
So now you know
BEA Tuxedo/Jolt is a messaging protocol

How the messages are moved around
How PeopleSoft has used it
How the PIA has been constructed
Options for Tuning and Sizing
Load Balancing
Multi-Node domains
Monitoring Scripts
BEA Documentation
is on your CD
http://e-docs.bea.com

23
More Information about PT8 Application Server

PeopleSoft 2000 Conference: Americas
8006: PeopleTools 8.1 Application Server

8010: Inside the Application Server
Buzz Voelker
PeopleTools Application Server Team

23
Final Caveat
PeopleSoft will not support any of the changes that I have
described in this presentation
Everything described in this presentation has been done, in

production, at UBS
If you want to use a multi-node domain you are strongly
recommended to arrange support/consultancy direct from

BEA.

23
Questions?

23
PeopleTools Application Server &

david.kurtz@go-faster.co.uk
http://www.go-faster.co.uk

Bea Internals

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Bea Internals

Încărcat de

Drepturi de autor:

Formate disponibile

PeopleTools Application Server &

BEA Tuxedo Internals

BEA Tuxedo Internals

if you think I have got something wrong

BEA Tuxedo Internals

BEA Tuxedo Internals

Brief Historical Background

BEA Tuxedo Internals

3-Tier Client Server

3-Tier Client Server

Where did Tuxedo come from?

BEA Tuxedo Internals

BEA Tuxedo Internals

PeopleSoft 2-Tier Client

BEA Tuxedo Internals

PeopleSoft 3-Tier Client

BEA Tuxedo Internals

PeopleSoft 2-Tier Client

1 database connection per window

BEA Tuxedo Internals

PeopleSoft 2-Tier Client

1 database connection per window

What happens in 3-Tier?

BEA Tuxedo Internals

PS/Tuxedo Internal Structure

Bulletin Board Liaison (BBL)

Shared Memory Segment

Bulletin Board (BB)

BEA Tuxedo Internals

PS/Tuxedo Internal Structure

Listens for connection request from clients on a specified port

BEA Tuxedo Internals

PS/Tuxedo Internal Structure

Client needs to know where to find the WSL

BEA Tuxedo Internals

PS/Tuxedo Internal Structure

handles traffic from client after connection

BEA Tuxedo Internals

PS/Tuxedo Internal Structure

BEA Tuxedo Internals

How does it work?

BEA Tuxedo Internals

How does PIA work?

Java Class Library

BEA Tuxedo Internals

Jolt Internal Structure

BEA Tuxedo Internals

New structure for PIA (Apache Webserver)

BEA Tuxedo Internals

What is the difference between the web servers?

Separate Servlet Engine

Contains an integral servlet container

BEA Tuxedo Internals

Progressive Connection Concentration

1000 concurrent connections

1000 users active in the last n minutes

assumes 10 clients per JSH

10-20 PSAPPSRV processes

each PSAPPSRV connects to database

contains configurable parameters

validation file for configurable values

configuration template for Tuxedo domain

configuration file for Tuxedo domain

compiled configuration file