Siebel 7 Performance and Scalability

Inside the Siebel Server

Richard Sands
Siebel Expert Services
Siebel Systems 2003 Do not distribute or re-use without permission

Siebel 7 Performance and Scalability

Inside the Siebel Server

Introduction to Siebel Architecture
How components scale
Within servers
Across servers

Siebel Performance
Hints
Monitoring
Diagnosis

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

Siebel 7 Infrastructure Overview

Wireless Web
Browser User Interface

Connected Web User

(Employee)

Connected Web User

(External)

Browser User Interface

Browser User Interface

PDA

Mobile (Disconnected)
Web User
Browser User Interface
Object
Manager

WAP Gateway
Server

Data Manager

Web Server(s)
Siebel Web Server
Extension

SIEB
SYNC
Local
DB

Gateway Name Server

External
Applications

Central
Load Dispatch
Balancer

Siebel Remote

Siebel Enterprise Server

Voice
Interaction

Siebel
eAI
Siebel
Replication

Object Manager
Data Manager

Object Manager
Data Manager

Regional
Siebel
DB Server

Central Siebel DB Server

Email
Interaction

Major Client Types

All accessed through a browser
High Interactivity (Employee facing)

Very demanding on browser

Can only run on strictly defined browser configurations
Rich user interface

Standard Interactivity (Customer facing)

Less demanding on browser
Can run on wide variety of browsers
Standard web user interface

Mobile Client

Has local copy of Siebel database

User-specific subset of data
Local server functionality
Quasi-web server
Business Object and Data Manager functionality
Uses High Interactivity interface

Siebel Enterprise Server SWSE

Web Server
SWSE

Gateway Server
Gateway
Name Server

Resonate Central
Dispatch

Enterprise Server
Siebel Server
Component

Siebel Server
Component

Siebel Web Server

Extensions
Web Server Plug-In
Manages
communications to
Siebel Enterprise
Includes cache for static
files (images, etc)

Siebel Enterprise Server Gateway Server

Serves as a single entry

point
Includes Siebel Gateway
Name Server

Web Server
SWSE

Gateway Server
Gateway
Name Server

Resonate Central
Dispatch

Component

Optionally includes
Resonate Central
Dispatch
Load balancing

Enterprise Server
Siebel Server

Siebel Enterprise
configuration data (static)
Siebel Enterprise
operations data (dynamic)

Siebel Server
Component

Siebel Enterprise Server Gateway Name Server

Holds Enterprise
Configuration
Stores component
definitions, parameters,
and connectivity
information

Web Server
SWSE

Gateway Server
Gateway
Name Server

Resonate Central
Dispatch

Enterprise Server
Siebel Server
Component

Siebel Server
Component

Stored in siebns.dat file

Provides Connectivity
information when
Resonate not used

Dynamically registers
Siebel Server and
component availability

Siebel Enterprise Server Resonate Central Dispatch

Web Server
SWSE

Gateway Server
Gateway
Name Server

Resonate Central
Dispatch

Enterprise Server
Siebel Server
Component

Siebel Server
Component

Load Balances Object

Manager Components
Provides resilience for
load balanced
components
Provides session
management
Simplifies firewall
configuration
Not always needed

Architecture Overview Siebel Server

Web Server
SWSE

Gateway Server
Gateway
Name Server

Resonate Central
Dispatch

Enterprise Server
Siebel Server
Component

Siebel Server
Component

Framework for running

server components
Obtains configuration
information from the
Gateway Name Server
Runs as a Windows
service
Siebel Enterprise Server
is a logical grouping of
Siebel Servers

Architecture Overview Server Components

Web Server
SWSE

Gateway Server
Gateway
Name Server

Resonate Central
Dispatch

Enterprise Server
Siebel Server

Siebel Server

Component

Component

A type of program, executed

as a task
Examples:
Object Manager Handling client
requests
Workflow Process
Manager
Processing
Workflows
File System Manager Handling access to
Siebel File System

Architecture Overview Server Component Types

Different types of component run in different ways:
Background
Background mode components execute tasks to perform
background operations for the Siebel Server. After a background
mode component task starts, it runs until you explicitly stop the
task, or until the Siebel Server itself is shut down.

Interactive
Interactive mode components start tasks automatically in response
to client requests. Interactive mode component tasks execute for as
long as the client maintains the session, and end when the client
disconnects.

Batch
Batch mode components execute tasks in response to requests.
Batch mode component tasks execute until they finish processing .
Internal batch tasks are initiated by other Siebel Server tasks
External batch tasks are initiated by Server Manager

Architecture Overview Component Execution Platforms

Different types of component execute in different platforms:

Single Threaded
Single threaded components have one execution stream per
process. So each operating system process supports a single Siebel
Task.
i.e. EIM

Multi-threaded
Multi-threaded components have multiple execution streams within
a single process.. So each operating system process can support
multiple Siebel Tasks.
i.e. Object Managers

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

Component Scalability

How Components Scale in a Siebel 7 Enterprise

Scaling within a server

Multi-threaded components
Single-threaded components

Scaling across servers

Load balancing
Resonate central dispatch
Load sharing
Server Request Broker
Server Request Processor

Scaling Within a Siebel Server

Single Threaded Components

Create multiple processes (tasks)

Some components are limited
i.e.
Transaction Processor max 1 per server
Workflow Monitor Agent max 1 per policy group per Enterprise
Can be started manually, through Server Request Broker, or
automatically (Default Tasks parameter)

Multi-Threaded Components

Create multiple threads &/or processes (tasks)

Control distribution through component parameters

Multi-Threaded Components

Can have multiple processes as well as multiple threads

Important to control ratio of threads to processes

Can have major impact on performance

Determined primarily by rate of switches between threads
100:1 good starting point for Client Object Managers
Assumes 30sec think time
For 15 sec think time use 50:1

Can set additional processes to spawn on demand

Will always start minimum number specified

Will start additional processes as needed to maintain process:thread
ratio
Limit on maximum number of processes

Memory Scalability

Multi-Process, Multi-Threaded model

All threads in a process share the same memory space

Each Process has a separate memory space

No single process needs a large memory space

Operating system manages memory allocation

If a single process needs more than 1GB theres normally something
wrong
No need for large process memory model (/3GB switch)
Can use Microsoft PAE for access to large memory capacities
Operating system can allocate memory to each process

Multi-Threaded Component Parameters

Typically set per component

Maximum number of tasks (MaxTasks)
Maximum number of Tasks per component per server
One thread per task
Some additional background system threads - not counted by MaxTasks

Maximum number of Multi-Threaded servers (MaxMTServers)

An MTServer is a multi-threaded component process

This defines the maximum number of MTServers per component per server

Minimum number of Multi-Threaded servers (MinMTServers)

This defines the minimum number of MTServers per component per server
Sets the number of MTServers started on server startup

Configuring the Object Managers

Set MaxTasks = peak (concurrent users + anonymous users)

Anonymous users are used for login screens before user authenticates
Typically set to 10%-15% of concurrent user count

Set MaxMTServers = MaxTasks / 100

An MTServer is equivalent to single instance of Object Manager

100 : 1 ratio is assuming average 30 second think time between user
operations
If average user think time is 15 seconds then ratio is 50 : 1 ( 50% of 100:1)
If average user think time is 60 seconds then ratio is 200 : 1 (200% of 100:1)

Set MinMTServers = MaxMTServers

Setting MinMT Servers < MaxMTServers may cause delay of service for
new users as MTServer gets initialized.

Multi-Threaded Component Parameter Example

Object Manager configuration for 800 Call Center users
Concurrent Users
800 Call Center Users

Object Manager Level

15%

MaxTasks
920
1000
100:1
MaxMTServers
10

MinMTServers
10

Anonymous Users
120

Round up to maintain
100:1 ratio

Scaling Across Siebel Servers

Depends on how component task initiated

Component Group Assignment

Which servers a component is available on

Object Managers

Load balanced (Resonate Central Dispatch)

Not valid for eConfigurator Object Manager

Siebel Remote

Manual allocation of mobile client to Siebel Server

Server Requests

Server Request Broker &/or Server Request Processor

Manual Requests (start task)

Manually specified server

Special cases

CTI
eConfigurator

Multi-Threaded Component Scalability

Vertical
Scalability

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Sales
Object
Manager

Siebel Server
Enterprise Server

Siebel Server

Siebel Server

Horizontal Scalability

Load Balancing

Web
Client

Web
Client

Web
Client

Web
Client

Web
Client

Web
Client

Load Balancing
Web Server + SWSE

Web Server + SWSE

Resonate Load Balancing

Thread
Process
Server

Sales Object
Manager

Sales Object
Manager

Siebel Server

Sales Object
Manager

Sales Object
Manager

Siebel Server

Enterprise Server

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

Server Request Broker & Server Request Processor

Server Request Broker (SRBroker)

Used to start synchronous Siebel Server tasks

Server Request Broker & Server Manager are the only components
which directly start tasks.
New in Siebel 7
Replaces Server Request Manager (SRMSynch) in Siebel 2000
Background component
Multi-threaded component
Need to set MaxTasks accordingly

Server Request Processor (SRProc)

Used to start asynchronous Siebel Server tasks

Manages queued requests
Calls SRBroker to manage task execution
Background component

Server Request Broker

Accepts requests from other Components

Will try to service request locally

If component is available on same Siebel Server then this is always

used

If not available locally then will use other Siebel Servers

Maintains internal table of components available on each Siebel
Server
Will route requests round Siebel Servers in turn (round-robin)

Multi-threaded component

May need to increase MaxTasks

Should always be running on all servers

Siebel Server Server Request Broker

Assignment
Manager
Server
Request
Broker
Workflow
Process
Manager

Run
Object Assignment
Manager Task
Is Assignment
available on
this server?
Run
Server
Server
Request Assignment Request
Broker on local server Broker
Assignment
Manager

Workflow
Process
Manager

Siebel Server Server Request Broker

Assignment
Manager

Run
Object Workflow
Is Workflow
Manager Process
Manager

Server
Request
Broker

Server
Request
Broker

Workflow
Process
Manager

Assignment
Manager

available on
Server
this server?
WhichRequest
other
serversBroker
have
workflow
Choose server
online?
onWorkflow
roundrobin
basis
Process
Manager

Server Request Processor

Processes asynchronous requests

Request submitted by creating record in table
S_SRM_REQUEST

Server Request Processor reads from table

Request must:
Be active (reached activation time)
Not be specified for a different Siebel Server
Not being processed by other Server Request Processor

Eligible requests submitted through Server Request Broker

Normally runs on all Siebel Servers

Siebel Server Server Request Processor

S_SRM_REQUEST

SRProc

Request
Queue
SRBroker

Task

Sleep
Interval

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

Connection Pooling

Siebel 7 can pool sessions at two levels:

Web server to Siebel Enterprise

SISNAPI Connection Pooling

Multiple SISNAPI (Siebel) sessions through one TCP session
Reduces operating system overhead and network traffic
Enabled by default
Set to 20
Controlled by component parameter:
Number of Sessions per SISNAPI Connection

Siebel Object Manager to Database

Database connection pooling

SQL traffic for multiple Siebel users through one database session
Reduces session overheads on database server
Disabled by default
Suitable for larger deployments (over 500 concurrent users)

Database Connection Pooling

Connections use native database protocols

Some components directly access native protocol
Object Managers

Siebel 7 supports its own database connection pooling

Used for connections from Object Managers
Two types of connection
Shared for general transactions
Specialized for long running

Not always appropriate

Should carefully evaluate tradeoffs

Benefits of less database sessions to maintain
Risk of database session contention

Database Connection Pooling

Database session uses login for first user to establish

session
Database connection pooling controls
Defined as component parameters
Set to -1 to disable (default)

Specialized (Dedicated) Database sessions

All valid per component process (MT Server) per Siebel Server
MaxTrxDbConns
- Maximum number of specialized DB sessions
MinTrxDbConns
- Minimum number of specialized DB sessions to
be kept in pool

Shared Database sessions

Valid per component per Siebel Server

MaxSharedDbConns- Maximum number of shared DB sessions
MinSharedDbConns
- Minimum number of shared DB
sessions
to be kept in pool

Database Network Architecture

ClientConnections
Siebel Server
Object Manager

Server
Request
Processor

Shared

Shared

Specialized

NativeDatabaseConnectivity
(ODBCforSQLServer)
Siebel
Database

Threads(sessions)
Processes(components)

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

General Siebel Server Optimisation

Disable unused components and component groups

These just use up system resources

Consider using common log file for multi-threaded

components that generate too many log files
Consider using multiple instances/server tasks for certain
activities
e.g. Parallel EIM, multiple Workflow Policy Groups, etc.

Monitor resource utilization by processes and threads

Object Manager Optimization

Enabling View Caching for High Interactive mode

Set in user preferences

Faster client response times, by as much as 1 second
Better network behavior
Expensive, 3MB per view cached
Most users use < 10 views / pages
Uses LRU mechanism to flush out cache

If not using eAdvisor or browser based eConfigurator

On cfg file set EnableCDA = FALSE

Set Timed Statistics Off

Set all Log Levels to 1

EAI Optimization
Minimize fields in Integration Components
Keep XML records small

XML is very verbose and needs to be parsed

Lump several records into one transaction

will achieve higher throughput

If possible keep records to 15 fields

More will reduce throughput

Keep in mind that an XML field translates to BC field

Integration Object initialize Business Objects and Business
Components
The more fields per record the slower it will process

EAI Optimization
Reduce SessPerSisnConn for EAI Adapters

Do this only if you are going to bombard adapter, i.e. little or no

think times

For very high throughput systems where you are

dedicating EAI machines

Consider letting the SRB forward BIM requests to dedicated machines

BIM processing can become bottleneck if BIM processing is complex
May be necessary to have a higher EAI Adapter:BIM machine ratio
(i.e. 1:x)

Use Session-Mode for high volume inbound HTTP

Greatly increased throughput

No overhead for authentication & session startup for each message

Workflow Process Optimisation

Use Run-Time Events for process automation

Minimize use of Workflow Policies since workflow policy work at the

database layer and do not take advantage of the level of abstraction
provided with the Object Manager
Performance gain since the decision event takes place on the
business object layer and so no trigger is created on the table
No need for scripting

Workflow Process Distribution

Workflow Policies cant be directly load balanced

Asynchronous Workflow Processes are distributed by SRProcessor
Synchronous Workflow Process requests are distributed by SRBroker

Application Performance - Configuration

Keep scripting small and tight

Avoid the use of script wherever possible

Remember it is still interpreted code and will execute as such
Ensure all objects are destroyed after use avoid memory leaks

MVG Applets

Keep to a minimum as they increase SQL complexity

Use primary joins as much as possible; reduces number of SQL
statements

Use SmartScript only where really required

E.g. Novice Call Center users

Avoid putting external news pages on login page

3rd party web sites can be slow

Application Performance - Configuration

Activate only fields that are absolutely necessary

Become active when displayed; Force Active in BC configuration

General rule, the more fields on a page the slower the page
Recently saw customer with 750 fields and 80 tables for one applet

Include only manageable set of fields on list applets

Many customers use several dozen fields; not usable and slow
performance

Pages with more than three applets will perform slower for
HI applications
Reason for this is not really the number of applets, rather the number
of fields

PickLists set Long List to TRUE if returning large data sets

Tree Applets can be slow

They generate Bill of Materials type queries

Network Performance Siebel Configuration

Browser Validation

Reduces the need for server communications to validate data entry

Implement through browser script

Immediate Posting of Changes

Where the Immediate Post Changes flag is set against a field data
will be transferred whenever a field is changed
Incurs additional round trip with approx 2KB data

Keep to no more than two Applets per View

Minimize Popups
Limit columns in List Applets

Network Performance Siebel Settings

View Caching

View definitions cached in browser memory

Requires approx 3MB memory per view
Typically around 10 cached views is enough
Uses LRU algorithm to maintain cache contents
Personalization and Applet Toggles wont use view caching
Enabled through Object Manager configuration (.cfg) file setting
[SWE]
EnableViewCache=TRUE
Controlled through:
User Preferences > Behaviour > View Cache Size
Default: 10

Network Performance Siebel Settings

Compression (Dynamic Content)
Performed by SWSE
Do not use web server dynamic compression (application files)
Enabled through SWSE configuration file (eapps.cfg)
[Defaults]
DoCompression = TRUE

Compression (Static Content)

Performed by web server (IIS only)

Network Performance Web Server / Browser Settings

Browser Settings
Dont clear cache except when necessary
Ensure Empty Temporary Internet Files Folder when browser is
closed option is not enabled.

Network Performance Web Server

Web Server Settings
Use HTTP keep-alive
Reduces the need to negotiate TCP sessions for each HTTP message

Network Performance Web Server

Web Server Settings
Set Content Expiration
2 days is typical setting
Set through Internet Information Services Administration
HTTP Headers > Content Expiration

Network Performance Web Caching

Uncached

GET:icon.gif

25KB
icon.gif
DATE:10/08/0307:14
RESPONSE:icon.gif
DATE:10/10/0309:25:08
LASTMODIFIED:10/08/0307:14

Network Performance Web Caching

Cached

GET:icon.gif
IFMODIFIEDSINCE:10/10/0309:25

2KB
icon.gif
DATE:10/10/0309:25
RESPONSE:Notmodified

icon.gif
DATE:10/08/0307:14

Network Performance Web Caching

CachedwithExpiration

0KB
icon.gif
DATE:10/10/0309:25
EXPIRES:10/12/0314:13:08

icon.gif
DATE:10/08/0307:14

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

Performance Problem Diagnosis

What is the problem?

Slow response times reported by User(s)?

Resource utilisation problem reported by Administrator?

Where is the problem?

Who is experiencing the problem? One user, a subset of users, all

users?
Network problem?
Database problem?
Siebel Server problem?
Web Server problem?

How do we resolve the problem?

Performance Problem Diagnosis continued

Problem Identification

User reports slow response times

Identify precise actions carried out by user at the precise time of the
problem set up a problem reporting process, with required
information to be supplied by user
Subset of Users report slow response times
Are users on the same LAN segment? Are other users experiencing
problems?
Are users all performing similar tasks?
All users report slow response times
Is it specific transactions (views) that are slow?
Do all transactions run slowly?
System Administrators should check system resource utilisation
System Administrator(s) report resource utilisation problems
Are users experiencing problems as a result?
Which area of the system is experiencing problems

Ask questions of the relevant people need more

information than The system seems to be slow

Performance Problem Diagnosis continued

Diagnosis Tools

Database SQL Trace (SQL Server Profiler)

Siebel Server Object Manager SQL Trace
Web Server SWSE Statistics Page
Web Server SWSE Log
Web Server Log (e.g. IIS log)
Server Resource Utilisation tools
Windows: PerfMon, Task Manager

TECHNOTE
361
Memory Utilization
in Siebel eBusiness
Applications

TECHNOTE
382
How can users
generate
Performance
Monitor information
into a log file?

Performance SQL Trace

Use Database SQL Trace or Siebel Component SQL Trace

to identify poor-performing queries
Correct Application Configuration
Add indexes (except unique indexes)
Refresh database statistics (not on Oracle)

Siebel Component SQL Trace

Set SQL Trace Flags parameter

Set Event Log Levels to 5 for %SQL%
Srvrmgr: change evtloglvl %SQL%=5 for component sccobjmgr_enu
server <siebel server name>
Scan log file for SQL Statement Execute Time

Need to use SQL Server Profiler for full capture

Siebel uses specialized ODBC connection models

Need to reproduce fully in order to get correct query execution plan

Performance Siebel Web Server Logs

SWSE Log

<Install Dir>/SWEApp/log/ssyymmdd_nn.log
Set log=details in eapps.cfg file for more detailed logging
Set environment variables for full detail
SIEBEL_SESSMGR_TRACE=1
SIEBEL_SISNAPI_TRACE=1
SIEBEL_LOGEVENTS=ALL
Restart Web Server

Web Server Log

IIS: <Windows Install Dir>\system32\Logfiles\W3SVC1\exyymmdd.log

In Internet Services Manager, enable extended logging to include
Client IP address, User Name, Time Taken
Use to identify long-running HTTP requests or individual user
machines experiencing problems

Performance Siebel SWSE Statistics Page

URL: <Application URL>/_stats.swe?verbose=high

<Application URL>/_stats.swe?verbose=high&reset=true to reset

stats

Eapps.cfg configuration file:

Allowstats=TRUE
SessionMonitor=TRUE
Allows individual user session statistics, in Current Sessions section:
Event

siebel.TCPIP.None.None://jmullisp4:2320/siebel/SCC
ObjMgr_enu/jmullisp4/!1.8c0.4023.3ec3708d
SADMIN

TotalTime

14.6779

GeneralStats
(count,mean,
stddev)

Frequency
(mean,std
dev)

54
0.2718
0.6826

11.33
59
44.64
84

Total Time: Time for all users HTTP requests

General Stats-Count: # user HTTP requests
General Stats-Mean: Average time for each user HTTP request
Frequency-Mean: Average time between each user HTTP request

Performance Siebel SWSE Statistics Page continued

Session Identification

!A.B.C.D
A = Siebel Server Id
B = Siebel Server Component Process Id
C = Siebel Server Component Task Id
D = Timestamp
All values in hexadecimal
Use to map session information to Siebel Server Task logfile

Section Current Operations Processing

Operations in bold type have been running > 10 seconds

Volume Performance Testing

Using Volume Performance Testing Tools, such as

Mercury LoadRunner
Segue SilkPerformer

Use to measure
Client Response times under high user load
Resource usage under high user load
Web Servers, Siebel Servers, Database Server, Network
Scalability of Siebel environment

Hardware
Use full duplication of production hardware, if possible
If not, scale down numbers of servers proportionately

Database

Tune Object Manager SQL prior to volume performance test exercise

Utilise Object Manager SQL trace flags

New Feature - SARM

Features

Siebel Application Response time

profiling SARM (7.5.3)
ARM Standard Support (7.7)
CPU and Memory Usage Profiling
(7.7)
Additional Siebel component
metrics (7.7)

Benefits

Proactive monitoring of
application response time
Diagnosis of response time
problems in the application and
infrastructure
Enables tuning of applications to
meet service level commitments
Enables diagnosis of memory
consumption

SARM Run-time Architecture

Siebel
Siebel Server
Server
SARM
Start

Server
Component
SARM
Stop
SARM Correlation Key
SARM
Start

SARM
Framework

3rd Party
ARM API
Library

SARM
Log

Server
Component
SARM
Stop

3rd Party
ARM Tool

ARM
Log

SARM Instrumentation for 7.5.3

Timers

User Interface
Physical UI Rendering

Web Server
Time

Logical UI
Network Time
SWE Time
Workflow Time
Script Time
SRB Time
Comp Time

Object Manager
Data Manager

Siebel
Database

External
Database

EAI

Legacy
Application

Database
Time

Performance and Scalability

ArchitectureOverview
ComponentScalability
ScalabilityAcrossComponents
NetworkScalability
PerformanceOptimization
PerformanceManagement

Questions and Answers

Siebel 7 Performance and Scalability

Inside the Siebel Server
Richard Sands
Siebel Expert Services
Siebel Systems 2003 Do not distribute or re-use without permission