WebLogic Server System Administration Top Ten Fundamentals Concepts Session ID# 11579

Scott Brinker, College of American Pathologist Shyam Kumar, AST Corporation

Agenda
Introduction About WebLogic Server WebLogic Server Administration Top Ten Concepts Hands-on Q&A

Introduction CAP College of American Pathologist the leading organization of board-certified pathologists, serves patients, pathologists, and advocating excellence in the practice of pathology and laboratory medicine
It is the worlds largest association composed exclusively of board-certified pathologists and is the worldwide leader in laboratory quality assurance. Serving more than 18,000 physician members and the global laboratory community

More than 7,000 laboratories are accredited by the CAP

Headquarters located in Northfield (Chicago), IL Implemented Oracle Middleware stack such as WebCenter Suite, SOA Suite, BPM Suite, IDM Suite, OBIEE, Hyperion

Specialized. Recognized. Preferred. The right partner makes all the difference.

Our Services
E-Business Suite Business Intelligence

Oracle Partnership
Oracle Platinum Partner Pillar Partner

Oracle Specialized
EBS Financial Management EBS Supply Chain Management

EPM-Hyperion
Fusion Middleware Managed Services Oracle University Project Advisory Services

SOA
Business Intelligence Hyperion Oracle University Approved Education Center Oracle University Reseller

EBS Human Capital Management

BI Applications BI Foundation Service-Oriented Architecture Public Sector

Oracle Accelerator Implementer

Certified OnDemand Implementer Small Business Strategy Council
Oracle Titan Award Winner 2011 & 2009 2011 Inc. Top Small Company Workplaces 2012 Inc. 5000 Fastest Growing Companies 2012 Best & Brightest Companies to Work For

WebLogic Server - Introduction

#1 in worldwide Application Server Software, with a growth rate of 11.7% The Foundation for Oracle Applications and Middleware Technologies Platform The Foundation for Cloud Applications and Application Infrastructure Consolidation
WebLogic 10.3.6

Supported features such as

Monitoring & Management Scalability/Clustering

Virtual Assembly Builder

Java EE Standards Enterprise Security Enterprise Messaging Linear Scalability Coherence etc

A Quick Poll?
How many of you are Business Users How many of you are Technical Users How many of you have actively been using WebLogic Server

WebLogic System Administration Top 10 Fundamentals Concepts

These are the concepts for WebLogic Server Administration based on our experience in implementing and supporting WebLogic Server platform These are generic concepts that would apply in any Oracle Middleware Implementations These are the Tips and Tricks

#10 Knowing Administration Console Important Elements

Tool Bar Appears on top of Console.

How do I The Panel includes the context sensitive link to Online Help. Change Center Lock to make any Change in WebLogic Server Domain Structure Admin Console Pages Tree Structure

System Status Reports the Health of system

#9 - WebLogic Domains - Fundamentals!!

WebLogic Server administration domain is a logically related group of WebLogic Server resources.
Domains include a special WebLogic Server instance called the Administration Server, which is the central point from which you configure and manage all resources in the domain.

Contents of a Domain
Administration Server Managed Servers Resource Services

#9 - WebLogic Domain Architecture!!

Depending on your particular business needs and system administration practices, you might decide to organize your domains based on criteria such as:
Logical divisions of applications Physical location Size

#8 - Installation of JVM, WLS, etc. Use standard paths between environments

DO NOT use Upper and Lower Case

Be consistent between the Paths

Use Environment variables consistently JAVA_HOME, DOMAIN_HOME, etc. Check for port conflicts before installing and configuring domain

Use JRocket as JVM and tune your JVM as per you Application requirements
Consistent log file configurations and proper log rotation sequence

#7 Resource Management

Mbean Mgmt/SNMP
JDBC - Connection pool waiters JDBC - Active Connections Current Count JVM - Heap Size JVM - Garbage Collection JMS ConnectionsCurrentCount JMS MessagesCurrentCount Threads PendingRequestCurrentCount Threads - QueueLength

Monitoring file system space, CPU and Memory

#6 - Thread Dumps for Troubleshoot & Debug WLS

Why Take Thread Dumps?
-

To understand what is actively executed in the system when under load When system is sluggish or slow or hangs
Java Virtual Process running but Server or App itself not responding

Pages take forever to load

Response time increases Application and or Server behavior erratic

Just to observe hot spots (frequently executed code) under load For Performance tuning & for Dependency analysis To capture snapshot of running state in server

#6 - Thread Dumps for Troubleshoot & Debug WLS

Methods to obtain thread dumps for analysis:
Use WLS Admin Console - Login to AdminConsole>Server > Monitoring > Threads

WLST create .py script

Weblogic.admin (WLS 8.1 and under)
java weblogic.Admin -url t3://localhost:7001 -username weblogic -password weblogic1 THREAD_DUMP

Jrockit JVM
jrcmd PID print_threads

Oracle/Sun Java JVM

kill -3 PID

Windows control + break in wls startup cmd script startweblogic.cmd, startmanagedweblogic.cmd Process Explorer

#6 - Thread Dumps for Troubleshoot & Debug WLS

Always take 4 or 5 thread dumps within a 20-25 sec period

Locking States
State
blocked

Description
This thread has tried to enter a synchronized block, but the lock was taken by another thread. This thread is blocked until the lock gets released. This is the same state as blocked, but with the additional information that the lock in question is a thin lock. This thread has called Object.wait() on an object. The thread will remain there until some other thread sends a notification on that object. This thread has called java.lang.Thread.sleep().

blocked (on thin lock) waiting

sleeping

parked
suspended

This thread has called java.util.concurrent.locks.LockSupport.park().

The threads execution has been suspended by java.lang.Thread.suspend() or a JVMTI/JVMPI agent call

#5 Log File Management

Consistent log locations across environments Shared file system/folder Leverage log rotation properly and efficiently Tools to view logs WLS Admin Console , EM Console, file system access and 3rd party tools (sawmill for example)

#4 - Back-Up/Restore Server Files

Perform a full offline backup shutdown all processes

- WLS - Node Managers - Listeners - EM and emAgent - OHS - WebCache

Perform file system backups using OS tools such as tar: tar zcvpf backup1.tar ... Test backup by performing recovery on another computer Configure automatic backup of WLS configuration it will create config-booted.xml and config-original.xml

#3 - Temporary Files and Cache Management

WebLogic creates the temporary files in various temp folders <XYZ>/bea/user_projects/domains/<yourdomain</servers/<yourse rver>/tmp
These Cache would often create issues so it would be recommended to delete all files/folders from tmp folders and Restart the server.

#2 WebLogic Diagnostic Framework (WLDF) Watch & Notification

WebLogic Diagnostics Framework (WLDF) consists of a number of components that work together to collect, archive, and access diagnostic information about a WLS Server instance & the applications it hosts such as Watches and Notifications - The Watch and Notification component of the WebLogic Diagnostics Framework (WLDF) provides the means for monitoring server and application states and then sending notifications based on criteria set in the watches

#2 WebLogic Diagnostic Framework (WLDF) Watch & Notification

A Watch identifies a situation that you want to trap for monitoring or diagnostic purposes & can be configured to analyze log records, data and events. A watch is specified as a watch rule, which includes:
A watch rule expression An alarm setting One or more notification handlers

A notification is an action that is taken when a watch rule expression evaluates to true. WLDF supports the following types of notifications:
Java Management Extensions (JMX) Java Message Service (JMS) Simple Mail Transfer Protocol (SMTP), for example, e-mail Simple Network Management Protocol (SNMP) Diagnostic Images

#1 WebLogic Diagnostic Framework (WLDF) Dashboard

Monitoring Dashboard provides views and tools for graphically presenting diagnostic data about servers & applications

WebLogic Reference Documents

WLS 11.1.1.6 http://docs.oracle.com/cd/E23943_01/wls.htm WLS 11.1.1.5 http://docs.oracle.com/cd/E21764_01/index.htm

Questions? Comments?

THANK YOU
Scott Brinker sbrinker@cap.org Shyam Kumar skumar@astcorporation.com