D73819GC10 SG PDF

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY.
COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
D74667
Edition 1.0
October 2011
D73819GC10
11
Student Guide
What's New in Oracle Solaris
Oracle University and ORACLE CORPORATION use only

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED
Copyright © 2011, Oracle and/or it affiliates. All rights reserved.

Authors
Michael Ernest Disclaimer
Gary Riseborough This document contains proprietary information and is protected by copyright and
other intellectual property laws. You may copy and print this document solely for your
Marcus Flieri
own use in an Oracle training course. The document may not be modified or altered
Bart Smaalders in any way. Except where your use constitutes "fair use" under copyright law, you
may not use, share, download, upload, copy, print, display, perform, reproduce,
Dave Miner publish, license, post, transmit, or distribute this document in whole or in part without
Nicolas Droux the express authorization of Oracle.
Dan Price The information contained in this document is subject to change without notice. If you
Cindy Swearingen find any problems in the document, please report them in writing to: Oracle University,
500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not

Glenn Fadden warranted to be error-free.
Liane Praza Restricted Rights Notice
If this documentation is delivered to the United States Government or anyone using

Technical Contributors the documentation on behalf of the United States Government, the following notice is
and Reviewers applicable:
Mike Tracey U.S. GOVERNMENT RIGHTS
Mike Carew The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or
disclose these training materials are restricted by the terms of the applicable Oracle
license agreement and/or the applicable U.S. Government contract.
Editor
Trademark Notice
Malavika Jinka
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names
may be trademarks of their respective owners.
Publishers
Nita Brozowski
Sumesh Koshy
Contents
Preface
1 Introduction
Oracle Solaris: The Mission Critical OS 1-2
Raising the Bar Set by Solaris 10 1-3

SPARC Enterprise Servers 1-4
SPARC T3 Servers: Scaling to New Heights 1-5
Oracle Solaris: Platform Choice and Flexibility 1-6
Serious About Oracle Solaris 1-7
Oracle Addresses Range of Customer Needs 1-8
Topic Outline 1-10
Module Structure 1-11
2 Image Packaging System (IPS) and Automated Installer (AI)

IPS Design Goals 2-2
IPS Implementation 2-3
IPS Package 2-4
Package Naming 2-5
IPS Repository 2-6
Starting the packagemanager GUI 2-7
Starting the packagemanager GUI - 2 2-8
pkg Subcommands 2-9
pkg Subcommands 2 2-10
Example: Search, List, and Install 2-11
Installing a Package with Dependencies 2-12
Verifying a Package 2-13
Fixing a Package 2-14
Listing Package Contents 2-15
Removing a Package 2-16
Updating a Package 2-17
Creating a Package 2-18
Group Packages 2-19
Other Commands and Utilities 2-20
AI: Why Replace JumpStart? 2-21
Rosetta Stone for Solaris 10 Users 2-22
AI Components and Features 2-23
AI Terminology 2-24
iii
Flow of Automated Installation 2-25

Creating an AI Service 2-26
Creating an IPS Repository 2-28
Creating AI Clients 2-29
JumpStart to AI Mapping 2-30
IPS References 2-31
AI References 2-32

3 Network Virtualization 1
Feature: Overview 3-2
Virtual NICs (VNICs) 3-3
Virtual NICs (VNICs) 2 3-4
Virtual Switches 3-5
Physical Wire, Physical Machines 3-6
Virtual Network: Example 3-7
Creating VNICs and Etherstubs 3-8
Unified Data Link Properties 3-9
Virtual Bridges 3-10
ipadm 3-11
Managing Interfaces and IP Addresses 3-12
Managing Interface Properties 3-13
Creating Flows 3-14
Data Link Vanity Naming 3-15
Resource Pools 3-16
dlstat(1M) 3-17
Other Network Observability Enhancements 3-18
Rethinking Zones 3-19
Other Solaris 11 Enhancements 3-20
4 ZFS Features in Solaris 11

Enhancements 4-2
Boot Environments 4-3
Boot Environments (BE) 4-4
Creating a Boot Environment 4-5
Activating a Boot Environment 4-6
Destroying a Boot Environment 4-7
Mounting and Unmounting a Boot Environment 4-8
Creating New Boot Environments 4-9
Creating New Boot Environments - 2 4-10
BE Upgrade with pkg-update 4-11
Deduplication 4-12
iv
Deduplication Example - 1 4-13

Deduplication Example - 2 4-14
Root Pool Mirroring 4-15
Snapshot Differences 4-16
zfs diff Output 4-17
Send Stream Enhancements 4-18
Send Stream: Override Example 4-19
Send Stream: Enforce Example 4-20

Send Stream: Ignore Example 4-21
Pool Import: Log Device Recovery 4-22
Pool Import Recovery: Example 4-23
Pool Import: Read-Only Mode 4-24
Synchronous Write Behavior Property 4-25
Values for sync Property 4-26
ZFS Synchronous Behavior: Tuning Caveats 4-27
RAIDZ/Mirror Performance 4-28
Integrating ZFS into Deployment 4-29
Performance Notes 4-30
Other ZFS Features 4-31
ZFS References 4-32
5 Zones
Changes Since Solaris 10 FCS 5-2
Design and Features 5-7
Storage 5-8
Networking: Exclusive IP Zones 5-9
Networking: Shared IP Zones – IPMP 5-11
Zones Observability 5-12
zonestat Command 5-13
zonestat Interval: Example 5-14
zonestat by Resource: Example 5-15
Resource Management 5-16
Zones Security 5-17
Solaris 10 Containers 5-18
Solaris 10 Container: Expected Migration Path 5-19
References 5-20
6 Network Virtualization 2
Advanced Network Features 6-2
ilbadm: L3/L4 Integrated Load Balancing 6-3
Load Balancing Components 6-4
v
ilbadm: Example 6-5

IP Filter, Forwarding in a Zone 6-6
Hardware Lanes and Dynamic Polling 6-7
Hardware Lanes 6-8
ipmpstat: Observability for IPMP Groups 6-9
ipmpstat: Example 6-10
Fiber Channel over Ethernet (FCoE) 6-11
Virtual Router Redundancy Protocol (VRRP) 6-12

IP over Infiniband (IPoIB) 6-13
Non-Uniform Memory Architecture (NUMA) I/O 6-14
NUMA I/O Architecture: Overview 6-15
GLDv3 Public Driver APIs 6-16
Network Performance Highlights 6-17
7 Security
Features 7-2
Root Implemented as a Role 7-3
File system encryption: zfs(1M) 7-4
Configuring ZFS Encryption 7-5
File system encryption: lofiadm 7-6
Network Spoofing Protection 7-7
Zones: Delegated Administration 7-8
SMF: Delegated Administration 7-9
SMF: Method Context 7-10
SMF: Firewall Integration 7-11
Least Privilege Changes 7-12
“In-kernel pfexec” 7-13
Basic Privileges: More is Less 7-14
Role-Based Access Control 7-15
Sandboxing Enhancements 7-16
Kerberos Improvements 7-17
Key Management: pkcs11_kms Provider 7-18
Other Enhancements 7-19
Oracle Solaris 11 Trusted Extensions 7-20
Trusted Extensions Changes 7-21
Trusted Platform Modules (TPM) 7-22
8 Services Management Facility (SMF)

SMF Design Goals 8-2
SMF Is the Glue in Solaris 11 8-3
Service Templates 8-4
vi
IPS Actuators 8-8

Fault Notification 8-7
Early Manifest Imports 8-5
SMF Enhanced Profiles 8-6
FMRI Stored in proc_t Structure 8-9
vii

Preface


Profile
Before You Begin This Course
You should be able to configure and manage a system running the Oracle Solaris
Operating system.
How This Course Is Organized
An understanding of Oracle Solaris features and working knowledge of the Oracle
Solaris 10 Operating System is beneficial, but not required
How This Course Is Organized
S What's New in Oracle Solaris 11 is an instructor-led seminar featuring lecture and

demonstrations. Online demonstrations and written practice sessions reinforce the
concepts and skills introduced.
• read.me files
• Oracle Magazine
Related Publications
• System release bulletins
• Installation and user’s guides
• International Oracle User’s Group (IOUG) articles

Introduction
Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Solaris: The Mission Critical OS

If It Must Work, It Runs on Solaris
• The #1 deployment platform for the

#1 mission critical Oracle Database
• Extreme data integrity: ZFS
• Hardened security: Secure by Default, Cryptographic

Framework, Least Privilege model
• Predictive Self Healing—FMA, SMF
• Complete Virtualization with application isolation and resource
management: Containers
• Production Safe Observability: DTrace
• Scalable to thousands of threads, terabytes of memory
What's New in Oracle Solaris 11 1 - 2

Raising the Bar Set by Solaris 10
Oracle Solaris 11
The Only Completely Virtualized OS
• Availability: Greatly improved with new packaging tools, safe
online upgrades, faster reboots

• Scalability and Performance: Thousands of threads, terabytes of
RAM, hundreds of Gbps network bandwidth
• Efficiency: Virtualized network, storage and server resources;
binary compatibility; advanced power management
• Security: On-disk data encryption, secure process execution, HW
certification of the OS at boot time

SPARC Enterprise Servers

The Leader in System Scalability
5 Year Trajectory
Cores 4x
Threads 32x
Memory Capacity 16x SPARC
Database TPM 40x 1-64 Sockets
Java Ops Per Second 10x +2x Throughput

+1.5x Single
M-Series Strand
8-64 Sockets
+2x
T-Series Throughput
1-8 Sockets
M-Series +3x
8-64 Sockets Throughput
+6x Throughput
T-Series +1.5x Single
1-4 Sockets Strand
M-Series +3x Single Strand
T-Series 1-64 Socket
1-4 Socket + 20%
+ 2x
Throughput
Solaris 11 Solaris 11 Solaris 11 Solaris 11 Solaris 11
Express Update Update Update
2010 2011 2012 2013 2014 2015

SPARC T3 Servers: Scaling to New Heights

Integrated, High Throughput SPARC Systems for Massive Scale
SPARC T3-4
World’s First 16
SYSTEM THROUGHPUT HIGH
Core Processor

• 64 cores
SPARC T3-2
• 512 threads
• Best scale
SPARC T3-1 • 32 cores
• Most security
• 256 threads
SPARC T3-1B Blade • Medium scale • Enterprise-
• 16 cores
for Blade 6000 ready
• 128 threads • Middleware
• Entry-level consolidation
• 16 cores
• Price/performa • Enterprise-
• 128 threads ready
nce
• Best density
• Best RAS
CONSOLIDATION HIGH
VIRTUALIZATION HIGH

Oracle Solaris: Platform Choice and Flexibility
Solaris Solaris Solaris Solaris 10

Zone 8 or 9 Zone Zone*
Zone*

Oracle SPARC x86 Oracle x86
• Built-in scalable, platform- • Consolidation path for older Solaris

independent virtualization versions
• Native, bare metal performance • Leverages server virtualization
technology
Binary Compatibility Guaranteed

Serious About Oracle Solaris

Investments in Oracle Solaris 11
• SPARC, x86 support
• Exadata and Exalogic
Compute, Storage, Network

• Over 2,700 projects, over 400 inventions
• Over 20 million hours of development
• Over 60 million hours of testing
• Over 56 million tests
• Over 11,000 applications
Solaris 11: Coming in 2011

Oracle Addresses Range of Customer Needs

High Performing Application-to-Disk Solutions from a Single Vendor
Engineered Systems

Efficiency HIGH
Oracle’s Optimized
Solutions
Applications
Fusion Middleware
Database
Compute, VM Solaris/OEL
Storage, Network, Server
Software Storage
Manageability and Simplicity HIGH

The preceding is intended to outline our general product

direction. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be

relied upon in making purchasing decisions.
The development, release, and timing of any features or
functionality described for Oracle’s products remain at the sole
discretion of Oracle.

Topic Outline
• Morning
– Image Packaging System
– Automated Installer
– Networking (Crossbow)

• Afternoon
– Solaris Containers
– ZFS
– Security
– SMF (Application Deployment)

Module Structure
Focus on enhancements since Oracle Solaris 10 9/10 release

Command-line examples included with slides
Feature demonstrations at instructor's discretion
• Use cases blogged daily

• Demo environment is generic
– VirtualBox instance
— Unless special arrangements are made
– Text install, slim_profile added

• Demo scripts available to those interested


Automated Installer (AI)

Image Packaging System (IPS) and

IPS Design Goals
Use one process for installing, patching, and upgrading

• Minimize system downtime
• Reverse install operations easily


IPS Implementation
Relies on ZFS for safety

• Makes fast, safe copies with snapshots and clones
• Can apply changes to cloned BEs when desired
• Avoids conditions imposed by patches that overwrite files

– Single-user mode to prevent untimely access
– Deferred activation to prevent uncoordinated access
— Problem: A file that has been patched is available immediately
for use. A program that depends on it, however, will not work
until the system is rebooted.
— http://blogs.oracle.com/patch/entry/deferred_activation_patching

IPS Package
New model incorporates all software change types

• Includes dependencies automatically
• Installs only what is required to complete a package
• Each package is associated with a publisher

• Replaces metacluster model with profiles that can overlap
• Supports signed packages
• Uses a fat package model
– All variations in one: SPARC/x86/debug/nondebug
• Available from a repository

Package Naming
• Packages use a Fault Management Resource Identifier

(FMRI)
– pkg://solaris/library/libc@5.11,5.11-
0.75:20071001T163427Z

• Package categories establish a namespace
– Similar to SMF service names
• Each version has its own tuple
– libc@5.11,5.11-0.75:20071001T163427Z
– <component>,<build>-<branch>:<time stamp>

IPS Repository
• Networked software catalog service

– Incremental or monolithic downloads
– Built-in software release versioning
– Avoids media size as a delivery constraint

– Publishes catalog of available software
– Automates retrieval of new dependencies, updates
– Download/unzip/install steps unnecessary
• Default publisher
– http://pkg.oracle.com/solaris/release/

or

Starting the packagemanager GUI


Starting the packagemanager GUI - 2

pkg Subcommands
• /usr/bin/pkg
• pkg list
– List packages installed on the system
•

pkg search <pkg_name|pattern>
– Identify the package that a file (or pattern) belongs to
– Install packages and configure repositories
– Limit search to local packages with -l option
• pkg info <pkg_name>
– Lists package details

pkg Subcommands 2
• pkg install <pkg_name>

• pkg uninstall <pkg_name>
• pkg verify <pkg_name>
– Validate a package’s installation

• pkg fix <pkg_name>
– Fix errors reported by pkg verify
• pkg contents <pkg_name>
– Display the objects making up a package

Example: Search, List, and Install
# pkg search /usr/bin/ncftp

INDEX ACTION VALUE PACKAGE
path file usr/bin/ncftp pkg:/network/ftp/ncftp@3.2.3-0.151.0.1
# pkg list pkg:/network/ftp/ncftp

pkg list: no packages matching 'pkg:/network/ftp/ncftp' installed

# pkg install ncftp
Packages to install: 1
Create boot environment: No
DOWNLOAD PKGS FILES XFER (MB)
Completed 1/1 13/13 0.5/0.5
PHASE ACTIONS
Install Phase 39/39
PHASE ITEMS
Package State Update Phase 1/1
Image State Update Phase 2/2

Installing a Package with Dependencies

# pkg install gimp
Refreshing catalog 1/1 solaris
Caching catalogs ...
Creating Plan
Services to restart: 6

library/desktop/libgweather 0/24 0/8732 0.0/68.0
...
image/library/gegl 23/24 8714/8732 68.0/68.0
Completed 24/24 8732/8732 68.0/68.0
PHASE ACTIONS
Install Phase 1/10557
...
PHASE ITEMS
...

Verifying a Package
# pkg verify ncftp

# ls -l /usr/bin/ncftp
-r-xr-xr-x 1 root bin 276012 Dec 7 20:39 /usr/bin/ncftp
# chmod 775 /usr/bin/ncftp

# pkg verify ncftp
Verifying: PACKAGE
STATUS
pkg://solaris/network/ftp/ncftp ERROR
file: usr/bin/ncftp
Mode: 0775 should be 0555

Fixing a Package
# pkg fix ncftp

Verifying: pkg://solaris/network/ftp/ncftp ERROR
file: usr/bin/ncftp
Mode: 0775 should be 0555
Created ZFS snapshot: 2010-12-07-23:29:09
Repairing: pkg://solaris/network/ftp/ncftp

Completed 1/1 2/2 0.1/0.1
PHASE ACTIONS
Update Phase 2/2
PHASE ITEMS
Package Cache Update Phase 1/1
# pkg verify ncftp

Listing Package Contents

# pkg contents ncftp
PATH
usr
usr/bin
usr/bin/ncftp
usr/bin/ncftpbatch

usr/bin/ncftpbookmarks
usr/bin/ncftpget
usr/bin/ncftpls
usr/bin/ncftpput
usr/bin/ncftpspooler
usr/sfw
usr/sfw/bin
usr/sfw/bin/ncftp
...

Removing a Package
# pkg uninstall ncftp
Creating Plan
Packages to remove: 1
PHASE ACTIONS
Removal Phase 1/33
Removal Phase 33/33

PHASE ITEMS
Package Cache Update Phase 1/1

PHASE ITEMS
Reading Existing Index 1/8
Indexing Packages 1/1

Updating a Package
Updating all installed packages to the latest version

# pkg update
Packages to update: 795
Create boot environment: Yes

Completed 796/796 4754/4754 205.2/205.2
PHASE ACTIONS
Removal Phase 2561/2561
Update Phase 6277/6277
...
A clone of solaris-39 exists and has been updated and activated.
On the next boot the Boot Environment solaris-40 will be mounted on '/'.
Reboot when ready to switch to this updated BE.

Creating a Package
• Easy to package existing software

$ pkgrepo -s file:/tmp/test-repo create
$ pkgrepo -s file:/tmp/test-repo set publisher/prefix=michael.oow.com
$ eval `pkgsend -s file:/tmp/test-repo open ilb_demo@1.0`
<exports a PKG_TRANS_ID value into shell environment>

$ pkgsend -s file:/tmp/test-repo import ~/ilb_demo
$ pkgsend -s file:/tmp/test-repo close
pkg://michael.oow.com/ilb_demo@1.0,5.11:20110912T012101Z
PUBLISHED
• Or emit a manifest
$ pkgsend generate ~/fu
file gnome_terminal_fu group=bin mode=0644 owner=root
path=gnome_terminal_fu pkg.size=326
file netbeans_fu group=bin mode=0644 owner=root path=netbeans_fu
pkg.size=283
file awk_fu group=bin mode=0644 owner=root path=awk_fu pkg.size=110

Group Packages
• Part of manual or automated install process

• Controls other installed packages (or package groups)
– babel_install installs slim_install
– slim_install is LiveCD content

• Must uninstall group packages to customize what they
control
– Remove babel_install to manage slim_install
– Remove slim_install to manage individual packages
– The automated installer will do this for you

Other Commands and Utilities
Other pkg(5) utilities

• pkg publisher
• pkg set-publisher
• pkgrepo(1)

• pkgsend(1)
• pkgrecv(1)
• pkgdepend(1)
• pkg.depotd(1M)
• pkgmogrify(1M)

AI: Why Replace JumpStart?
• To make updating/patching:
– Faster
– More reliable
– Easily reversible

• To leverage current technology
– Integrate with ZFS
– Leverage the IPS repository
– Apply SMF naming scheme
• To separate client and server dependencies
– Make the installer platform-neutral
– Let clients select their software repository

Rosetta Stone for Solaris 10 Users
Solaris 10 Solaris 11
SVR4 Packages IPS (SVR4 still supported)

Install media Starter image + IPS repository

Live Upgrade beadm(1M)
Upgrade option pkg update, Update Manager

JumpStart Automated Installer(AI)
JumpStart Profiles AI Manifests
Flash Install replication No equivalent yet
Blueprints for custom DVDs Distribution Constructor

AI Components and Features
• Three service components

– DHCP server (requires mDNS)
– SMF-based installer
– IPS repository

• Tools for managing and observing process
– Configure with installadm(1M)
– Observe clients using livessh install parameter
– Manage image with beadm(1M)
• AI is WAN Boot-ready

AI Terminology
• Client (installation target)

– Can be physical or virtual (not zones, yet)
• SMF Services
– svc:/network/dhcp-server:default

– svc:/system/install/server:default
– svc:/application/pkg/server
• Manifest – SMF-named install configuration
• Criteria – Properties that match client details to an
appropriate manifest


Flow of Automated Installation

Creating an AI Service
• Use Oracle Solaris DHCP or ISC DHCP

• installadm(1M) will manage DHCP if:
– svc:/network/physical:default (Not nwam)
– svc:/network/dns/multicast:default

– /etc/netmasks entry exists
– Default route is set
• Use AI-specific image
– sol-11-exp-201011-ai-{x86|sparc}.iso
– Server and client platforms do not have to match
– Cannot super-size the AI image from Text or LiveCD

Creating an AI Service
# pkg verify installadm

…
# installadm create-service -a sparc -n solaris_11 \
> -i 192.168.1.10 -c 3 -s ai_sparc_image.iso \
> /export/ai/sparc/solaris_11

…
# installadm list
…
-n <name> Install service name

-i <IP> DHCP start address
-c <count> DHCP range
-s <file.iso> AI source image
<target_directory>

Creating an IPS Repository
• Download Repository Image (two files)

– http://www.oracle.com/technetwork/server-
storage/solaris11/downloads/index.html
• Combine the files and:

– Burn it to media
– Or, mount it by using lofiadm(1M)
– Or, copy it to a ZFS file system with rsync(1)
• Enable repository service
– svc:/application/pkg/server:default
• For more details, see “How to Copy An Oracle Solaris 11
Software Package Repository.”

Creating AI Clients
• The client will get AI service location from DHCP.

• The client will get boot image, configuration, and repository
location from AI service.
• AI service identifies clients by MAC address.

– x86 clients can add other boot parameters.
• AI service binds clients to a named install service.
# installadm create-client -b "console=ttya,livessh=enable" \

> -e 0:e0:81:5d:bf:e0 -n s11-x86
…
# installadm create-client -e 00:14:4f:a7:65:70 -n s11-sparc
…

JumpStart to AI Mapping
JumpStart AI
setup_install_server installadm create-service
add_install_client installadm create-client

Manifests, driver updates, custom image
begin script
from Distribution Constructor
Client profiles, rules Manifests with client criteria
pkg actuators (before reboot)
finish script
“First-boot” SMF services
sysidcfg file SMF profile

IPS References
Adding and Updating Oracle Solaris 11 Software Packages

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=AUOSS


AI References
• Creating a Custom Oracle Solaris Installation Image

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=CCOSI
• Transitioning From Oracle Solaris 10 JumpStart to Oracle
Solaris 11 Automated Installer

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=MFJAI
• Creating and Administering Oracle Solaris 11 Boot
Environments
http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=CMBEA
• Installing Oracle Solaris 11 Systems
http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=IOSUI

Network Virtualization 1

Feature: Overview
• Virtualized NICs, switches, and bridges

• Dynamic IP address management
• Quality of Service (QoS)
– Control bandwidth by transport, service, protocol, or

connection
• Vanity naming for devices
• Fencing compute resources
– Assign NICs/VNICs to processor sets or pools
• Real time usage and history

Virtual NICs (VNICs)
• Same control as a physical NIC

– Private TCP/IP stack
– Managed with ifconfig, dladm, and so on
• Dedicated MAC address

– May be random, chosen, or device-assigned
• Can be bound to hardware and kernel resources

Virtual NICs (VNICs) 2
• Private TCP/IP stack

– Data path is separate, does not rely on modules added to a
global stack
• A complete, standards-based virtualization solution

– VLAN tags supported
– Priority Flow Control (PFC)
– With supporting hardware, can be fully encapsulated to the
switch

Virtual Switches
• VNICs sharing a VLAN id on one data link need a switch

• MAC layer provides built-in switching semantics
– Data path among VNICs sits on top of the data link
– Connects VNIC to physical network

– Isolates broadcast domains
• Want an explicit virtual switch? Use an etherstub:
– Makes any virtual network topology possible
– Can reduce or eliminate trips to physical NIC
– Can also manage resource controls

Physical Wire, Physical Machines
Client Router Host 1 Host 2
Port 6 Port 9 Port 3 Port 1 Port 2

20.0.03 20.0.01 10.0.03 10.0.01 10.0.02
1 Gbps 1 Gbps 1 Gbps 100 Mbps 1 Gbps

Switch 3 Switch 1
Virtual Wire, Virtual Machines
Virtual
Client Router Host 1 Host 2
VNIC6 VNIC9 VNIC3 VNIC1 VNIC2

20.0.03 20.0.01 10.0.03 10.0.01 10.0.02
1 Gbps 1 Gbps 1 Gbps 100 Mbps 1 Gbps
Etherstub 3 Etherstub 1


Virtual Network: Example

Creating VNICs and Etherstubs
# dladm create-vnic -l bge1 vnic1

# dladm create-vnic -l bge1 -m random –p maxbw=100M -p cpus=4,5,6 vnic2
# dladm create-etherstub vswitch1

# dladm show-etherstub
LINK
vswitch1
# dladm create-vnic -l vswitch1 -p maxbw=1000M –p cpus=4,5,6 vnic3
# dladm show-vnic
LINK OVER MACTYPE MACVALUE BANDWIDTH CPUS
vnic1 bge1 factory 0:1:2:3:4:5 - -
vnic2 bge1 random 2:5:6:7:8:9 max=100M 4,5,6
vnic3 vswitch1 random 4:3:4:7:0:1 max=1000M -
# dladm create-vnic -l ixgbe0 -v 1055 -p maxbw=500M -p cpus=1,2 vnic9

Unified Data Link Properties
• dladm [set,reset,show]-linkprop
• Alternative to ndd(1M) utility
• Single, stable interface for network property consumers
• Changes can be made temporary or persistent

$ dladm show-linkprop e1000g0
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
e1000g0 speed r- 1000 1000 --
e1000g0 duplex r- full full half,full
e1000g0 state r- up up up,down
e1000g0 flowctrl rw no bi no,tx,rx,bi
e1000g0 maxbw rw -- -- --
e1000g0 priority rw high high low,medium,high
e1000g0 protection rw -- -- mac-nospoof,
restricted,
ip-nospoof,
dhcp-nospoof
e1000g0 rxrings rw -- -- --

Virtual Bridges
• Data Link (Layer 2), 802.1D

• Detects MAC addresses VNIC VNIC VNIC
• Connects NICs, etherstubs,

link aggregations

• Lets you move a VNIC Bridge
without changing IP address
• Supports RBridges
(TRILL – Transparent etherstub
Interconnect of Lots of
Links)
• Manages with dladm NIC NIC

ipadm
• Consolidates management of
– Network interface state
– IP address assignment
– TCP/IP protocol properties

• Uses action-object subcommands like dladm
– create-if, show-if, disable-addr, and so on
• Supercedes various commands and files
– ifconfig
– /etc/hostname.<interface>
– ndd

Managing Interfaces and IP Addresses
# dladm create-vnic –l bge0 play1

# ipadm create-addr –T static –d –a 10.2.3.5/24 play1/v4static2
# ipadm show-if
IFNAME STATE CURRENT PERSISTENT

lo0 ok -m-v------46 ---
bge0 ok bm--------46 ---
play1 down bm--------46 -46
# ipadm show-addr
ADDROBJ TYPE STATE ADDR
play1/v4static2 static down 10.2.3.5/24
#
# ipadm up-addr play1/v4static2
# ipadm show-addr play1/v4static2
ADDROBJ TYPE STATE ADDR
play1/v4static2 static ok 10.2.3.5/24

Managing Interface Properties
# ipadm show-ifprop play1

IFNAME PROPERTY PROTO PERM CURRENT PERSISTENT DEFAULT POSSIBLE
play1 arp ipv4 rw on -- on on,off
play1 forwarding ipv4 rw off -- off on,off

play1 metric ipv4 rw 0 -- 0 --
play1 mtu ipv4 rw 1500 -- 1500 68-1500
play1 exchange_routes ipv4 rw on -- on on,off
play1 usesrc ipv4 rw none -- none --
play1 forwarding ipv6 rw off -- off on,off
play1 metric ipv6 rw 0 -- 0 --
play1 mtu ipv6 rw 1500 -- 1500 1280-1500
play1 nud ipv6 rw on -- on on,off
play1 exchange_routes ipv6 rw on -- on on,off
play1 usesrc ipv6 rw none -- none --

Creating Flows
• Define a flow by:

– Service (protocol + port address)
– Transport type (TCP, UDP, SCTP, iSCSI, and so on)
– IP address/subnet

– Differentiated Service Code Point (DSCP) label
• Flows can assign bandwidth caps (maxbw)
• Flows maintain their own kstat counters
– Use flowstat(1M)
– Use extended accounting for historical reference
# flowadm create-flow -l bge0 protocol=tcp,local_port=443 -p maxbw=50M http-1

# flowadm set-flowprop -l bge0 -p maxbw=100M http-1

Data Link Vanity Naming
• Vanity naming
– Set desired name via dladm(1M)
– List device interfaces in /dev/net
• Supports alternative to so-called PPA hack

– PPA: Physical Point of Attachment
– Name calculated with (VID*1000 + instance)
– Example: bge + (487 * 1000 + 1) = bge487001
knickknack@os11e:/dev/net$ ls -l
total 0
crw-rw-rw- 1 root sys 58, 1001 2010-12-19 17:37 beatnic0
crw-rw-rw- 1 root sys 20, 1 2010-12-19 14:22 e1000g0

Resource Pools
• Assigned CPUs process network traffic for a data link

– Both kernel threads and network interrupts
• Configured through pools data link property
– # dladm show-linkprop –p pool <datalink>

– Alternative to manual setting (cpus property)
• Pool configuration determines the CPUs selected
– svc:/system/pools:default
– Automatically updated if CPUs migrate to other pools
• Some zones use dynamic pools
– svc:/system/pools/dynamic:default
– Assigns CPUs on zone bootup, releases on shutdown

dlstat(1M)
• Observability for data link and flow statistics

– Measured per hardware/software ring
— For VirtualBox instance:
# kstat -n mac_rx_ring0

• Includes network traffic spread to other CPUs (aka fanout)
• Hardware lane counters (if NIC supports them)
$ dlstat -i 30
LINK IPKTS RBYTES OPKTS OBYTES
bge0 25.89K 16.90M 18.23K 4.42M
play0 5.64K 1.51M 226 15.61K
play1 5.55K 1.49M 131 7.63K
bge0 81 13.29K 19 7.13K

play0 62 9.37K 0 0
play1 62 9.37K 0 0

Other Network Observability Enhancements
• IP-layer observability
– Snoop loopback traffic between zones using shared-IP
— # snoop -I lo0
• Network DTrace providers

– udp: send, receive probes
– ip: send, receive, drop-in, drop-out probes
– tcp: send, receive, state-change,connect-
[request|refused|established|, accept-
[refused|established]
• tcpdump and wireshark are IPS packages
• Observe flows with flowstat
• Observe IPMP groups with ipmpstat

Rethinking Zones
• Consider using the global zone (GZ) as a system service

processor
– NGZs isolate processes, software stacks
– Resource controls cap NGZ consumption

— CPU binding, psets, or pools
— Virtual, resident set size (RSS), or paging memory
— Shared memory, semaphores
– An exclusive TCP/IP stack completes the picture.
• L2/L3 boundary: Data links (exclusive-IP property)
— Per-NIC in Solaris 10, per-VNIC in Solaris 11
• One example: the Immutable Service Container
– http://blogs.sun.com/video/entry/immutable_service_containers

Other Solaris 11 Enhancements
• Still more stuff in dladm(1M)

– VLAN, WiFi, IP tunnel management
• Network Auto-Magic (NWAM) service
– svc:/network/physical:nwam

– Automagic setup
– User can modify security, name services
— Manual control (CLI or GUI)
— Location-specific configurations

ZFS Features in Solaris 11

Enhancements
Key enhancements discussed in this module:

• Root pool boot environments (BE)
• Deduplication
• Root pool mirroring

• Snapshot diff capability
• Synchronous write behavior property
• Send stream enhancements
• Improved pool recovery

Boot Environments
• Makes updates safe, reliable, and recoverable

• Similar to Solaris 10 Live Upgrade
– ZFS only
• Managed by beadm(1M)

• Subcommands provide means to:
– List
– Activate
– Create, Destroy, Rename
– Mount, Unmount

Boot Environments (BE)
• ZFS is required.
• A BE is a special-purpose ZFS snapshot.
– beadm(1M) replaces lu* commands.
• All BEs reside in the root pool.

– No need to maintain partitions
• Integrated with IPS
– New BEs with package actuators
• Make new BE with pkg image-update or pkg update

Creating a Boot Environment
• Initial boot environment after installation

# beadm list
BE Active Mountpoint Space Policy Created
-- ------ ---------- ----- ------ -------
solaris NR / 2.81G static 2010-12-06 03:48

• Create a new boot environment by using beadm create
# beadm create S11-BE-1 && beadm list
-- ------ ---------- ----- ------ -------
S11-BE-1 - - 110.0K static 2010-12-09 04:23
solaris NR / 2.81G static 2010-12-06 03:48
• Active flags
– N = Active Now
– R = Active next Reboot

Activating a Boot Environment
• Activating a boot environment

# beadm activate S11-BE-1
# beadm list

-- ------ ---------- ----- ------ -------
S11-BE-1 R - 2.81G static 2010-12-09 04:23
solaris N / 120.5K static 2010-12-06 03:48
• After reboot
# beadm list
-- ------ ---------- ----- ------ -------
S11-BE-1 NR / 2.82G static 2010-12-09 04:23
solaris - - 7.37M static 2010-12-06 03:48

Destroying a Boot Environment
Destroying a boot environment
# beadm destroy solaris

Are you sure you want to destroy solaris? This action cannot be

undone(y/[n]): y
# beadm list
-- ------ ---------- ----- ------ -------
S11-BE-1 NR / 2.83G static 2010-12-09 04:23

Mounting and Unmounting a Boot Environment
Mounting and unmounting a boot environment

# beadm create S11-BE-2 && beadm list
-- ------ ---------- ----- ------ -------
S11-BE-1 NR / 2.83G static 2010-12-09 04:23

S11-BE-2 - - 45.0K static 2010-12-09 04:53
# beadm mount S11-BE-2 /mnt && beadm list

-- ------ ---------- ----- ------ -------
S11-BE-1 NR / 2.83G static 2010-12-09 04:23
S11-BE-2 - /mnt 11.67M static 2010-12-09 04:53
# beadm unmount S11-BE-2 && beadm list

-- ------ ---------- ----- ------ -------
S11-BE-1 NR / 2.83G static 2010-12-09 04:23
S11-BE-2 - - 12.08M static 2010-12-09 04:53

Creating New Boot Environments
Create a new BE with an IPS package change

# beadm list
-- ------ ---------- ----- ------ -------
S11-BE-1 NR / 2.84G static 2010-12-09 04:23

S11-BE-2 - - 12.08M static 2010-12-09 04:53
# pkg install --require-new-be --be-name=S11-BE-3 ncftp

Create boot environment: Yes
Completed 1/1 13/13 0.5/0.5
PHASE ACTIONS
Install Phase 39/39
PHASE ITEMS

Creating New Boot Environments - 2
PHASE ITEMS
Indexing Packages 1/1
A clone of S11-BE-1 exists and has been updated and activated.

On the next boot the Boot Environment S11-BE-3 will be mounted
on '/'.
# beadm list
-- ------ ---------- ----- ------ -------
S11-BE-1 N / 352.0K static 2010-12-09 04:23
S11-BE-2 - - 12.08M static 2010-12-09 04:53
S11-BE-3 R - 2.85G static 2010-12-09 05:19

BE Upgrade with pkg-update
New BE names are incremented by default

# pkg update …
A clone of zfsBE exists and has been updated and
activated.

On the next boot the Boot Environment zfsBE-1 will be
mounted on '/'.
# init 6
# beadm list
-- ------ ---------- ----- ------ -------
zfsBE - - 9.38M static 2010-10-15 09:18
zfsBE-1 NR / 10.76G static 2010-11-05 09:57

Deduplication
• Drops redundant data blocks

• Enabled per-file system: dedup property
• To determine benefit on the existing ZFS storage:
– # zdb -S <pool>

– http://hub.opensolaris.org/bin/view/Community
+Group+zfs/dedup
• Benefit is expressed similarly to compressratio
• Observable via zpool status
– Dedup operations have pool scope.

Deduplication Example - 1
bayle@os11e:~$ ls -l /usr/java/src.zip
-rw-r--r-- 1 root bin 19160179 2010-12-06 04:44
/usr/java/src.zip
bayle@os11e:~$ zfs set dedup=on rpool1/home/deirdre
bayle@os11e:~$ cp /usr/java/src.zip /home/deirdre/src1.zip

<copy in src[23456].zip>
bayle@os11e:~$ zfs list rpool1/home/deirdre
NAME USED AVAIL REFER MOUNTPOINT
rpool1/home/deirdre 110M 8.10g 110M /home/deirdre

Deduplication Example - 2
bayle@os11e:~$ zpool list

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
rpool1 15.9G 6.61G 9.27G 41% 6.00x ONLINE -
bayle@os11e:~$ rm /home/deirdre/*zip
bayle@os11e:~$ zpool list

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
rpool1 15.9G 6.61G 9.27G 41% 1.00x ONLINE -
bayle@os11e:~$ zfs list rpool1/home/deirdre
rpool1/home/deirdre 31K 8.12G 31K /home/deirdre

Root Pool Mirroring
• Root pools can be mirrored after installation

# zpool attach rpool <root_disk> <new_disk>
• Allow resilvering to complete

# zpool status rpool
• Boot blocks are installed automatically
• Verify bootability

Snapshot Differences
The zfs diff command lists differences between two

snapshots.
$ ls /home/timh
fileA

$ zfs snapshot tank/home/timh@old
<Create fileB>
$ ls /home/timh
fileA fileB
$ zfs snapshot tank/home/timh@new
$ zfs diff tank/home/timh@old tank/home/timh@new
M /tank/home/timh/
+ /tank/home/timh/fileB

zfs diff Output
Differences listed for files and directories:

• M: Modification or link count change
• -: Object is present in the first snapshot only
• +: Object is present in the second snapshot only

• R: Object has been renamed

Send Stream Enhancements
• Modify property values in a received dataset

• Enforce property value(s) in a sent dataset
• Disable property settings in a received dataset


Send Stream: Override Example
File compression is off for the tank/data file system. You

want to enable compression for the bpool/data file system.
# zfs get compression tank/data
NAME PROPERTY VALUE SOURCE

tank/data compression off default
# zfs send -p tank/data@snap1 | zfs recv -o
compression=on -d bpool
# zfs get -o all compression bpool/data
NAME PROPERTY VALUE RECEIVED SOURCE
bpool/data compression on off local

Send Stream: Enforce Example
The -b option declares the file system as a property source.

# zfs send -b bpool/data@snap1 | zfs recv -d restorepool
# zfs get -o all compression restorepool/data
NAME PROPERTY VALUE RECEIVED SOURCE

restorepool/data compression off off received

Send Stream: Ignore Example
• The receive -x option ignores property settings.

– Applies recursively to contained file systems
• For example: Ignore quota property setting:
# zfs send -R tank/home@1020 | zfs recv -x quota

bpool/home
# zfs get -r quota bpool/home
NAME PROPERTY VALUE SOURCE
bpool/home quota none default
bpool/home@1020 quota - -
bpool/home/cindys quota none local
bpool/home/cindys@1020 quota - -
bpool/home/tom quota none local
bpool/home/tom@1020 quota - -

Pool Import: Log Device Recovery
• Importing a pool with a missing log causes an error.

# zpool import dozer
The devices below are missing, use '-m' to import the
pool anyway:

c3t3d0 [log]
cannot import 'dozer': one or more devices is currently
unavailable
• Now, you can import the pool as-is (-m).
• Attach the missing log device.
• Use zpool clear to resolve errors.
• Works for mirrored log devices

Pool Import Recovery: Example
Example: Import Pool With Missing Log Device

# zpool import -m dozer
# zpool status dozer
pool: dozer
state: DEGRADED

status: One or more devices could not be opened. Sufficient replicas
exist for the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'. see:
http://www.sun.com/msg/ZFS-8000-2Q
config:
NAME STATE READ WRITE CKSUM
dozer DEGRADED 0 0 0
mirror-0 ONLINE 0 0 0
c3t1d0 ONLINE 0 0 0
c3t2d0 ONLINE 0 0 0
logs
14685044587769991702 UNAVAIL 0 0 0 was c3t3d0

Pool Import: Read-Only Mode
• May help in recovering a damaged pool

• All datasets are mounted in the read-only mode.
• Disables pool transaction processing
– No pending synchronous writes in the intent log are played.

– Ignored attempts to set a pool property
# zpool import -o readonly=on tank
# zpool scrub tank
cannot scrub tank: pool is read-only
• To revert to read-write, export, and import the pool

Synchronous Write Behavior Property
• The sync property defines per-file system write behavior

• Replaces the zil_disable tunable parameter
• The default setting is standard
– Write synchronous transactions to the intent log, flush

devices
# zfs set sync=always tank/home/perrin

Values for sync Property
Possible sync property values include:

• standard – Synchronous-write transactions: all
fsync(3C) calls, open(2) calls flagged with O_DSYNC,
O_SYNC.

• always – Write and flush all transactions to stable
storage. The system call returns upon completion.
• disabled – Commit transactions to stable storage with
the next flush, regardless of delay. Fast performance, no
risk of pool corruption. Data corruption is another matter.

ZFS Synchronous Behavior: Tuning Caveats
• A sync property value of disabled on the active BE or

/var may produce undefined behavior.
– Increases vulnerability to replay attacks
– Understand all the risks before using this value

• Processes that rely on synchronous behavior can lose
data with the disabled value.

RAIDZ/Mirror Performance
• Latest-and-greatest RAIDZ pools automatically mirror

latency-sensitive metadata.
– Pools created with b148 or later
– Pool version 29 or later

• Boosts I/O throughput
• Applies to all newly-written data
• Trades off space for time
– Does not improve resilience to failure

Integrating ZFS into Deployment
• Consider a separate file system per significant application.

• Monitor with fsstat(1M).
• Use snapshots for easy rollbacks.
• Use zfs diff to monitor changes.

• Apply encryption if appropriate.
• Use zfs send/receive for replication or backup.

Performance Notes
• On-disk encryption costs ~7% on random I/O and ~3% on

sequential I/O.
• RAID-Z mirror allocation – Some workloads show 2-4x
speedup on directory searches.

• Scrub/resilver ops now prefetch their metadata.
• System duty cycle (SDC) scheduler balances thread
priorities for CPU time.
• Slim ZIL reduces metadata I/O if data blocks are not full.
• Explicit ZIL behavior is controlled via sync property.

Other ZFS Features
• Dynamic LUN expansion

– autoexpand property
• Splittable mirrored pools (zpool split)
• Triple-parity RAID-Z (raidz3)

• Improved ACL compatibility with CIFS
• Automatic snapshots/Time Slider
– SMF service auto-snapshot
• User/group quotas
– Via userspace and groupspace subcommands

ZFS References
Oracle Solaris Administration: ZFS File Systems

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=ZFSADMIN


Zones

Changes Since Solaris 10 FCS
• Core
– Configurable privileges (limitpriv)
— Supports DTrace inside a zone
– Zone rename and move operations

– Zone migration (attach, detach)
– Software update on attach
— Default update is conservative
— Option -U will update all
– Boot arguments (bootargs)
• Packaging
– Parallel patching, turbo SVR4 packaging
– Live Upgrade support

• Resource management
– Overhauled and simplified (zone.*)
– CPU Caps added
— zone.cpu-cap, zone.cpu-shares

— See resource_controls(5)
– Enhanced observability
— Supported by getvmusage(2)
• Integration with ZFS
– Assign datasets to zones
– Faster provisioning with clones and snapshots

• Networking
– ip-type
– defrouter
• Brands

– Oracle Solaris 8 Containers
– Oracle Solaris 9 Containers
– Trusted extensions
– Sun Cluster integration
• Oracle Enterprise Manager Ops Center 2.5 Integration

Physical to virtual (p2v) migration

• Consolidate legacy instances as zones onto new hardware
– Available for Oracle Solaris 8, 9, and (other) 10 instances
• Process

– Create a system image
– Transfer to zonepath location
– Install the zone
• Image automatically updated during installation
– User-land/kernel need to be in sync
• Need to emulate Host ID

Solaris 11
Changes in Oracle


Design and Features
• lofiadm support
• v2v and p2v migration
• Branded Oracle Solaris 10 containers
• Exclusive-IP network stack enhancements

• zonestat
• IPMP support for ip-type

Storage
• lofiadm(1M), lofi(7D) supported

• New resource control to limit lofi devices
– zone.max-lofi
zonecfg:zone1> add rctl

zonecfg:zone1:rctl> set name=zone.max-lofi
zonecfg:zone1:rctl> add value (priv=privileged, limit=10, action=none)
zonecfg:zone1:rctl> end
zonecfg:zone1>

Networking: Exclusive IP Zones
Exclusive-IP options
• allowed-address property defines usable
address/range.
• defrouter property supports ip-type=exclusive.

# zonecfg -z zone1
zonecfg:zone1> set ip-type=exclusive
zonecfg:zone1> add net
zonecfg:zone1:net> set allowed-address=192.168.1.10/32
zonecfg:zone1:net> set physical=vnic1
zonecfg:zone1:net> set defrouter=192.168.1.1
zonecfg:zone1:net> end

Networking: Exclusive IP Zones
• Administration/tools available inside a zone

– dladm, flowadm, ipadm
– IP Tunnels
– IPMP

• Zones are ideal for virtual networking
– Configurable with multiple vnics
– Internal namespace for flows
• Layers 2 and 3 network protection
– Prohibit mischievous traffic from exclusive-IP zones
– (Try dladm show-linkprop protection)

Networking: Shared IP Zones – IPMP
• Solaris 10 IPMP, interface name changes on failover,

creating issues for some users
– For example: Using interface ce0:2 one moment, ce1:1
the next

– Zone admin has no control
• Solaris 11 IPMP
– Zone retains same interface
— ipmp0:2 remains ipmp0:2 for the zone session
– Zone admin can test interface for IPMP flag
— If set, the address is highly available.

Zones Observability
• Improved utilization monitoring

– CLI and Oracle Enterprise Manager integration
– Uses extended accounting (see acctadm)
— Also svcs extended-accounting

– Reports on both shared and dedicated resources
– Measures utilization against configured limits
• zonestat(1M)

zonestat Command
• zonestatd daemon performs monitoring

– Nonroot users and nonglobal zone users can see (some of)
the information
• zonestat can monitor:

– Virtual, physical, and locked memory
– Pools, psets, LWPs, and processes
– Shared-memory, semaphore, and message resources
• Can report specific zones, resource types
• Supports sorting by column
• Machine-parseable output is also available

zonestat Interval: Example
End-of-run reporting for average, high, and total usage
$ zonestat 5
Collecting data for first interval...

Interval: 1, Duration: 0:00:05
SUMMARY Cpus/Online: 32/32 Physical: 32.0G Virtual: 47.9G
----------CPU---------- ----PHYSICAL----- -----VIRTUAL-----
ZONE USED %PART %CAP %SHRU USED PCT %CAP USED PCT %CAP
[total] 1.57 4.92% - - 5660M 17.2% - 9.9G 20.6% -
[system] 0.09 0.28% - - 5086M 15.5% - 9275M 18.8% -
kodiak-dp 1.00 100% - 100% 46.0M 0.14% 4.49% 36.2M 0.07% 1.17%
global 0.48 1.56% - 1.56% 419M 1.27% - 673M 1.37% -
kodiak-ab 0.00 0.00% - 0.01% 67.0M 0.20% - 115M 0.23% -
kodiak-rie 0.00 0.00% - 0.02% 41.6M 0.12% - 62.4M 0.12% -

zonestat by Resource: Example
Example: Monitor lwps and processes

$ zonestat -r processes,lwps 5
PROCESSES SYSTEM LIMIT
system-limit 292K
ZONE USED PCT CAP %CAP

[total] 191 0.63% - -
[system] 0 0.00% - -
global 167 0.55% - -
foo 24 0.08% 300 8.00%
LWPS SYSTEM LIMIT

system-limit 2047M
ZONE USED PCT CAP %CAP
[total] 713 0.00% - -
[system] 0 0.00% - -
global 618 0.00% - -
foo 95 0.00% 1000 9.50%

Resource Management
• New max-processes resource control

# zonecfg -z zone1
zonecfg:zone1> set max-processes=300
…
• prctl now reports resource utilization

# prctl -i zone foo
zone: 4: foo
NAME PRIVILEGE VALUE FLAG ACTION
zone.max-lofi
usage 0
system 18.4E max deny
zone.max-swap
usage 28.3MB
privileged 3.00GB - deny
system 16.0EB max deny
…

Zones Security
• Delegated administration
• Authorizations can be configured directly in zonecfg
– login, manage, clonefrom

# zonecfg -z zone1
zonecfg:zone1> add admin
zonecfg:zone1:admin> set user=jack
zonecfg:zone1:admin> set auths=login,manage
zonecfg:zone1:admin> end
zonecfg:zone1> commit
• Authorizations are added to user/role entry in

/etc/user_attr by zonecfg.

Solaris 10 Containers
• Solaris 10 branded zone

– Similar to the existing solaris8 and solaris9 brand settings on
Solaris 10
• Promote adoption and compatibility of Oracle Solaris 11

– Leverage existing investment in Solaris 10
— Infrastructure, training, support
– Allow new technology to support Oracle Solaris 10 context
— Virtualized networking among Solaris 10 instances
– Application recertification for Solaris 11 unnecessary
• Use p2v installation process
– Or v2v for moving the existing Solaris 10 zones
• Support instances on Solaris 10 10/09 or later

Solaris 10 Container: Expected Migration Path
zone: db27-prod redeploy

Solaris 10 Solaris10 Brand
zone: db27-prod zone: db27-prod
p2v
Solaris 11 Solaris 11
Solaris 10
db27-prod

References
Oracle Solaris Administration: Oracle Solaris Zones, Oracle

Solaris 10 Zones, and Resource Management
http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=SYSADRM


Network Virtualization 2

Advanced Network Features
• ilbadm
• IP Filtering, forwarding in a zone
• Hardware Lanes and dynamic polling
•

ipmpstat
• Fiber Channel over Ethernet (FCoE)
• VRPP support
• NUMA I/O
• Public GLDv3 APIs

ilbadm: L3/L4 Integrated Load Balancing
• Operational modes
– Stateless Direct Server Return (DSR)
– Half or Full NAT
• Algorithms supported

– Round robin
– IP hashing: Source address or source address + port
• Health-checking built-ins
– TCP, UDP, ICMP probes
– Apply as parameters to user-scripted tests
• Performance comparable to IP forwarding

Load Balancing Components
• pkg://solaris/service/network/load-
balancer/ilb@0.5.11,5.11-0.148:…
• To configure:
– Server group: list of host+port addresses

– Virtual IP (aka “logical host”)
– Algorithm, operational type
– Healthcheck program and parameters (optional)
• The configured elements form a rule.
• ilbadm subcommands follow dladm model.

ilbadm: Example
# ilbadm create-servergroup \
> -s servers=apache-zone1:80,apache-zone2:80 \
> apache_group

#
# ilbadm create-rule \
> –e –p –I vip=10.1.2.3,port=80 \
> -m lbalg=rr,type=HALF-NAT \
> -h hc-name=/var/hc/apache_check \
> -o servergroup=apache_group \
> apacheload_rrobin

IP Filter, Forwarding in a Zone
• Same operational semantics as the GZ

• For IP Filter in a zone
– # pkg install ipfilter; pkg contents ipfilter
– Filter/NAT configuration files in the /etc/ipf directory

—See /usr/share/ipfilter/examples
– # svcadm enable ipfilter
• Or just forwarding
– # svcadm enable ipv4-forwarding

Hardware Lanes and Dynamic Polling
• A Hardware Lane is defined by

– NIC-supported partitions (Receive/Transmit Rings, DMA)
– Kernel queues/threads bound to CPU, pset, or pool
• Same CPUs assigned to a VNIC or a flow

• Dynamic polling
– Switches from interrupt handling to polling rate in low traffic
• Reduces context switching and lock contention
mpstat output with NIC and legacy driver:
intr ithr csw icsw migr smtx srw syscl usr sys wt idl
10818 8607 4558 1547 161 1797 289 19112 17 69 0 12
mpstat with NIC and GLDv3-based driver:

intr ithr csw icsw migr smtx srw syscl usr sys wt idl
2823 1489 875 151 93 261 1 19825 15 57 0 27

Hardware Lanes
Intended for multicore platforms with multi-10gigE NICs

• Hardware Lanes + dedicated resources = linear scaling
• Integrated with virtualization and QoS controls
• Dynamic polling, packet chaining boost efficiency

Physical Machine
Physical NIC
C Hardware Kernel Threads Virtual
L Rings/DMA VNIC
and Queues Machine/Zone
A
S Hardware Lane
Hardware Kernel Threads Virtual
VNIC
S Rings/DMA and Queues Machine/Zone
Switch
I
VLAN F
Separated I
E Hardware Kernel Threads
Flow Application
R Rings/DMA and Queues

ipmpstat: Observability for IPMP Groups
• Reads sockets opened by in.mpathd

• Five output modes
– Address (-a)
– Group (-g)

– Interface (-i)
– Probe (-p)
– Target (-t)
• VNICs are valid IPMP group members.
– Useful for testing

ipmpstat: Example
# ifconfig blut0 ipmp

# ifconfig play0 group blut0
# ifconfig play1 group blut0

# ipmpstat -a
ADDRESS STATE GROUP INBOUND OUTBOUND
fe80::897f:b644:ae41:e0b up blut0 -- --
10.2.3.5 up blut0 play1 play1 play0
10.9.8.7 up blut0 play0 play1 play0
# ifconfig play0 group ""
# ipmpstat -a
ADDRESS STATE GROUP INBOUND OUTBOUND
fe80::897f:b644:ae41:e0b up blut0 -- --
10.2.3.5 up blut0 play1 play1
10.9.8.7 up blut0 play1 play1
#

Fiber Channel over Ethernet (FCoE)
MAC Layer APIs To Create VNICs, App Leadville

Dedicate Resources, Bandwidth Fiber
Channel
for both Network Stack and FCoE Network Stack
Stack
Virtual FCoE

NIC Glue
Virtualized Data Link Layer MAC MAC

Client Client
MAC Layer
Rx/Tx Ring Rx/Tx Ring
DMA DMA
Channel Channel
H/W Flow Classifier
Pseudo FC instance presented to storage

10gB10g Ethernet
Port Port Port
FCoE

Virtual Router Redundancy Protocol (VRRP)
• HA support for routers and load balancers

• Treats active server as a primary
– Other servers are passive
• Solaris framework monitors control messages

• Upon primary failure, framework elects a new primary
– Moves the Virtual IP address (VIP)
• Each VRRP router associates a VNIC with the VRRP id
– VNIC attributes are set via dladm(1M).

IP over Infiniband (IPoIB)
• Used in Exalogic systems (BOND0 interface)

• Runs on top of IB's verb layer
• Control over IB partitions in dladm(1M)
– *-part subcommands

– IB data links show up as Host Channel Adapter (HCA) ports
– Create partition data links over IB data links
— Plumb them with IP addresses, assign them to zones
– All dladm(1M) link properties apply

Non-Uniform Memory Architecture (NUMA) I/O
• On NUMA platforms, I/O performance factors include:

– Kernel resource location (memory placement)
– Hardware topology
– Device location (backplane attachment)

• NUMA I/O Framework
– Defines “affinity” for all I/O subsystems
– I/O subsystems register affinity to needed resources
– Framework uses affinity to determine memory placement
– Consumer-transparent process

NUMA I/O Architecture: Overview
I/O I/O I/O topology
Kernel Affinity APIs

Subsystem topology constructor
Admin
Interface Core NUMA I/O

Framework
CPUS/pool
constraints NUMA
I/O Bind topology
Subsystem interrupt NUMA lgrp
sub-system
Device Interrupt PCI/DDI

Driver handles Framework

GLDv3 Public Driver APIs
• Dynamic polling
• Packet chaining
• Hardware checksumming offload
– Large Send Offload (LSO)

• Revamped driver property interface
– Simplify driver development
– Extensibility for future releases
• First supported in Solaris 10 U9 (09/10 release)
• See Chapter 19, Document #816-4854

Network Performance Highlights
• Dynamic polling on receive rings boosts efficiency

• Aggregation, flow control on transmit rings
• Binding available to psets or pools
– Supports Message Signaled Interrupts (MSI)

— Used in PCI Express (PCIe) hardware
— Alternative to traditional Pin-Based Interrupt
• Hardware Lanes
– Improve cache locality, isolates traffic


Security

Features
• Root as a role
• On-disk file encryption
• Network spoofing protection
• Delegated administration

– Zones, SMF services
• “In-kernel” pfexec
• Forced Privilege and Stop Profile

Root Implemented as a Role
• User defined during installation receives the root role

• sudo is enabled with 5-minute grace
installer@os11e:~$ roles
root

installer@os11e:~$ profiles
Console User
Suspend To RAM
Suspend To Disk
Brightness
CPU Power Management
Network Autoconf User
Network Wifi Info
Desktop Removable Media User
Basic Solaris User
All

File system encryption: zfs(1M)
• Applicable to datasets or volumes

• Need a wrapper key to mount file system
– Passphrase or file-based, delegatable key control
• See man page examples 22-27 for zfs(1M)

$ zfs create -o encryption=on rpool1/home/fng
Enter passphrase for 'rpool1/home/fng':
Enter again:
$ zfs list rpool1/home/fng
rpool1/home/fng 31K 8.29G 31K /export/home/fng
fir@os11e:/$ zfs get all rpool1/home/fng | grep key
rpool1/home/fng keysource passphrase,prompt local
rpool1/home/fng keystatus available -
rpool1/home/fng rekeydate Fri Dec 10 10:35 2010 local

Configuring ZFS Encryption
You can also write a key to a file

• keysource attribute specifies format and file path
• Encryption policy is inherited and read-only

# pktool genkey keystore=file outkey=/dmkey.file
keytype=aes keylen=256
# zfs create -o encryption=aes-256-ccm -o
keysource=raw,file:///dmkey.file rpool1/home/fng
# zfs clone rpool1/home/fng@final rpool1/home/delivered
Enter passphrase for 'rpool1/home/delivered':
Enter again:
# zfs set encryption=off rpool1/home/delivered
cannot set property for 'rpool1/home/delivered:
'encryption' is readonly

File system encryption: lofiadm
Full scenario: Example 6, lofiadm(1M) man page
marty@os11e:/$ mkfile 64m /var/tmp/setec

marty@os11e:/$ lofiadm -c aes-256-cbc -a /var/tmp/setec
Enter passphrase:

Re-enter passphrase:
/dev/lofi/1
marty@os11e:/$ newfs /dev/rlofi/1
newfs: construct a new file system /dev/rlofi/1: (y/n)? y
...
marty@os11e:/$ lofiadm
Block Device File Options
/dev/lofi/1 /var/tmp/setec Encrypted

Network Spoofing Protection
• mac-nospoof: Cannot change MAC address

• restricted: Outbound ipv4, ipv6, and ARP packets only
• ip-nospoof: Checks outbound packets against allowed-
ips property

• dhcp-nospoof: Multiple conditions apply. See
dladm(1M).
# dladm show-linkprop -p protection play0
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
play0 protection rw -- -- mac-nospoof,
restricted,
ip-nospoof,
dhcp-nospoof
# dladm set-linkprop -p protection=mac-nospoof play0

Zones: Delegated Administration
• Per-user, per-zone authorizations

• Limits NGZ access from the GZ
• zonecfg(1) syncs with GZ /etc/user_attr file.

zonecfg:webber> info
zonename: webber
zonepath: /home/webber/zone
...
admin:
user: hen3ry
auths: login,manage
zonecfg:webber> verify; exit
UX: /usr/sbin/usermod: hen3ry is currently logged in,
some changes may not take effect until next login.

SMF: Delegated Administration
• Set authorizations in manifest

– Enable/disable (value_authorization)
– Restart/refresh (action_authorization)
– Modify values in all or select property groups

• Assign auths to profiles/users via rbac(5)
• Complete list in smf_security(5)
<property_group name='general' type='framework'>

<propval name='action_authorization' type='astring'
value='solaris.smf.manage.myservice' />

<propval name='value_authorization' type='astring'
value='solaris.smf.manage.myservice' />
</property_group>

SMF: Method Context
Execution attributes include:

• Security
– User, group, privileges
• Also resource management and environment

<exec_method type='method' name='start'
exec='/lib/svc/method/foobar start'
timeout_seconds='60' >
<method_context>
<method_credential
user='foo'
group='bar'
privileges='basic,sys_net_config,net_rawaccess' />
</method_context>
</exec_method>

SMF: Firewall Integration
• Application-specific attributes
$ svcadm enable ipfilter
$ svccfg -s ipfilter:default setprop
firewall_config_default/policy = allow

$ svcadm refresh network/ipfilter
$ svcadm enable ftp
$ svccfg -s ftp setprop firewall_config/policy = allow
$ svccfg -s ftp setprop firewall_config/apply_to =
network:192.168.1.0/24
• Applications can participate in automatic firewall policy

– Define firewall_context/name for RPC services.
– Implement firewall_context/ipf_method for other
services.
– See svc.ipfd(1M) for more information.

proc_fork
proc_exec
net_priv_addr

Least Privilege Changes

“In-kernel pfexec”
• New PRIV_PFEXEC process flag

• Set by any profile shell, inherited across exec(2)
• Applies RBAC attributes transparently
– No need for pfexec

• Other profile shells now available:
– pfbash(1)
– pftcsh(1)
– pfzsh(1)

Basic Privileges: More is Less
• basic privilege set expanded

– file_read, file_write, file_link_any
– proc_exec, proc_fork
– proc_info, proc_session

– net_access
• Easier to disable certain privileges:
– Read-only process: !file_write
– Host-only process: !net_access

Role-Based Access Control
Software Installation
DTrace Analysis
Developer

Audit Review
File Integrity Verification Internal
Auditor
Dataset Management
Backup Operator
Sys
Admin

Sandboxing Enhancements
• User profiles are cumulative, processed in list order

– /etc/user_attr, /etc/security/policy.conf
• Ignored any profiles assigned after Stop is read
– Either by file (policy.conf) or by command

• Provides an explicit limit to a user's authorizations

Kerberos Improvements
• “Zero-configuration” client via DNS

– Authentication via Active Directory available
– Enhancements to PAM configurations
– Better interoperability for Windows clients

• Initial authentication possible with public keys
– RFC 4556 (PKINIT) implemented
• New kdcmgr(1M) tool
– Sets up Kerberos Key Distribution Center

Key Management: pkcs11_kms Provider
• Consumer for Key Management Server (KMS)

• Configured with kmscfg(1M)
– pkg:/system/library/security/crypto/pkcs11_kms@...
• KMS configuration required for each consumer

– See “KMS 2.2 Administration Guide” for details
– http://docs.sun.com/app/docs/doc/316195103AA

Other Enhancements
• NSA Suite B algorithms support

• Internet Key Exchange
– Accepts Elliptic Curve Cryptography (ECC)
– Also RSA and DSA

• AES Cipher Feedback (CFB) mode
– Available on SPARC T3 processor
– Used by Oracle Database Advanced Security Option
– Supports acceleration of table-level encryption

Oracle Solaris 11 Trusted Extensions
• Mandatory Access Control

(MAC)
Need-to- Internal Public • Zones are classified (“labeled”)
know Use
• Processes need proper

Multilevel Desktop Services clearance to access labelled
(Global Zone) assets
Solaris Kernel • Networks, printers also labeled
• Runs all Solaris applications
net net net net
• Designed for defense and
intelligence industry
requirements
• Meets Common Criteria
Certifications at EAL4+ levels

Trusted Extensions Changes
• GNOME replaces CDE as Desktop

– GNOME login manager asserts labeling
– X server uses same X Access Control Extension (XACE)
policy hooks as SELinux

• New ZFS attribute: mlslabel
– Prevents remounting on the wrong label
• Labeled IPsec
– Multilevel IKE daemon negotiates Security Associations
– Maintains the label’s confidentiality and integrity
– CIPSO data does not need to be sent in the clear
– Allows the use of single physical network

Trusted Platform Modules (TPM)
Support for Trusted Platform Modules (TPM)

• TSS 1.2 API
• tpmadm(1M) CLI
• pkcs11_tpm(5) Crypto module



Services Management Facility (SMF)

SMF Design Goals
• Increase application availability

– Monitor services in run time
– Restart failed processes
• Graph-dependent services

– Start independent service paths concurrently
• Common naming for all services
– Not just daemon processes
– It is either disabled or some variation of enabled.

SMF Is the Glue in Solaris 11
• Services are first-class objects

– Health monitoring
– FMRI-based naming
– Universal lifecycle

– Tools to observe services, not just processes
• Automated restarts after errors and faults
– Integrated refresh upon reconfiguration
• Control for many service attributes
– Privileges
– User/group delegation
– Resource controls

Service Templates
• Service properties include:

– Decorations
– Descriptions
– Simple constraints

• Online help
– Store property descriptions with the service
• Catch errors during configuration:
– Validate constraints in APIs and commands
smf_template(5)

Early Manifest Imports
• Two import services

– svc:/system/early-manifest-import:default
– svc:/system/manifest-import:default
• Solves potential race condition with manifest upgrades

• Reads new manifest location
– /lib/svc/manifest
– /var/svc/manifest remains for compatibility
— manifest-import service reads /lib/svc/manifest, and
then /var/svc/manifest.

SMF Enhanced Profiles
• Customize configuration for mutliple services

– Example: enabling/disabling services in one action
— # netservices limited | open
• Easy deployment of services configurations

– Drop-in during system deployment
– Installer support for SMF profiles in the works
– /etc/svc/profile
– Use site/ subdirectory for local customization

Fault Notification
• Set and list notification types for SMF/FMA faults.

• Default parameters kept as a service
– svc:/system/svc/global:default

# svccfg setnotify -g to-maintenance mailto:admin@domain.com
# svccfg listnotify -g
Event: to-maintenance (source: svc:/system/svc/global:default)
Notification Type: smtp
Active: true
to: admin@domain.com

IPS Actuators
• Signals additional behavior, usually on a live system

• restart_fmri prompts a service restart.
– Per-file attribute
Remember that IPS only updates objects as needed.

—
• reboot-needed indicates that a reboot is required.
dir group=bin mode=0755 owner=root path=opt timestamp=20101109T051058Z

dir group=bin mode=0755 owner=root path=opt/app timestamp=20101109T051110Z
file opt/app/app-bin group=bin mode=0555 owner=root path=opt/app/app-bin
pkg.size=48088 reboot-needed=true
file opt/app/app.conf group=bin mode=0644 owner=root path=opt/app/app.conf
pkg.size=267
file lib/svc/manifest/application/lianep-app.xml mode=0444 owner=root
path=lib/svc/manifest/application/lianep-app.xml
restart_fmri=svc:/system/manifest-import:default

FMRI Stored in proc_t Structure
#!/usr/sbin/dtrace –s
inline string fmri =

stringof(curthread->t_procp->p_ct_process->conp_svc_fmri->rs_string);
syscall:::entry
{

@[fmri] = count();
}
dtrace: script '/var/tmp/foo' matched 228 probes

^C
…
svc:/system/sysevent:default 10
svc:/network/smtp:sendmail 21
svc:/network/physical:nwam 40
svc:/network/ntp:default 50
svc:/system/hal:default 65
svc:/network/datalink-management:default 428
svc:/application/graphical-login/gdm:default 274792

D73819GC10 SG PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

D73819GC10 SG PDF

Încărcat de

Drepturi de autor:

Formate disponibile

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY.

COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or it affiliates. All rights reserved.

Oracle University and ORACLE CORPORATION use only

If this documentation is delivered to the United States Government or anyone using

Oracle University and ORACLE CORPORATION use only

2 Image Packaging System (IPS) and Automated Installer (AI)

Flow of Automated Installation 2-25

Oracle University and ORACLE CORPORATION use only

4 ZFS Features in Solaris 11

Deduplication Example - 1 4-13

Oracle University and ORACLE CORPORATION use only

ilbadm: Example 6-5

Oracle University and ORACLE CORPORATION use only

8 Services Management Facility (SMF)

IPS Actuators 8-8

FMRI Stored in proc_t Structure 8-9

Oracle University and ORACLE CORPORATION use only

Oracle University and ORACLE CORPORATION use only

Oracle University and ORACLE CORPORATION use only

Oracle University and ORACLE CORPORATION use only

• International Oracle User’s Group (IOUG) articles

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and ORACLE CORPORATION use only

Oracle Solaris: The Mission Critical OS

• The #1 deployment platform for the

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 2

Raising the Bar Set by Solaris 10

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 3

SPARC Enterprise Servers

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 4

SPARC T3 Servers: Scaling to New Heights

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 5

Oracle Solaris: Platform Choice and Flexibility

Solaris Solaris Solaris Solaris 10

Oracle University and ORACLE CORPORATION use only

• Built-in scalable, platform- • Consolidation path for older Solaris

Binary Compatibility Guaranteed

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 6

Serious About Oracle Solaris

Oracle University and ORACLE CORPORATION use only

Solaris 11: Coming in 2011

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 7

Oracle Addresses Range of Customer Needs

Oracle University and ORACLE CORPORATION use only

Manageability and Simplicity HIGH

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

What's New in Oracle Solaris 11 1 - 8

The preceding is intended to outline our general product

Oracle University and ORACLE CORPORATION use only

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.