Data Center Server Architecture

Cisco Unified Computing Sales Training Series

Jim Anderson Mike Wilson, Firefly

Agenda

 Overview of the Server Market  What Drives Server Opportunities? Server Technology Overview Questions and Answers

Server ApplicationsWhat and Where

DATA CENTER BRANCH OFFICE
File/Print

Email

Manufacturing

CRM

Email

Database

REGIONAL OFFICE
Business Intelligence Web Farm Cloud HPC Appliances Database File/Print Email Collaboration

Cisco UC

Servers Are Everywhere

Branch / WAN
Customer Hits Buy Billing Notification Notify Sales Rep
Process Process Order Order Check Check Credit Credit Ship Order Bill Customer Update Cust Svce Cust Svce

Data Center

Confirm Shipment

Order Complete

Premium Customer? Update Records

CRM Order ERP Update

Enter

WAN
Credit Approved Update Call Center Update Contracts Deliver Order Update Call Center

Check Inventory Credit Override required

Inventory

SCM
Initiate Accts Billing

Intranet
Check Customer DB

Check Account Balance

Check Availability

Customer Master
Check Credit Purchasing

XML Logistics EDI Warehouse

Pack & Ship Order

Extranet

Procure Material

SOAP Credit

Check Credit History

Extended Enterprise

Why Are Servers Important to Cisco?

Relative Market Values for Data Center Purchasing

Source: IDC, Gartner

Gartner Magic Quadrant

Others 12.4% HP 30.7%

Fujitsu 5.1% Sun 9.5% Dell 12% IBM 30.2%

Source: IDC, December 2008

 Leaders offer a range of standardized options  IBM and HP 80+ share of blade server market  Visionaries focus on a single approach:
 Data Warehouse appliances: Teradata and Netezza  Dedicated rack/server solutions: Egenera, Rackable Systems

 Overview of the Server Market  What Drives Server Opportunities? Server Technology Overview Questions and Answers

Increasing Pressures for the Data Center

New Business Requirements

Focus on Costs

Green Initiatives

SLA Metrics

Global Availability

Compliance

Increasing Pressures for the Data Center

New Business Requirements

Focus on Costs

Green Initiatives

SLA Metrics

Global Availability

Compliance

Operational Limitations

Power & Cooling

Asset Utilization

Provisioning

Threat Prevention

Bus. Continuance

Data Center Evolution

Data Center 1.0
Mainframe
 $5B annual market in 1970s  Making a comeback  IBM 90% market share

Centralized
Application Architecture Evolution

Data Center Evolution

Data Center 1.0
Mainframe
 $5B annual market in 1970s  Making a comeback  IBM 90% market share

Data Center 2.0

Client-Server and Distributed Computing

 Dominant from the1990s through 2004  In & out of the DC  Focused now on SMB market

Centralized

Decentralized
Application Architecture Evolution

Data Center Evolution

Data Center 1.0
Mainframe
 $5B annual market in 1970s  Making a comeback  IBM 90% market share

Data Center 2.0

Data Center 3.0

Client-Server and Distributed Computing

 Dominant from the1990s through 2004  In & out of the DC  Focused now on SMB market

Virtualized and Service-Oriented  $54B Server Market  Increasing demand  Driven by SOA concept

Centralized

Decentralized
Application Architecture Evolution

Virtualized

Data Center Evolution

Data Center 1.0
Mainframe
 $5B annual market in 1970s  Making a comeback  IBM 90% market share

Data Center 2.0

Client-Server and Distributed Computing

 Dominant from the1990s through 2004  In & out of the DC  Focused now on SMB market

Centralized

Decentralized
Application Architecture Evolution

Data Center Evolution

Data Center 1.0
Mainframe
 $5B annual market in 1970s  Making a comeback  IBM 90% market share

Centralized
Application Architecture Evolution

Data Center Evolution

Data Center 1.0
Mainframe
 $5B annual market in 1970s  Making a comeback  IBM 90% market share

Data Center 2.0

Data Center 3.0

Client-Server and Distributed Computing

 Dominant from the1990s through 2004  In & out of the DC  Focused now on SMB market

Virtualized and Service-Oriented  $54B Server Market  Increasing demand  Driven by SOA concept

Centralized

Decentralized
Application Architecture Evolution

Virtualized

Server Consolidation
DATA CENTER BRANCH OFFICE

File Email Manufacturing CRM ERP Authentication

Print

DC

REGIONAL OFFICE

Email

Database

Sales

Web Farm

HPC

File Authentication

Print

Server Consolidation
DATA CENTER BRANCH OFFICE

Email Manufacturing

CRM

ERP

Authentication

DC File/Print Database Sales

REGIONAL OFFICE

Web Farm

HPC

File Authentication

Print

Benefits and Challenges with Server Consolidation

 Consolidation is a huge driver for server purchasing today:
80% of IT organizations actively involved $25B market in 2009 Stepping stone to virtualization and the DC 3.0 vision

Benefits and Challenges with Server Consolidation

 Consolidation is a huge driver for server purchasing today:
80% of IT organizations actively involved $25B market in 2009 Stepping stone to virtualization and the DC 3.0 vision

 Result is increasingly higher densities:

More compute power per sq. ft. But also more power and cooling about 20%+ of DC costs Environmentals are a barrier to consolidation

Power Requirements Continue to Grow

2000
2U servers

2002
1U servers

2006
Blades

2009
Blades

2kW

6kW

24kW

42kW

Heading towards 50kW

Its not just about greening its about the bottom line.
Total electricity consumption
Worldwide (2007)

15,746 Billion US (2008) 4,581 kWH

Sample Energy Costs Per Rack Per Rack Watts per Energy Cost Sq Ft

% of WW electricity usage for Data Centers

2kW 4kW 6kW 24kW 40kW

67 133 200 800 1333

$3,094 $6,199 $12,064 $48,257 $98,955

(Estimates vary) 0.8 3%

Cost of electricity
US Commercial Sector Avg (2008)

$0.0957/kWH

Its not just about greening its about the bottom line.
Total electricity consumption
Worldwide (2007)

15,746 Billion US (2008) 4,581 kWH

Sample Energy Costs Per Rack Per Rack Watts per Energy Cost Sq Ft

% of WW electricity usage for Data Centers

2kW 4kW 6kW 24kW 40kW

67 133 200 800 1333

$3,094 $6,199 $12,064 $48,257 $98,955

(Estimates vary) 0.8 3%

Cost of electricity
US Commercial Sector Avg (2008)

$0.0957/kWH

Cost of powering Data Centers

Estimated, US only

$2.9 10.9B

What is Driving and Enabling Server Consolidation?

Multi-core Architecture

Blade Servers

Consolidation

What is Driving and Enabling Server Consolidation?

Multi-core Architecture

Blade Servers

Server Virtualization `

Consolidation

Increasing Adoption of Server Virtualization

VM Penetration of Installed Workloads
61%

60%

40%

20%
7%

0% 2005 2007 2009 2011 2013

Increasing Adoption of Server Virtualization

VMs (finally) making inroads into the DC
60% 40% 20%
7%

VM Penetration of Installed Workloads

61%

The VM installed base was 2.9 million in 2007 only about 7% of the total DC server market Anticipated to be over 60% within 4 years

Impact on server architecture

VMs enable full usage of multicore VMs driving higher memory and I/O requirements

Servers more commoditized

No lock-in with workloads Blades more attractive lock in with technology

0% 2005 2007 2009 2011 2013

Server Financials

What Drives Server Purchases?

Traditionally, application deployment but is that changing?

More users More powerful applications

Virtualization Multicore

More servers needed More servers needed More servers needed

Rewrite Applications

Decision Makers and Influencers

Apps Team
Vendor sales force, Partners, VARs, First to know Vendor relationships Define physical scope Owns servers after purchase App Admin Traditionally owns serversstill influences Manages budgets Sets requirements

Facilities Data Center Mgr

Server Admin

CxO
Drives initiatives Controls priorities

 Overview of the Server Market  What Drives Server Opportunities? Server Technology Overview Questions and Answers

Criteria for Choosing Servers

What Drives Server Technology Development?

Environmental

Data Center Arch. & Design

Management

Provision Visibility

Performance & Scalability

CPU / L2 Cache Memory I/O

Form Factor

Simplicity

Types of ServersForm Factor

Small Business/Entry Level
 Pedestal or Rack  Single, Dual, or Quad Processor  Internal Storage

Mid to High Range

 Rack Optimized  1U, 2U, 4U, or 8U size  Wide variety of configurations  Internal Storage

Enterprise Blade Systems

 Rack Optimized  7 U enclosures  8 16 blade slots need a minimum  Server, Storage, Interconnect, Tape, Management Blades

Blade Initiatives Drive Modular Building Block Approaches in Data Centers

CHASSIS RACK DATA CENTER

 Most blades today  Server farm in a   

chassis Shared power and cooling Fixed I/O (usually) Proprietary

 "Data center in a rack  Integrates compute,  

storage, memory & I/O CPU and memory aggregation Virtualized I/O

 Data center modules  Preconfigured zones  "Just add water"  

(and some power) Integrated, but not always "integrateable" Suited (but not limited) to blades as building blocks

Rack Server Architecture

Monolithic architecture
CPU, memory, and I/O are all integrated through onboard chipsets

CPU
Chipset Chipset

Limited upgrade paths:

CPUs can be upgraded (rarely done) Memory can be upgraded I/O cards can be added but on-board I/O architecture is not upgradeable

Memory
I/O
SATA PCI-X 10GigE

Not scalable or flexible as a data center solution

Blade Server Architecture

Memory CPU
NIC HBA
Blades

Networking Midplane Fans

I/O

LAN

LAN

SAN

SAN

 Decouples compute power from network and storage I/O  Offer customers more flexibility and scalability  Reduces cabling requirements

Onboard Admin

Power Module Power Backplane Network Module

Multi-core CPU Architecture

Traditional SMP Application
Scheduler CPU L1 L2 CPU L1 L2
CPU Interconnect

Multicore System
Core/Cache/Memory Footprint of a virtualized OS Instance

I/O

Memory Controller

Memory I/O
System Memory (NUMA) X86 Core with L1 Cache L2 Cache

Server Memory Architecture

 Application needs determine memory requirements  Narrows server choices  Server are selected based on current and future needs  Memory scalability is critical  Server sales are driven by the ability to scale
Maximum memory (# of DIMM slots) is tied to # of CPUs limits scalability

Server I/O Requirements

Parallel LAN SAN infrastructure:
FC HBA FC HBA

 Currently many DC servers have 5+ connections per server  Virtualization drives higher bandwidth to the server

GE GE GE GE

Leads to an inefficient and costly network infrastructure:

 Higher adapter and cabling costs  Plus downstream port costs  Multiple fault domains; complex diagnostics  Management complexity; firmware, driver patching, version management

10/100

Server I/O Cabling

Pedestal Servers 1U Pizza Box Servers Blade Servers

4-9 cables per server 4-9 cables per server (LAN + SAN + KVM) Up to 336 cables per rack No cable pathways Rack cable management

2-8 uplinks per chassis 8-32 uplinks per rack Uplinks are increasingly 10GEhigher TCO vs GE

Virtualization Drives 10GE and I/O Consolidation to the Server

FC HBA FC HBA FC HBA FC HBA GE GE GE 10GE

OS+App OS+App Hypervisor

GE

OS+App OS+App OS+App OS+App Hypervisor

10GE

10/100

OS+App OS+App OS+App OS+App OS+App OS+App OS+App OS+App OS+App OS+App Hypervisor

FCoE FCoE

10/100

10/100

Low CPU utilization CPU utilized more I/O capacity is the 10GE mitigates I/O bottleneck bottleneck Still too many cables!

Single (redundant) I/O channel for LAN and SAN Optimized system

Server Management Challenges

Discovery

Reporting

Provisioning

Blade Chassis Management In-Band/Outof-Band Management Manage LAN/SAN Ports

Server Management

Inventory

Software Distribution System Monitoring

Summary
 The server market is undergoing a major paradigm shift  Traditionally, simple scaling drove server purchases:
Number of users, new application deployments

 Today, server consolidation is driving the market:

Logical architectureenabled by virtualization Physical architectureDC facilities requirements 60% of deployed servers are opportunities for DC consolidation

 Blade servers are becoming dominant in the DC  Blade servers are the only growing server segment:
Agilityindependently scale compute, memory, and I/O Simplified managementservice-oriented provisioning

 Overview of the Server Market  What Drives Server Opportunities? Server Technology Overview  Questions and Answers