Best Practices with Designing Exchange Server for Virtualized Environments

David Zazzo Senior Consultant, Microsoft david.zazzo@microsoft.com

Disclaimer

2012 Microsoft Corporation. All rights reserved. Microsoft, Office 365, and other product and service names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Exchange 2013 Disclaimer

Performance testing and validation is currently in progress on Exchange 2013 the Exchange Product Group has not yet finalized their guidance on virtualizing E2013.
Guidance may change, keep an eye on TechNet for the latest on virtualization guidance

About me

Senior Consultant with Microsoft Consulting Services At Microsoft for nearly five years and counting Instructor for Microsoft Certified Master | Exchange
Virtualization Load Balancing RBAC

Why Virtualize Exchange?

Money, Money, Money
Power, Space, Cooling

Licensing benefits multiple servers under one license Server Consolidation

Maximize Utilization of Hardware

Underutilized processors == too much money spent Underutilized memory == too much money spent

Politics

We bought it, were going to use it everywhere CIOs / managers reading CIO Magazine again Senior company leadership

Whats supported, whats not?

SUPPORTABILITY

Virtualization Host Support Guidelines

Hypervisor must be one of the following platforms:

Windows Server Hyper-V (2008, R2, or 2012) Hyper-V Server (2008, R2, or 2012) SVVP-certified hypervisor http://www.windowsservercatalog.com/svvp.aspx

Using Windows Azure, Amazon EC2, etc. is not supported

Virtualization Guest Support Guidelines

Exchange 2010 Guest OS:

Guest OS: Windows Server 2008 SP2 or R2
Exchange Server 2010 SP3 will support 2012, not yet released

E2010 RTM: All roles except UM E2010 SP1+: All roles including UM

Exchange 2013 Guest OS:

Guest OS: Windows Server 2008 R2 or Windows Server 2012 All roles (CAS + MBX)

Virtualization Guest Support Guidelines

Using Host-Based High Availability (Live Migration, vMotion, etc)

Exchange 2010 RTM: Can not mix with DAG Exchange 2010 SP1+: Can mix with DAG* Exchange 2013: Can mix with DAG* * You can mix DAG and HA with certain constraints more on this later in the presentation

Virtualization Guest Support Guidelines

Maximum CPU ratio of 2:1 vCPU:pCPU

Example: If you have 8 cores, maximum supported is 16 vCPUs assigned to guests Recommended ratio is 1:1, dont overcommit CPU in production

iSCSI initiator in guest is supported

Check with your vendor, make sure VM network stack supports jumbo frames, full network fidelity

Whats Not Supported?

- Dynamic memory & memory over-commit

- Not supported for any Exchange role

- Hypervisor snapshots (Time Travel)

- Not supported for any Exchange role

- Differencing/delta/dynamic disks
- Not supported for any Exchange role

- Other applications installed on the root

- Only deploy your management, monitoring, A/V, etc.

Time Traveling (Hypervisor Snapshots)

Hypervisor snapshots make lab testing much easier Resist the temptation to use hypervisor snapshots in production they arent supported! Aspects of the Exchange system do not handle time travel well (notably log shipping) Use caution with snapshots in the lab (you may need to roll back more than one VM)

PLANNING FOR EXCHANGE

Planning Guidance

Historically, our planning guidance has been very simple: Determine your baseline requirements, then have the physical/virtual discussion. Why?
You still need enough CPU You still need enough memory You still need enough disk I/O You still need enough network bandwidth

Unified Messaging Virtualization

Requires Exchange 2010 SP1+ for support VM should have a minimum of:
(4) Virtual Processors/Cores Requires 1:1 vCore:pCore allocation 16GB of memory

Unified Messaging server role must stand alone (2010)

Hub, CAS, and Mailbox can not be in the same VM as Unified Messaging role

VM Placement Recommendation (2010)

Use common sense when placing your VMs

Deploy VMs with the same role across multiple roots

Do not deploy MBX VMs in the same DAG on the same root server

CAS HUB MBX MBX HUB

HUB

CAS CAS MBX CAS CAS MBX

MBX

VS

HUB HUB

MBX

Host-Based High Availability What is Host-Based High Availability? Automatic failover of virtual machines to another virtualization host in the event of a critical hardware failure (virtualization platform independent)

Host-Based High Availability Concepts Key Concepts

Cold boot: Bringing a system from a power-off state to a clean, fresh start of the OS Saved state: Many hypervisors allow you to save state, or hibernate a VM rather than shutting it down or turning it off. On resume, youre resuming from the saved state. Planned migration: Using Live Migration or vMotion to move a VM from one host to another by administrator action.

Host Based High Availability What you need to be aware of: Not an Exchange-aware solution
No knowledge of transaction logs, clean/dirty database dismount, database checksums, Exchange-health, etc.

Only protects against server hardware/network failure No protection against storage failure / data corruption Trend is larger mailboxes = larger database sizes = longer time to fully recover from data loss (cache warming) Requires the guest VM to perform a cold boot (this is HA?)

Host Based Failover Clustering

What is Supported: Automatically failing VMs to an alternate cluster node in the event of a critical host hardware issue
Important! VM must come up from a cold start, not saved state or warm start

Live Migration / vMotion / etc. What is Not Supported: Quick Migration (Windows Server 2008 pre-R2) Anything that pauses/saves state, migrates, and then resumes Time Traveling

Live Migration Considerations

Adjust cluster heartbeat if necessary

Target less than 5 seconds for Live Migration / vMotion Adjust if Live Migration takes longer than 5 seconds, but no more than 10 seconds

Enable jumbo frames on the Live Migration network(s) Use very fast networks (5Gb, 10Gb)

Which HA to choose?

Microsoft recommends Exchange HA (DAG)

Exchange-aware HA (log shipping, single page restore, no more -1018s, best copy selection, etc.) Understand failure domains
Power, network, storage, rack, blade chassis, etc

Make sure your DAG members are placed where you think they should be, and that they dont migrate where they shouldnt be

Disaster Recovery / Backup Considerations Design your backup solution carefully!

Use Exchange-aware VSS solutions in guests to get the Exchange stores and guest OS instance Use VSS on the host to backup the hypervisor and configuration

Interesting DR scenarios
Some physical, some virtual Ensure that if you have a virtual machine as a target for multiple physical server in your DR scenario, that it can handle the worst case DR

Hardware Considerations Disk subsystem must still support required IOPS

Jetstress is still the gold standard

Test as a system
Jetstress and LoadGen all VMs on the system to stress the entire system with a realistic peak load Just like with physical, one or the other is not enough Unless your clients are going to be virtual in production (VMware VDI for instance), test with physical LoadGen clients

Watch over-oversubscribe of processors

Remember the vCore : pCore ratio (2:1 max supported, 1:1 recommended)

Hardware Considerations, Cont

Network
Bandwidth Take into account iSCSI when necessary!
High bandwidth customers should avoid 1Gb iSCSI Newer FC4Gb, FC8Gb, 10Gb iSCSI or FCoE better

New hardware vs reuse/upgrade of old

Will older procs support 64-bit guests (watch new procs, too) Can you upgrade the procs in an old server to work? Will upgrading the memory cost more than a new server?

Deployment Recommendations Exchange Server 2010/2013 is not virtualization aware Core Exchange Design Principles Still Apply
Design for Performance, Reliability and Capacity Design for Usage Profiles (CAS/MBX) Design for Message Profiles (Hub/Edge)

Virtualization Design Principles Now Apply

Design for Performance, Reliability and Capacity Virtual machines should be sized specific to the Exchange role (EDGE, HUB, CAS, MBX, multi-role) Hosts should be sized to accommodate the guests that they will support

Mailbox Server Guidelines (2010)

Database Cache requirements are the same for physical and virtual deployments
Total Send + Receive (75k message size) 50 Database Cache Per Mailbox (MB) 3

100
150 200 250 300 350 400

6
9 12 15 18 21 24

450
500

27
30

Mailbox Server Guidelines (2010)

Virtual Processor Logical Processor
Hypervisor and the Virtualization Stack consume CPU Reduce recommended mailbox count by ~10%
Megacycles per Active Mailbox Physical MBX Role Megacycles per Passive Mailbox Physical MBX Role Megacycles per Active Mailbox Virtual MBX Role Megacycles per Passive Mailbox Virtual MBX Role

Total Send + Receive (75k message size)

50

0.15

1.1

0.17

100
150 200 250

2
3 4 5

0.3
0.45 0.6 0.75

2.2
3.3 4.4 5.5

0.33
0.5 0.66 0.83

300
350 400 450

6
7 8 9

0.9
1.05 1.2 1.35

6.6
7.7 8.8 9.9

0.99
1.2 1.32 1.49

500

10

1.5

11

1.7

Mailbox Server Guidelines (2013)

Unfortunately, too early guidance coming soon, watch TechNet and EHLO Expect that the 10% rule of thumb will continue

Mailbox Storage Configuration

Virtual SCSI (passthrough or fixed disk VHD/VMDK)

Recommended configuration for database and log volumes

iSCSI
Standard best practice for iSCSI connected storage apply (dedicated NIC, jumbo frames, offload, etc.) iSCSI initiator in the guest is supported but need to account for reduced performance
Clarify support between hypervisor vendor and storage vendor (certain combinations are not supported)

Mailbox Server Role Requirements Calculator

Mailbox Server Role Requirements Calculator

The storage calculator but much, much more now! The de facto Exchange calculator Created during Exchange 2007 product cycle Properly calculates host CPU, memory, storage requirements, network bandwidth for replication Newer versions generate database/dag deployment scripts

Using the Calculator to Plan for Virtualization

Get the Exchange Processor Query Tool

http://msexchangeteam.com/archive/2010/10/27/456738.aspx

This allows you to put in a processor and number of cores, and have the tool give you the average SpecInt value you can use in the calculator

Using the Calculator to Plan for Virtualization

X/(N*Y) = per virtual processor SPECInt2006 Rate value

Where X is the SPECInt2006 rate value for the hypervisor host server Where N = the number of physical cores in the hypervisor host Where Y = 1 if you will be deploying 1:1 virtual processor-to-physical processor on the hypervisor host Where Y = 2 if you will be deploying up to 2:1 virtual processor-tophysical processor on the hypervisor host

Getting the Calculator to Plan for Virtualization

Latest Mailbox Server Role Requirements Calculator (v17.2) now includes support for virtualization!

Hypervisor CPU Adjustment Factor is the virtualization overhead (rule of thumb is 10%)

Now that weve talked about virtualizing Exchange

WHY NOT VIRTUALIZE?

Why NOT Virtualize Exchange?

Virtualization does add complexity Virtualization does impact performance You do still have to manage those virtual servers as if they were physical
Licensing, Hotfixes, Service Packs, Updates, Monitoring, etc.

You also have to manage that virtualization platform Are there alternatives? Do they make sense?
Can a physical architecture make full use of hardware? Would use of blade server technologies make more sense than virtualization?

The Alternative: Exchange Multi-Role Server

Current Microsoft guidance is to deploy multi-role servers (CAS, HUB, MBX) instead of virtualization Your core math works out:
Example 8-core Physical Machine Mailbox Server Role gets 4 cores Client Access Server Role gets 3 cores (3:4 sizing) Hub Transport gets 1 core (1:5 sizing, this is better)

Take full advantage of the physical machine

Additional reading
Windows Server 2008 R2 Hyper-V
http://www.microsoft.com/windowsserver2008/en/us/hyperv-main.aspx

Windows Virtualization Team Blog

http://blogs.technet.com/virtualization

Infrastructure Planning and Design Guides for Virtualization

http://technet.microsoft.com/en-us/solutionaccelerators/ee395429.aspx?SA_CE=VIRT-IPD-WEB-MSCOM-2009-09-21

Exchange Server 2010 Guidance

http://technet.microsoft.com/en-us/library/bb124558(EXCHG.140).aspx

Exchange Team Blog

http://blogs.technet.com/exchange

Best Practices Whitepaper for Virtualizing Exchange 2010

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=8647c69d-6c2c-40ca-977e-18c2379b07ad

QUESTIONS? VAA VPRAANJA?

Thank you!

Hvala za udelebo in prosim izpolnite ankete.