Vmware Validated Design: For Software-Defined Data Center 2.0

VMware Validated Design
for Software-Defined Data Center 2.0
Logical Component Architecture

Within each region, the VMware Validated Design instantiates two Platform Service Controllers and two vCenter Server systems in the appliance
form factor. This includes one PSC and one vCenter Server for the management pod and one PSC and one vCenter Server for the compute and edge
pods. The design also joins the Platform Services Controller instances to the same vCenter Single Sign-On domain and points each vCenter Server
instance to its respective Platform Services Controller instance.
Core vSphere Management Systems
In both regions, two separate NSX Managers instances are deployed, one for the Management pod and one for the Compute and Edge pods, along with an
associated NSX Universal Controller Cluster. In the Region B the secondary NSX Manager instances automatically imports the configurations of
the NSX Universal Controller Clusters from Region A.
NSX for vSphere
Common vCenter Single Sign-On Domain
Management Stack
vCenter Server
Appliance
(Ring Topology)
Region A
Region A
Platform Services
Controller
Appliance
Compute Stack
vCenter Server
Appliance
Management Stack
vCenter Server
Appliance
Platform Services
Controller
Appliance
Management Stack
vCenter Server
Appliance
Management Stack
vSphere Data
Protection
NSX Edge
Services Gateways
(Routing)
Management Stack
NSX Universal
Controller Cluster
NSX Edge Services

Gateway w/ HA
(One-Arm Load Balancer)
Compute Stack
vCenter Server
Appliance
Management Stack
NSX Manager
(Secondary)
NSX Manager Pairing
Management Stack
vSphere Data
Protection
Region A Management Pod
Region A
Management /
Compute
vCenter Servers
Remote Collector Cluster
Clctr
Node
Clctr
Node
NSX
for vSphere
NSX
for vSphere
Clctr
Node
Virtual
SAN
Analytics Cluster
Master
Node
Replica
Node
Data
Node
Region B Management Pod
Worker
Node
NSX Edge
Services Gateways
(Routing)
Compute Stack NSX

Universal Controller
Cluster
vRealize
Automation
vRealize
Orchestrator
NSX Edge
Services Gateways
(Routing)
Import of Compute Stack

NSX Controller Configuration
from Primary NSX Manager
Region A Edge Pod
Worker
Node
Master
Node
Worker
Node
Worker
Node
Virtual
SAN
Virtual SAN
NFS
Virtual SAN
NFS
Primary Storage
Log Archives
Primary Storage
Log Archives
Production Reservation
vRealize
Automation
vRealize
Business
VRO
VRA
IWS
IMS
DEM
IAS
VRO
VRA
IWS
IMS
DEM
IAS
Compute Pod(s)
Compute Pod(s)
Region A
Compute Stack
vCenter Server
SQL
BUS
IAS
BUC
IAS
Production Reservation
Region B
Compute Stack
vCenter Server
NSX
for vSphere
Edge Pod
Region B Edge Pod
NSX
for vSphere
vRealize Log Insight Cluster
This VMware Validated Design instantiates an extensible Cloud Management Platform layer to deliver multi-platform and multi-vendor cloud services through the use of vRealize Automation, vRealize Orchestrator and vRealize Business for Cloud. It provides comprehensive and purpose-built
capabilities that provide standardized resources in a short time span, service delivery methods that integrate with existing enterprise management systems, and user-centric and business-aware governance for all private and public cloud services.
vRealize Automation, vRealize Orchestrator

and vRealize Business for Cloud
Event
Forwarding
via Ingestion
API
Region B
Data
Node
NSX Edge Services

Gateway w/ HA
(One-Arm Load Balancer)
Master
Node
Development Reservation
Management /
Compute
vCenter Servers
vRealize
Automation
vRealize Log Insight Cluster
vRealize
Operations
Shared
Storage
Systems
NSX Edge
Services Gateways
(Routing)
Compute Stack
NSX Manager
(Secondary)
NSX Manager Pairing
vRealize Log Insight
Region B
Shared
Storage
Systems
Compute Stack
vCenter Server
Appliance
Compute Stack
NSX Manager
(Primary)
Remote Collector Cluster

Clctr
Node
Import of Management Stack

NSX Controller Configuration
from Primary NSX Manager
Compute Stack
vCenter Server
Appliance
Management /
Compute
vCenter Servers
vRealize
Automation
In a multi-region SDDC, a vRealize Log Insight cluster is deployed in each region that consists of three nodes, enabling
continued availability and increased log ingestion rates. vRealize Log Insight collects data from ESXi hosts using the syslog
protocol, connects to vCenter Server instances and integrates with vRealize Operations Manager to send notication events and
enable the last mile of root cause analysis. Content packs for Virtual SAN, NSX and vRealize Automation are also configured.
Region B
NSX
for vSphere
Management Stack
NSX Manager
(Primary)
Platform Services
Controller
Appliance
Management /
Compute
vCenter Servers
Region A
Management Stack
vCenter Server
Appliance
Region B
Platform Services
Controller
Appliance
VMware Validated Designs use several VMware solutions for network, storage and cloud management. You can monitor and
perform diagnostics on all of them by using vRealize Operations and solution management packs. In these designs, vRealize
Operations is configured with management packs for NSX, vRealize Log Insight, vRealize Automation and Storage Devices.
vRealize Operations
Development Reservation
NSX
for vSphere
Edge Pod
Edge Reservation
Edge Reservation
Core and Pod Architecture
Spine
Spine
Spine
WAN/
MPLS
Spine
Management Transport Zone in Hybrid Mode
WAN/
MPLS
VTEP
VTEP
ESXi
40 GigE
Leaf
40 GigE
Leaf
Leaf
L3
Leaf
Leaf
Leaf
Leaf
Leaf
IGMP
IGMP
L2
L2
10 GigE
IGMP
IGMP
IGMP
IGMP
IGMP
VTEP
ESXi
VTEP
VTEP
ESXi
VTEP
VTEP
ESXi
VTEP
ESXi
VTEP
VTEP
ESXi
VTEP
VTEP
ESXi
VTEP
VTEP
VTEP
ESXi
Virtual SAN
10 GigE
L2
10 GigE
VTEP
VTEP
ESXi
Virtual SAN
VTEP
ESXi
VTEP
Routed Uplinks (ECMP)
VTEP
Server
Edge Pod
(4 VSAN Ready Nodes)
Each ESXi host in the Management and Edge Pod is

redundantly connected to the SDDC network fabric and the
WAN/MPLS via two 10 GbE uplinks.
Edge Pod ESXi Host
ESXi
Management Cluster
Edge Cluster
Compute Cluster n
Minimum 4 VSAN Ready Nodes | vSphere HA and DRS Enabled

| Virtual SAN Enabled
Business Workload Requirements | Minimum 4 Nodes per Pod

VSAN Ready Nodes Recommended
Management Pod
Edge Pod
Compute Pod n
The management pod hosts the infrastructure components used to instantiate,

manage and monitor the SDDC. This includes the core infrastructure
components, such as the Platform Services Controllers, vCenter Server Instances,
NSX Managers, NSX Controllers for the management stack, vSphere Replication,
Site Recovery Manager and as well as the SDDC monitoring andautomation
solutions like vRealize Operations, vRealize Log Insight and vRealize Automation.
The edge pod provides a centralized gateway through which workloads

running in the SDDC are able to access external networks. The NSX Manager instance,
for the Edge Cluster, is deployed in the management pod. The NSX Controller Cluster
runs inside the Edge Pod, alongside the NSX Edge Service Gateways that
have been deployed to support business workloads running in the compute pods.
Within the SDDC, all business and end-user workloads run inside
the compute pods. By design, business and end-user workloads running in
the SDDC are isolated on their own network and do not have direct access to
external networks. To access external networks, traffic must be routed
through the edge pod using a shared NSX transport zone.
Management Stack
Compute Stack
Managed by Management Stack vCenter Server
Managed by Compute Stack vCenter Server
nic1
10 GigE
vRealize Operations and vRealize Log Insight
Management Distributed Switch
VLAN 1611
VLAN 1612
VLAN 1613
VLAN 1614
VLAN Trunk (802.1Q)
ESXi Host
Internet or Enterprise WAN/MPLS
Spine
Switches
Region B
To Edge Pod
Internet or
Enterprise
WAN/MPLS
To Edge Pod
To Compute Pods
To Compute Pods
Spine
Switches
L2
Region Independent Application Virtual Network

Universal Logical Switch / VXLAN 5 xxx
NSX Edge Services Gateway
One-Arm Load Balancer
L3
172.16.11.0/24
172.17.11.0/24
L2
BGP
Peering
VDP
OS
PSC
OS
VC
OS
Universal Transit Network

Universal Logical Switch / VXLAN 5xxx
192.168.10.0/24
Management Universal Distributed Logical Router

192.168.11.0/24
APP
APP
APP
APP
APP
OS
OS
OS
OS
OS
VC
OS
PSC
OS
VDP
OS
NSXM
OS
ECMP
NSX Edge
Services Gateways

192.168.10.0/24
APP
APP
APP
APP
OS
OS
OS
OS
Master
Node
Replica
Node
Data
Node
Data
Node
Management
172.16.11.0/24
DGW:
172.16.11.253
192.168.10.0/24

192.168.31.0/24
APP
192.168.11.0/24


IWS
Reserved for Disaster Recovery
OS
OS
Collector
Node
APP
vRealize Operations
OS
OS
Collector
Node
APP
APP
OS
OS
APP
APP
OS
OS
IMS
APP
APP
OS
OS
DEM
APP
APP
OS
OS
APP
APP
OS
OS
SQL
MTU
9000
VLAN NFS
MTU
9000
VLAN Management
MTU
9000
VLAN Management
MTU
9000
VLAN Management
MTU
9000
VLAN vMotion
MTU
9000
VLAN vMotion
MTU
9000
VLAN vMotion
MTU
9000
VLAN VTEP (VXLAN)
MTU
9000
VLAN VTEP (VXLAN)
MTU
9000
VLAN VTEP (VXLAN)
MTU
9000
VLAN VSAN
MTU
9000
VLAN VSAN
MTU
9000
VLAN VSAN
MTU
9000
vMotion
172.16.12.0/24
DGW:
172.16.12.253
VXLAN
172.16.13.0/24
DGW:
172.16.13.253
VSAN
172.16.14.0/24
DGW:
172.16.14.253
VLAN Customer 1
MTU
9000
VLAN Customer n
VLAN Uplink 01
VLAN Uplink 01
VLAN Uplink 02
VLAN Uplink 02
VMware Validated Designs use NFS storage as a secondary storage tier for management and compute pods. NFS is used as the target for vSphere Data Protection backups
and vRealize Log Insight log archives in management pods. NFS is also used to host the virtual machine templates in the IT Automation Cloud validated design.
APP
APP
APP
APP
APP
APP
APP
APP
OS
OS
OS
OS
OS
OS
OS
OS
OS
Application Virtual Networks for SDDC Management Solutions in Region B

(vRealize Operations, vRealize Log Insight, vRealize Automation & vRealize Business)
Cluster
VIP
APP
APP
APP
APP
APP
OS
OS
APP
OS
OS
OS
Master
Node
OS
Worker
Node
Worker
Node
Master
Node
Worker
Node
Worker
Node
ECMP
NSX Edge
Services Gateways

BGP
Peering
PCIe
Ultra DIMM
Caching
Tier
Read and Write Cache
192.168.11.0/24
Region B
NFS Storage Array
NFS Storage Array

Volume 1
VRA
Volume 2
Volume 1
Volume 2
Capacity
Export
(vRealize
Automation)
Data
Persistence
Tier
IWS
Export
(vRealize
Log Insight)
Export
(vSphere
Data Protection)
Export
(vRealize
Automation)
Export
(vRealize
Log Insight)
Export
(vSphere
Data Protection)
IMS
Region Dependent Application Virtual Network
DEM
VRO
APP
OS
OS
BUS
APP
APP
APP
OS
OS
OS
IAS
IAS
BUC
APP
APP
APP
OS
OS
OS
IAS
IAS
BUC
Reference
Networks
VRA
VIP: 192.168.11.53
192.168.11.51 > Active Node

192.168.11.52 > Active Node
IWS
VIP: 192.168.11.56
192.168.11.54 > Active Node

192.168.11.55 > Active Node
IMS
VIP: 192.168.11.59
192.168.11.57 > Active Node

192.168.11.58 > Passive Node
VRO
VIP: 192.168.11.65
192.168.11.63 > Active Node

192.168.11.64 > Active Node
Notable Acronyms
External Transit Network(s)
Universal Transit Network VXLAN
Management Distributed Port Group
Validated Products, Integrations and Versions
VRA
vRealize Automation Appliance
VMware vCenter Server 6.0 Update 2
IWS
vRealize Automation IaaS Web Server
VMware vSphere 6.0 Update 2
IMS
vRealize Automation IaaS Manager Service
IAS
vRealize Automation IaaS vSphere Proxy Agent
DEM
vRealize Automation Distributed Execution Manager
VRO
vRealize Orchestrator Appliance
BUS
vRealize Business Appliance
BUC
vRealize Business Data Collector
SQL
Microsoft SQL Server Database
Management Application Virtual Network VXLAN
Copyright 2016 VMware, Inc. All rights reserved.
Region A

vRealize Operations
Cluster
VIP
External NFS Storage
192.168.32.0/24
APP
192.168.32.0/24
APP
(vRealize Operations, vRealize Log Insight, vRealize Automation & vRealize Business)
192.168.11.0/24
192.168.31.0/24
OS
Application Virtual Networks for SDDC Management Solutions in Region A
192.168.10.0/24

APP
Compute Distributed Switch
VLAN NFS
VMware Validated Designs use rack mount Virtual SAN Ready Nodes to ensure seamless compatibility and support. The
configuration and assembly for each node is standardized with all components installed the same manner to eliminate system
variability. Virtual SAN enables both tiered-hybrid and all-flash architectures.
APP
Collector
Node
nic1 10 GigE

192.168.31.0/24
192.168.10.0/24
192.168.32.0/24
APP
Collector
Node
Region B

VRO

Region A

192.168.11.0/24

BGP
Peering
nic0
MTU 9000
Edge Distributed Switch
VLAN External Management
SSD




BGP
Peering
10 GigE
Storage
vRealize Operations
BGP
Peering
Region B
10 GigE
The two 10GbE NICs on each host are connected across the top-of-rack leaf switches and teamed on the vSphere Distributed Switch via an active-active configuration. All port groups except for the ones that carry VXLAN traffic are configured for the 'Route based on physical NIC load' teaming algorithm. VTEP kernel ports and VXLAN traffic use the Route based on
SRC-ID' algorithm. The vSphere Distributed Switch has a MTU of 9000 configured for Jumbo Frames along with with necessary VMkernel ports.
ECMP
NSX Edge
Services Gateways
VRA
BGP Peering
NSXM
OS
192.168.11.0/24
Top-of-Rack
Leaf Switches
BGP Peering
ECMP
NSX Edge
Services Gateways
192.168.10.0/24
L3
Top-of-Rack
Leaf Switches
Region A
BGP
Peering
Internet or Enterprise WAN/MPLS
Region A
Internet or
Enterprise
WAN/MPLS
nic1
MTU
9000
VLAN vSphere Replication
vRealize Automation, vRealize Orchestrator and vRealize Business for Cloud
ECMP
NSX Edge
Services Gateways
nic0
VLAN NFS
Virtual SAN Ready Nodes

ECMP
NSX Edge
Services Gateways
10 GigE
MTU 9000
Distributed Logical Routing and Application Virtual Networks for Management, Operations and Automation Solutions
Distributed Logical Routing and Application Virtual Networks
Compute Pod ESXi Host
L3
plus NFS
Minimum 4 VSAN Ready Nodes | vSphere HA and DRS Enabled

| Virtual SAN Enabled
nic0
MTU 9000
Layer 3 ToR Switch
Any Supported Storage
plus NFS
Management Pod
(4 VSAN Ready Nodes)
Compute Pods
(Up to 19 2RU Hosts or 19 VSAN Ready Nodes)
VTEP
L2
L3
Server
VTEP
ESXi
L2
10 GigE
Each ESXi host in the Compute Pod is redundantly

connected to the SDDC network fabric via two 10
GbE uplinks.
VTEP
10 GigE
Compute Distributed Switch
Leaf
L3
IGMP
VTEP
Edge Distributed Switch
40 GigE
Leaf
L3
The leaf switches of each rack acts as the Layer 3 interface for the corresponding subnet. Management and Edge Pods are provided with externally accessible VLANs for access to the Internet and/or MPLS-based corporate networks.
Compute Transport Zone in Hybrid Mode
Management Distributed Switch
40 GigE
Host Connectivity
Management Pod ESXi Host
WAN/
MPLS
Spine
VMware Validated Designs use a small set of common, standardized building blocks called pods.
Span of VLANs
Spine
Spine
Pods
The physical network architecture in the VMware Validated Designs is tightly coupled with the pod-and-core architecture and uses a Layer 3 leaf
and spine network model for an efficient and distributed core.
Span of VLANs
Leaf-and-Spine Network
VMware vSphere Update Manager 6.2

VMware vSphere Data Protection 6.1.2
VMware Virtual SAN 6.2
VMware vRealize Log Insight 3.3.1

- VMware NSX for vSphere Content Pack 3.3
- VMware vRealize Automation 7.0 Content Pack 1.1
- VMware vRealize Orchestrator 7.0 Content Pack 1.1
- VMware vRealize Operations Manager 6.x Content Pack 1.6
- VMware Virtual SAN Content Pack 2.0
VMware NSX for vSphere 6.2.2
VMware vRealize Automation 7.0.1
VMware vRealize Operations 6.2.1
VMware vRealize Orchestrator 7.0.1
- VMware NSX for vSphere Management Pack 3.0.2
- VMware vRealize Orchestrator Plug-in for NSX 1.0.3
- VMware vRealize Log Insight Management Pack 1.0.1
- VMware vRealize Orchestrator Plug-in for vRealize Automation 7.0.1
- VMware vRealize Automation Management Pack 2.0

- Management Pack for Storage Devices 6.0.4
VMware vRealize Business for Cloud 7.0.1
vmware.com/go/vvd

Vmware Validated Design: For Software-Defined Data Center 2.0

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Vmware Validated Design: For Software-Defined Data Center 2.0

Încărcat de

Drepturi de autor:

Formate disponibile

VMware Validated Design

for Software-Defined Data Center 2.0

Logical Component Architecture

Core vSphere Management Systems

NSX for vSphere

Common vCenter Single Sign-On Domain

NSX Edge Services

NSX Manager Pairing

Region A Management Pod

Remote Collector Cluster

Region B Management Pod

Region B Management Pod

Compute Stack NSX

Import of Compute Stack

Region A Edge Pod

Region A Management Pod

Region B Management Pod

Region A Management Pod

Region B Edge Pod

vRealize Log Insight Cluster

vRealize Automation, vRealize Orchestrator

Region B Management Pod

Region A Management Pod

NSX Edge Services

Region A Management Pod

vRealize Log Insight Cluster

NSX Manager Pairing

vRealize Log Insight

Remote Collector Cluster

Import of Management Stack

Region B Management Pod

Core and Pod Architecture

Management Transport Zone in Hybrid Mode

Routed Uplinks (ECMP)

Each ESXi host in the Management and Edge Pod is

Edge Pod ESXi Host

Minimum 4 VSAN Ready Nodes | vSphere HA and DRS Enabled

Business Workload Requirements | Minimum 4 Nodes per Pod

The management pod hosts the infrastructure components used to instantiate,

The edge pod provides a centralized gateway through which workloads

Managed by Management Stack vCenter Server

Managed by Compute Stack vCenter Server

vRealize Operations and vRealize Log Insight

Management Distributed Switch

VLAN Trunk (802.1Q)

Internet or Enterprise WAN/MPLS

Region Independent Application Virtual Network

Universal Transit Network

Management Universal Distributed Logical Router

Region Independent Application Virtual Network

Universal Transit Network

Universal Transit Network

Management Universal Distributed Logical Router

NSX Edge Services Gateway

NSX Edge Services Gateway

Reserved for Disaster Recovery

VLAN VTEP (VXLAN)

VLAN VTEP (VXLAN)

VLAN VTEP (VXLAN)

Application Virtual Networks for SDDC Management Solutions in Region B

vRealize Log Insight

Universal Transit Network

Read and Write Cache