The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable,

and more
predictable customer deployments. For more information visit

http://www.cisco.com/go/designzone.

ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, "DESIGNS")

IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES,
INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT
SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAG-
ES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR
INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.

THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICA-
TION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF
CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLE-
MENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.

CCDE, CCENT, Cisco Eos, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, the Cisco logo,
DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are
service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP,
CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco
Systems Capital, the Cisco Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast
Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study, IronPort, the IronPort
logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Net-
work Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The
Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Sys-
tems, Inc. and/or its affiliates in the United States and certain other countries.

All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner
does not imply a partnership relationship between Cisco and any other company. (0809R)

© 2015 Cisco Systems, Inc. All rights reserved.

2
3
4
5
Executive Summary

Background
Objective and Benefits

Audience

VersaStack Components

Figure 1

7
8
 Figure 2

Data Center Virtualization and Cloud Management

VMware vSphere ESXi and VMware vCenter Server

9
IBM Storwize V7000 Unified Storage

10
Cloud Overview and Considerations

Overview

Figure 3

Cloud Model (IaaS Features)

11
Essential Characteristics

Elasticity

Broad Network Access

12
Measured Services

1.

13
2.

14
On-Demand Self-Service Provisioning and Automation

1.

15
2.

16
3.

4.

17
Platform Modularity

Integration Points

18
Figure 4

III Party Products

ITSM Tools
Open API for Integration
Billing

Backup/Recovery

PXE
UCS Director BMA

VCenter
Virtual
ESXi Bare Metal
VersaStack
Cisco UCS

Cisco Nexus 9K & MDS

Control
IBM Storwize V7000 Unified
Update

19
Solution Architecture and Design

Architecture

20
 Application Ready Infrastructure

Chargeback
Dashboard Provisioning Management
& Reporting

Common Infrastruc-
U Portal
Self-Service ture

Bare Metal Agent UCS Director 5.3.1 A/D, Nexus

DNS, 1000v
VCenter 5.5 U2 DHCP VSM

OS 1 OS 2 ESXi5.5 U2

VersaStack
Cisco UCS B-200/C-220 M3/M4 Servers

Cisco Nexus 9396 & MDS Switches

IBM Storwize V7000 Unified Storage

Tenant Design

All policy admin

cloudadm

Development Group Admin Test Group Admin Production Group Admin

devadm testadm prodadm

End-user End-user End-user

devuser1 ……………………. testuser1 ……………………. produser1 …………………….

21
22
Cloud Management Environment Sizing

Minimum System Requirements for a Single-Node Setup

Reference

For optimal performance, reserve additional CPU and memory resources. We recommend that you reserve
the following resources in addition to the minimum system requirements listed in the tables below: CPU re-
sources of more than or equal to 3000MHz, and additional memory of more than or equal to 4GB.

Up to 2,000 Virtual Machines

Table 1

Up to 5,000 Virtual Machines

Table 2

23
Table 3

24
Deployment of Cloud Services

Base Platform

High-Level Architecture
Figure 5

25
Figure 6

26
Cisco UCS Director Installation and Configuration

You must obtain a license to use Cisco UCS Director. Please see sections titled About Licenses and Fulfilling
the Product Access Key (PAK) at the following link before you begin:
http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-director/vsphere-install-guide/5-
3/b_Installing_UCSDirector_on_vSphere_5_3.pdf

1.

2.

27
Initial Cisco UCS Director Setup

1.

2.

3.

28
 Note: Upgrade the reserved resources for the newly created VM.

4.

5.

29
6.

30
7.

8.

Note: UCSD version 5.3.1 comes with a set of wizards that help with setup after installing the product.

9.

31
 10.

Create a Converged Pod

1.

Note: Number of credential policies * Number of addresses should be less than 1000 for discovery.
If this value exceeds 1000, break the discovery process into smaller batches.

32
2.

Note: MDS 9148S switches have the Account type of Cisco Nexus OS.

3.

33
4.

34
5.

35
6.

7.

36
8.

37
 Note: For some browsers you may need to add the web URL to trusted sites to display correctly.

Create Local Users and Groups

1.

38
2.

3.

4.

39
5.

6.

7.

40
 8.

9.

Note: The User Role determines whether an account is specific to a group or not. Therefore, only accounts
with privileges that can be limited to the group will be presented with the User Group field and a drop-down list.

LDAP Integration

Note Users that do not belong to a group or a domain user’s group display in LDAP as User With No Group.
These users are added under the domain user’s group in Cisco UCS Director.

1.

2.

41
3.

4.

5.

6.

7.

42
 8.

9.

10.

11.

Note: local groups and users can also be added and managed.

Virtual Data Center (vDC)

43
IBM Storwize V7000 Storage Tiering

44
45
 Note: For the Gold tier, the second option consisting of Flash and SAS without NL-SAS disks was validated.
Based on performance requirements, tiers could be deployed in various configurations. For example, Gold
could be all flash followed by Silver consisting of all SAS and Bronze with all NL-SAS disks.

IBM Storwize V7000 Storage Pool Setup

1.

2.

Note: you need to have candidate/free disks available to assign to the pool.

3.

46
4.

47
5.

Note: Retain some enterprise drives for the Silver tier.

48
6.

49
7.

8.

50
9.

51
10.

52
11.

53
12.

54
13.

55
14.

Note: Enterprise located in the last column shown below points to SAS disks.

56
Case Study - Infrastructure Management for In-House Software Development

57
58
1.

2.

3.

4.

59
60
 All policy admin - Cloudadm

Dev_vDC Test_vDC Prod_vDC

Development Group Admin Test Group Admin Production Group Admin

devadm testadm prodadm

Service End-User (devuser1) Service End-User (testuser1) Service End-User (produser1)

Catalog Catalog Catalog

App db App db App db

Compute Policy Compute Policy Compute Policy Compute Policy Compute Policy Compute Policy Compute Policy Compute Policy Compute Policy
2 vCPU 4 vCPU 8 vCPU 2 vCPU 4 vCPU 8 vCPU 2 vCPU 4 vCPU 8 vCPU
4 GB Memory 8 GB Memory 16 GB Memory 4 GB Memory 8 GB Memory 16 GB Memory 4 GB Memory 8 GB Memory 16 GB Memory

Storage Policy Storage Policy Storage Policy Storage Policy Storage Policy
Storage Policy Storage Policy Storage Policy Storage Policy
40 GB Bronze 80 GB Bronze 160 GB Bronze 160 GB Silver 80 GB Gold
40 GB Silver Tier 80 GB Silver Tier 40 GB Gold Tier 160 GB Gold Tier
Tier Tier Tier Tier Tier

Network Policy Network Policy Network Policy Network Policy Network Policy Network Policy Network Policy Network Policy Network Policy
VM Network VM Network VM Network VM Network VM Network VM Network VM Network VM Network VM Network

Cost Model Cost Model Cost Model Cost Model Cost Model Cost Model Cost Model Cost Model Cost Model
$x/mth/instance $2x/mth/ $4x/mth/ $y/mth/instance $2y/mth/ $4y/mth/ $Z/mth/instance $2Z/mth/ $4Z/mth/
(say) instance instance (say) instance instance (say) instance instance

Note: Cost presented as $x < $y <$z where $x is for Development group with bronze storage tier capped by
a budget. $y is when silver tier is used with partial budget capping as in the case of the Test group. $z is for
production where gold tier storage is required. Thus, total cost for each vDC is determined by number of in-
stances predicated by performance and resiliency needs (multiple instances for redundancy/D.R).

5.

6.

61
7.

8.

9.

10.

62
11.

63
 12.

Create System Policy

1.

64
65
 2.

Create Compute Policy

1.

66
2.

3.

67
Create Network Policy
1.

2.

68
69
3.

4.

5.

70
6.

71
 7.

Note: Adaptor Type needs to be matched with the corresponding item mapped to the template in
vCenter. This will allow for a mapping of the previously cloned template (through vCenter) to
the service request to prevent the operator/self-service user from altering resources used by the
instance.

Create Storage Policy

1.

72
2.

3.

73
 4.

5.

6.

7.

Note: The disk size specified should be larger than what is used in the template created in vCenter. This will
allow for a mapping of the previously cloned template (in vCenter) to the service request to prevent the opera-
tor/self-service user from altering resources used by the instance.

Create Cost Model Policy

1.

74
 2.

Note: Skip the User Action Policy at this stage. It is used as a post-provisioning option within the vDC. The
option to delete inactive VM’s allows for the reclaiming of resources that are not powered on (inactive) be-
tween 1 to 90 days, if desired.

Create End User Self -Service Policy

1.

2.

75
3.

76
 4.

5.

Clone Cisco UCS Director Policies

1.

2.

3.

4.

5.

6.

7.

77
8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

Note: There is the option to filter storage selections on disk characteristics of capacity, performance and us-
age.

21.

22.

23.

24.

25.

26.

78
 27.

Note: There is the option of picking required port-groups and addressing scheme (static/DHCP) for desired
resiliency, bandwidth and scale.

28.

29.

30.

31.

32.

33.

34.

35.

Note: Above policies are created for TestGroup from DevGroup. Edit cloned policies as required. Follow the
same process and create another set of policies for the ProdGroup.

Clone vDC
1.

2.

3.

79
4.

5.

80
Catalog Publishing

Note: All users have the same catalog items with slight variations due to tier of storage with corresponding
cost differences. Given that we use template based catalog items in Cisco UCS Director, there is more flexibil-
ity. This is because they deliver different functions (development, test and production) on the same applica-
tion. However, if the tenants happen to have different requirements, Cisco UCS Director has the flexibility to
accommodate different catalog items for each group.

81
 Cloud Admin

Tenant-A: Development Tenant-C:Production

Tenant-B: Test
Group Admin Group Admin
Group Admin
devadm prodadm
testadm

Service end-user Service end-user Service end-user

devuser1 testuser1 Produser1

Catalog Catalog
Catalog

Approved Budget
Budget Approvers
&
&
Resource Limits No Budget
Resource Limits

App App App

db db db

1.

2.

82
3.

4.

83
5.

84
Publishing Advanced Catalog

1.

2.

85
3.

86
4.

87
5.

88
Self-Service Portal

Design

Figure 7

Budget & Re-

Cost Model source Limits
System Policy

Compute Pol- vDC Self- Groups &

icy Service Op- Users
Network Poli-
tion
cy
Storage Policy
Cloud Portal Catalog

VersaStack
Platform

Implementation

Create Service Request

1.

89
2.

3.

90
4.

5.

91
 The administrator sets a permitted operation shown above in the end-user policy and it can be modified as
required. Service requests created by the user can be seen under the Services button above. Service requests
created by the administrator will not be visible to the end-user. However, outcome of such requests is seen.

Workflow Triggers and Schedules

1.

92
2.

93
 Note: Type of Object to Monitor can be one of the following with relevant parameters.

3.

4.

5.

6.

94
7.

95
Scheduled Workflow
1.

96
2.

97
3.

98
4.

99
Scenarios for triggers and Schedulers

1.

2.

3.

4.

5.

100
Setting Quotas

1.

2.

101
 3.

4.

Configure Budget Policy

Resource Used Unit Cost Resource Cost

X =
(Resource Limits) (Chargeback) (Budget)

1.

102
 2.

3.

Dashboard

1.

103
 2.

3.

4.

5.

Resource Monitoring

1.

104
2.

105
3.

106
4.

107
5.

108
UCS Director Bare-Metal Provisioning

Cisco UCS Director-BMA Configuration

1.

109
2.

3.

4.

5.

6.

7.

8.

110
Bare-Metal Image Upload

1.

2.

1.

2.

3.

4.

5.

111
6.

7.

8.

112
9.

113
10.

114
Bare Metal Workflow Orchestration

1.

2.

3.

4.

115
5.

6.

116
7.

117
8.

118
 Note: only screen-shots where input is required are shown. Please continue to the next screen (click Next)
for each task until you arrive at a screen as shown below to provide required inputs.

9.

119
10.

120
11.

121
12.

122
13.

123
14.

15.

124
16.

17.

125
18.

19.

20.

126
Summary

127
128
Cloud Use Cases

Table 4

129
Account Services

Open an account

130
1.

131
2.

3.

132
Close an account

133
134
Terminate an account

Data Services

Copy data into the cloud

135
Erase data in the cloud

136
Identity Management

User account provisioning

User authentication

137
Virtual machine lifecycle services

Provision virtual machine

138
Manage/Reconfigure an existing virtual machine

Decommission a Virtual Machine

139
Bill of Material

Part Number Product Description Quantity

Cisco Nexus 9300 Switching

N9K-C9372PX Nexus 9300 with 48p 10G SFP+ and 6p 40G QSFP+ 2

N9KDK9-612I3.1 Nexus 9500 or 9300 Base NX-OS Software Rel 2

6.1(2)I3(1)

Part Number Product Description Quantity

Cisco MDS FC Switch

DS-C9148S-12PK9 MDS 9148S 16G FC switch, w/ 12 active ports 2

M91S5K9-6.2.9 MDS 9100 Supervisor/Fabric-5, NX-OS Software Re- 2

lease 6.2.9

Part Number Product Description Quantity

Cisco UCS Unified Compute System

UCSB-5108-AC2 UCS 5108 Blade Server AC2 Chassis, 0 PSU/8 fans/0 1

FEX

UCS-IOM-2208XP UCS 2208XP I/O Module (8 External, 32 Internal 10Gb 2

Ports)

UCSB-B200-M4 UCS B200 M4 w/o CPU, mem, drive bays, HDD, mezz 4

140
 UCS-CPU-E52650D 2.30 GHz E5-2650 v3/105W 10C/25MB Cache/DDR4 8
2133MHz

UCS-MR-1X162RU-A 16GB DDR4-2133-MHz RDIMM/PC4-17000/dual 32

rank/x4/1.2v

UCSB-MLOM-40G-01 Cisco UCS VIC 1240 modular LOM for blade servers 4

Part Number Product Description Quantity

Cisco UCS UCS-FI-6248UP Fabric

Interconnect

UCS-FI-6248UP UCS 6248UP 1RU Fabric Int/No PSU/32 UP/ 12p LIC 2

N10-MGT012 UCS Manager v2.2 2

Part Number Product Description Quantity

Cisco FEX (optional)

N2K-C2232PF 2
Nexus 2232PP with 16 FET, choice of airflow/power

Part Number Product Description Quantity

Cisco UCS Rack Servers

UCSC-C220-M4S UCS C220 M4 SFF w/o CPU, mem, HD, PCIe, PSU, rail 2
kit

UCS-CPU-E52640D 2.60 GHz E5-2640 v3/90W 8C/20MB Cache/DDR4 4

1866MHz

141
UCS-MR-1X162RU-A 16GB DDR4-2133-MHz RDIMM/PC4-17000/dual 16

rank/x4/1.2v

UCSC-PCIE-CSC-02 Cisco VIC 1225 Dual Port 10Gb SFP+ CNA 2

142
Software Revisions

143
Conclusion

144
References

145
Appendix –A

Orchestration - Task Library

146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
Appendix – B

Bare-metal workflow for Linux:

163
Note: In this section, only screenshots where default inputs are not taken is shown. Screenshots are in simi-
lar sequence as tasks in above workflow with task name shown on top left within parenthesis.

164
165
166
167
168
169
170
171
Summary

172
173
174
About the Author

Acknowledgements

175