Documente Academic
Documente Profesional
Documente Cultură
Orran Krieger, Senior Staff Engineer, VMware, Inc.; Adjunct Professor, CMU Tichomir Tenev, Senior Staff Engineer, VMware, Inc.
Overview
1. 2. 3. 4.
What is cloud computing VMware vCloud abstractions VMware vCloud implementation challenges
Research opportunities
Confidential
What is cloud?
IT dept.
Host it R us.
Host it R us.
Host it R us.
Host it R us.
$5/day
Host it R us.
$5/day
Host it R us.
Host 4 Less
$5/day
$3/day
Host it R us.
Host 4 Less
Host it R us.
Host 4 Less
Host it R us.
Host 4 Less
Host it R us.
Host 4 Less
Snoopys Startup
14
Host it R us.
Host 4 Less
Dogspace
Snoopys Startup
15
Host it R us.
Host 4 Less
Dogspace
DogTube
Snoopys Startup
16
Host it R us.
Host 4 Less
Dogspace DogFlix
DogTube
Snoopys Startup
17
Host it R us.
Host 4 Less
Dogspace DogFlix
DogTube dBay
Snoopys Startup
18
Host it R us.
Host 4 Less
Dogspace DogFlix
DogTube dBay
Snoopys Startup
19
Host it R us.
Host 4 Less
Snoopys Startup
20
Host it R us.
Host 4 Less
DogTube dBay
Snoopys Startup
21
Host it R us.
Host 4 Less
dBay
Snoopys Startup
22
Host it R us.
Host 4 Less
dBay
Snoopys Startup
23
Host it R us.
Host 4 Less
dBay
Snoopys Startup
24
Host it R us.
Host 4 Less
dBay
Snoopys Startup
25
Host it R us.
Host 4 Less
26
27
28
Test
Release
Install
Configure
Operate
SaaS model
Develop
Test
Operate
29
Develop
Test
Operate
Enormous investment required in application level to scale. So, successful SaaS vendors started building re-usable
platforms
30
Cloud Front
Simple DB
Amazon AWS
31
Example
32
Not a utility!
Very rich set of incompatible services from each vendor Locks out everyone else from innovating Doesnt work for all applications/workloads
33
34
John McCarthy, MIT Centennial in 1961 Virtualization converts computation into a fungible commodity
35
Nicholas Carr:
As with the factory-owned generators that dominated electricity production a
century ago, today's private IT plants will be supplanted by large-scale, centralized utilities.
Enables long-tail in SW
36
Challenges
Security/multitenancy
37
PHYSICAL
VIRTUALIZED
INTERNAL CLOUD
R P 1
R P 2
HW HW HW HW
HW HW HW
HW HW
HW HW
HW HW HW
HW HW
HW
HW
38
Requirement
Overprovisioning Support SLA Power management Ethernet networks controlled by network admin
Product/technology
VMotion & storage VMotion Distr. Resource Scheduling (DRS) Distr. Power Management (DPM) vNetwork Distributed Switch
VMware studio
VMsafe vShield Zones VMware View 3
39
Org 2 Org 1 Company Compan Access Control Access Control Message B yDB service A Self Service UI
Users Users
vDC
Service
vDC
Alloca tion
Alloc ation
Automation from system administrators self service UIs, VMs that implement IaaS, PaaS, SaaS Let the world innovate on implementing DB service,
content distribution service, blob storage service, messaging service
Plug-in
Technologies vSphere
Can replace any part of the service. Can replace the entire implementation under the API Federation between multiple sites and multiple clouds.
40
40
Key Challenges
41
Overview
1. 2. 3. 4.
What is cloud computing VMware vCloud abstractions VMware vCloud implementation challenges
Research opportunities
42
Confidential
Takes the benefits of virtualization: encapsulation, isolation and mobility higher up the stack 1. Product: eCommerce
Policies
Properties:
Comprised of one or more VMs
(may be multi-tier applications)
2.
Topology
3.
4. 5. 6. 7.
Tomcat
Built by:
ISVs / Virtual Appliance Vendors IT administrators SI/VARs
SAP
43
vDC
vApp vApp
Type of Commodity
Compute, Storage and Network
SLAs
Quantity
MB of RAM, MHz of CPU, GB
foo bar (isolated)
of Storage
VDCNet (fenced)
44
45
46
Organization 1
vDC
vDC vApp
Storage
vDC
Resource Pool
47
Standardizable
Extensible Secure Scalable Supports legacy applications and enterprise users
48
Request
POST https://vcloud.example.com/vapp/413/power/action/powerOn
49
Request
POST https://vcloud.example.com/vapp/413/power/action/powerOn
50
Response
202 Accepted <?xml version="1.0" encoding="UTF-8"?> <Task href="https://vcloud.example.com/task/389"
type="application/vnd.vmware.vcloud.task+xml"
startTime="2009-7-31T09:30:47Z" status="running" ...> <Link rel="task:cancel href="htt.../task/389/action/cancel"/> <Owner href="https://vcloud.example.com/vapp/413" type="application/vnd.vmware.vcloud.vapp+xml name="My vApp"/> </Task>
51
Request
52
Request
53
Response
<VApp name="My vApp" status="1" href="https://vcloud.example.com/vapp/413" ...> <Link rel="up" href= "https://vcloud.example.com/vdc/128"/> <NetworkSection>...</NetworkSection> <ovf:OperatingSystemSection ...> <Link rel="edit" href="http..." ... />
54
Entity Model
UML Notation:
whole-part relationship part is exclusively owned by whole
group-member relationship
member maybe shared between groups
55
Entity Model
56
Entity Model
57
Entity Model
58
List of Operations
vApp Operations
POST <vapp-uri>/action/{deploy, undeploy} POST <vapp-uri>/power/action/{powerOn, powerOff} POST <vapp-uri>/power/action/{reset, suspend} POST <vapp-uri>/power/action/{shutdown, reboot} GET <vapp-uri>/screen POST <vapp-uri>/screen/action/acquireTicket
Upload/Download/Provisioning Operations
POST <vdc-uri>/action/composeVApp POST <vdcuri>/action/instantiateVAppTemplate POST <vdc-uri>/action/instantiateOvf POST <vdc-uri>/action/annotate POST <vdc-uri>/action/uploadVAppTemplate POST <vdc-uri>/media PUT <upload-uri> GET <download-uri> DELETE <resourceEntity-uri>
Task Management
GET <tasks-list-uri> GET <task-uri> POST <task-uri>/action/cancel
Inventory Listing
GET <vapp-uri> GET <vdc-uri> GET <vAppTemplate-uri> GET <media-uri> GET <network-uri>
Catalog Management
GET <catalog-uri> POST <catalog-uri>/catalogItems
59
Mapping of Abstractions
New Layer above vSphere
vDC
New Technologies
Provider vDC
vSphere
vSphere
Scalability Security
Physical
Physical Infrastructure
VLAN Host SAN
Provider vDC: Compute resources and Storage vDC: allocation out of provider vDC given to Org: Sub-allocation of storage and compute Networks
60
Overview
1. 2. 3. 4.
What is cloud computing VMware vCloud abstractions VMware vCloud implementation challenges
Research opportunities
61
Architecture Overview
UI
API
VMRC
Image Transfer
HTTPS
HTTPS
HTTPS
HTTPS
Firewall
webapp servlet
Console Proxy
Transfer Service
cell
VC Proxy
Inventory
Task Mgr
cell
62
Architecture Overview
UI
API
VMRC
Image Transfer
Firewall
cellcell
cell
cell cell
cell cell
Console Proxy
cell
63
Architecture Overview
UI
API
VMRC
Image Transfer
Firewall
Load Balancer
cell
cell
cell
cell
Database
Message Bus
64
Architecture Overview
UI
API
VMRC
Image Transfer
Firewall
Load Balancer
cell
cell
cell
cell
Database
Message Bus
additional can for pick up the role If aSome cells haveOperationsroles,be handled by any cell and to run task scheduler cell with special role dies, other cellsexample monitoring (some performance gain can be achieved by directing same session to same cell)
65
Architecture Overview
UI
API
VMRC
Image Transfer
Firewall
Load Balancer
cell
cell
Database
Message Bus
66
Substrate
UI API VMRC Img xfer
Firewall
Load Balancer
cell
Substrate
cell
cell
cell
Database
VC
Any cellOnly make a VIM call can one cell listening for updates for a VC keeps open connection
ESXi
ESXi
67
Firewall
Load Balancer
cell
cell
cell
cell
Database
VC
VC
ESXi
ESXi
ESXi
ESXi
ESXi
ESXi
68
System Security
UI
Public Internet
cell
Image Transfer
Uploaded image can be scanned for vulnerabilities. Custom moderation process can be configured to inspect images prior to addition into the system.
Storage
VC and ESX
69
Middle Tier
Outer Firewall Inner Firewall vCloud Client
vCloud Substrate
ESX VC ESX
vCloud Client
target identification
Cell
LB
Connections carry
ESX
vCloud
Cell
VC
ESX
Client
ESX
Port 443 Ports 902, 903
70
Image Transfer
Client
Embeddable client to allow users to upload vApps (OVF) and media (iso, flp) into cloud. Supports chunked resumable uploads
Steps:
vCloud cell
vDC service Transfer Server
Transfer session
Message Bus
Client PUTs file chunks: Transfer server writes to spooling FS On File complete posts message on bus After last file, vDC service: Validates OVF
vSphere Datastore
Spooling FS
71
Image Transfer
All state in DB to handle failures in cell OVF validated at multiple points Upload of completes before sending bits to Datastore
Client
vCloud cell
vDC service
Transfer Server
Transfer session
Message Bus
vSphere Datastore
Spooling FS
72
Overview
1. 2. 3. 4.
What is cloud computing VMware vCloud abstractions VMware vCloud implementation challenges
Research opportunities
73
Security: detection, not trusting service provider, ... Rest API: versioning, efficiency, client interoperability, enabling
ecosystem
74
applications
75
General
Simple
Portable Apps & services
enormously complicated. They are enormously difficult to customize to support new workloads, or to exploit new specialized HW. Massive investment to support all the different OSes, e.g., validation in application, device driver development Come with substantial management overhead.
Windows
Linux
Solaris
MacOS
DD execution
DD execution
76
Windows
Linux
Solaris Hypervisor
MacOS
DD execution
DD execution
DD execution
DD execution
77
Java App
Storage
Appliance
HPC App
Security SVC
Hypervisor
Applications with reduced needs can be moved off of general purpose OS. Application OS can be a reduced OS, or a highly customized library OS: more easily exploit new HW, massive multi-core, extra blades Java applications require restricted interfaces, native code that invokes OS services can be shipped to legacy environment. Cluster services require highly deterministic real-time environment. HPC applications require specialized services (e.g., scheduling & memory management) Security services can be implemented with a reduced TCB
78
Research in OS development
Scalable deployment for HPC. Communication protocol. Developing library OS that is re-usable. Scalability for massive multi-core. Migration to and from generic OS.
Real time Scalability
Control of TLB for managed code Code and file system sharing Examples: Managed Code (Libra IBM, Liquid VM BEA,
Maxine VM SUN), HEC/HPC (UNM/Sandia, Cambridge) Games (Sony PS3), Denali from UofWashington
79
What metrics can be collected presented to user? How do we characterize application? How do we characterize physical capacity?
80
Example: Grid/batch
New scheduler for grid/batch tasks New scheduler for data intensive supercomputing: e.g., Hadoop Fork task across 100s of nodes: e.g. SnowFlock from UofToronto
81
Exploiting long tail for data deduplication Encryption/security for data Trading floor/futures market for capacity
82
When your desktop is in the cloud, what changes? Disaster recovery Being a system admin for your mom.
83
Architecture can evolve without OS stifling innovation Implicitly managed storage hierarchy Scale out storage
84
Concluding Remarks
Cloud computing is going to be transformative to our industry Lots of changes needed before enterprise adopts Lots of new research opportunities
Vendor lock in will damage innovation and kill the research community
Beneath all these clouds, we need utility computing Open source vCloud API (www.vmware.com/go/vcloudapi)
85