TEQIP II Sponsored Faculty Development Programme On Hands on Cloud &

Big Data Analytics 12.5.2015 to 25.5.2105

PRactical Approaches of Cloud Computing Using Open Stack

12-05-2015 & 13-05-2015

By
D.KESAVARAJA B.E , M.E ,M.B.A,(PhD)
Assistant Professor / CSE
Dr Sivanthi Aditanar College of Engineering , Tiruchendur .
EMail :

admin@k7cloud.in

ORGANIZED BY

DEPARTMENT OF INFORMATION TECHNOLOGY

ANNA UNIVERSITY, BIT CAMPUS
TIRUCHIRAPPALLI-620024
Web:www.annauniv.edu/www.aubit.edu.in

Welcomes U All

Agenda Day -1
1. Cloud Computing
2. Open Stack
3. Nova
4. Dashboard Demo
5. Open Stack - SWIFT
6. Installation Cent OS7
7. Pack Stack RDO

Agenda Day -2
1. OpenStack rc
2. KeyStone
3. Neutron
4. Instance cirros , fedora
5. SSH , CURL , Rdesktop
6. Deployment
7. E-Resources

Introduction
Cloud computing means storing and accessing data and
programs over the Internet instead of your computer's hard
drive.
Cloud computing is a computing term that evolved in the late
1990s
According to Gartner's Hype cycle, cloud computing is a top
most technology
Cloud computing is a research topic, as it is a market offering.
6

Use the right tool for the job

Computer/Server & CLOUD

What is Cloud Computing?

Applications maintained by expert partner,
so always up to date

What is Cloud
Computing?

Delivered over a secure

high quality network
Operational in weeks
not months
Available anywhere with
an internet
connection, even
when on the move

No installation of hardware or software at

your premises
What are the
benefits?

A set of applications
managed and hosted
externally by a
specialist partner

No specialist IT support required

No capital expenditure
pay on a subscription basis
50% reduction in total ownership
costs

Lets
Use
Cloud

What is OpenStack
Dashboard
(horizon)

Network
(Neutron)

Provides UI
for other projects

Provides
network
connectivity

Compute
(nova)
Block
Storage
(cinder)

Provides
Images

Provides
volumes
Provides Authentication and Service
Catalog for other Projects

Identity
(keystone)

Slide 11

Image
repo
(glance)

Stores
Images
as
Objects

Object
Storage
(Swift)

12

Cloud Computing

Technology to provide
everything as a service
via Internet
XaaS

Every
morning

what
Cloud
Computing
Utility Computing
Utility
do
youService
read?

Cloud -- a Utility Computing

Cloud Computing
Cloud Computing s characteristics:

1. Scale with increasing demand

2. Accessible anytime
3. Have Cloud operating system to manage
resources and hide details from users
4. Pay Per Uses

1. Scalability
Processing
resources
for 1

Processing
resources for
an organization

2. Accessible Anytime
Required
Resources

Time

Required
Resources

Time

3. Cloud Operating Systems

Cloud OS

4. Pay Per Uses

May
Required
Resources

Feb
Jan

Mar Apr

Jun
Time

May
Required
Resources

Feb
Jan

Mar Apr

Jun
Time

Type of Services: Software

SaaS

Type of Services: Platform

PaaS

Type of Services: Infrastructure

IaaS

Cloud Layers

SaaS
PaaS
IaaS

Service Space
Private Cloud
Use within org
Org pays
Share resources within
org

Lower
demands

Public Cloud
Use on Internet
Pay per uses
Share resources with
everyone

Hybrid Cloud

Higher
demands

Cloud Computing Challenges: Dealing with

too many issues
Scalability
Reliability
Billing

Utility & Risk

Managem
ent
Programming Env.
& Application
Dev.
Software Eng.
Complexity

Top Ten Strategies

Cloud Computing.
Advanced Analytics.
Client Computing.
IT for Green.
Reshaping the Data Center.
Social Computing.
Security Activity Monitoring.
Flash Memory.
Virtualization for Availability.
Mobile Applications.
REF: http://www.gartner.com

Source: Trends in Technology of cloud Computing ETRI

OpenStack Overview
Cloud OS developed by Rackspace and
NASA
Infrastructure as a Service
Support Private Cloud and Public Cloud
Open Source (Apache 2.0 license)
OpenStack Foundation
Popular and widely supported

What OpenStack provide?

1. manage virtual machines

2. manage virtual networks

3. manage virtual storages

4. Multi-tenents

What is OpenStack

The open source software for building private and public clouds
Controls large pools of compute, storage, and networking resources

OpenStack Conceptual Architecture

OpenStack Logical Architecture

OpenStack Components

Computing

Networking

OpenStack Networking (Neutron)

Storing

OpenStack Compute (Nova)

OpenStack Image service (Glance)

OpenStack Object Storage (Swift)

OpenStack Block Storage (Cinder)

IdentityKeystone
Dashboard Horizon

OpenStack Components (Cont.)

Nova Compute

Glance

Compute resource management and Scheduler

VM life cycle management and VNC proxy
Discovering, registering, and retrieving VM images

Neutron

Manage VMs Network, assign floating IP, Iptables,

openvswitch

OpenStack Components (Cont.)

Swift

Cinder

Provides persistent block storage to VM

Keystone

Object Storage (ex. Amazon S3)

User Identity
Components need register to keystone

Horizon

OpenStack - Compute

API

nova-api : supports OpenStack Compute API, Amazon's EC2 API

and a special Admin API
nova-api-metadata : accepts metadata requests from instances

Computing core

nova-compute : creates and terminates virtual machine

instances(KVM, qemu, XEN and etc)
nova-schedule : schedule the VM to run on a host
nova-conductor : mediator between nova-compute and the
database

OpenStack Compute(Cont.)

Networking for VMs

nova-network : setting up bridging interfaces or

changing iptables rules -> Neutron
nova-dhcpbridge : tracks IP address leases and
records -> Neutron

Console Interface

nova-consoleauth : authorizes users tokens that

console proxies provide
nova-novncproxy : provides a proxy for accessing

OpenStack Compute(cont.)

Image Management (EC2 scenario)

nova-objectstore : provides an S3 interface for registering images

onto the image management service
euca2ools client : euca2ools can be used to issue cloud
management commands

Command Line Interpreter/Interfaces

nova client
nova-manage client
Queue service : RabbitMQ
Database : sqlite3, MySQL and PostgreSQL

OpenStack Object Store

Swift

Distributed file system and prevent any single point of

failure
swift-proxy-server) accepts incoming requests via the
OpenStack Object API or HTTP

OpenStack Image Store

glance-api

glance-registry

stores, processes and retrieves metadata about images (size, type, etc.)

A database to store the image metadata

accepts Image API calls for image discovery, image retrieval and image
storage

sqlite3, MySQL and PostgreSQL

image repository

Swift, normal filesystems, RADOS block devices, Amazon S3 and HTTP

OpenStack Identity

keystone

handles API requests as well as providing configurable

catalog, policy, token and identity services

pluggable backend

LDAP or SQL and Key Value Stores

OpenStack Network

Neutron-server

accepts API requests and then routes them to the

appropriate OpenStack Networking plugins for action

Networking plugins and agents

Plugins : Cisco virtual and physical switches, Nicira NVP

product, NEC OpenFlow products, Open vSwitch, Linux
bridging and the Ryu Network Operating System
Agents : L3 (layer 3), DHCP (dynamic host IP
addressing)

OpenStack Block Storage

cinder-api

cinder-volume

accepts API requests and routes them to cinder-volume for action

maintain Cinder database state
interacting with other processes (like cinder-scheduler) through a
message queue
access upon block storage

cinder-scheduler

picks the optimal block storage provider node to create the volume
on

OpenStack Dashboard

Horizon

Horizon is a modular Django web application that

provides an end user and administrator interface to
OpenStack services

OpenStack Layers
User IT Infrastructure
Cloud OS
Virtualization/OS
Hardware/Storage/Network

OpenStack Components
DASHBOARD

KEYSTONE

 OpenStack Dashboard (Horizon) :

Web-based User interface

OpenStack Identity Service

(Keystone) : Authentication and
authorization

OpenStack Components
NOVA

GLANCE

Neutron

 OpenStack Compute (Nova):

create and manage VMs
OpenStack Image Service
(Glance): manage VM images
and snapshots
OpenStack Network
(Neutron): manage virtual
network

OpenStack Components
SWIFT

SCIENCE
CLOUD

CINDER

 OpenStack Object Storage

(Swift): manage storage
Object
OpenStack Block Storage
(Cinder): manage Virtual
Storage Device

OpenStack Operation
1.User Login

2. Authen and
give Token

OpenStack Operation

3. Issue create
VM request

4. Nova choose
machine to run VM

OpenStack Operation

5. config network
on compute and
net nodes

6. Nova copy image

from glance to
compute node

7. Nova run VM on
compute node

OpenStack: The Mission

"To produce the ubiquitous Open Source

cloud computing platform that will meet the
needs of public and private cloud
providers regardless of size, by being
simple to implement and massively
scalable."

OpenStack Founding Principles

Apache 2.0 license (OSI), open development process
lOpen design process, 2x year public Design Summits
lPublicly available open source code repository
lOpen community processes documented and
transparent
lCommitment to drive and adopt open standards
lModular design for deployment flexibility via APIs
l

Community with Broad Commercial Support

OpenStack Cloud Architecture

Compute

OpenStack Compute Key Features

ReST-based API
Asynchronous
eventually consistent
communication
Horizontally and
massively scalable
Hypervisor agnostic:
support for Xen ,XenServer, Hyper-V, KVM,
UML and ESX is coming

Hardware agnostic:
standard hardware, RAID not required

Public Network

Server Groups
Dual Quad Core
RAID 10 Drives
1 GigE Public
1 GigE Private
1 GigE Management

Management

Example OpenStack
Compute Hardware
(other models possible)

Private Network
(intra data center)

User Manager

Cloud Controllers: Global state of

system, talks to LDAP, OpenStack
Object Storage, and
compute/storage/network workers
through a queue

ATAoE / iSCSI

API: Receives HTTP requests,

converts commands to/from API
format, and sends requests to cloud
controller

OpenStack Compute

Host Machines: workers

that spawn instances
Glance: HTTP + OpenStack Object
Storage for server images

System Components

API Server: Interface module for command and control requests

Designed to be modular to support multiple APIs
In current release: OpenStack API, EC2 Compatibility Module
Approved blueprint: Open Cloud Computing Interface (OCCI)
Message Queue: Broker to handle interactions between services
Currently based on RabbitMQ
Metadata Storage: ORM Layer using SQLAlchemy for datastore abstraction
In current release: MySQL
In Diablo: PostgreSQL
User Manager: Directory service to store user identities
In current release: OpenLDAP, FakeLDAP (with Redis), Database
Scheduler: Determines the placement of a new resource requested via the
API
Modular architecture to allow for optimization
Base schedulers included in Bexar: Round-robin, Least busy

System Components (Cont.)

Compute Worker: Manage compute hosts through commands
received on the Message Queue via the API
lBase features: Run, Terminate, Reboot, Attach/Detach Volume, Get
Console Output
lNetwork Controller: Manage networking resources on compute
hosts through commands received on the Message Queue via the
API
lSupport for multiple network models
lFixed (Static) IP addresses, VLAN with NAT, DHCP
lVolume Worker: Interact with iSCSI Targets to manage volumes
lBase features: Create, Delete, Establish
lImage Store: Manage and deploy VM images to host machines
l

Object Storage Summary

To Load Balancers

Example OpenStack
Object Storage Hardware
Proxies

5 Zones
2 Proxies per 25
Storage Nodes
10 GigE to Proxies
1 GigE to
Storage Nodes
24 x 2TB Drives
per Storage Node

Example Large Scale Deployment -- Many Configs Possible

Object Storage Key Features

ReST-based API

Data distributed evenly throughout

system
Scalable to multiple
petabytes, billions of
objects

Account/Container/Object structure
(not file system, no nesting) plus
Replication (N copies of accounts,
containers, objects)
No central
database

Hardware agnostic: standard

hardware, RAID not required

System Components

The Ring: Mapping of names to entities (accounts,

containers, objects) on disk.
l

Stores data based on zones, devices, partitions, and replicas

lWeights can be used to balance the distribution of partitions
lUsed by the Proxy Server for many background processes
l

Proxy Server: Request routing, exposes the public API

Replication: Keep the system consistent, handle failures

lUpdaters: Process failed or queued updates
lAuditors: Verify integrity of objects, containers, and accounts
l

System Components (Cont.)

Account Server: Handles listing of containers, stores as SQLite DB

lContainer Server: Handles listing of objects, stores as SQLite DB
lObject Server: Blob storage server, metadata kept in xattrs, data in
binary format
lRecommended to run on XFS
lObject location based on hash of name & timestamp
l

Evolution of Object Storage Architecture

Version 1: Central DB
(Rackspace Cloud Files 2008)

Version 2: Fully Distributed

(OpenStack Object Storage 2010)

Example Small Scale Deployment

OpenStack
Architecture

OpenStack Arhitecture

Send/receive
packets with
outside world

interface with
users and
make
management
decisions

run VM and
store files

Hardware Recommendations

Recommended hardware configurations for a minimum

production deployment for the cloud controller nodes
Server

Recommended Hardware

Notes

Cloud Controller node (runs

network, volume, API, scheduler
and image services)

Processor: 64-bit x86

32-bit processors will work for the

cloud controller node.
A quad core server with 12 GB
RAM would be more than
sufficient for a cloud controller
node.

Memory: 12 GB RAM
Disk space: 30 GB (SATA or SAS or
SSD)
Volume storage: two disks with 2
TB (SATA) for volumes attached to
the compute nodes
Network: one 1 GB Network
Interface Card (NIC)

Two NICS are recommended but

not required.

http://docs.openstack.org/

Hardware Recommendations

Recommended hardware configurations for a minimum

production deployment for the compute nodes
Server

Recommended Hardware

Notes

Compute nodes (runs virtual

instances)

Processor: 64-bit x86

Note that you cannot run 64-bit

VM instances on a 32-bit compute
node. A 64-bit compute node can
run either 32- or 64-bit VMs,
however.

Memory: 32 GB RAM
Disk space: 30 GB (SATA)
Network: two 1 GB NICs

With 2 GB RAM you can run one

m1.small instance on a node or
three m1.tiny instances without
memory swapping, so 2 GB RAM
would be a minimum for a testenvironment compute node.

http://docs.openstack.org/

ScienceCloud System Architecture

Data Network
Cloud Controller:

External
Network

nova-compute
nova-network
nova-scheduler
nova-api
Keystone
Dashboard
glance
Neutron-agents
Neutron-plugin

Compute Node:
nova-compute
Neutron-plugin

Mangmt Network

Compute Node:
Compute
Node:
nova-compute
Compute
Node:
nova-compute
Neutron-plugin
nova-compute
Neutron-plugin
Neutron-plugin

OpenStack Releases

Series

Status

Releases

Date

Liberty

Under discussion

Due

Q4 2015

Kilo

Under development

Due

Apr 30, 2015

2014.2.3

Apr 13, 2015

2014.2.2

Feb 5, 2015

2014.2.1

Dec 5, 2014

2014.2

Oct 16, 2014

2014.1.4

Mar 12, 2015

2014.1.3

Oct 2, 2014

2014.1.2

Aug 8, 2014

2014.1.1

Jun 9, 2014

2014.1

Apr 17, 2014

2013.2.4

Sep 22, 2014

2013.2.3

Apr 03, 2014

Juno

Icehouse
Havana

Current stable
release, securitysupported

Security-supported
EOL

Havana

Grizzly

Folsom
Essex

EOL

EOL

EOL
EOL

2013.2.4

Sep 22, 2014

2013.2.3

Apr 03, 2014

2013.2.2

Feb 13, 2014

2013.2.1

Dec 16, 2013

2013.2

Oct 17, 2013

2013.1.5

Mar 20, 2014

2013.1.4

Oct 17, 2013

2013.1.3

Aug 8, 2013

2013.1.2

Jun 6, 2013

2013.1.1

May 9, 2013

2013.1

Apr 4, 2013

2012.2.4

Apr 11, 2013

2012.2.3

Jan 31, 2013

2012.2.2

Dec 13, 2012

2012.2.1

Nov 29, 2012

2012.2

Sep 27, 2012

2012.1.3

Oct 12, 2012

2012.1.2

Aug 10, 2012

2012.1.1

Jun 22, 2012

2012.1

Apr 5, 2012

2011.3.1

Jan 19, 2012

2011.3

Sep 22, 2011

Diablo

EOL

Cactus

Deprecated

2011.2

Apr 15, 2011

Bexar

Deprecated

2011.1

Feb 3, 2011

Austin

Deprecated

2010.1

Oct 21, 2010

User/Tenant/Role
User is a login name to OpenStack
Tenant or Project represents user capability
and resource quota
Role represent users authority over the
system

OpenStack Network Model

Fixed IP is an internal IP address
inOpenStack
Floating IP is Public IP address for external
accesses

Use Case: Provider Router with Private

Networks

Use Case: Per-tenant Routers with

Private Networks

OpenStack - Juno

October 2, 2014

OpenStack Cloud Platform

Cloud Management Platform :

NCTU OpenStack Introduction

With its 10th release, OpenStack supports the widest

set of technologies, enabling new use cases across
finance,
manufacturing,
technology
and
many
industries.

10th Release of OpenStack Key Themes

Enterprise
Maturity

Networking Advances

New Data Processing &

Container Support

Most widely-supported cloud platform, expanded

testing for plugins
Storage policies
Federated identity enhancements
Operational improvements
OpenStack infrastructure natural home for
implementing NFV
NFV workgroup established, new features landing
in Nova
Neutron parity with Nova Network
New Data Processing capability part of integrated
release. Supports Hadoop and Spark
Improved Docker support via Compute driver in
StackForge

New Features by Category

Compute
Network Functions Virtualization

Subteam formed in Atlanta

Multiple use cases split out (9)

Operational Updates

Improvements for rescue mode: boot from alternate image and attach all
local disks
Improve nova-network code to allow per-network settings

Other updates

Ironic driver added

Rolling upgrade improvements
Scheduling updates to support scheduling services and extensibility

Internationalization updates

Storage
Object Storage

Storage policies: major update, allows flexibility to use different types

of storage devices, replication settings
Ongoing work on erasure coding, potentially coming in Kilo
New features:

Keystone v3 support
Account to account copy

Block Storage

Cinder v2 API in Nova

Several new storage technologies support and improved testing of
third-party storage systems
Project maturing, consistent contributors building out core functionality

Networking
nova-network to Neutron Migration Path

Initial path for eventual deprecation

Back-end plug-in enabled

NFV Work

Support for IPv6 networking

Third-party driver testing ensures consistency and reliability across network
implementations
Focus on Compute during Juno release cycle, but updates for Networking
coming too

L3 High Availability

Networking layer now allows a distributed operational mode

Shared Services

Identity service: Federated authentication improvements

Orchestration: Rollback on failed deployment, delegation
improvements for non-admin users
Telemetry: Efficiency improvements; increase in performance
Dashboard: Data Processing integration, RBAC support for Block
Storage and Images
Data Processing: Faster ability to scale based on custom parameters,
support for additional drivers (e.g., Spark), Neutron support, Big data
contributors
Image Service: Image service expanding to broader artifact catalog
service

Kilo and Beyond

Kilo is expected to be released April 30, 2015. New
capabilities integrated in the Kilo release:
Bare Metal (Ironic); note that the Compute driver is
available in the Juno release
Additional projects being incubated, expected to land in late
2015 and beyond:
Manila (shared file system)
Zaqar (queue service)
Designate (DNS service)
Barbican (key management)

Key Dates & Housekeeping

Publication soft launched in May; steadily growing

readership, contributors and content
Now that we have solid base of content and
feedback, starting to promote and drive more traffic
to the site
First Superuser Awards to be given at the Paris
Summit
superuser.openstack.org

Car Cloud Turns Big Data Into Smart Insights

Top 10 Auto Manufacturer

http://openstack.org/enterprise

106

Good luck!

k7cloud@gmail.co
m