Grid Computing Middleware

Xon Xay Nguyen Tuan Viet Tram The Phien

Presenter:

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite
2

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite
3

Grid environment

Introduction

Three main issues to confont in Grid environment:

Heterogeneity multiplicity of resources Scalability Adaptability - Resource managers or applications dynamic to extract the maximum performance from the available resources and services.
5

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite
6

Grid Middleware

System software between applications and operating system

Provide services to application

Discovery, storage, execution, information, service integration, resource monitoring, failure detection and recovery,

Hide heterogeneous of the Grid environment Provide standardised interfaces to services.

7

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite
8

Purposes of Middleware

Open, general-purpose and standard Standard protocols

Defines the contents and sequence of message exchanges used to request remote operation Important and essential to achieve the interoperability that Grid depends on Interfaces to code libraries Facilitate construction of Grid components by allowing code components to be reused
9

Standard APIs

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite
10

Grid layered architecture

- The neck: a small set of core abstractions and protocols - Top of hourglass: many different highlevel behaviors can be mapped - Base of hourglass: can be mapped onto many different underlying technologies
11

Component of Middleware

Connectivity layer Resource layer Collective layer

12

Connectivity layer

Communicating easily and securely Communication protocol: exchange data Authentication protocol: cryptographically secure mechanism for verifying the identity of users and resources.

Single sign-on Delegation Integration with various local security solutions User-based trust relationships
13

Resource layer

Sharing single resources Define protocols (and APIs and SDKs) for the secure negotiation, initiation, monitoring, control, accounting, and payment of sharing operations on individual resources Information protocols - obtain information about the structure and state of a resource Management protocols - negotiate access to a shared resource
14

Collective layer

Coordinating multiple resources Implement a wide variety of sharing behaviors without placing new requirements on the resources being shared Example Directory services, co-allocation,
scheduling, brokering services, Monitoring and diagnostics services, Data replication services, Grid-enabled programming systems,

15

Collective layer- Example

Application use coreservation service protocol to request endto-end network reservation

We define a coreservation service protocol Implement coreservation service Call co-allocation API to implement co-allocation operation
16

Outline

Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite
17

Globus Toolkit

Evolution Globus Toolkit version 2 Globus Toolkit version 3 Globus Toolkit version 4

18

Globus evolution

GT2.0 1997 Proof of concept

GT3.0 OGSA 2003

GT4.x WSRF 2005 Standardization

19

Globus Toolkit version 2

Community-based, open-architecture, open source set of services and software libraries that support Grids and Grid applications. De facto standard for Grid computing Focus on usability and interoperability Middleware

GT2 Connectivity layer GT2 - Resource layer GT2 - Collective layer

20

GT2 Connectivity Authentication protocol

Public-key based GSI protocols

GSI - Grid Security Infrastructure authentication, communication protection, authorization TLS - Transport Layer Security TLS - defines a protocol to provide privacy and data integrity between two communicating applications.
21

GSI builds on and extends the TLS protocols

GT2 Connectivity layer Communication protocol

Internet layered protocol architecture

Internet layer - IP and ICMP Transport - TCP, UDP Application layer - DNS, OSPF, RSVP,

22

GT2 - Resource layer

A Grid Resource Information Protocol base on Lightweight Directory Access Protocol (LDAP) HTTP-based GRAM protocol

GridFTP - a management protocol for data access

GRAM - Grid Resource Access and Management allocation of computational resources monitoring and control of computation on resources

An extended version of the File Transfer Protocol

23

GT2 - Collective layer

build on Globus Connectivity and Resource protocols Meta Directory Service

introduces Grid Information Index Servers (GIISs) LDAP information protocol

Access resource-specific GRISs to obtain resource state GRRP used for resource registration.

DUROC resource co-allocation library provides an SDK and API for resource co-allocation
24

Globus Toolkit version 3

Build on and significantly extend GT2 concepts and technologies Base on Open Grid Services Architecture

Factoring of component behaviors identify essential Grid functions and express Service orientation enable a uniform treatment of all network entities Align with Web services WSDL (Web Services Definition Language)
25

Globus Toolkit version 4

developed by The Globus Alliance OGSA requires stateful services WSRF Web Service Resources Framework

developed by OASIS (http://www.oasis-open.org) WSRF provides the stateful services that OGSA needs
26

Relationship between OGSA, GT4, WSRF, and Web Services

27

gLite Toolkit

The Enabling Grids for E-sciencE project (EGEE) Middleware stack that combines components developed in various related projects.

28

gLite middleware

A layer between services and resources gLite follows a Service Oriented Architecture

29

gLite - guiding principles

Service oriented approach

Lightweight (existing) services

Allow for multiple interoperable implementations

Easily and quickly deployable Use existing services where possible
Condor, EDG, Globus, LCG,

Portable

Security

Being built on Scientific Linux and Windows Sites and Applications

30

gLite - guiding principles

Performance/Scalability & Resilience/Fault Tolerance

Comparable to deployed infrastructure

Co-existence with deployed infrastructure

Co-existence with LCG-2 and OSG (US) are essential for the EGEE Grid services
Reduce dependence on global, central services

Site autonomy

Open source license

31

gLite Architecture

32

gLite - Access

All of the gLite services are accessible via APIs and CLIs.

API - Application Programming Interface CLI - Command Line Interface

33

gLite - Security Services

34

gLite - Security Services

Authentication

Authorization

identify entities PKI (X.509) infrastructure with CA third party for signature
allows or denies access to specific services three basic authorization models, classified as agent, push and pull. tackle the problems that arise when connectivity is restricted by the resource owner. monitoring and providing for post-mortem analysis of security related events.
35

Dynamic Connectivity Service

Auditing

gLite - Infomation and Monitoring Services

36

gLite - Infomation and Monitoring Services

Infomation and Monitoring Services a vital low-level component of any grid

37

gLite - Infomation and Monitoring Services

Job Monitoring

use the APIs offered by the apache logging services (log4j)

Service Discovery

a facility offered to both end users and to other services to locate suitable services.
define a standard interface to monitoring frameworks. NPM Publisher for use by gLite components.
38

Network Performance Monitoring

gLite - Helper Services

39

gLite - Helper Services

Configuration and Instrumentation Service

common, standard-based configuration and instrumentation functionality to the gLite services. control and balance the usage of the network and categorise and prioritise traffic flows.

Bandwidth Allocation and Reservation Service

Agreement Service

implements the communication protocol used to exchange information about Service Level Agreements (SLAs) and defines the SLA structure.
40

gLite - Data Services

41

gLite - Data Services

Catalog

store information about the data and metadata is the Grid service responsible for saving/retrieving files to/from some data store which can provide a wide range of quality of service to the user.

Storage Element

Data movement

provide scalable and robust managed data transfer between Grid sites
42

gLite - Job Management Services

43

gLite - Job Management Services

Accounting

accumulates information about the usage of Grid resources

representing a computing resource job management and other capabilities comprises a set of Grid middleware components responsible for the distribution and management of tasks across Grid resources

Computing element

Workload Management

44

gLite - Job Management Services

Job Provenance

keep track of the definition of submitted jobs, execution conditions and environment.

Package Manager

is a helper service that automates the process of installing, upgrading configuring, and removing software packages from a shared area

45

Comparison

Globus Toolkit - faster and more efficient gLite - sophisticated and developer friendly Document

Globus documents - sketchy and missing gLite documents - sufficient, available

46

Summary

Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology

Globus GLite

47

Reference

I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. Morgab Kaufmann Publishers, 1999. Fran Berman, Anthony J. G. Hey and Geoffrey C. Fox, Grid computing: Making the Global Infrastructure a Reality. John Wiley & Sons Ltd, 2003. Globus project: http://www.globus.org/alliance/ Grid Computing Slides Nguyen Tuan Anh Grid Computing Slides Pham Tran Vu vonLaszewski--grid-middleware 03-grid-concepts-architecture-middleware-adler-vortrag Websites of gLite (http://www.eu-egee.org)

48

Q&A

49