1

GRID COMPUTING
2
Hour 1: Introduction
What is Grid Computing?
Who Needs It?
An Illustrative Example
Grid Users
Current Grids
3
What is Grid Computing?
Computational Grids
Homogeneous (e.g., Clusters)
Heterogeneous (e.g., with one-of-a-kind
instruments)
Cousins of Grid Computing
Methods of Grid Computing
4
Computational Grids
A network of geographically distributed
resources including computers, peripherals,
switches, instruments, and data.
Each user should have a single login account
to access all resources.
Resources may be owned by diverse
organizations.
5
Computational Grids
Grids are typically managed by gridware.
Gridware can be viewed as a special type of
middleware that enable sharing and manage
grid components based on user requirements
and resource attributes (e.g., capacity,
performance, availability)
6
Cousins of Grid Computing
Parallel Computing
Distributed Computing
Peer-to-Peer Computing
Many others: Cluster Computing, Network
Computing, Client/Server Computing, Internet
Computing, etc...
7
Distributed Computing
People often ask: Is Grid Computing a fancy
new name for the concept of distributed
computing?
In general, the answer is no. Distributed
Computing is most often concerned with
distributing the load of a program across two
or more processes.
8
PEER2PEER Computing
Sharing of computer resources and services by
direct exchange between systems.
Computers can act as clients or servers
depending on what role is most efficient for
the network.
9
Methods of Grid Computing
Distributed Supercomputing
High-Throughput Computing
On-Demand Computing
Data-Intensive Computing
Collaborative Computing
Logistical Networking
10
Distributed Supercomputing
Combining multiple high-capacity resources
on a computational grid into a single, virtual
distributed supercomputer.
Tackle problems that cannot be solved on a
single system.
11
High-Throughput Computing
Uses the grid to schedule large numbers of
loosely coupled or independent tasks, with
the goal of putting unused processor cycles to
work.
12
On-Demand Computing
Uses grid capabilities to meet short-term
requirements for resources that are not locally
accessible.
Models real-time computing demands.
13
Data-Intensive Computing
The focus is on synthesizing new information
from data that is maintained in geographically
distributed repositories, digital libraries, and
databases.
Particularly useful for distributed data mining.
14
Collaborative Computing
Concerned primarily with enabling and
enhancing human-to-human interactions.
Applications are often structured in terms of a
virtual shared space.
15
Logistical Networking
Global scheduling and optimization of data
movement.
Contrasts with traditional networking, which
does not explicitly model storage resources in
the network.
Called "logistical" because of the analogy it
bears with the systems of warehouses,
depots, and distribution channels.
16
Who Needs Grid Computing?
A chemist may utilize hundreds of processors
to screen thousands of compounds per hour.
Teams of engineers worldwide pool resources
to analyze terabytes of structural data.
Meteorologists seek to visualize and analyze
petabytes of climate data with enormous
computational demands.
17
An Illustrative Example
Person X, a IICT/CCMB research scientist,
collected microbiological samples in the place of
East Godavari Dist, AP.
She needed the high-performance microscope
located at the for Research Centre at JNU, Delhi
She used Garuda Grid to view and control the
output of the microscope from her desk on
Wallops Island. Thus, in addition to viewing the
samples, she could move the platform holding
them and make adjustments to the microscope.
18
Example (continued)
She used Garuda Grid to view and control the
output of the microscope from her desk on
Wallops Island. Thus, in addition to viewing
the samples, she could move the platform
holding them and make adjustments to the
microscope.
19
Grid Users
Grid developers
Tool developers
Application developers
End Users
System Administrators
20
Grid Developers
Very small group.
Implementers of a grid protocol who
provides the basic services required to
construct a grid.
21
Tool Developers
Implement the programming models used by
application developers.
Implement basic services similar to
conventional computing services:
User authentication/authorization
Process management
Data access and communication
22
Tool Developers
Also implement new (grid) services such as:
Resource locations
Fault detection
Security
Electronic payment
23
Application Developers
Construct grid-enabled applications for end-
users who should be able to use these
applications without concern for the
underlying grid.
Provide programming models that are
appropriate for grid environments and
services that programmers can rely on when
developing (higher-level) applications.
24
System Administrators
Balance local and global concerns.
Manage grid components and infrastructure.
Some tasks still not well delineated due to the
high degree of sharing required.
25
Some Highly-Visible Grids
The NSF PACI/NCSA Alliance Grid.
The NSF PACI/SDSC NPACI Grid.
The NASA Information Power Grid (IPG).
The Distributed Terascale Facility (DTF)
Project.
Garuda Grid (India)
26
Using the Grid
Globus
Condor
Harness
Legion
IBP
NetSolve
Others
27
Globus
A collaboration of Argonne National
Laboratorys Mathematics and Computer
Science Division, the University of Southern
Californias Information Sciences Institute, and
the University of Chicago's Distributed
Systems Laboratory.
Started in 1996 and is gaining popularity year
after year.
28
Globus
A project to develop the underlying
technologies needed for the
construction of computational grids.
Focuses on execution environments
for integrating widely-distributed
computational platforms, data
resources, displays, special
instruments and so forth.
29
The Globus Toolkit
The Globus Resource Allocation Manager
(GRAM)
Creates, monitors, and manages services.
Maps requests to local schedulers and computers.
The Grid Security Infrastructure (GSI)
Provides authentication services.
30
The Globus Toolkit
The Monitoring and Discovery Service (MDS)
Provides information about system status,
including server configurations, network status,
and locations of replicated datasets, etc.
Nexus and globus_io
provides communication services for
heterogeneous environments.
31
The Globus Toolkit
Global Access to Secondary Storage (GASS)
Provides data movement and access mechanisms
that enable remote programs to manipulate local
data.
Heartbeat Monitor (HBM)
Used by both system administrators and ordinary
users to detect failure of system components or
processes.
33
Condor
The Condor project started in 1988 at the
University of Wisconsin-Madison.
The main goal is to develop tools to support
High Throughput Computing on large
collections of distributively owned computing
resources.
34
Condor
Runs on a cluster of workstations to glean
wasted CPU cycles.
A Condor pool consists of any number of
machines, of possibly different architectures
and operating systems, that are connected by
a network.
Condor pools can share resources by a feature
of Condor called flocking.
35
The Condor Pool Software
Job management services:
Supports requests about the job queue .
Puts a job on hold.
Enables the submission of new jobs.
Provides information about jobs that are already
finished.
A machine with job management installed is
called a submit machine.
36
The Condor Pool Software
Resource management:
Keeps track of available machines.
Performs resource allocation and scheduling.
Machines with resource management
installed are called execute machines.
A machine could be a submit and an
execute machine simultaneously.
37
Condor-G
A version of Condor that uses Globus to
submit jobs to remote resources.
Allows users to monitor jobs submitted
through the Globus toolkit.
Can be installed on a single machine. Thus no
need to have a Condor pool installed.
38
Legion
An object-based metasystems software
project designed at the University of Virginia
to support millions of hosts and trillions of
objects linked together with high-speed links.
Allows groups of users to construct shared
virtual work spaces, to collaborate research
and exchange information.
39
Legion
An open system designed to
encourage third party development
of new or updated applications, run-
time library implementations, and
core components.
The key feature of Legion is its
object-oriented approach.
40
Harness
A Heterogeneous Adaptable Reconfigurable
Networked System
A collaboration between Oak Ridge National
Lab, the University of Tennessee, and Emory
University.
Conceived as a natural successor of the PVM
project.
41
Harness
An experimental system based on a highly
customizable, distributed virtual machine
(DVM) that can run on anything from a
Supercomputer to a PDA.
Built on three key areas of research: Parallel
Plug-in Interface, Distributed Peer-to-Peer
Control, and Multiple DVM Collaboration.
42
IBP
The Internet Backplane Protocol (IBP) is a
middleware for managing and using remote
storage.
It was devised at the University of Tennessee
to support Logistical Networking in large scale,
distributed systems and applications.
43
IBP
Named because it was designed to enable
applications to treat the Internet as if it were a
processor backplane.
On a processor backplane, the user has access
to memory and peripherals, and can direct
communication between them with DMA.
44
IBP
IBP gives the user access to remote
storage and standard Internet
resources (e.g. content servers
implemented with standard sockets)
and can direct communication
between them with the IBP API.
45
IBP
By providing a uniform, application-
independent interface to storage in the
network, IBP makes it possible for applications
of all kinds to use logistical networking to
exploit data locality and more effectively
manage buffer resources.
46
NetSolve
A client-server-agent model.
Designed for solving complex scientific
problems in a loosely-coupled heterogeneous
environment.
47
The NetSolve Agent
A resource broker that represents
the gateway to the NetSolve system
Maintains an index of the available
computational resources and their
characteristics, in addition to usage
statistics.
48
The NetSolve Agent
Accepts requests for computational
services from the client API and
dispatches them to the best-suited
sever.
Runs on Linux and UNIX.
49
The NetSolve Client
Provides access to remote resources through
simple and intuitive APIs.
Runs on a users local system.
Contacts the NetSolve system through the
agent, which in turn returns the server that
can best service the request.
Runs on Linux, UNIX, and Windows.
50
The NetSolve Server
The computational backbone of the system.
A daemon process that awaits client requests.
Runs on different platforms: a single
workstation, cluster of workstations,
symmetric multiprocessors (SMPs), or
massively parallel processors (MPPs).
51
The NetSolve Server
A key component of the server is the Problem
Description File (PDF).
With the PDF, routines local to a given server
are made available to clients throughout the
NetSolve system.
Policy based Management
Loose in Internet
Managing Centrally might
be difficult
Resource Management is
difficult
Distributed policy
repository
Strong In Grid
Best for larger Networks for
Centrally Management
Better Resource
Management
Dedicated policy
repository
Policy-based management provides a way to
allocate network resources, primarily network
bandwidth, QoS, and security (firewalls),
according to defined business policies
Configuration Management
In Internet, the
Configuration Management
is distributed
In Grid, Central and
Distributed Management
Configuration Management is a set of utilities that manage
network configurations, diagnose and monitor network etc
Application(s)
Applications in Internet are well known, where
as in Grid environment still few and closed
environment
Known vulnerabilities are high in Internet
where as low in Grid environment
Other security issues too
Data Flow
Replication
The data replication may be high in Internet
QoS
Must in Internet, and also in Grid
Volume
Size may be larger in Grid
Periodicity
Perimeter Security
Public and Private Networks in Internet, where
as closed environment for Grid
Perimeter Security is required for all private
Networks where as Grid is private, but needed
security to avoid malware attacks
Middleware Services
PKI
Job Scheduling
Grid monitoring and management
Security Issues
DoS/DDoS
Buffer Overflows
Information Gathering
Gaining privileges
Malware
Data Theft
Any such attacks on Grid
environment?
Attacks against new Grid components..
High compute resources..(DoS..)
Junk program/data consuming resources..
A grid is an automated error amplifier
Grid applications behavior
Is it possible to profile normalcy of traffic
Like bimodal distribution nature
Target schedulers
PKI framework
Formal MoUs between agencies
management level understanding
Hence enforcing technical
configurations..possibly easy..
Common policy and standardization
Automatic threat assessment
Adaptive and Dynamic Firewall
New techniques for identifying intrusions
Dynamic configuration facilities
User, Resource, Flow, Scheduler, Application
profilers..
Enhanced Audit capabilities
Vigilance, monitoring and patrolling agents
Good progress in middleware framework for
authentication and authorization
Evolving Standardization initiatives such as Grid Trust
Federation that establishes the Policy management
authorities
Security Operations : to be streamlined and
automated..
Control points for Policy framing, policy enforcement,
monitoring and forensics : to be strengthened further
Research Challenges
Grid Traffic flow analysis
Policy based Security framework (Policy specification language
ASL, ISPS, KAoS, LaSCO, PCIM, PDL, PMAC, PPL..)
Dynamic firewall
Secure Programming and application security
Grid specific security solutions
Virtual community specific security solutions
Threat & Attack Modeling, Attack prediction and mitigation
Self-healing and immune architectures
Cyber (Grid) Forensics
Simulations of security (OptorSim, ChicagoSim, Simgrid, Gridnet)