Parallel Processing

sp2016
lec#8
Dr M Shamim Baig

1.1

Interconnection Networks (INs)

for
Parallel Computers

1.2

Static INs:
Evaluation Parameters
Degree : Max of links connected
at any node of the network
Diameter: Distance (shortest path)
between the farthest nodes in the
network.
Link Cost: Total Number of links
required to implement the network.

1.3

Static INs: Topologies

Completely Connected
Star
Tree
Linear Array & Ring
2-D/ 3-D Mesh & Torus
Hypercube
k-d Mesh
1.4

Static INs topologies: Completely Connected

Each processor is connected to
every other processor,(number
of processors=p)
Diameter=1, number of links in
the network are p (p-1) / 2
While performance scales very
well, hardware complexity is not
realizable for large values of p.
In this sense, these networks
are static counterparts of
crossbars (dynamic IN)

(a) A completely-connected network of eight nodes;

(b) a star connected network of nine nodes.

1.5

Static INs topologies: Star

Every node is connected
only to a common node at the
center.
Distance between any pair
of nodes is 2. However, the
central node becomes a
bottleneck.
In this sense, star networks
are static counterparts of
buses.
(a) A completely-connected network of eight nodes;
(b) a star connected network of nine nodes.

1.6

Static INs topologies: Tree

Tree is a graph without any loops.
Good for hierarchical comm model
Tree involves root, leaves, parent,
child, levels, height
In Complete binary tree each node
has 3-neighbors (degree=3),
1-parent & 2-children (except root has
no parent & leaves have no child)
A p-node complete binary tree has
2i nodes at ith level & log(p+1) levels;
(root level is 0)

Tree has link cost = p-1

Tree has Diameter= 2 log(p+1) - 1
Trees can be laid out in 2D with no
wire crossings, ease of manufacture.
(a) Complete binary tree IN

(b) Fat-tree IN
1.7

Static INs topologies:

Fat-tree
Links higher up the
tree potentially
carry more traffic
than those at the
lower levels.
For this reason, a
variant called a fattree, fattens the
links as we go up
the tree.
Popular topology
(a)
binary tree IN
forComplete
Data Centers

(b) Fat-tree IN
1.8

Static INs topologies:

Linear Array / Ring

Linear arrays: (a) with no wraparound links;

(b) with wraparound link (ring)

1.9

Static INs topologies:

Linear Array & Ring
In a linear array, each node has two neighbors,
one to its left & one to its right(except end-nodes
which have one neighbor)
In an array, even if a single node/link fails array
partitions in two parts.
If array nodes at either end are connected
together, it becomes a ring (in which all nodes
are identically connected to two neighbors)
In a ring, if a single node/link fails, remaining
nodes are still connected as an array.
In an array each pair of nodes has only one path
while in ring they have two alternate paths
Ring is symmetrical & has better redundancy
1.10

Static INs topologies:

Linear Array & Ring
A p-node Linear array has: Degree= 2
Diameter= p-1
Link cost= p-1
A p-node Ring has: Degree= 2
Diameter= floor[p/2]
Link cost= p
1.11

Static INs topologies:

2-D Mesh/ Torus
In 2-dimension mesh
each node has 4neighbors each to the
north, south, east, &
west (except the edge
nodes)
A 2-D mesh with
wrapped around links
at corresponding edge
nodes in each
dimension is a Torus.
In torus, all nodes are
identically connected
to 4-neighbors
2-Dimensional Mesh:
(a) 2-D mesh with no wraparound (b) 2-D mesh with wraparound link (torus)
1.12

Static INs topologies:

2-D Mesh/Torus
A p-node 2-D square Mesh(
has:
Degree= 4
Diameter= 2( -1)
Link cost= (2p 2 )= 2(p -

A p-node 2-D square Torus(

has:
Degree= 4
Diameter= 2 x floor[
Link cost= 2p

/2]

How about non-square (m x n) Mesh & torus???

1.13

Static INs topologies: Hypercubes

Construction of hypercubes from hypercubes of lower dimension.

1.14

Static INs topologies:

Hypercube
A d-dimension hypercube has p=2d nodes
p-node hypercube has logp= d dimensions
Each node has logp neighbors.
The distance between any two nodes is
given by number of bit positions in which
two nodes binary labels differ.
The distance between the two farthest
nodes is log p= diameter.
Link cost of hypercube= (p logp) / 2
1.15

Static INs Evaluation:

2-D, 3-D, ..k-D Mesh

2-Dimensional Mesh:
(a) 2-D mesh with no wraparound (b) 2-D mesh with wraparound link
(c) 3-D mesh no wrap around

1.16

Static INs Evaluation: k-d Mesh

Mesh/Torus can be generalized to more dimensions
1dim, 2dim, 3dim ..d-dim. If # of nodes in each dim
is k, called k-d mesh (non-wrapped/ wrapped)
A k-node array can be considered as 1D mesh, a k
x k node square mesh is 2D mesh,
akxk
x k node mesh is 3D mesh,
akxkx
.. (d-times) node mesh is k-d mesh.
A (symmetric) k-d mesh has k-nodes in each of ddimensions
p= kd nodes
A d-dimension Hypercube is a special case of (symmetric) k-d
mesh having 2-nodes (k=2) in each dim, giving
total number of nodes= 2 x 2 x (d-times)
p= 2d nodes
1.17

k-d Mesh: Evaluation parameters

A (symmetric) k-d mesh has k-nodes in each of ddimensions
p= kd nodes
A (symmetric) k-d mesh (with wrap around) has:Degree= 2d
Diameter= d x floor[k/2]
Link cost= ((2d) x p) / 2 = d x p
How about evaluation parameters of (symmetric) k-d mesh
(without wrap around)???
A (non-symmetric) k-d mesh has ki nodes in di dimension
and therefore will have different formulae to compute the
evaluation parameters???
1.18

Summary of Evaluation Parameters:

Static Interconnection Networks
Network

Completely-connected
Star

Diameter

Bisection
Width
Degree

Link-cost
(no. of links)

p-1
p-1

Complete binary tree

Linear array / Ring
2-D mesh, no wraparound
2-D mesh wraparound
Hypercube
k-d mesh wraparound

3
2
4
4
logp
2d

1.19