Data-Parallel Architectures and Algorithms

Group Members:

S.Ameed Uddin S.Shujah Haider S.Tauqeer Abbass Usman Khalil

Parallel Computing
Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). There are several different forms of parallel computing:
DATA
BIT INSTRUCTION TASK

Data Parallelism
Data parallelism (also known as loop-level parallelism) is a form of parallelization of computing across multiple processors in parallel computing environments. Data parallelism focuses on distributing the data across different parallel computing nodes. Data parallelism is achieved when each processor performs the same task on different pieces of distributed data.

An Example
For instance, consider a 2-processor system (CPUs A and B) in a parallel environment, and we wish to do a BINARY SEARCH on some data d. Now, A B

Contd.
Here CPU A traverses the Head while CPU B traverses the Tail. Data parallelism emphasizes the distributed (parallelized) nature of the data. Data Parallelism is opposite to task parallelism.

Clarification!
One can confuse data parallelism with the multithreaded programming in C# and JAVA, but the two are different. In multithreading programming we alter the durations of threads on processor as needed but in data parallelism we are distributing the data to our resources.

Types of Data Parallel Arch.

Data Parallel Architecture

Vector
Architecture

Associative and neural architecture

SIMDs

Systolic Architecture

SIMD(Single Instruction Multiple Data)

Single instruction, multiple data (SIMD), is a class of parallel computers. It describes computers with multiple processing elements that perform the same operation on multiple data simultaneously. Thus, such machines exploit data level parallelism.

Contd.

Vector Processor
A vector processor, or array processor, is a central processing unit (CPU) that implements an instruction set containing instructions that operate on one-dimensional arrays of data called vectors. This is in contrast to a scalar processor, whose instructions operate on single data items. The vast majority of CPUs are scalar

Systolic Architecture
In computer architecture, a systolic array is a pipe network arrangement of processing units called cells. Here cells (i.e. processors), compute data and store it independently of each other. A good example would be matrix multiplication

Contd.

Advantages and Disadvantages

Pros Faster Scalable Cons Expensive Highly specialized for particular applications Difficult to build

Neural Architecture
An artificial neural network (ANN), usually called neural network (NN), is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks.

Applications
Parallel discrete event simulation

Contd.
Compute results at c,d,e before output from a is known. Therefore concurrent processing results in quicker completion of task.

Matrix Addition
As matrices are stored by the help of multi dimensional arrays, different processors add the smaller chunks of huge matrix. Sparse matrix( most values are 0) are also resolved by data parallel approach.

GPU(Graphics Processing Unit)

GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display. GPUs are used in embedded systems, mobile phones, personal computers, workstations and game consoles. Modern GPUs are very efficient at manipulating computer graphics, and their highly parallel structure makes them more effective than generalpurpose CPUs.

Contd.

Contd.
All modern Video Accelerating cards, Stream processors ,and graphic cards to enhance gaming experience use parallel architecture to process visual data.

Algorithms Implementation
Identify concurrency in program Map concurrent pieces to parallel processes Distribute input, output and intermediate data Manage accesses to shared data by processors Synchronize processors as program executes.

An Example
Model =civic and year=2001 and (color=green or color=white)

Contd.

Contd.

Granularity

Contd.

THANK YOU!