ASSIGNMENT

Parallel and Distributed Computing

Submitted To: Sir Shoaib
Date: 25-03-2019

Umer Shahzad
BCS 8th (A)
Roll No: 7432
Q1. What is Parallel Computing?
Parallel Computing is the simultaneous use of multiple compute resources to solve a
computational problem.
 Parallel computing helps in performing large computations by dividing the workload
between more than one processor, all of which work through the computation at the
same time.

Figure 1. Parallel Computing Architecture

Multiprocessing:
In Multiprocessing systems, two or more processors shared the work to be done.

Master/Slave Configuration:
The earliest versions had a master/slave configuration. One processor (the master) was
programmed to be responsible for all of the work in the system; the other (the slave)
performed only those tasks it was assigned by the master. This arrangement was necessary
because it was not then understood how to program the machines so they could cooperate
in managing the resources of the system.
 Symmetric Multiprocessing System (SMP)
 Massively Parallel Processing Systems (MPP)
 Non-Uniform Memory Addressing
 SMP machines

Explanation: (Purpose & Mechanism)

Parallel Computing is to increase the available computation power for faster application
processing or task resolution. Processors in parallel computing can have access to the shared
memory. To deal with the problem of power consumption and overheating the major central
processing unit (CPU or processor) manufacturers started to produce power-efficient
processors with multiple cores. The core is the computing unit of the processor and in multi-
core processors, each core is independent and can access the same memory concurrently. It
is generally implemented in operational environments/scenarios that require massive
computation or processing power.
Advantages of Parallel Computing:

 Parallel computing saves time, allowing the execution of applications in a shorter and
solve larger problems in a short point of time.

 You can do many things simultaneously by using multiple computing resources and is
much better suited for modeling, simulating and understanding complex, real-world
phenomena.

 It has massive data storage and quick data computations and can help keep you
organized. If you have Internet, then communication and social networking can be
made easier.

 It can take advantage of non-local resources when the local resources are finite.
Speed up and better cost per performance in the long run.

Disadvantages of Parallel Computing:

 The extra cost (i.e. increased execution time) incurred are due to data transfers,
synchronization, communication, thread creation/destruction, etc.

 The use of parallel computing can solve problems computationally and data-intensive
problems using multicore processors, but, sometimes this effect on some of our
control algorithm and does not give good results.

 They're harder to debug or prove correct, and they often perform worse than their
serial counterparts due to communication and coordination overhead.

Daily Life Examples of Parallel Computing:

 Web search engines/databases processing millions of transactions every second.

 Collaborative Networks provide a global venue where people from around the
world can meet and conduct work "virtually".

 Modeling, Simulating.

Q2. What is Distributed/Grid or Cluster Computing?

Distributed computing also refers to the use of distributed systems to solve computational
problems. In distributed computing, a problem is divided into many tasks, each of which is
solved by one or more computers, which communicate with each other via message passing.
 Grid, Distributed or Cluster Computing works on various tasks within a network, but
it is also capable of working on specialized applications. In distributed computing
processors usually have their own private or distributed memory.
Significant Characteristics of Distributed Systems:
Three significant characteristics of distributed systems are:
 Concurrency of components
 Lack of a global clock
 Independent failure of components.

Explanation: (Purpose & Mechanism)

A distributed system requires concurrent Components,
communication network and a synchronization
mechanism. A distributed system allows resource sharing,
including software by systems connected to the network.
The goal of distributed computing is to make such a
network work as a single computer. A distributed system is
designed to tolerate failure of individual computers so the
remaining computers keep working and provide services to
the users. Database access and processing is conducted in
another computer that provides centralized access for
many business processes.

Advantages of Distributed Computing: Figure 2 Distributed Computing Architecture

 Enhanced performance tasks can be executed

concurrently; load distribution to reduce response time.
 Easier modular expansion hardware and software resources can be easily added
without replacing existing resources.
 Data used locally can be stored locally and network traffic kept to a minimum if data
lost on central site, it can be reduplicated from local site.

Disadvantages of Distributed Computing:

 It is more difficult to implement a distributed database system; thus it is more costly.

 The deployment cost of a distribution is higher than a single system.
 Distributed computing systems are difficult to deploy, maintain and
troubleshoot/debug than their centralized counterparts.

Daily Life Examples of Distributed Computing:

 Telecommunication networks: Telephone networks and Cellular networks.

 Intranets, Internet, WWW, Email.
 Electronic banking.

____________________________________