Distributed Systems Design

COMP 6231
Introduction
Lecture 1

Essam Mansour

1
2
Who am I?
Dr. Essam Mansour
Assistant Professor
www.emansour.com
@DrEssam_Mansour
2019 - Now
Concordia Data
2013 - 2019 Systems (CoDS) Lab
2009 - 2013
Large-scale Analytics
2008 - 2009 on Strings
Ireland

2004 - 2008 Querying

Ph.D. in Computer Science Geo-Distributed Graphs
Data Management
Egypt
Semi-structured data
Active Databases Mobile Data Management
1997 - 2003 Energy-aware algorithms Elastic OLTP Systems
B.Sc. and M.Sc.
Database Systems Data Integration and Discovery
Software Engineering
Temporal Query Processing

3
Who am I?
Dr. Essam Mansour
Assistant Professor
www.emansour.com
@DrEssam_Mansour
2019 - Now
Concordia Data
2013 - 2019 Systems (CoDS) Lab
2009 - 2013

2008 - 2009 30+ Publications

Ireland

Expertise Venues
2004 - 2008
• Big data • VLDBJ
Ph.D. in Computer Science
• PVLDB
Egypt
Data Management • Database systems
Semi-structured data • SIGMOD
• Data discovery and integration • ICDE
Active Databases
1997 - 2003 • Distributed and parallel systems • WWW
B.Sc. and M.Sc.
Database Systems
• Graph and web data management • CIKM
Software Engineering • …….
Temporal Query Processing

4
Supervision/Mentoring Experience
- Co-supervising postgraduate students:
3 PhD students
 2 students in the string analytics project (2015, and 2017)
 1 student in the decentralized graph management (2018)
 1 VLDBJ, 2 PVLDB, 1 ICDE, and 1 CIKM research papers
 1 SIGMOD, 1 VLDB, and 1 ICDE demo/poster papers

2 Master students
 1 student in the string analytics project (2012)
 1 student in the mobile data management project (2009)
 1 PVLDB and 1 ICSOFT research papers

- Mentoring different postdocs.

5
 Academic Service
• A Program Committee member of:

• I also regularly review papers for a number of top international journals including:
• ACM Transactions on Database Systems (TODS)
• The International Journal on Very Large Data Bases (VLDB Journal)
• IEEE Transactions on Knowledge and Data Engineering (TKDE)
• IEEE Transactions on Parallel and Distributed Systems (TPDS)
• IEEE Transactions on Cloud Computing (TCCSI)
• IEEE Transactions on Big Data (TBD)

6
Teaching Style
• I like interaction in class
• I like to ask questions
• I like to be asked questions

• I like to know (and memorize) your names 

• I like to give practical assignments and projects

• I like to learn …

7
… and now

8
Guest Lecture, CMPT 523 Distributed Systems, Spring 2016 9
2009
10
On the Verge of A Disruptive Century :
Breakthroughs

Ubiquitous
Astronomy
Computing

Smaller, Faster,
Cheaper Sensors

Gene
Sequencing and
Biotechnology

11
 A Common Theme is Data
The amount of data is only growing…
1.2 Zettabytes (1021 B or 1 Billion TB)

12
We Live in a World of Data…

13
What Do We Do With Data?

Store Share

Access Process

…. and
Encrypt more!

We want to do these seamlessly...

14
Using Diverse Interfaces & Devices

Mobile Devices
Computers

…and even appliances

Consumer Electronics Personal Monitors and

Sensors

We also want to access, share and process our data from all of
our devices, anytime, anywhere!

15
Data Becoming Critical to Our Lives

Health Science
Domains
Education of Data
Work

Environment Finance

… and more

16
 How to Store and Process Data at Scale?

• A system can be scaled:

• Either vertically (or up) 
• Can be achieved by hardware upgrades (e.g., faster CPU, more
memory, and/or larger disk)

• And/Or horizontally (or out)

• Can be achieved by adding more machines

17
 Vertical Scaling
• Caveat: Individual computers can still suffer from limited resources
with respect to the scale of today’s problems

1. Caches and Memory:

L1
16-32 KB/Core, 4-5 cycles
Cache

L2 Cache 128-256 KB/Core, 12-15 cycles

L3 Cache 512KB- 2 MB/Core, 30-50 cycles

Main Memory 8GB- 128GB, 300+ cycles

18
 Vertical Scaling
• Caveat: Individual computers can still suffer from limited resources
with respect to the scale of today’s problems

2. Disks-- some advancements, but still:

 Limited capacity

 Limited number of channels

 Limited bandwidth

19
 Vertical Scaling
• Caveat: Individual computers can still suffer from limited resources
with respect to the scale of today’s problems

2. Processors:
 Moore’s law still holds

 Chip Multiprocessors (CMPs) are now available

P P P P
P L1 L1 L1 L1
L1
Interconnect
L2
L2 Cache

A single Processor Chip

20
A CMP
 Vertical Scaling
• Caveat: Individual computers can still suffer from limited resources
with respect to the scale of today’s problems

2. Processors:
 But up until a few years ago, CPU speed grew at the rate of 55%
annually, while the memory speed grew at the rate of only 7%

Processor-Memory speed gap

M

21
 Vertical Scaling
• Caveat: Individual computers can still suffer from limited resources
with respect to the scale of today’s problems

2. Processors:
 But up until a few years ago, CPU speed grew at the rate of 55%
annually, while the memory speed grew at the rate of only 7%

 Even if 100s or 1000s of cores are placed on a CMP, it is a challenge

to deliver input data to these cores fast enough for processing
 Vertical Scaling

P P P P
10000 Suffer from
L1 L1 L1 L1 seconds (or limited scalability
3 hours) to
Interconnect
load data
L2 Cache
A Data Set
of 4 TBs

Memory

4 100MB/S IO Channels

23
 How to Store and Process Data at Scale?

• A system can be scaled:

• Either vertically (or up)
• Can be achieved by hardware upgrades (e.g., faster CPU, more
memory, and/or larger disk)

• And/Or horizontally (or out)

• Can be achieved by adding more machines 

24
 Horizontal Scaling

P 100
P
Splits Only 3
L1 Machines L1
minutes to
L2 L2 load data

Memory Memory
A Data Set (data)
of 4 TBs

25
 Requirements
• But, this necessitates:
• A way to express the problem in terms of parallel processes and execute them
on different machines (Programming and Concurrency Models)

• A way to organize processes (Architectures)

• A way for distributed processes to exchange information (Communication

Paradigms)

• A way to locate and share resources (Naming Protocols)

26
 Requirements
• But, this necessitates:
• A way for distributed processes to cooperate, synchronize with one another,
and agree on shared values (Synchronization)

• A way to reduce latency, enhance reliability, and improve performance

(Caching, Replication, and Consistency)

• A way to enhance load scalability, reduce diversity across heterogeneous

systems, and provide a high degree of portability and flexibility
(Virtualization)

• A way to recover from partial failures (Fault Tolerance)

27
 Degree of Parallelism

DATA D A T A

Task Parallelism Data Parallelism

28
 So, What is a Distributed System?

A distributed system is:

One in which components

A collection of independent
located at networked
computers that appear to its
computers communicate and
users as a single coherent
coordinate their actions only
system
by passing messages

29
 Features
• Distributed Systems imply four main features:

1 Geographical Separation

2 No Common Physical Clock

3 No Common Physical Memory

4 Autonomy and Heterogeneity

30
 Parallel vs. Distributed Systems
• Distributed systems contrast with parallel systems, which entail:

1 Strong Coupling

2 A Common Physical Clock

I am not sure.
Can you verify that?

3 A Shared Physical Memory

4 Homogeneity
Some Administrivia!

32
 Distributed Systems Design
Considered: a reasonably critical and
comprehensive perspective. .9. Fault Tolerance
Thoughtful: Fluent, flexible and efficient .8. Distributed Frameworks
perspective. .7. Replication
Masterful: a powerful and illuminating .6. Caching
perspective. .5. Synchronization
.4. Naming
3. Architectures
.2. Remote Procedure Calls
.1. Networking
.0. Introduction
33
 Course Objectives
The course aims at providing an in-depth
understanding and hands-on experience on

How modern
distributed
Distributed systems meet the
system demands of
Principles on programming contemporary
which models and distributed
Principles on distributed analytics applications
which systems are engines
distributed optimized
systems are
based

34
 Teaching Team
COMP
Instructor:
Essam Mansour Teaching 6231
Assistants Teaching
(EM)
Team

EM Office Hours
• Monday, 2:30 - 4:00PM
• Welcome when my office door is open
• By appointment
• My office: EV 3.251
TAs Office Hours
• To be announced
• By appointment

35
Textbooks

36
Paper Reading and Reviews

37
 Teaching Methods
13 Lectures
• Motivate learning
• Provide a framework or roadmap to organize the information of
the course
• Explain subjects and reinforce the critical big ideas

12 Labs
•Get you to reveal what you don’t understand, so that we can help you
•Allow you to practice skills you will need to become an expert

38
 Assignments and Projects

Assignments

• required problem solving and reading assignments

Projects

• large programming project

39
 Projects
Project Proposal

Introduction and Related Work

Proposed Solution

Experiential Evaluation

Paper Demo
40
 Assessment Methods
 How do we measure learning?
Type # Weight
Project 1 35%
Final Exam 1 30%

Problem Set 4 20%

Presentations/reviews 3 15%

41
 Next Lecture
• Networking- Part I

Questions?

42