11/13/2014

ECEG-5501
Introduction to Algorithm
Analysis & Design

Introduction

What is Algorithm?
a sequence of unambiguous instructions for
solving a problem, i.e. for obtaining a required
output for any legitimate input in a finite
amount of time.

Introduction

What is a problem?
Roughly, a problem specifies what set of outputs is
desired for each given set of inputs.
Example: The Sorting Problem

A problem instance is just a specific set of inputs.

Solving a problem instance consists of specifying a
procedure for converting the inputs to an output of the
desired form (called a solution).

An algorithm that is guaranteed to result in a solution
for every instance is said to be correct.

Note that a given instance may have either no
solutions or more than one solution.

11/13/2014

Introduction

The First Algorithm

Euclids Algorithm
Euclid(m,n){
while n does not divide m
r m%n;
mn
nr
end
return n
}
while n 0 do
r m mod n
mn
n r
return m

Simple factoring algorithm

Input: 2 integers m,n
Output: Largest integer that
divides both without a remainder
Algorithm 1:
1. factorize m=m1Xm2XXmP
2. factorize n=n1Xn2XXnk
3. Identify the common factors,
multiply the result and return

Introduction

What is a program?
A program is the expression of an algorithm in
a programming language
A set of instructions which the computer will
follow to solve a problem

Introduction

Important points
Non-ambiguity
Range of inputs
Several algorithms for solving the same problem

11/13/2014

Introduction

How Algorithm is Specified?

An algorithm is specified:

In words (in a free and also a step-by-step form)

In pseudocode, i.e. a mixture of a natural language and
programming language-like construct
if then
if then else
for do

while do
repeat until

Introduction

Pseudo code
C

false

true

true

S1

next statement

S2

next statement

if then

if then else

Introduction

Pseudo code
for k 1 to n do

.
.
.

done n times

11/13/2014

Introduction

Pseudo code
while do
false

C
true

next statement

Introduction

Pseudo code
repeat until

true

C
false
next statement

Introduction

Steps Problem Solving using Algorithm

Understand the problem
Decide on computational means
Exact vs approximate solving,
Algorithm design technique

Design an algorithm
Prove correctness
Analyze the algorithm
Code the algorithm

11/13/2014

Introduction

Fundamentals of Algorithmic Problem Solving

What does it mean to understand the problem?
Read the problems description carefully
Do a few small examples by hand
Think about special cases

Capabilities of the Computational Device

- Random access machines
- Sequential algorithms
- parallel algorithms
- exact vs Approximate algorithms

Introduction

Fundamentals of Algorithmic Problem Solving

Algorithm Design Techniques
Build a computational model of the solving
process

Prove correctness
Correct output for every legitimate input in finite
time
By Mathematical induction

Introduction

Fundamentals of Algorithmic Problem Solving

Analyze the algorithm
Efficiency: time and space
Simplicity
Generality:
range of inputs, it accepts
Problems the algorithm solve

Optimality
(What is the minimum amount of effort any algorithm will need to exert
to solve the problem?)

11/13/2014

Introduction

Fundamentals of Algorithmic Problem Solving

Analyze the algorithm
recognize limitations of various algorithms for
solving a problem
understand relationship between problem size and
running time
how to analyze an algorithm's running time without
coding it

Coding
How the objects and operations in the algorithm are
represented in the chosen programming language?

Introduction

Important Problem Types

Sorting
Searching
String Processing
Combinatorial Problems
Geometric Problems
Numerical Problems

Introduction

Important Problem Types

Sorting
rearrange items of a given list in a specific order, based on some key

Searching
find a given search key in a given set

String Processing
String is a sequence of characters from alphabet
search a word in a text (string matching)

Graph Problems
Collections of vertexes some of them connected by line segments
includes graph traversal problems, shortest-path algorithms, traveling
salesman problem, graph coloring problems (event scheduling)

11/13/2014

Introduction

Important Problem Types

Combinatorial Problems
Use permutation, combination, subset that satisfies certain
constraints(either maximize a value or minimize a cost)
e.g. traveling salesman

Numerical Problems
solve equations & systems of equations, evaluate
functions, etc.[ difficulty: round-off error]

Geometric problems
Deals with geometric objects such as points, lines and
polygon
E.g. Closest pair problem

Introduction

Fundamental Data Structures

Linear Data Structures
Array
Linked list
Stack
Queue

Introduction

Fundamental Data Structures

Non Linear Data Structures
Graphs G=<V, E>
-graph representation: adjacency lists,
adjacency matrix
- weighted graph
Trees : connected graph without cycles
Rooted trees
Ordered trees
Binary trees

Tree representation: as graphs; binary nodes

11/13/2014

Introduction

Fundamental Data Structures

Sets, Bags, Dictionaries
Set: unordered collection of distinct elements
Operations: membership, union, intersection
Representation: bit string; linear structure

Bag: unordered collection, elements may be

repeated
Dictionary: a bag with operations search, add,
delete

Introduction

Algorithm classification
By types of problems: Study sorting algorithms, then
searching algorithms, etc.
By design technique: Study algorithms by design
strategy.
A general approach to solving problems
Examples of design strategies include:
Brute force
Divide-and-conquer
The greedy method
Dynamic programming

Introduction

Example: ALGORITHM Input / Output

Example: ALGORITHM Euclid (m, n)
// compute GCD(m, n) by Euclids algorithm
// input: two non-negative, not-both-zero integers m and n
// output: Greatest Common Divisor of m and n
while n 0 do
r m mod n
m n
n r
return m
Input : m and n
Output: the greatest common divisor of m and n

Time : ?

11/13/2014

Introduction

Example: ALGORITHM Input/Output

Example: sequential search
Algorithm sequential search (A[0..n-1], K)
// searches for a given value in a given array by sequential search input: an array A[0..n-and
//a search key K output: returns the index of the first element of A that matches K
or
//-1 if there are no matching elements
i0
while i n and A[i] K do
ii+ 1
if i n return i
else return -1

Input: A[0..n-1], K output: j such that A[j] = K or 1

Time: ?

Introduction

End of Slide Show