Algorithm Design Techniques

CSC 3102 0.1 B.B. Karki, LSU

 Algorithm Design Techniques

Various design techniques exist:

Classifying algorithms based on design ideas or commonality.
General-problem solving strategies.

Brute force
Divide-and-conquer
Decrease-and-conquer
Transform-and-conquer
Space-and-time tradeoffs
Dynamic programming
Greedy techniques

CSC 3102 0.2 B.B. Karki, LSU

 Algorithm Design Techniques
Brute force
Selection sort, Brute-force string matching, Convex hull problem
Exhaustive search: Traveling salesman, Knapsack, and Assignment problems
Divide-and-conquer
Master theorem, Mergesort, Quicksort, Quickhull
Decrease-and-conquer
Insertion sort, Permutations (Minimal change approach, Johnson-Trotter algorithm)
Fake-coin problem (Ternary search), Computing a median, Topological sorting
Transform-and-conquer
Gaussian elimination, Heaps and Heapsort, Problem reduction
Space-and-time tradeoffs
String matching: Horspools algorithm, Boyer-Moore algorithm
Dynamic programming
Warshalls algorithm for transitive closure
Floyds algorithms for all-pairs shortest paths
Greedy techniques
MST problem: Prims algorithm, Kruskals algorithm (Sets and set operations)
Dijkstras algorithm for single-source shortest path problem
Huffman tree and code
More on algorithms.
CSC 3102 0.3 B.B. Karki, LSU
 Brute Force

CSC 3102 0.4 B.B. Karki, LSU

 The Simplest Approach

Brute force - the simplest of the design strategies

Is a straightforward approach to solving a problem, usually directly based on
the problems statement and definitions of the concepts involved.
Just do it - the brute-force strategy is easiest to apply.
Results in an algorithm that can be improved with a modest amount of time.

Brute force is important due to its wide applicability and simplicity.

Weakness is subpar efficiency of most brute-force algorithms.

Important examples:
Selection sort, String matching, Convex-hull problem, and Exhaustive search.

CSC 3102 0.5 B.B. Karki, LSU

 Selection Sort

Scan the list repeatedly to find the elements, one at a time, in an non-
decreasing order.
On the ith pass through the list, search for the smallest item among the last
n - i elements and swap it with A[i]. After n - 1 passes, the list is sorted.

Algorithm SelectionSort (A[0..n-1])

//Sorts a given array
//Input: An array A[0..n-1] of orderable elements
//Output: Array A[0..n-1] sorted in ascending order
for i 0 to n - 2 do
min i
for j i + 1 to n - 1 do
if A[j] < A[min] min j
swap A[i] and A[min]

n2 n1 n2 n2
(n 1)n
C(n) = 1 = [(n 1) (i + 1) + 1] = (n 1 i) = (n 2 )
i= 0 j= i+1 i= 0 i= 0
2
CSC 3102 0.6 B.B. Karki, LSU
 Brute-Force String Matching
Align the pattern against the first m characters of the text and start matching the corresponding
pairs of characters from left to right until all m pairs match. But, if a mismatching pair is found,
then the pattern is shift one position to the right and character comparisons are resumed.

The goal is to find i - the index of the leftmost character of the first matching substring in the text
such that ti = p0,.ti+j = pj, .. Ti+m-1 = pm-1

Algorithm BruteForceStringMatching (T[0..n - 1], P[0..m - 1])

//Implements string matching
//Input: An array T[0..n - 1] of n characters representing a text
// an array P[0..m - 1] of m characters representing a pattern
//Output: The position of the first character in the text that starts the first
// matching substring if the search is successful and -1 otherwise.
for i 0 to n - m do
j0
while j < m and P[j] = T[i + j] do
jj+1
if j = m return i
return -1

In the worst case, the algorithm is (mn). But the average-case efficiency is shown to
be linear, i.e., (m + n) = (n) for searching in random texts.
CSC 3102 0.7 B.B. Karki, LSU
 Convex-Hull Problem
P6
The convex hull of any P7
set S of n points is a
convex polygon with the P2
P8
vertices at some of the
points (called extreme P5
points) of S so that the P3
polygon contains all
points (either inside or on P4
its boundary)
A set of points on the
P1
plane is called convex if
for any two points in the
set, the entire line The convex hull for the set of eight points is the
segment with the end convex polygon with its vertices at P1, P5, P6,
points at P and Q P7 and P3
belongs to the set.

CSC 3102 0.8 B.B. Karki, LSU

 Brute-Force Approach for Convex-Hull Problem

A brute-force approach is based on the fact:

A line segment connecting two points i and j is a part of its convex hulls boundary if and only
if all other points of the set lie on the same side of the straight line through these points.
Repeating this test for every pair of points yields a list of line segments that make up the
convex hulls boundary.

Find the equation of the straight line through two points (x1, y1) and (x2, y2)
ax + by = c, where a = y2 - y1, b = x1 - x2 and c = x1y2 - y1x2.

This line divides the plane into two half-planes:

For all the points in one plane, we have ax + by > c whereas for all other points in the other
plane, we have ax + by < c.
For all points on the line itself, ax + by = c.
Check whether the expression ax + by - c has the same sign at each of these points
To check whether certain points lie on the same side of the line.
If yes, the line forms one side of the polygon.

Efficiency of the algorithm: O(n3)

For each of n(n -1)/2 pairs of distinct points, one needs to find the sign of ax + by - c for each
of the other n - 2 points.
CSC 3102 0.9 B.B. Karki, LSU
 Exhaustive Search

Exhaustive search is a brute-force approach to combinatorial problems.

It suggests generating each and every combinatorial object of the problem,
selecting those of them of that satisfy the problems constraints and then
finding a desired object.
Impratical for all but applicable to very small instances of problems.

Examples:
Traveling salesman problem
Finding the shortest tour through a given set of n cities that visits each city
exactly once before returning to the city where it started.
Knapsack problem
Finding the most valuable list of out-of n items that fit into the knapscak.
Assignment problem
Finding an assignment of n people to execute n jobs with the smallest total
cost.

These problems are the examples of so-called NP-hard problems.

CSC 3102 0.10 B.B. Karki, LSU

 Traveling Salesman Problem
2
Traveling salesman problem a b
Asks to find the shortest tour through a given
7 8
set of n cities that visits each city exactly
5 3
once before returning to the city where it
started.

c d
Finding the shortest Hamiltonian circuit of 1
the graph
A cycle that passes through all the vertices of
the graph exactly once.
A sequence of n +1 adjacent vertices v0, v1, Tour Length
v2, vn-1, v0.
Get all tours by generating all the abcda 2+8+1+7 = 18
permutations of n - 1 intermediate cities, abdca 2+3+1+5 = 11
compute the tour lengths, and find the
shortest among them. acbda 23
acdba 11
adbca 23
Efficiency:
adcba 18
Total number of permutations = (n - 1)!/2
Impractical but ok very small values of n.
More than one optimal solutions.
CSC 3102 0.11 B.B. Karki, LSU
 Knapsack Problem

Given n items of known weights W = 10

w1, ., wn and values v1, , vn and
a knapsack of capacity W, find the
most valuable subset of the items w=7 w=3 w=4 w=5
that fit into the knapsack. v = $42 v = $12 v = $40 v = $25

Knapsack item1 item2 item3 item4

Consider all the subsets of the set Subset Total weight Total value
of n items given,
Null 0 $0
Computing the total weight of
{1} 7 $42
each subset in order to identify {2} 3 $12
feasible subsets (the ones with the {3} 4 $40
total not exceeding the knapsacks {4} 5 $25
capacity). {1,2} 10 $36
{1,3} 11 not feasible
Finding a subset of the largest
{1,4} 12 not feasible
value among them. {2,3} 7 $52
The number of subsets of an n- {2,4} 8 $37
element set is 2n so the exhaustive {3,4} 9 $65
search leads to (2n) algorithm. {1,2,3} 14 not feasible
{1,2,4} 15 not feasible

{1,2,3,4} 19 not feasible
CSC 3102 0.12 B.B. Karki, LSU
 Assignment Problem
Given n people to be assigned to execute n A small instance of the problem:
jobs, one person per job, find an assignment Job 1 Job 2 Job 3 Job 4
with the smallest total cost. Person 1 9 2 7 8
Person 2 6 4 3 7
C[i, j] is the cost for the ith person assigned to
Person 3 5 8 1 8
the jth job. Person 4 7 6 9 4

An instance of the assignment problem is

completely specified by its cost matrix C.

Select one element in each row so that all

selected elements are in different columns
and the total sum of the selected elements is Total possible iterations: 4! = 24
the smallest possible. Remaining iterations:

Permutation cost
<2 1 3 4> 2+6+1+4 = 13
Generate all the permutations of integers 1, <2 1 4 3> 2+6+8+9 = 25
2, n, computing the total cost of each .
assignment by summing up the <3 1 2 4> 7+6+8+4 = 25
.
corresponding elements of the cost matrix, <4 1 2 3> 8+6+8+9 = 31
and finally select the one with smallest sum.
The optimal solution is <2 1 3 4> with the minimum cost
of 13. So person 1 to job 2, person 2 to job 1, person 3 to
Complexity: n! operations. job 3 and person 4 to job 4.

CSC 3102 0.13 B.B. Karki, LSU

 Brute Force - Summary

Simplest approach: just do it

Selection sort: n2

String matching: mn

Convex hull problem: n3

Exhaustive search
Traveling salesman problem: (n-1)!/2
Knapsack problem: 2n
Assignment problem: n!

CSC 3102 0.14 B.B. Karki, LSU