Binary Trees and

Recurrence Equations
Lecture #2 of Algorithms, Data structures and Complexity

July 7, 2015

⃝
c GfH&JPK
#2: Binary trees + Recurrence equations

Plan for today

• Binary trees, an important class of objects

– what are they, and what makes them important
– what do typical methods on trees look like
– how do we count steps in these recursive methods

• Formal approach to step counting: recurrence equations

– Example: Fibonacci
– General problem: (solving) recurrence equations

⃝
c GfH&JPK 1
#2: Binary trees + Recurrence equations

Binary trees – 1

Storage structures: arrays, lists, (binary) trees, ....

Typical properties

• An array allows random access, a list and a tree have a single starting point.
• A tree has a good ratio between the number of elements and the average distance
to the starting point, a list has not.

⃝
c GfH&JPK 2
#2: Binary trees + Recurrence equations

Binary trees – 2

A binary tree: each data object has two pointers (left and right) to
successive objects, this yields a storage structure like:

parent 12
of B and C A
key
left right
left right
child of A child of A
6 225
B C

⃝
c GfH&JPK 3
#2: Binary trees + Recurrence equations

Binary trees – 2

A binary tree is a set of nodes that is empty or else satisfies

• there is a single distinguished node called root

• remaining nodes are divided into a left sub-tree and a right sub-tree;
we also speak of children instead of sub-trees (and use parent for the
node itself).

• the left and right sub-tree of a leaf are empty

• the in-degree of a node is 1 (except the root); its out-degree is

maximally 2

⃝
c GfH&JPK 4
#2: Binary trees + Recurrence equations

Binary trees – 4
• The level (or depth) of a node is its distance to the root
• The level of a tree is the maximum level of its leafs
root

102
level 0
level of the binary tree = 3

internal node
6 225
level 1

34 103 1056
level 2

23 40 level 3

leaf

⃝
c GfH&JPK 5
#2: Binary trees + Recurrence equations

The benefits of binary trees – 1

Suppose you want to hold 30 data objects:

• Level 0 (root) holds 1 data object

• Level 1 holds 2 data objects total: 3

• Level 2 holds 4 data objects total: 7

• Level 3 holds 8 data objects total: 15

• Level 4 holds 16 data objects total: 31

• a data object can be traced in 5 steps (instead of 31) if stored handy

⃝
c GfH&JPK 6
#2: Binary trees + Recurrence equations

The benefits of binary trees – 2

Some facts about binary trees:

• there are at most 2d nodes at level d

• a binary tree of level h contains at most 2h+1 − 1 nodes

• a binary tree with n nodes has level at least ⌈log(n+1)⌉ − 1

⃝
c GfH&JPK 7
#2: Binary trees + Recurrence equations

Preorder traversal of a binary tree

102 102 102

6 225 6 225 6 225

34 103 1056 34 103 1056 34 103 1056

23 40 23 40 23 40

102 102
102

6 225 6 225
6 225

34 103 1056 34 103 1056

34 103 1056

23 40 23 40
23 40

parent, left subtree, right subtree

⃝
c GfH&JPK 8
#2: Binary trees + Recurrence equations

Inorder traversal of a binary tree

102 102 102

6 225 6 225 6 225

34 103 1056 34 103 1056 34 103 1056

23 40 23 40 23 40

102 102
102

6 225 6 225
6 225

34 103 1056 34 103 1056

34 103 1056

23 40 23 40
23 40

left subtree, parent, right subtree

⃝
c GfH&JPK 9
#2: Binary trees + Recurrence equations

Postorder traversal of a binary tree

102 102 102

6 225 6 225 6 225

34 103 1056 34 103 1056 34 103 1056

23 40 23 40 23 40

102 102
102

6 225 6 225
6 225

34 103 1056 34 103 1056

34 103 1056

23 40 23 40
23 40

left subtree, right subtree, parent

⃝
c GfH&JPK 10
#2: Binary trees + Recurrence equations

Inorder-Tree Walk

We have binary trees with integer keys that are sorted preorder.

We have node objects with the following features:

• key, an integer key value,

• left, points to the left child,
• right, points to the right child

We want to print out all the keys in a binary tree in the inorder traversal
by a simple recursive algorithm, i.e., the algorithm will print the key of
the root of a subtree between printing the values in its left subtree and
printing those in its right subtree.

⃝
c GfH&JPK 11
#2: Binary trees + Recurrence equations

Inorder-Tree Walk

Inorder-Tree Walk (x)

{ if (x ̸= null )
Inorder-Tree Walk (x.left)
print x.key
Inorder-Tree Walk (x.right)
}

To use the above procedure to print all the elements in a binary tree T ,
we call Inorder-Tree Walk (T.root).

Notice that the procedure calls itself recursively exactly twice for each
node in the tree — once for its left child and once for its right child.

⃝
c GfH&JPK 12
#2: Binary trees + Recurrence equations

Recursive algorithm – what to remember

You can often solve a problem by making clever use of the fact that it is
sufficient to solve a smaller copy of the same problem (or a number of
smaller copies of the same problem).

In such a case you can write a recurrence equation for the number of
steps in the solution of the original problem.

But solving the recurrence equation is not always necessary. If the

recursion reduces the problem from a complete structure to smaller
parts, it is often sufficient to understand what happens in every visit
of the basic components of the structure, and to count how often the
components are visited.

⃝
c GfH&JPK 13
#2: Binary trees + Recurrence equations

General problem – Solving recurrence equations

Typical case:
(n)
T (n) = a·T b + f (n) with a > 1 and b > 1

• problem is divided into a similar problems of size nb each

• non-recursive cost f (n) to split problem or combine solutions of sub-problems

⃝
c GfH&JPK 14
#2: Binary trees + Recurrence equations

The recursion tree method – 1

Visualize the back-substitution process as a tree, keeping track of

• the size of the remaining arguments in the recurrence

• the nonrecursive costs
⇒ very useful to obtain a good guess for substitution method

The recurrence tree of T (n) = 3·T (n/4) + n looks like:

remaining argument T (n) n
for recurrence
nonrecursive cost

T (n/4) n/4 T (n/4) n/4 T (n/4) n/4

T (n/16) n/16 T (n/16) n/16 T (n/16) n/16 .................. T (n/16) n/16 T (n/16) n/16 T (n/16) n/16

⃝
c GfH&JPK 15
#2: Binary trees + Recurrence equations

Intermezzo: Recursion tree properties

(n)
T (n) = a·T b + f (n) with a > 1 and b > 1

The level of the recursion tree for T (n) is the least k such that bk > n.
So the level of this recursion tree is ⌈blog n⌉ = ⌈log n/ log b⌉.

The number of nodes at level m in the recursion tree for T (n) is am.

The number of leaves in the recursion tree for T (n) is

a(log n/ log b) = n(log a/ log b).

x z z log y z log x
(Calculus: log y = log y/z log x and x =y )

⃝
c GfH&JPK 16
 #2: Binary trees + Recurrence equations

The recursion tree method – 2

T (n ) n n

T (n/4)n/4 T (n/4)n/4 T (n/4)n/4 3n/4

4 log n

T (n/16)n/16 T (n/16)n/16 T (n/16)n/16 .................. T (n/16)n/16 T (n/16)n/16 T (n/16)n/16 9n/16

T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) .................. T (1) T (1) T (1) T (1) T (1) T (1) T (1)
4
n log 3 leaves

4 log n−1 ( )i
∑ 3 4 log 3
T (n) = ·n + c·n
| {z }
4
{z } | level cost
| i=0 {z } total cost leafs
sum levels

⃝
c GfH&JPK 17
#2: Binary trees + Recurrence equations

The recursion tree method – 3

An upper bound on the complexity can now be obtained by:
4 log n−1 ( )i
∑ 3 4 log 3
T (n) = ·n + c·n
i=0
4
⇒ (* ignore the smaller terms *)
∑∞ ( )i
3 4 log 3
T (n) < ·n + c·n
i=0
4
≡(* geometric series *)
1 4 log 3
T (n) < ·n + c·n
1 − (3/4)
≡ (* calculus *)
4 log 3
T (n) < 4·n + c·n
≡ (* obtain the order *)
T (n) ∈ O(n)
The algorithm thus has a linear worst-case complexity

⃝
c GfH&JPK 18
#2: Binary trees + Recurrence equations

The Master theorem

(n)
Consider: T (n) = a · T b + f (n) with a > 1 and b > 1

• problem is divided into a similar problems of size nb each

• non-recursive cost f (n) to split problem or combine solutions of sub-problems
• we saw that # leafs in the recursion tree is nE with E = log a/ log b

The Master theorem says:

if then
1. f (n) ∈ O(nE−ε) for some ε > 0 T (n) ∈ Θ(nE )
2. f (n) ∈ Θ(nE ) T (n) ∈ Θ(f (n)· log n)
3. f (n) ∈ Ω(nE+ε) for some ε > 0
and af ( nb ) 6 cf (n) for some c < 1 T (n) ∈ Θ(f (n))

If none of the cases apply, the Master theorem gives no clue!

⃝
c GfH&JPK 19
#2: Binary trees + Recurrence equations

Applying the Master theorem

T (n) = 4·T (n/2) + n

• so a = 4, b = 2 and f (n) = n; E = log 4/ log 2 = 2

• since f (n) = n ∈ O(n2−ε), case 1 applies: T (n) ∈ Θ(n2)

T (n) = 4·T (n/2) + n2

• so a = 4, b = 2 and f (n) = n2; E = log 4/ log 2 = 2

• since n2 ̸∈ O(n2−ε), case 1 does not apply
• but since f (n) = n2 ∈ Θ(n2), case 2 applies: T (n) ∈ Θ(n2· log n)

T (n) = 4·T (n/2) + n3

• so a = 4, b = 2 and f (n) = n3; E = log 4/ log 2 = 2

• clearly, cases 1 and 2 do not apply since E = 2
• since f (n) = n3 ∈ Ω(n2+ε) for ε = 1, case 3 might apply
• . . . and as 4(n/2)3 = 1/2n3 6 cf (n) for c = 1/2, case 3 applies:
T (n) ∈ Θ(n3)

⃝
c GfH&JPK 20
#2: Binary trees + Recurrence equations

The Master theorem does not always apply

n2
T (n) = 4·T (n/2) + log n

• we have a = 4, b = 2 and f (n) = n2/(log n); E = 2

• n2/(log n) ̸∈ O(n2−ε) since f (n)/n2 = (log n)−1 ̸∈ O(n−ε)
⇒ case 1 of the Master theorem does not apply
• n2/(log n) ̸∈ Θ(n2)
⇒ case 2 of the Master theorem does not apply
• f (n) ̸∈ Ω(n2+ε) since f (n)/n2 = (log n)−1 ̸∈ Ω(n+ε)
⇒ case 3 of the Master theorem does not apply

[⇒] The Master theorem does not apply to this case at all!

By substitution one obtains: T (n) ∈ Θ(n2· log(log n))

⃝
c GfH&JPK 21
#2: Binary trees + Recurrence equations

Understanding the Master theorem – 1

(n)
Consider: T (n) = a · T b + f (n) with a > 1 and b > 1

Suppose f (n) is small enough, i.e., f (n) << nE

• to get a feeling let f (n) = 0, then

• only the costs at the leafs will count!
• there will be b log n levels in the recursion tree
• the # nodes at level k equals ak
b log n
⇒ T (n) = a = nE

This yields case 1 of the Master theorem

f (n) < nE is insufficient; f (n) must be polynomially smaller than nE

⃝
c GfH&JPK 22
#2: Binary trees + Recurrence equations

Understanding the Master theorem – 2

(n)
Consider: T (n) = a·T b + f (n) with a > 1 and b > 1

Suppose f (n) is large enough, i.e., f (n) >> nE

• if f (n) is large enough, it will exceed a·f (n/b)

• for instance f (n) = n3 > (n/3)3 + (n/3)3 + (n/3)3 unless a > 27

This yields case 3 of the Master theorem

⃝
c GfH&JPK 23
#2: Binary trees + Recurrence equations

Understanding the Master theorem – 3

If f (n) and nE grow equally fast, the depth of the tree is important

• this perfect balance is what one wants to achieve with divide-and-conquer

algorithms!

This yields case 2 of the Master theorem

⃝
c GfH&JPK 24