Algorithms

R OBERT S EDGEWICK | K EVIN W AYNE

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
F O U R T H E D I T I O N

! deletion

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu

! deletion

Binary search trees

Definition. A BST is a binary tree in symmetric order.
root a left link a subtree

A binary tree is either:

Empty. Two disjoint binary trees (left and right).

null links

right child of root

Anatomy of a binary tree

Symmetric order. Each node has a key, and every nodes key is:

parent of A and R left link of E A C keys smaller than E H E R

S
9

key X value associated with R

Smaller than all keys in its right subtree.

Larger than all keys in its left subtree.

keys larger than E

Anatomy of a binary search tree

BST representation in Java

Java definition. A BST is a reference to a root Node. A Node is comprised of four fields:

A Key and a Value. A reference to the left and right subtree.

smaller keys larger keys

private class Node { private Key key; private Value val; private Node left, right; public Node(Key key, Value val) { this.key = key; this.val = val; } }
Key and Value are generic types; Key is Comparable

BST
Node key val

left

right

BST with smaller keys

BST with larger keys

Binary search tree

4

BST implementation (skeleton)

public class BST<Key extends Comparable<Key>, Value> { private Node root; private class Node { /* see previous slide */

root of BST

public void put(Key key, Value val) { /* see next slides */ } public Value get(Key key) { /* see next slides */ } public void delete(Key key) { /* see next slides */ } public Iterable<Key> iterator() { /* see next slides */ } }

Binary search tree demo

Search. If less, go left; if greater, go right; if equal, search hit.

successful search for H

Binary search tree demo

Insert. If less, go left; if greater, go right; if null, insert.

insert G

BST search: Java implementation

Get. Return value corresponding to given key, or null if no such key.

public Value get(Key key) { Node x = root; while (x != null) { int cmp = key.compareTo(x.key); if (cmp < 0) x = x.left; else if (cmp > 0) x = x.right; else if (cmp == 0) return x.val; } return null; }

Cost. Number of compares is equal to 1 + depth of node.

8

BST insert
Put. Associate value with key.
inserting L

S E A R C search for L ends at this null link E A C create new node H M L P S E A C reset links on the way up H M L P R X R H M P S X X

Search for key, then two cases:

Key not in tree !

Key in tree ! reset value. add new node.

Insertion into a BST

9

BST insert: Java implementation

Put. Associate value with key.
public void put(Key key, Value val) { root = put(root, key, val); }
concise, but tricky, recursive code; read carefully!

private Node put(Node x, Key key, Value val) { if (x == null) return new Node(key, val); int cmp = key.compareTo(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; return x; }

Cost. Number of compares is equal to 1 + depth of node.

10

Tree shape

C A E R

S X

Many BSTs correspond to same set of keys. typical case S best case is equal to 1 + depth of node. Number of compares for search/insert H E X
C A E R S X A C H R

best case

H S E R X

typical case

S E X R H

A C E

worst case

C A

A C

H R S X

typical case

S E X R H

A C E

worst case

A C

BST possibilities H R S

X A Tree shape worst case Remark. depends on order of insertion. C E H R BST possibilities
11

BST insertion: random order visualization

Ex. Insert keys in random order.

12

Correspondence between BSTs and quicksort partitioning

H D A C L E I M O

T U Y

Remark. Correspondence is 1-1 if array has no duplicate keys.

13

BSTs: mathematical analysis

Proposition. If N distinct keys are inserted into a BST in random order, the expected number of compares for a search/insert is ~ 2 ln N. Pf. 1-1 correspondence with quicksort partitioning.

Proposition. [Reed, 2003] If N distinct keys are inserted in random order, expected height of tree is ~ 4.311 ln N.
How Tall is a Tree? How Tall is a Tree?
reed@moka.ccr.jussieu.fr reed@moka.ccr.jussieu.fr
CNRS, Paris, France

Bruce Reed

CNRS, Paris, France

Bruce Reed

purpose of this note ABSTRACT have: purpose ofsearch this note that for /3 -- ! + ~ 3, Let H~ be the height of a random binary tree to onprove n nodes. We show that there constants a = 4.31107... have: Let H~ be the height of a random binary search tree on n exists THEOREM 1. E(H n d / 3 = 1.95... such that E(H~) = c~logn - / 3 1 o g l o g n + nodes. We show that there existsa constants a = 4.31107... Var(Hn) O(1) . O(1), We also show that Var(H~) = O(1). THEOREM 1. E(H~) = ~ l o g n / 3 1 o g l= og n + O(1 a n d / 3 = 1.95... such that E(H~) = c~logn - / 3 1 o g l o g n +

ABSTRACT

O(1), We also show that Var(H~) = O(1).

Var(Hn) = O(1) .

But Worst-case height is NE.2 . [Data Structures]:

(exponentially small chance when keys are inserted in random order)

For more informatio as we will see. may consult [6],[7], [ For more information on random binary search trees 1. THE RESULTS R e m a r k After I ann may consult [6],[7], [1], [2], [9], [4], and [8]. A binary search tree is a binary tree to each node of which 1. THE RESULTS developed an alterna R e m aaxe r k drawn After I from announced results, Drmota(unp we have associated a key; these keys some these O(1) using complete A binary search tree is a binary tree to each node of which anbe alternative proof of the fact that Var(H totally ordered set and the key developed at v cannot larger than proofs illuminate dif we have associated a key; these keys axe drawn from some 14 O(1) using completely different techniques. As ou the key at its right child nor smaller than the key at its left decided to submit we th totally ordered set and the key at v cannot be larger than proofs illuminate different aspects of the problem, child. Given a binary search tree T and a new key k, we and asked that they the key at its right child nor smaller than the key at its left

Categories and Subject Descriptors given is more R e m a r k By the definition of a, nition /3 = 7"g~" 3~ The firs as we will see. Categories and Subject Descriptors E.2 [ D a t a S t r u c t u r e s ] : Trees nition given is more suggestive of why this value is co
Trees

R e m a r k By the def

ST implementations: summary

guarantee implementation search sequential search (unordered list) binary search (ordered array) insert

average case ordered ops? search hit insert operations on keys

N/2

no

equals()

lg N

lg N

N/2

yes

compareTo()

BST

1.39 lg N

1.39 lg N

next

compareTo()

15

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu

! deletion

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu

! deletion

Minimum and maximum

Minimum. Smallest key in table. Maximum. Largest key in table.

min()
min

S E A C H M R X

max() max

Examples of BST order queries

Q. How to find the min / max?

18

Floor and ceiling

Floor. Largest key " to a given key. Ceiling. Smallest key # to a given key.

floor(G)

min()

S E A C H M
floor(D)

max()

X R
ceiling(Q)

Examples of BST order queries

Q. How to find the floor /ceiling?

19

Computing the oor

Case 1. [k equals the key at root] The floor of k is k.
A C H M
finding floor(G)

S X R
G is less than S so floor(G) must be

Case 2. [k is less than the key at root] The floor of k is in the left subtree.
E

on the left X

S A C H M S E A C H M
floor(G)in left subtree is null

Case 3. [k is greater than the key at root] The floor of k is in the right subtree (if there is any key " k in right subtree); otherwise it is the key in the root.

G is greater than E so floor(G) could be

on the right

X R

S E A C result R H M
20

Computing the floor function

Computing the oor

finding floor(G)

S X R H M

public Key floor(Key key) { Node x = floor(root, key); if (x == null) return null; return x.key; } private Node floor(Node x, Key key) { if (x == null) return null; int cmp = key.compareTo(x.key); if (cmp == 0) return x; if (cmp < 0) return floor(x.left, key);

E A C

G is less than S so floor(G) must be

on the left X

S E A C H M S E A C H M
floor(G)in left subtree is null

G is greater than E so floor(G) could be

on the right

X R

Node t = floor(x.right, key); if (t != null) return t; else return x; }

S E A C result R H M
21

Computing the floor function

Subtree counts
In each node, we store the number of nodes in the subtree rooted at that node; to implement size(), return the count at the root.

node count
6 2

S
3 2

E
1

R
1

Remark. This facilitates efficient implementation of rank() and select().

22

BST implementation: subtree counts

private class Node { private Key key; private Value val; private Node left; private Node right; private int count; }
number of nodes in subtree

public int size() { return size(root);

private int size(Node x) { if (x == null) return 0; ok to call return x.count; when x is null }

private Node put(Node x, Key key, Value val) { if (x == null) return new Node(key, val); int cmp = key.compareTo(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; x.count = 1 + size(x.left) + size(x.right); return x; }
23

Rank
Rank. How many keys < k ? Easy recursive algorithm (3 cases!)
node count
6 2 8

S
3 2

E
1

R
1

public int rank(Key key) { return rank(key, root);

private int rank(Key key, Node x) { if (x == null) return 0; int cmp = key.compareTo(x.key); if (cmp < 0) return rank(key, x.left); else if (cmp > 0) return 1 + size(x.left) + rank(key, x.right); else if (cmp == 0) return size(x.left); }
24

Inorder traversal

Traverse left subtree. Enqueue key. Traverse right subtree.

public Iterable<Key> keys() { Queue<Key> q = new Queue<Key>(); inorder(root, q); return q; } private void inorder(Node x, Queue<Key> q) { if (x == null) return; inorder(x.left, q); q.enqueue(x.key); inorder(x.right, q); }
BST
key val

left

right

BST with smaller keys smaller keys, in order

key

BST with larger keys larger keys, in order all keys, in order

Property. Inorder traversal of a BST yields keys in ascending order.

25

BST: ordered symbol table operations summary

sequential search
search

binary search lg N N 1 lg N lg N 1 N

BST

N N N N N N N log N

h h h h h h N
h = height of BST (proportional to log N if keys inserted in random order)

insert

min / max

floor / ceiling

rank

select

ordered iteration

order of growth of running time of ordered symbol table operations

26

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu

! deletion

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu

! deletion

ST implementations: summary

guarantee implementation search sequential search (linked list) binary search (ordered array) insert delete

average case search hit insert delete

ordered iteration?

operations on keys

N/2

N/2

no

equals()

lg N

lg N

N/2

N/2

yes

compareTo()

BST

1.39 lg N

1.39 lg N

???

yes

compareTo()

Next. Deletion in BSTs.

29

BST deletion: lazy approach

To remove a node with a given key:

Set its value to null. Leave key in tree to guide searches (but don't consider it equal in
search).
E A C H N S
delete I

E A C R H N S

tombstone

Cost. ~ 2 ln N' per insert, search, and delete (if keys in random order), where N' is the number of key-value pairs ever inserted in the BST. Unsatisfactory solution. Tombstone (memory) overload.
30

Deleting the minimum

To delete the minimum key:

Go left until finding a node with a null left link. Replace that node by its right link. Update subtree counts.

go left until reaching null left link A C

S E R H M X

return that nodes right link A

S E R H M X

public void deleteMin() { root = deleteMin(root);

available for garbage collection

private Node deleteMin(Node x) { if (x.left == null) return x.right; x.left = deleteMin(x.left); x.count = 1 + size(x.left) + size(x.right); return x; }

update links and node counts after recursive calls

5

S X R H M

E C

Deleting the minimum in a BST

31

Hibbard deletion
To delete a node with key k: search for node t containing key k. Case 0. [0 children] Delete t by setting parent link to null.

deleting C

update counts after recursive calls S E X R H M replace with null link S E A H M R X

1 5

S X R H M

A C

node to delete

available for garbage collection C

32

Hibbard deletion
To delete a node with key k: search for node t containing key k. Case 1. [1 child] Delete t by replacing parent link.

deleting R

update counts after recursive calls S E X R C H M node to delete S E A C replace with child link R H M available for garbage collection X A C
5

S X H M

33

Hibbard deletion

deleting E

node to delete

S R

X key k. To delete a node with key k: search for node tEcontaining A C H M

t

Case 2. [2 children]

search for key E

x has no left child but X don't garbage collect x

Find successor x of t. Delete the minimum in t's right subtree. A C Put x in t's spot.
deleting E

S E
x

R H M
still a BST successor
min(t.right)

node to delete E A C H M
t

S X R search for key E

go right, then go left until reaching null left link

x
t.left

S E H R X
deleteMin(t.right)

A S C X R
5

M
7

E A C go right, then go left until reaching null left link

x
t.left

S X R M update links and node counts after recursive calls

34

H M

successor
min(t.right)

H A C

S E X
deleteMin(t.right)

Deletion in a BST

Hibbard deletion: Java implementation

public void delete(Key key) { root = delete(root, key);

private Node delete(Node x, Key key) { if (x == null) return null; int cmp = key.compareTo(x.key); if (cmp < 0) x.left = delete(x.left, key); else if (cmp > 0) x.right = delete(x.right, key); else { if (x.right == null) return x.left; if (x.left == null) return x.right; Node t = x; x = min(t.right); x.right = deleteMin(t.right); x.left = t.left; } x.count = size(x.left) + size(x.right) + 1; return x; }

search for key

no right child no left child

replace with successor

update subtree counts

35

Hibbard deletion: analysis

Unsatisfactory solution. Not symmetric.

Surprising consequence. Trees not random (!) ! sqrt (N) per op. Longstanding open problem. Simple and efficient delete for BSTs.
36

ST implementations: summary

guarantee implementation search sequential search (linked list) binary search (ordered array) insert delete

average case search hit insert delete

ordered iteration?

operations on keys

N/2

N/2

no

equals()

lg N

lg N

N/2

N/2

yes

compareTo()

BST

1.39 lg N

1.39 lg N

!N

yes

compareTo()

other operations also become !N if deletions allowed

Red-black BST. Guarantee logarithmic performance for all operations.

37

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu

! deletion

Algorithms

R OBERT S EDGEWICK | K EVIN W AYNE

3.2 B INARY S EARCH T REES

! BSTs ! ordered operations

Algorithms
F O U R T H E D I T I O N

! deletion

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu