Progress in Computer Science

No.6

Edited by
J. Bentley
E. Coffman
R.L.Graham
D.Kuck
N. Pippenger

Springer Science+Business Media, LLC

Luc Devroye
Lecture Notes on
Bucket Algorithms

1986 Springer Science+Business Media, LLC

Author:

Luc Devroye
School of Computer Science
McGill University
Montreal H3A 2K6
Canada

Library of Congress Cataloging in Publication Data

Devroye, Luc.
Lecture notes on bucket algorithms.

(Progress in computer science ; no. 6)

Bibliography: p.
Includes index.
1. Data structures (Computer science) 2. Algorithms.
I. Title. II. Title: Bucket algorithms. III. Series.
QA76.9.D35D48 1985 005.7'3 85-26850
ISBN 978-0-8176-3328-8 ISBN 978-1-4899-3531-1 (eBook)
DOI 10.1007/978-1-4899-3531-1
CIP-Kurztitelaufnahme der Deutschen Bibliothek

Devroye, Luc:
Lecture notes on bucket algorithms / Luc Devroye. -

(Progress in computer science ; No.6)

ISBN 978-0-8176-3328-8

NE:GT
All rights reserved.
No part of this publication may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, recording or otherwise, without prior
permission of the copyright owner.

© 1986 Springer Science+Business Media New York

Originally published by Birkhiiuser Boston, Inc. in 1986

ISBN 978-0-8176-3328-8
 TABLE OF CONTENTS

o. INTRODUCTION. 1

1. ANALYSIS OF BUCKET SORTING AND SEARCHING. 10

1.1. Expected values. 10
1.2. Weak convergence. 17
1.3. Variance. 23
1.4. Large deviation Inequalltles. 25
1.5. Double bucketing. 30

2. DENSITIES WITH UNBOUNDED SUPPORT. 37

2.1. Main results. 37
2.2. Proofs. 45
2.3. A superllnear number of buckets. 52

3. MULTIDIMENSIONAL BUCKETING. 55
3.1. Main theorem. 55
3.2. Sorting and searching. 54
3.3. The travellng salesman problem. 73
3.4. Closest point problems. 80

4. THE MAXIMAL CARDINALITY. 93

4.1. Expected value and lnequalltles. 93
4.2. An example: the selection problem. 105
4.3. Nonllnear functions of the maximal cardlnallty. 109
4.4. Extremal point problems. 114

5. AUXILIARY RESUL TS FROM PROBABILITY THEORY. 127

5.1. Properties of the multinomial distributIon. 127
5.2. Properties of the Poisson distribution. 132
5.3. The Lebesgue density theorem. 134

REFERENCES. 135

INDEX. 143
 PREFACE
Hashing algorithms scramble data and create pseudo-uniform data distribu-
tions. Bucket algorithms operate on raw untransformed data which are parti-
tioned Into groups according to membership In equl-slzed d-dlmenslonal hyperrec-
tangles, called cells or buckets. The bucket data structure Is rather sensitive to
the distribution of the data. In these lecture notes, we attempt to explain the
connection between the expected time of various bucket algorithms and the dis-
tribution of the data. The results are Illustrated on standard searching, sorting
and selection problems, as well as on a variety of problems In computational
geometry and operations research.
The notes grew partially from a graduate course on probability theory In
computer science. I wish to thank Elizabeth Van Gulick for her help with the
manuscript, and David Avis, Hanna AYukawa, Vasek Chvatal, Beatrice Devroye,
Hossam EI Glndy, Duncan McCallum, Magda McCallum, Godfrled Toussaint and
Sue Whltesldes"for making the School of Computer Science at McGill University
such an enjoyable place. The work was supported by NSERC Grant A3456 and
by FCAC Grant EQ-1679.
INTRODUCTION 1

INTRODUCTION

It Is not a secret that methods based upon the truncation of data have good
expected time performance. For example, for nice distributions of the data,
searching Is often better done via a hashing data structure Instead of via a search
tree. The speed one observes In practice Is due to the fact that the truncation
operation Is a constant time operation.
Hashing data structures have not received a lot of attention In the 1970's
because they cannot be fit Into the comparison-based computational model. For
example, there Is no generally accepted lower bound theory for algorithms that
can truncate real numbers In constant time. The few analyses that are avallable
(see Knuth (1973), Gonnet (1981,1984) and the references found there) relate to
the following model: the data pOints are uniformly distributed over either [O,IJ or
{l, ... ,M}. The uniform model Is of course motivated by the fact that It Is often
possible to find a good hash function h (.), l.e. a function of the data pOints which
distributes the data evenly over Its range. In the vast majority of the cases, h (.)
Is not a monotone function of Its argument when the argument Is an Integer or a
real number. Non monotone functions have the undesirable side-effect that the
data are not sorted. Although this Is not Important for searching, It Is when the
data need to be llsted In sorted order rather frequently. If the data form a data
base, I.e. each data point can be considered as a point In R d with d > 1, then
range queries can be conveniently handled If the data are hashed via monotone
functions. There Is an ever Increasing number of appllcatlons In computational
geometry ( see the general survey articles by Toussaint (1980,1982) where appll-
cations In pattern recognition are hlghllghted ; and the survey article on bucket
methods by Asano, Edahlro, Imal, Irl and Murota (1985)) and computer graph-
ics, In which the data pOints should preserve their relative positions because of
the numerous geometrical operations that have to be carried out on them. Points
that are near one another should stay near. In geographic data processing,
the cellular organization Is particularly helpful In storing large amounts of data
such as satelllte data (see the survey article by Nagy and Wagle, 1979). Many
tests In statistics are based upon the partition of the space In equal Intervals,
and the counts of the numbers of pOints In these Intervals. Among these, we cite
the popular chi-square test, and the empty cell test. See for example Kolchln,
Sevast'yanov and Chlstyakov (1978) and Johnson and Katz (1977) for appllca-
tlons In statistics. In economic surveys and management science, the histo-
gram Is a favorite tool for vlsuallzlng complex data. The histogram Is also a
superb tool for statisticians In exploratory data analYSis. In all these examples,
the order In the data must be preserved.
2 INTRODUCTION

Monotone hash function

Figure 0.1.

If we use monotone or order-preserving hash functions, or no hash functions

at all, the uniform distribution model becomes suspect. At best, we should
assume that the data points are random vectors (or random variables) that are
Independent and Identically distributed. The randomness Is Important because
we are not Interested here In worst-case performance. Expected time can only be
analyzed If some randomness Is assumed on the part of the data. The Indepen-
dence assumption can be defended In some situations, e.g. In the context of data
bases for populations. Unfortunately In some geometrical appilcatlons, particu-
larly Image processing, the Independence assumption Is just not good enough.
Notice that If pixels In a screen were selected Independently and according to a
given distribution, then the composed picture would be a "pure noise" picture.
In a sense, the more Information we have In a picture, the more dependence we
see between the pixels. Finally, If we accept the Independence assumption, we
might as well accept the Identical distribution assumption, except If there Is some
nonstatlonary (time-dependent) element In the data collection process.
We will only deal with d-dlmenslonal real numbers and with distributions
that have densities. The complexities of various algorithms are measured In
terms of fundamental operations. Typically, truncation or hashing Is one such
operation. We will of course assume that real numbers can be truncated and / or
hashed In time Independent of the size or the precision of the number - recall
that a similar assumption about comparing two real numbers Is needed In the
well-known comparison-based complexity theory. Densities are convenient
because they free us from having to consider discretization problems: If a distri-
bution Is atomic (I.e., It puts Its mass on a countable set), and enough data points
INTRODUCTION 3

are drawn from this distribution, the number of colllding values Increases
steadlly. In fact, If n Independent Identically distributed random vectors are
considered with any atomic distribution, then N In --+ 0 almost surely as
n --+ 00 where N Is the number of different values. Meaningful asymptotlcs are
only possible If either the atomic distribution varies with n , or the distribution Is
non-atomic. There Is another key argument In favor of the use of densities: they
provide a compact description of the dIstrIbutIon, and are easIly visuallzed or
plotted.
When Independent random vectors with a common density are partitioned
by means of a d-dlmenslonal grid, the number of grid locations (or buckets) with
at least 2 points has a distribution which depends upon the density In question.
The density affects the frequency of colllsions of data pOints In buckets. For
example, If the density Is very peaked, the buckets near the peak are more likely
to contain a large number of pOints. We want to Investigate how this crowding
affects the performance of algorithms of bucket or grid algorithms.
Throughout this set of notes, we wlll consider a d-dlmenslonal array of equl-
sized rectangles (which we wlll call a grid), and within each rectangle, points are
kept In a chain (or linked list). The number of rectangles wlll be denoted by m,
and the data size by n. We wlll not consider Infinite grids such as
([i ,i +1) I i Integer} because Infinite arrays cannot be stored. However, because
data may grow not only In size but also In value as n --+ 00, we wlll consider at
times grid' sizes m that are data value dependent. In any case, m Is usually a
function of n .
4 INTRODUCTION

Figure 0.2.
2d Grid

The purpose of this collection of notes Is to give a variety of probability

theoretical ~echnlques for analyzing various random variables related to the
bucket structure described above. Such random variables Include for example,
the average search time, the time needed for sorting, the worst-case search time
and other nonlinear functions of the cardinalities N 11 • . . ,Nm of the buckets.
The probability theoretical techniques have several features: they are general (for
example, the Lebesgue density theorem Is needed In crucial places In order to
avoid having to Impose any smoothness conditions on the densities), and when-
ever possible, appropriate probability Inequalities are Invoked (for example, heavy
use Is made of Jensen's Inequality (see e.g. Chow and Teicher (lg78)) and
Chernoff's exponential bounding technique (Chernoff (lg52))). Since
N l' . . . ,Nm Is multlnomlally distributed for a data-Independent grid, and the
N; 's are thus not Independent, It Is sometimes useful to use an embedding
method that relates the multinomial vector to a vector of Independent Poisson
random variables. This method Is commonly called Polssonlzatlon. Even In our
Polssonlzatlon, we choose to rely on Inequalities because only Inequalities will
INTRODUCTION 5

help us In the assessment of the expected time performance for particular values
of n.
The point Is that we do not wish to gIve an exhaustIve description of known
results In the field, or to present a list of exotic applications. We start very
slowly on standard problems such as one-dImensional sorting and searchIng, and
wlll move on to multidimensional applications towards the end of the notes.
These applications are In the areas of computatIonal geometry, operatIons
research (e.g. the traveling salesman problem) and pattern recognItIon (e.g. the
all-nearest neighbor problem).
In chapter 1, we have the sImplest of all possible settings: the random vari-
ables X l' . . . ,Xn have a common density f on [0,1), and [0,1) Is dIvIded Into

m equal Intervals Ai = [ --;;;-' i)

i-I -;;;- ,1 ~ i ~ m. We are concerned with the
sImplest possIble measures of performance In searchIng and sortIng such as the
average successful search time (called Ds) and the number of element comparis-
ons for sorting (called C). If m = n, and f Is uniform on [0,1), then each
Interval receIves on the average one data point. It Is well-known that
E(Ds) = 0(1) and E(C) = O(n) In that case. It Is also known that the den-
sity f alIects the dIstribution of quantItIes such as Ds and C. We wlll see that
E (Ds ) ,....., 1 + 2:.. Jf J
2 and E (C) ,....., !!:. f 2 as n -+00. The factor f 2, whIch Is J
2 2
a measure of the peakedness of the densIty f , alIects the performance In a
J
dramatic way. For example, when f 2 = 00, we have E(C)ln -+ 00 and
E (Ds) -+ 00 as n -+ 00. In other words, bucket sorting takes linear expected
tIme If and only If f 2 < 00. J
While most users wlll be quite satisfied wIth Information about E (C), some
may doubt whether the expected value Is a good measure of the state of alIalrs.
After all, E (C) Is an estimate of the time taken per sort If averaged over a large
number of sorts. The actual value of C for one Individual sort could be far away
from Its mean. Fortunately, this Is not the case. We will see that
C In
.
-+ 2:..
2
Jf 2 In probablilty as n -+ 00: thus, If Jf 2 < 00 , C IE (C) -+ 1 In
probablllty. For large n, even If we time only one sort, It Is unlikely that
C IE (C) Is far away from 1. Of course, similar results are valld for D s and the
other quantities.
We can take our analysIs a bit further and ask what the variation Is on ran-
dom variables such as C. In other words, how small Is C - E (C) or
Ds - E (Ds)? ThIs too Is done In chapter 1. The answer for C Is the followIng:
6 INTRODUCTION

In other words, C - E (C) Is of the order of Vn whereas E (C) Itself Is of the

order of n. Variances are used by statisticians to obtain an upper bound for

P(C-E(C) ~ E)

via the Chebyshev-Cantelll Inequality:

Var (C)
P(C-E(C) ~ E) ~
E2+ Var (C)

Sometimes, this Inequality Is very loose. When E Is large compared to Vn , there

are much better (exponential) Inequalities which provide us with a lot of
confidence and security. After all, If C Is extremely unlikely to be much larger
than E (C), then the usual worst-case analysis becomes almost meaningless.
We close chapter 1 with an attempt at reducing the dependence upon f .
The Idea Is to apply the bucket method again within each bucket. This wlll be
called double bucketing. The rather surprising result Is that double bucketing
J
works. For example, when f 2 < 00, we have

Je- f
1
E(C) ~ 2:.. < 1
n 2 0 2

The detailed analysis of chapter 1 Is well worth the effort. The development
given there can be mimicked In more complicated contexts. It would of course be
unwise to do so In these notes. Rather, from chapter 2 on, we will look at vari-
ous problems, and focus our attention on expected values only. From chapter 2
onwards, the chapters are Independent of each other, so that Interested readers
can Immediately skip to the subject of their choice.
In chapter 2, the data XII ... I Xn determine the buckets: the Interval
[min Xi ' max Xi 1 Is partitioned Into n equal Intervals. This Introduces addi-
tional dependence between the bucket cardinalities. The new factor working
against us Is the size of the tall of the distribution. Infinite tails force min Xi
and max Xi to diverge, and If the rate of divergence Is uncontrolled, we could
actually have a situation In which the sizes of the Intervals Increase with n In
some probabilistic sense. The study of E (Ds), E (C) and other quantities
requires auxiliary results from the theory of order statistics. Under some condi-
J
tions on f , Including f 2 < 00, we will for example see that
INTRODUCTION 7

E(C) "-' E( max Xi - min Xi ). If 2,

n l~i~n l~i~n

I.e. the asymptotic coeffiCient of n Is the expected range of the data (this meas-
ures the heaviness of the tall of f ) times If 2, the measure of peakedness.
Unless f vanishes outside a compact set, It Is Impossible to have
E(C)=O(n).
In chapter 3, we look at multidimensional problems In general. The appIlca-
tlons are so different that a good treatment Is only possible If we analyze

m
E g(Ni )
i=l

where g (.) Is a "work function", typically a convex positive function. The main
.result of the chapter Is that for m = n, the expected value of this sum Is 0 (n)
If and only If f has compact support, and

Ig(f)<oo

provided that g C.} Is a "nice" function. Some appIlcatlons In computational

geometry and operations research are treated In separate sections of the chapter.
In some problems, we need to have assurances that the expected worst-case
Is not bad. For example, In the simple one-dimensional bucket data structure,
the worst-case search time for a given element Is equal to the maximal cardinal-
Ity. Thus, we need to know how large

maxCNi )

Is. This quantity Is analyzed In chapter 4. If f

Is bounded on a compact set of
d log n
R ,and m =n then Its expected value Is asymptotic to . If f Is not
log log n
bounded, then Its expected value could Increase faster with n. This result Is for
example applied to Shamos' two dimensional convex hull algorithm.
8 INTRODUCTION

Figure 0.3.
Binary trie for points distributed on [0,1].

It Is sometimes Important to have bucket structures which are allowed to

grow and shrink dynamically, I.e. structures that can handle the operations Insert
and delete emclently. The essential Ingredient In such a structure Is an auxiliary
array of bucket cardinalities. One can choose to split Individual buckets once a
certain threshold value Is reached. This leads to a tree structure. If a bucket can
hold at most one element, then one obtains In fact a binary trle (Knuth, 1973).
Another strategy consists of splitting all buckets In two equl-slzed buckets simul-
taneously as soon as the global cardinality reaches a certain level. In this manner,
the number of buckets Is guaranteed to be a power of two, and by manipulating
the threshold, one can assure that the ratio of points to buckets Is a number
between 1 and 2 for example. This has the additional advantage that Individual
bucket counts are not necessary. Also, no pointers for a tree structure are needed,
since data points are kept In linked lists within buckets. This dyadic dynamic
structure Is at the basis of the extendible hash structure described and analyzed
In Fagin, Nlevergelt, Pippenger and Strong (1979), Tammlnen (1981) and FlaJolet
(1983). Tammlnen (1985) compares extendible hashing with ordinary bucketing
and various types of tries. See Tammlnen (1985) and Samet (1984) for multidi-
mensional tries. To keep these notes simple, we will not analyze any tree struc-
tures, nor will we specifically deal with dynamic bucket structures.
A last remark about the grid size m. Usually, we will choose m such that
m = m (n) ~ en for some constant e > o. (The ratio m In will be called en'
INTRODUCTION 9

so that c n -+ C as n -+ 00.) We do so because we are mainly Interested In

searching and sorting. Roughly speaking, we can expect to sort the data In time
o (n) and to search for an element In time 0 (1). If m = 0 (n), the average
number of points per Interval grows unbounded, and we cannot hope to sort the
data In time 0 (n). On the other hand, If min -+ 00, the overhead due to
housekeeping (e.g., traveling from bucket to bucket), which Is proportional to m,
and the storage requirements are both superllnear In n. Thus, there are few
situations that warrant a subllnear or superllnear choice for m .
While we do generally speaking have some control over m , the grid size, we
do not have the power to determine d, the dimension. Raw bucket algorithms
perform particularly poorly for large values of d. For example, If each axis Is cut
Into two Intervals, then the grid size Is 2d. There are problems In which 2d Is
much larger than n, the sample size. Thus, storage limitations will keep us from
creating fine mazes In large dimensions. On the other hand, If rough grids are
employed, the distribution of points Is probably more uneven, and the expected
time performance deteriorates.
1.0 CHAPTER-l

Chapter 1

ANALYSIS OF BUCKET SORTING

AND SEARCHING

1.1. EXPECTED VALUES.

In thIs chapter, f Is a densIty on [0,1], whIch Is dIvIded Into m Intervals

Aj =
i-I
[ -;;-' i)
-;; ,1~i ~m .

The quantItIes of Interest to us here are those that matter In sorting and search-
Ing. If sorting Is done by performing a selection sort within each bucket and con-
catenating the buckets, then the total number of element comparisons Is

m N j (Nj -1) 1
C = :E = -(T-n)
j=1 2 2

where, by· definition,

m
T=:E Nj 2 •
j=1
 CHAPTER 1 11

N =2
3

Figure 1.1.
Bucket structure with n=17 points, m=12 buckets.

The other work takes time proportional to m , and Is not random. Selection sort
was only chosen here for Its simplicity. It Is clear that for quadratic comparlson-
based sorting methods, we will eventually have to study T.
To search for an element present In the data, assuming that all elements are
equally likely; to be queried, takes on the average
12 CHAPTER 1

comparisons. This wlll be rererred to as the ASST (Average Successful Search

Time). Note that Ds Is a function or the Ni'S and Is thus a random variable.
To search ror an element not present In the data (I.e., an unsuccessful
search), we assume that the element queried Is Xn +1' Independent or the data
and distributed as X l' The expected number or comparisons conditional on the
data Is

m m
Du E
i=1
Ni Jf
A,
= E
i=1
Ni Pi

where only comparisons with non-empty cells In the data structure are counted.
Du wlll be called the AUST (Average Unsuccessrul Search Time), and Pi Is the
Integral or f over Ai .
The properties or this simple bucket structure ror sorting and searching have
been studied by Maclaren (lg66), Doboslewlcz (lg78) and Akl and MeIjer (lg82).
In this chapter, we wlll unravel the dependence upon f. To get a rough Idea or
the dependence, we wlll start with the expected values or the quantities defined
above.

Theorem 1.1.
Let f be an arbitrary density on [0,1]. Then, even Ir J f 2 = 00,

E(C)jn ......, _l-J

2 c
f 2;
CHAPTER 1 13

Furthermore, E(T) = o(n2), E(C) = o(n2), E(Du) = o(n) and

E (Ds ) = 0 (n ).

Density with low value for Density with high value for
square integral square integral

Figure 1.2.

Theorem 1.1 sets the stage for this paper. We see for example that
J
E (T) = 0 (n ) If and only If f 2 < 00. Thus, for hashing with chaining, f 2 J
measures to some extent the Influence of f on the data structure: It Is an Indi-
cator of the peakedness of f. In the best case (j f 2 < 00), we have linear
expected time behavior for sorting, and constant expected time behavior for
searching. This fact was flrst pointed out In Devroye and KlIncsek (lIl81). Under
stricter conditions on f (f bounded, etc.), the given expected time behavior was
established In a series of papers; see e.g. Doboslewlcz (1977), Weide (1978), Meijer
and Akl (1980) and Akl and Meijer (1982). Theorem 1.1 gives a characterization
J
of the densities with f 2 = 00 In terms of quantities that are Important In com-
puter science. It also provides us with the form of the "best .. density. Because
J J
f 2 2: (j f )2 = 1 (Jensen's Inequality), and f 2 = 1 for the uniform density
14 CHAPTER 1

on [0,1], we see that all the expected values In Theorem 1.1 are minimal for the
uniform density.
Theorem 1.1 does not give the rate of Increase of E (T) as a function of n
m
when If 2 = 00. However, even though T = E Ni 2 can reach Its maximal
i=l
value n 2 (Just set N 1=n, N 2=··· =Nm=o), we have E(T) = o(n2) for all
densities f. Thus, hashing with chaining when used for even the most peaked
density, must dramatically Improve the expected time for sorting and searching
when n Is large.

Proof of Theorem 1.1.

The proof Is based upon a fundamental Lemma that will be useful In several
places:

Lemma 1.1.

(I) max Pi = 0 (1) as m --+00.

i

m
(II) For all r > I, n r E p/::;
i=l

(III) For all r > I,

r -1
.E .E p{-l) .
m
E (nPi r ( ~) n If r, and p{ = 0 (
i=l 1=1 1=1

Proof of Lemma 1.1.

(I) follows from the absolute continuity of f , I.e. for each E > 0 we can find
a 8 > 0 such that for all sets A with dx < 8, we have I < E. If
A A
(II) follows from Jensen·s Inequality:

E
i=l
(npi r = E (.E:..m Y (m I f Y ::; (.E:..m Y E m I f
i=l A, i=l A,
r = (.E:.. Y m
m
If r
CHAPTER 1 15

(lU) follows from (II) and a small additional argument: the upper bound In

c
r- J
(II) ~ (.!.. 1 n Jr. Furthermore, by Fatou's Lemma and the Lebesgue density
.
theorem (see Lemma 5.10 for one version of this theorem), we have

E
r

lim Inf.!..
R-+OO n i=1
(nPi r = lim Inf .!.. [..E:...)
n--+oo n m
E (mJ f)'
i=1 A.

r
= 11m Inf
n --+00
.!.. [..E:...)
n m
m Jf n r (where f n (x )=mpi for x EAj )

r-l
2: lim Inf [ ..E:... )
n-+oo m
J lim Inf f n
n-+oo
r

= [ ~ ) r-l J f r (because f n -+ f for almost all x).

Note that f n Is the histogram approximation of f .

16 CHAPTER 1

f
n

A A
1 A2 Figure 1.3. 10
Density f and its histogram approximation

The second half of (III) follows from (I) and the Inequallty
m m
.E Pi r :S max Pi· .E Pi r-1.
i=1 i=1
CHAPTER 1 17

The proof of Theorem 1.1 Is simple. Observe that each Ni Is a binomial

(n ,Pi) random variable and thus E (Ni 2)=(npi )2+npi (1-Pi)' Thus,

m ) m m
E (T) = E [ i~1 Ni 2 = i~1 (n 2Pi 2 + nPi (I-Pi » = (n 2 - n) i~1Pi 2 +n
m
~ ~ (nPi)2 + n ~ .!: Jf 2 + n
i=1 c

m
by Lemma 1.1 (III). Also, by Lemma 1.1 (III), ~ Pi 2 = 0 (1), so that
i=1
E (T) = 0 (n 2). All the other statements In the Theorem follow from the rela-
tions:

1 m 1
C = - ~ (N- 2_N-) = -(T - n) .
2 i=1 I I 2

1 m 1 2 1 T
Ds = - ~ -(Ni + Ni ) = - + - ,
n i=1 2 2 2 n

and
m m
Du ~ Pi Ni (E (Du) = n ~ Pi 2) .
i=1 i=1

1.2. WEAK CONVERGENCE.

In the previous section, we obtained an asymptotic expression for E (T ).
One should not exaggerate the Importance of such a quantity unless It Is known
that T -E (T) Is usually "small". For example, If we could show that
T / E (T) -> 1 In probability, then we would be satisfied with our criterion
E (T). In addition, since T / E (T) Is closed to 1 for large n, the value of T
obtained In one particular case (I.e., run; simulation) Is probably representative of
nearly all the values that will be obtained In the future for the same n. The
main result here Is
18 CHAPTER 1

Theorem 1.2.

Let 1/ 2 < 00. Then:

TIn -+ 1+1..1/2 In probablllty ;

c

C In -+ _1_
2 c
1/ 2 In probablllty ;

Ds -+ 1 + _1_ 1/2 In probablllty ;

2 c

and

Du -+ .!.1/ 2 In probablllty .
c

The proof of the Theorem uses Poissonization to handle the fact that
N l ' . . . ,Nm are dependent random varlables. For some propertIes of the PoIs-
son dIstrIbutIon used here, we refer to section 5.1. We proceed now by extractIng
a key Lemma:

Lemma 1.2.
Let 1/2 < 00. Let Nj be PoIsson (npj) random variables 1 <i ~ m.
Then

1 m
11m 11m sup - :E E (Yj ) = 0
K-+oo 1&-+00 n j=1

where Y j Is eIther
CHAPTER 1 19

(II) E (Nj 2)P (Nj 2~K) ; or

(Ill) E (N j 2) Inp,?K '

and I Is the Indicator function.

Proof of Lemma 1.2.

It Is useful to recall a simple association Inequality: If ¢>,1/J are nondecreaslng
nonnegative functions of their arguments, and X Is an arbitrary real-valued ran-
dom variable, then E(¢>(X)1/J(X)) ~ E(¢>(X))E(1/J(X)) (see e.g. Lehmann (IQ66),
Esary, Proschan and Walkup (IQ67), and Gurland (IQ68)). For example, applied
here,

Thus, we need not consider (II). We will deal with (III) first.

~.~ E(Nj 2) Inp.?K = ~ .~ (n2pj2+npi)Inp.?K

1=1 1=1

(where f n Is the function of section 1.1)

1 m n2
< n.E (-;:;;)If 2 +(nIf)Ip.?Kln (Jensen's Inequality)
1=1 A. Ai

= 1(: f 2 + f )If.?Kmln
o

Now, n /m -+ 1/ c. Also, If. ?Km In ~ If ?Kc 12 for almost all x for which
f (x) > 0, and all n large enough (this uses the fact that f n -+ f for almost
20 CHAPTER 1

all x; see sectIon 5.3.} SInce Jf 2 < 00, we thus have by the Lebesgue dom-
Inated convergence theorem,

1 1

Um sup
n -+00
J (.!!:....
0 m
f 2+ f }lj • ?Km In ~ J ( Ie f
0
2+ f }lj ?Kc 12

and thIs can be made arbltrarlly small by choosIng K large enough.

ConsIder now (I). Let L > 0 be an arbItrary constant, dependIng upon K
only.

A sImple appllcatlon of (ll!) shows that the first term on the rlght-hand-slde has a
llmlt supermum that Is 0 (I) as L -> 00. Thus, we should choose L In such a
way that L -> 00 as K -> 00. The second term on the rlght-hand-slde Is

1
:E E j2(np,), e'" Ii!
n j =1 J?,fj{
,,2,.2+1&,; <L

< 2..( :E I}. [ :E P(,;y;}i e -..;L / j!)

n n 2 p!<L. j ~m j?VK

~ (e +0 (1» E (y z1y ?VK) (where Y Is PoIsson (VL) dIstrIbuted}

~ (e +0 (1» E (y 3 /JK) (by Chebyshev·s Inequallty)

ThIs tends to 0 as K -> 00 when we choose L = K1/4. The proof of Lemma 1.2
Is complete.
CHAPTER 1 21

Proof of Theorem 1.2.

The results for C and Ds follow from the result for T. One possible Pols-
sonlzatlon argument goes as follows: let n' = n -n 3/4 , n" = n +n 3/4. Let
N' ,N" be Poisson (n' ) and Poisson (n' , ) respectively. Let N;' be a
number of X j ' s, 1:S j :S N" belonging to A; . It Is clear that
N l' , . . . , N m ' are Independent Poisson random variables with parameters
m 12m
n" p;,l:Si:Sm. Finally, let T' = EN; ,T" = EN;" 2. For arbl-
;=1 ;=1
trary E > 0 we have

N-n' ,
Using Theorem 5.5, we have P (N' , < n) = P ( " < -n 3/4In' , )
n
n 3/ 4
:S 2 exp(-n 3/ 2/(2n' , (1+--))). Thus, for all n large enough,
n' ,

T I T ' ,
n c
J
P ( - > (HE)(l+- f 2)) :S 0 (1) + P (-,-,-
n
>
E
(H-)(H-
2 c
1
Jf 2)).

Similarly,

:S 2 exp(-n 3/2/(2+0 (l))n' ) :S P (-,- < (1--

n
T'
2 )(H-
c
E 1
Jf 2)),

all n large enough. Now, all the probabilities Involving T' and T" are 0 (1)
If both T' In' and T' , In" tend to 1+2.
c
Jf 2 In probability. Thus, the
statements about T, C and Ds are valid If we can show the statement about T
m
where T = EN; 2 and N 1> • . • • N m are Independent Poisson random varl-
;=1
abies with parameters np; , 1 :S i :S m.
First, we note that by Lemma 1.1,
22 CHAPTER 1

To show that (T -E (T »/n --+ 0 In probability (which Is all that Is left). we

could verify the conditions of the weak law of large numbers for triangular arrays
of non-Identically distributed random variables (see e.g .• Loeve (1963. p. 317».
Instead. we will proceed In a more direct fashion. We will show the stronger
result that E (I T -E (T )1)/ n --+ o. We have

m
IT-E(T)I:S I:E (Nj2-E(Nj2»IIN"_E(N")I~KI
j=1

m
+ I :E (Nj 2_E (Nj 2»IIN"_E(N")I~K I = II I + II •
j=1

m
E(III):S :E [E(Nj2IN.. ~K/2) + E(Nj2)IE(N.~~K/2+E(Nj2)P(Nj2~K/2)
j=1

m m m
II = I:E Yj I :S I:E (Yj-E(Yj))1 + I:E E(Yj)1 = III + IV •
j=1 j=1 j=1

IV = IE(I)I.
and

E(IV) :S E(III).

Now. first choose K large enough so that 11m supE (II I)/n < E • where E Is an
n-->oo
arbitrary positive Integer. (Thl1> can be done In view of Lemma 1.2.) Now. we
need only show that for every K. E (lII)/n --+ o. But this Is an Immediate
consequence of the fact that the Y j -E (Yj ) terms are Independent zero mean
bounded random variables (see e.g. section 16 of Loeve (1963».
This completes the first part of the proof of Theorem 1.2. The argument for
Du Is left as an exercise: first. argue again by Poissonization that It sumces to
CHAPTER! 23

consider Independent Ni's that are Poisson (nPi) distributed. Then note that we
m
need only show that E Pi (Ni -npi )-+0 In probablllty.
i=1

1.3. VARIANCE.
The results obtained so far are more qualltatlve than quantitative: we know
n (1+- Jf
1
now for example that E (T) grows as 2) and that IT -E (T )I/n tends
c
to 0 In probablilty and In the mean. Yet, we have not establlshed just how close
T Is to E (T). Thus, we should take our analysis a step further and get a more
reflned result. For example, we could ask how large Var (T) Is. Because of the
relations between C, Ds and T, we need only consider Var (T) as
Var (C)= Var (T )/4 and Var (Ds)= Var (T )/(4n 2). Var (Du ) Is treated
separately.

Theorem 1.3.
A. For all f , we have

where the rlght-hand-slde remains valld even If Jf 2 or Jf 3 are Inflnlte. (To

avoid 00-00, consider only the lowest bound In such situations.)
B. For all f

Here, the right-hand-side should formally be considered as 00 when either

J J
f 2 = 00 or f 3 = 00.

(J
We note that for all f , f 2)2 :s J
f 3 (Jensen's Inequallty), and that equal-
Ity Is reached for the uniform density on [0,1). Thus, once again, the uniform
24 CHAPTER 1

density mlmlnlzes the "cost", now measured In terms of variances. In fact, for
4 6
the uniform density, we have Var (Du) = 0, all n, and Var (T )=2n-4--+-
n n2
when c=l, m=n.
For the proof of Theorem 1.3, the reader should consult section 5.1 first. We
note here that the Polssonlzatlon trick of section 1.2 Is no longer of any use
because the variance Introduced by It, say, Var (T' - T) for n' =n (see nota-
tion of the proof of Theorem 1.1), grows as n, and Is thus asymptotically nonne-
gllglble.

Proof of Theorem 1.3.

Consider T first. We will repeatedly use Lemma 5.1 because N l' . . . , N m
are multinomial (n ,P l' . . . , Pm). Thus, omitting the fact that we are con-
stantly summing for i and j from 1 to m we have

E2(T) = 2:,E2(Nj ) + 2:, E (Nj 2)E (N/)

j'fj

+ 2:, [n 2(n_1)2 pj 2p/ + n 2(n-1)(PiP/+Pi 2Pj) + n 2PiPj]

i¥-j

where we used the fact that E2(Nj ) = n (n -l)Pj 2 + npj. Using various expres-
sions from Lemma 5.1, we have

E (T2) = 2:,E (Nj 4) + 2:, E (Nj 2N/)

j'fj

= 2:, [npj +7n (n -l)pj 2+6n (n -l)(n -2)pj 3+ n (n -l)(n -2)(n -3)pj 4J

+ 2:, [n (n -l)(n -2)(n -3)pj 2p/+ n (n -l)(n -2)(pj p/+Pj 2pj )+n (n -l)pj Pj J.
j¥-j

Because Var(T)=E(T 2)-E 2(T), we have

+ 2:, [(-4n 3+10n 2_6)pj 2p/+(-2n 2+2n )(Pj 2pj +Pj p/)+(-n )Pj Pj J.
j'fj
CHAPTER 1 25

By Lemma 1.1, we have for all constants r 2': 1, :BPi r ~ (nc )-{r-l) Jf r .

J
Thus, If f 2 < 00,

which gives us our expression. The rlght-hand-slde of this expression Is nonsense

J J
If both f 2 and f 3 are 00. In that case, note that (:BPi 2)2 ::; :BPi 3 (by
Jensen's Inequality), and that thus, because :BPj 3 = 0 (:BPi 2),

so that Var (T )/n -+ 00. This concludes the proof of the first half of Theorem
1.3.
We have E (Du ) = :BnPi 2 ~ .!.. Jf 2, and
c

E(Du 2 ) = E (:BPj2Nj 2) + E(:B Pi Pi NiNi )

ih

= :BPi 2 (nPi +n (n-1)Pi 2 ) + :B Pi 2 p/n(n-1)

ih

Thus,
26 CHAPTER 1

If Jf 3 < 00, then Var (Du ) ~ _l-cJ f 3_cJ f 2)2).

nc 2
If

J J
f 3 = 00 but f 2 < 00, this Is still true. If both Integrals are Infinite, we need
an additional argument. For example, let J be the collection of Indices for which
Pi > a 1m, where a > 0 Is a constant. We have, by the Inequality
(u +V)2 ~ 2u 2+2v 2,

Var (Du) ~ nEpi 3 -2n (Epi 2)2 + nEpi 3_2n (Epi 2)2
J J J' J'

where JC Is the complement of J. By Jensen's Inequality,

EPi 3 EPi ~ (Epi 2)2 ,

J J J

and similarly for JC. Thus, we have

Var (Du ) ~ n (EPi 2)2((Epi r 1 -2) + n (EPi 2)2((Epi r 1 -2).

J J J' J'

It Is a simple exercise to show that mEpi 2 --+ J f 2, EPi --+ J f,

J' f ~a J' f ~a
EPi --+ J
f, mE Pi 2 --+ 00. For any choice of a with f J E (0,1), we have
J f >a J f >a
thus n Var (Du ) --+ 00.

1.4. LARGE DEVIATION INEQUALITIES.

Often, one would llke to know the likelihood of the event [C > x 1 (or of
[Ds > x 1 or [Du > x D, and In the absence of an exact answer, good upper
bounds for the corresponding probablllties P (C > x), P (Ds > x) and
P (Du > x) will do too. For the sake of simpliCity, we will derive such upper
bounds for P (Du > x). The analysis for C and Ds Is considerably more com-
plicated.
FIrst, we observe that there Is little hope to get a small bound unless x
exceeds E (Du ) ~ -J
c
1
f 2. Thus, we will ask for upper bounds for the proba-
n
blllty
CHAPTER 1 27

P(D u > _l-J

C
f 2 (HE)), E > O.
n

From Markov's Inequality and Theorem 1.1, we have

P(D u > _l-J f 2 (HE)) ::; E (Du ) <

Cn _l-J
c
f 2 (HE)
n

valid for all f. Unfortunately, thIs bound requIres large values of E to be useful.
By restrIctIng ourselves to smaller classes of densItIes, we can obtaIn smaller
upper bounds.
For example, by the Chebyshev-Cantelll Inequality and E (Du) ::; C n -1 Jf 2,

we have

P (DU 2: (HE)C n -1 Jf 2) ::; P (DU-E (DU ) 2: EC n -1 Jf 2)

::; Var (DU )/( Var (DU )+E2(j f 2)2C n -2)

If Jf 2 < The upper bound Is obvIously useless when f 3 = 00. When

00. J
J f 3 < 00, It decreases wIth n for every E > O. Unfortunately, the decrease Is
only as lin. Better rates can be obtaIned at the expense of strIcter condItIons
on f. For example, we can hope to obtaIn bounds that decrease as (n E2 r for r
J
arbItrary r > 1 provIded that f P < 00 for an approprIately bIg constant p .
J
The condItIons f P < 00, p > 1, are condItIons restrIcting the size of the
InfinIte peaks of f. The strongest possIble peak condItIon Is .. f ::; C for some
constant C'. In that case, we can obtaIn an exponentIal Inequality:
28 CHAPTER 1

Theorem 1.4.
Assume that sup f ~ C < 00. For all € > 0, we have

where

A (E) = sup
r >0
rEI f 2 - -I f 3 e rC
r2

2
> o.

In particular, If € = En varies with n In such a way that En ! 0, then

and If En i 00, then

Proof of Theorem 1.4.

The proof Is based upon ChernotJ·s bounding technique and a simple expres-
sion for the moment generating function of the multinomial distribution (see
Lemma 5.2). Let t > 0 be an arbitrary number. Then

P(D u

~ E (exp(-t (HE)_l_I f 2 +t ~ Ni Pi ))
Cn i=l

1 m
= exp(-t-J f 2(H€)) ( E Pi exp(tpi ))n .
Cn i=l
CHAPTER 1 29

Let us recall the definition of the function f n from Lemma 1.1. Using the fact
2
that e U -1 :S u + .!... e U for u > 0, we have the following chain of equalities
2
and Inequalities (where the first expression Is equal to the last expression of the
chain given above):

exp(-tc n -1(1+E)J f 2). (j f n exp(...!.. f n )dx)n

m

t t2 t
:S exp(-tc n -1(1+E)J f 2), (1+-
m
Jf n 2 + --J f n 3 exp( - f n ))n
2m2 m

t2 t
:S exp(-tc n -IE Jf + n--J
2
2m2
f 3 exp( -
m
C)).

Here we used the Inequality (1+u):S exp(u), and the fact that
also
Jf n :S Jf
8 for all 8 ~ 1 (Lemma 1.1). The first half of the Theorem follows
8

from the choice t = rm. Now, as E L 0, we see that the supremum Is reached
for r =r (E) > 0, and that A (E) Is asymptotic to the value sup r EJ f 2__ r2
r >0
1
2
Jf 3.
The latter supremum, for each € > 0, Is reached for r = € Jf 21 Jf 3. Resubstl-
tutlon gives the desired solution, A (€) ~ -€2(J
2
1
f 2)21 Jf 3.

When € i 00, It Is easy to see that the supremem In the expression for A (€)
Is reached for r (E) i 00. By standard functional Iterations, applied to the equa-
tion r (E)= ~ log( E Jf 2/(r (E)J f 3)), we see that A (E) ~ the value of the expres-
sion to be optimized, at r = ~IOg(EJ f 2/(j f 3 ~IOgE))' which gives us our solu-
tion.
30 CHAPTER 1

Remark.
The Inequality of Theorem 1.4 for fn ! 0, n fn 2 i 00, Is called a moderate
deviation inequality. It provides us with good Information about the tall of
the distribution of Du for values of the order of magnitude of the mean of Du
plus a few standard deviations of Du. On the other hand, when fn Is constant
or tends to 00, we have large deviation inequalities. As a rule, these should
give good Information about the extreme tall of the distribution, where the cen-
tral limit theorem Is hardly at work. For example, It appears from the form of
the Inequality that the extreme tall of Du drops off at the rate of the tall of the
Poisson distribution.

1.5. DOUBLE BUCKETING.

The results that we have obtained until now qualifY the statement that T Is
close to n (1+2.
c
Jf 2) when Jf 2 < 00. The presence of Jf 2 In this expression Is
disappointing. Perhaps we could hope to reduce the direct Influence of f on the
quantities that are of Interest to us by hashing the n Intervals a second time:
each Interval Ai Is subdivided Into Ni equal subintervals. This method will be
referred to as the "double bucketing" method. The Idea of double bucketing Is
obviously not novel (see for example Maclaren, 1966). In fact, we could keep on
dividing Intervals until all data points are In separate Intervals. The structure
thus obtained Is called an N-tree (Ehrlich (1982), Tammlnen (1982)). Some
analysis for restricted classes of densities Is given In these papers. Recursive
bucketing when applied to sorting Is analyzed In Doboslewlcz (1978) and Van
Dam, Frenk and Rlnnooy Kan (1983).
What we will try to show here Is that most of the beneflts of recursive buck-
eting are obtained after two passes, I.e. with double bucketing. The structure
that we will analyze Is obtained as follows:

Step 1.
- i-I i
Let Ai =[--,-), 1
n n
i :s :s
n. For each Ai' keep a lined list of XJ.I s fai-
ling In It. Let Ni be the cardinality of Ai .
Step 2.
For i = 1 to n do : If Ni ~ 1, divide Ai Into Ni equal Intervals A ij , and
keep for each Aij linked lists of the data pOints In It. Let Nij be the cardi-
nality of Aij •
CHAPTER 1 31

Double bucket structure.

n=17 data points (€i»)
6 original buckets
bucket with cardinality N .divided into N. inteNals
I I

Figure 1.4.

The quantItIes that we will consIder here are

n N.
T :E:E N i/ ,
i=1 j=1

and
n N.
Du = :E :E Pij N jj
i=1 j=1

N.
where all the summatIons :E for N j = 0 must be omItted, and
j=1
Pij = J f when Aij Is defined. We note that the first dIvIsIon Is Into n
A.}
32 CHAPTER 1

Intervals. The generalization towards a division Into m Intervals Is stralghfor-

ward.

Theorem 1.5.
If J1 2 < 00, then the double bucketing structure gives

1 1 1

E(T)/n ~l+Je-f ;E(C)/n ~ ~ Je- f ;E(Ds)~..!..(2+Je-f)

o 0 2 0

and

E(Du) -+ 1 .

If we compare these asymptotic expressions with those for ordinary bucket-

Ing when m =n, I.e. E (T )/n ~ 1+J 1 2, we see that double bucketing Is
strictly better for all I. This follows from Jensen's Inequality and the fact that
e -II <
-
1-u +.!..u 2:
2

1 1

JI2 > ~J12 ~ J e- f ~ exp(-JI)=..!...

o 0 e

For all f with Jf 2 < 00, we have

11m E(T) E [1+..!.., 2) .

n-oo n e

Thus, the limit of E (T)/ n Is uniformly bounded over all such I. In other
words, double bucketing has the effect of eliminating all peaks In densities with
J f 2 < 00. Let us also note In passing that the lower bound for E (T )/ n Is
reached for the uniform density on [0,1], and that the upper bound can be
approached by considering densities that are uniform on [0,1], and that the upper
bound can be approached by considering densities that are uniform on
CHAPTER 1 33

1
[0, ~ I cJ e - f =
1- ~ + ~ e -K) and letting K ---+ 00. The fact that the proper-
o
ties of the double bucketing structure are basically Independent of the density 1
was observed Independently by Tammlnen (1985). The same Is a fortiori true for
N -trees (Ehrlich (1981), Van Dam, Frenk and Rlnnooy Kan (1983), Tammlnen
(1983)).

Proof of Theorem 1.5.

N,
In the proof, all summations E
for which N j = 0 should be omitted, to
j=1
avoid trivialities. We start with a lower bound for E (T ).

n N,
E (T) = E E (IN,:;::1 E [(Nj 2_Nj )(pjj Ipj )2 + N j pjj Ipj ))
j=1 j=1
n n N,
E E (Nj ) + E E «Nj 2_Nj ) E (pjj Ipj )2)
j=1 j=1 j=1

n
= n + EE«Nj -l)+) (where u+=max(u,O))
j=1
n n
=n + EE(Nj -l) + EP(Nj=O)
j=1 j=1
n
= n + E P (Nj =0)
j=1
n
= n + E (l_pj)n (where Pj = J1 )
j=1 A,

n
~ n + E exp(-npj 1(I-Pj)) (because l-u ~exp(-u 1(I-u )), O::;u <1)
j=1
1

= n +n JexP(-1 n 1(1-1 n In)) (where 1n (x)=npj' xEAj)

o
1
~n+nJe-f
o
34 CHAPTER 1

by the Lebesgue domInated convergence theorem and Lemma 5.10.

We now derIve an upper bound for E (T). For any Integer K. we have

n n
E(T)=n + EE(V/)+ EE(V;")
;=1 ;=1

where
N,
V;' = (N; 2_N; ) E (p;j /p;)2 IN, ~K
j=1

and
N,
v·"I = (N; 2_N;) E (p;j /p;)2 IN, >K.
j=1

The statements about E (T). E (C) and E (Ds ) In Theorem 1.5 are proved If we
can show that

!lm !lm -
K-+oo n-+oo n ;=1
1
E
n ,
E (Vi ) = J0 e- I
1 n ,
E
I
!lm !lm sup - E ( V; ) = O.
K-+oo n-+oo n ;=1

We wlll use the functIon gn (x) = E (V;' ). X EA;. Clearly.

gn (x) ~ K E (N; ) = Knp; = Kf n (x) • x EA; •

Jf n = 1. all n ; f n -+ f almost all x.

Thus. by an extended versIon of the Lebesgue domInated convergence theorem

(see e.g. Royden (1968. p. 89)). we have

n 1
!lm
n -+00
1.. E E (V;' ) =
n ;=1
!lm Jgn
n -+00 0
= J0 n-+oo
!lm gn
CHAPTER 1 35

provIded that the limIt of gn exIsts almost everywhere. ConsIder now a sequence
of couples (i ,j) such that x EAjj 5;Aj for all n. We have by Lemma 5.11,
nNj pjj -+ f (x) for almost all x, unIformly In N j , 1 Nj K. From thls,we :s :s
conclude that

gn (x) "" E ((Nj -l)+IN , ~K ) , almost all x.

ConsIder only those x I s for whIch f (x) > 0, and Lemma 5.11 applies.
Clearly, N j tends In dIstrIbutIon to Z where Z Is a PoIsson (f (x)) random varI-
able (thIs follows from npj -+ f (x) (Chow and TeIcher (1978, p. 36-37))). SInce
(Nj -1)+IN , ~K forms a sequence of bounded random varIables, we also have con-
vergence of the moments, and thus,

for all such x , I.e. for almost all x (f). Thus,

11m 11m ..!..E(Vj ') = 11m J(f (x)-1+e-f(xLE((Z-l)+Iz>K)) dx

K -+()() n -+()() n K -+()() 0

Here we needed the fact that 11m

K -+()()o
JE ((Z -l)+Iz >K ) dx = 0, whIch Is a sImple

consequence of the Lebesgue domInated convergence theorem (note that

1

J E (Z )dx = .1). Also,

o

Define the functIon hn (x) = E (Nj 2 IN, >K ), x EA j , and the functIon
h (x) = E (Z2 I Z >K) where Z Is PoIsson (f (x)) dIstrIbuted. We know that
hn (x) :s
E (Nj 2) :s
npj + (npj)2 = f n (x)+ f n 2(X) -+ f (x) + f 2(x), almost
J J
all x; and that f n + f n 2 -+ f + f 2. Thus, by an extensIon of the Lebesgue
domInated convergence theorem, we have
36 CHAPTER 1

provided that the almost everywhere limit of hn exists. For almost all x, Ni
tends In distribution to Z. Thus, for such x ,

00
Ih n -h 1< :E j 2 1P(Ni =n-p (Z =j)1 --+ 0
j=1

(see e.g. Simons and Johnson, 1971). But I h --+ 0 as K --+ 00 since
o
1 1

IE (Z2) = If +f 2 < 00, and E (Z2JZ >K )--+0 for almost all x. This concludes
o 0
the proof of

1 n
:E E (Vi
I I
11m sup 11m sup - ) = O.
K --+00 n --+00 n i =1

We will only sketch the proof for

n N, n N,
E (Du )=E (:E :E Pij N ij ) = E ( :E Pi Ni :E (Pij /Pi )2) .
i=1 j=1 i=1 j=1

First, It Is easily seen that

n n
E (Du) ~ E ( :E Pi N,./ Ni ) = :E Pi = 1.
i=1 i=1

Also, If we follow the treatment to obtain an upper bound for E (T), we come
I I 2
across terms Vi and Vi I In which (Ni -Ni ) Is now replaced by Pi N i . Mim-
Icking the Poisson approximation arguments for E (T ), we obtain
11m sup E (Du ) ~ 1 when If 2 < 00. This concludes the proof of Theorem 1.5.
n --+00
CHAPTER 2 37

Chapter 2

DENSITIES WITH UNBOUNDED SUPPORT

2.1. MAIN RESULTS.

In chapter 1, we have analyzed In some detail what happens when f Is
known to have support contained In [0,1]. In first approximation, the main term
In the asymptotic expressions for E (T )/ nand E (D u) contain the factor f 2,J
which Is scale-dependent. If we were to divide the Interval [Mn ,Mn '] =
[min Xi ,max Xi] Into m equal-sized sub-Intervals, these expected values would
obviously not be scale-dependent because the distribution of N 11 . . . ,Nm Is
scale Invariant.
We could repeat all of chapter 1 for this more general setting If tedium was
no deterrent. There Is a new Ingredient however when f has Infinite tails
because Mn and / or Mn' diverges In those cases. The results In this chapter
rely heavily on some results from the theory of order statistics. The technicalities
are deferred to section 2.2. The following notation will be Introduced:

Mn = min Xi'
l::;i::;n

Pi = Jf ,1~i ~m ,
x,

M••

P Jf
M.
38 CHAPTER 2

s = ess sup XI - ess Inf XI = wIdth of support of f·

Area p.
I
• :13 points X.
I

4 buckets

Figure 2.1.

Theorem 2.1.
Let f be a densIty on R 1 wIth 1/ 2 < 00. Then

C\) 11m Inf 11m Inf _ _ _ _ _E--'-CT-:.-)- - - - > 1

6LO n-+oo (n(I+E(mln(R n cn - 1I/2,8n)))

and

(II) lImsup E(T) :SI.

n -+00 n (HE (mln(R n Cn -11/ 2 ,n )))

In partIcular, If s < 00, we have

CHAPTER 2 39

Um lnf E (T) = Um sup E (T) = 1 + 8 .!.. I f 2.

" -+00 n " -+00 n e

Theorem 2.1 shows that there ls a close relatlon between E (T) and the
range R". For densltles wlth no talls, we have a generaUzatlon of Theorem 1.1.
It ls noteworthy that 1 +.!.
e
If
2, the Umlt value of E (T)/ n, ls scale lnvarlant.

When 8 = 00, It ls not clear at all how E (mln(R" e" -11 f 2,n )) varles wlth n.
For example, ls thls quantlty close to E (R" )e" -11 f 2 (whlch ls easler to handle)?
Thus, to apply Theorem 2.1 In concrete examples, some results are needed for
R". Some of these are stated In Lemma 2.1.
We wlll work wlth the followlng quantltles: X = X 1 has densIty f and dIs-
trIbutIon functIon F (x) = P (X ::; x) = I-G (x); the Integrals

00

F (x) = I F (t )dt ; 0 (x) = I G (t )dt

-00

wlll also be useful. We recall that

00 o
E(IXD = o (O)+F (0) = IG(t)dt + I F(t)dt
o -00

Lemma 2.1
Let fI > 0 be arbItrary. Then:

(1) E (mln(R" ,fin ))j.

(11) 11m sup E (mln(R" fin)) < 00 If and only If 8 < 00 .
n-+oo .
(lU) 11m sup E (R" ) < 00 If and only If 8 < 00 .
"-+00

(Iv) E (R" ) = 00 for all n ~ 2 If and only If E (R" ) = 00 for some n > 2
If and only If E (IX D= 00.

(v) E (IX D< 00 Imp11es E (R" ) = 0 (n ).

40 CHAPTER 2

(vi) E (IX D< 00,

11m 11m Inf

G (ax)
~~..!.... = 00
o
(- = 00)
a Lo % -+00 G (x ) o
and

11m 11m Inf F (-ax) = 00

aLo %-+00 F(-x)
Imply
E (mln(Rn ,8n » ,....., E (Rn ) for all 8 > o.
(VII) Are equivalent:
11m sup E (mln(Rn ,8n »/n > 0 for all 8 > 0 ;
n-+oo

11m sup E (mln(Rn ,8n »/n > 0 for some 8 > 0 ;

n-+oo

11m sup
%-+00
Ix IP (IX I > x) > o.
(vIII) Are equivalent:
11m Inf E (mln(R n ,8n »/n > 0 for all 8 > 0 ;
n-+oo

11m Inf E (mln(Rn ,8n »/n > 0 for some 8 > 0 ;

n-+oo

11m Inf
%-+00
Ix IP(IXI > x) > o.

Lemma 2.1 In conjunction with Theorem 2.1 gives us quite a bit of Informa-
tion about E (T). For example, we have

The-orem 2.2.
If J/ 2 < 00, then are equivalent:

11m Inf E(T)/n < 00;

n-+oo

11m sup E(T)/n < 00;

n-+oo

8 < 00 .
CHAPTER 2 41

(And If s < 00, thIs 11m Inf Is equal to thIs 11m sup. Its value 1+~ Jf 2.)
C

Theorem 2.2 follows from Lemma 2.1 (I), (II) and Theorem 2.1. In Devroye
and Kllncsek (lQSO), one finds a slightly stronger result: E(T)=O(n) If and only
J
If s < 00 and f 2 < 00. In the next chapter, thIs will be generalized to R d , so
we don't have to bother wIth an R 1 versIon of It here.
We also have

Theorem 2.3.
If Jf 2 < 00, then condItIon (vI) of Lemma 2.1 Implies that

Theorems 2.2 and 2.3 cover all the small-tailed dIstrIbutIons wIth little oscil-
lation In the tails. In Akl and MeIjer (lQS2) the upper bound part of Theorem
2.3 was obtaIned for bounded densItIes. The actual limItIng expressIon of E (T)
shows the InteractIon between the effect of the peaks cJ
f 2) and the effect of the
)J
tails (E (Rn )). Note that E (Rn f 2 Is a scale-InvarIant and translatlon-
InvarIant quantIty: It Is solely determIned by the shape of the densIty. It Is
perhaps InterestIng to see when condItIon (vI) of Lemma 2.1 Is valid.

Example 2.1. (RelatIvely stable dIstrIbutIons.)

A relatIvely stable dIstrIbutIon Is one for whIch

(I) 11m G(ax) = 00, all aE(O,l); (.Q.=oo);

x --->00 G (x) 0

and

(II) 11m F (-ax) = 00 , all a E(O,l) .

x--->oo F (-x)
42 CHAPTER 2

If we use the notation Mn * = max(X 1+' . . . I Xn +) where u + =max( u ,0) then

It should be noted that If P (X > 0) > 0, (I) Is equivalent to

(III) Mn * -+ 1 In probability for some sequence an

1
(Gnedenko, 1943). In that case we can take an = Inf(x: G (x):S -) where
n
G (x )=P (X ~ x), or In short, an = G- I(..!.) (Dehaan, 1975, pp. 117). We note
n
that (I) Is equivalent to G (0) < 00, G (x )/(xG (x)) -+ 0 as x -+ 00; or to
00

G (0) < 00, JtdF (t )/(xG (x)) -+ 1 as x -+ 00.

x

For relatively stable distributions, we have E (R ) ~ F -I(..!. )+ G -I(..!.)

n n n
(Plckands, 1968). It Is very easy to check that condition (Vi) follows from the
relative stability of the distribution of X. When

(Iv) 11m xl (x) = 00 ,

x--+oo G(x)

we know that (III) holds (GetTroy, 1958; Dehaan, 1975, Theorem 2.9.2). condition
(Iv) comes close to being best possible because If I Is nonlncreaslng and positive
for all x, then (III) Implies (Iv) (Dehaan, 1975, Theorem 2.9.2).

Example 2.2. (Normal distribution.)

For the normal distribution with density (21l't i / 2 exp(-x 2 /2), we have rela-
tive stability and square Integrability. In particular,
E (Rn ) ~ 2G- I £) ~ 2V2 log n (see e.g. Galambos, 1978, pp. 65), and thus
n

E(T) ~n (H2V2log n J12) = n (1 +V! log n ) ~ ~ n Vlog n .

Example 2.3. (Exponential distribution.)

For density f (x) = e -x, x > 0, we have relative stability and square
Integrability. Thus, because E (Rn ) ~ log n,
CHAPTER 2 43

E(T) '" n(1+ log n I 12) '" 1..2 n log n .

Example 2.4. (Regularly varying distribution functions.)

Condition (vi) of Lemma 2.1 Is satisfied for all distributions for which
(I) G (X) = 0 for all x large enough; or G Is regularly varying with
coemclent p < 0 (I.e., G (ax )/G (x) -+ a P for all a > 0 as x -+ 00).
(U) F (x) = 0 for all x large enough; or F Is regularly varying with
coemclent p < 0 (I.e., F (ax)lF (x) -+ a P for all a > 0 as x -+ 00).
In (I) and (U) we can replace the functions G and F by G and F If we wish pro-
vided that we add the condition that the coemclent of regular variation be
p < -1. The latter fact follows from the observation that as x -+ 00,
G (x) '" xG (x )/(-p-l) (Dehaan, 1975, Theorem 1.2.1).

Example 2.5. (Upper bounds for E (RII ).)

One of the by-products of Theorem 2.1 Is that

E(T)
11m sup - - _ ................_ - ~ 1.
11-+00 n E (R
II
)1..c II 2

Thus, good upper bounds for E (R II ) give us good upper bounds for E (T )/ n .
For example, we have

E (R II ) ~ E (m~ Xi + - m~n Xi -)
I I

~ E l/ r (m~ x,. +') + E l / r «-m!n Xi-Y) ,all r ~ 1 ,

I I

Thus, depending upon the heaviness of the taU of X, we obtain upper bounds for
E (T ). that Increase as n Hl/r. We can do better when the moment generating
function of X Is finite In a neighborhood of the origin, I.e.

E (e t IXI) < 00 , for all t In some Interval [O,E).

44 CHAPTER 2

Since u r ~ ( e r t) e ttl ,u 2:: O,we have

E(Rn) ~ :rtn1/r El/r(e tIXI )

log n log(E (e t IXI))

= 2 - - exp( ) ,for such t , and all n > e,
t log n

where we took r = log n. For the t ' 8 In the Interval [O,f), we have as
n -+ 00,

E (Rn ) ~ (2+0 (1)) IO~ n.

Thus, the best result Is obtained by setting t equal to Eo In particular, If

E(e tIXI ) < 00 for all t > 0 (such as for the normal density), then

E (Rn ) = 0 (log n ),

and thus

E(T) = o(n log n).

Theorem 2.2 treats densities with compact support, while Theorem 2.3 cov-
ers quite a few densities with finite moment. We will now skip over some densi-
ties In a gray area: some have a finite first moment but do not satisfy (vi) of
Lemma 2.1, and some have Infinite first moment E (IX I), but have relatively
small tails. The worst densities are described In Theorem 2.4:
CHAPTER 2 45

Theorem 2.4.
Let Jf 2 < 00. Then

(1) I1msupE(T)/n 2
n -t>00
> IxIP(IXI > x) > 0;
o If and only If 11m sup
Z -tooo
(11) 11m Int E (T )/n 2 > 0 If and only If 11m Int Ix IP (IX I > x) > 0;
n~oo z~oo

(lU)E(T) = o(n2) If and only If 11m sup IxIP(IXI > x)=O;

%-+00

(Note that T:S n 2 for all densities, and that statement (1) Impl1es
E(IXI> = 00.)

Thus, the Cauchy density f (x )=..!..(1+X 2

1r
t 1 , which satlsfl.es (11), must have
E (T) 2: en 2 for some positive constant e. If we compare Theorem 2.4 with the
results of chapter 1, we notice that heavy taUs are much more of a nuisance than
Infl.nlte peaks: Indeed, regardless of which f Is chosen on [0,1], we have
E (T) = 0 (n 2); but even moderately taUed densities can lead to a lower bound
for E (T) of the form en 2. Let us also point out that there are densIties with
E (IX I> = 00 for all n, but E (mIn(Rn ,on» = 0 (n) for all 0 > 0: just take
F (x) = l-l/«l+X) log (x +e x >0. »,
We conclude this section by noting that

E(D s ) ....... E(T)/(2n) + 1/2,

E(C) ....... E(~)-n ....... ~ n2 .f. (pdp)2,

1=1

and
m
E(D u ) ....... E(n E Pi 2/P)·
i=1

Nearly all that was saId about E(T) remaIns easUy extendIble to E(C), E(Ds)
and E (D u ). For example, If s < 00,

E (Ds ) ....... 1 + ..!- Jf 2 ,

2e
46 CHAPTER 2

and

1
If 8 = 00, we have E(C)"""" E(Ds ),......, E(T)/(2n) and E(D u )"""" E(T)/n.

We finally note that the quantity 8 Jf 2 Is scale Invariant and that for all
densities It Is at least equal to 1, In view of

1 = ( J f )2 ~ J f 2 J dx = 8J f 2 .
support of f support of f

2.2. PROOFS.

Proof of Lemma 2.1.

Fact (I) Is trivial. For fact (II), we note that If 8 = 00, we have Rn -+ 00
almost surely, and thus, 11m Inf E (mln(R n ,on)) 2': E (11m Inf mln(Rn ,on )) = 00.
n~oo n~oo

Also, In all cases, 8 2': Rn ' and we are done. Fact (m) Is proved as (II).
For Item (Iv), we note that E (Rn ) ~ 2nE (IX D, that E (Rn) i and that
E (R 2) = E (IX eX 2D2': Inf E (IX -x D= 00 when E (IX I) = 00.
x
To show (v), It suffices to prove that E (max( I X I I ! • • '! I Xn I)) = 0 (n ).
Let IXII have distributed function F on [0,00). Then for all E > 0,

00

E(max(IXIL·· '! IXn I)) = J1-(1-F(x))n dx

a
00 00

~ nE + J(l-(l-F(x))n)dx ~ nE +nJ F(x)dx = nE + o(n),

nf

and we are done.

CHAPTER 2 47

We wlll now prove (vI). SInce mIn(Rn ,Em) ~ Rn ' we need only show that
Um Inf E (mIn(Rn ,on ))/ E (Rn)~ 1 for all 0 > o. Let us define x +=max(x ,0),
n--+oo
x-=mln(x,O), R+=max(X 1+, ... , Xn +), R-=mln(X 1-, • . . , Xn -). We will
show that E (Rn -mIn(R n ,On ))/E (Rn ) -+ 0 for all 0 > 0 and all nondegenerate
dIstrIbutIon wIth 8 = 00 ( for otherwise, the statement Is trIvIally true). Clearly,
It sumces to show that for all 0 > 0, E (R +-mineR +,on ))/E (Rn) -+ o. If x+
has finIte support, we see that this follows from (ll). Thus, we need only consIder
the case that X+ has Infinite support. Now, E(Rn) ~ E«R+-X)IR+>o)
00

o
00 00

~ J1-(1-G (t)? dt. Also, E (R +-mineR +,on)) = J1-(1-G (t nn dt. We have

o h
reduced the problem to that of showIng that for all 0 > 0,

00 00

J1-(1-G(t))n dt / J1-(1-G(t))n dt -+ O.
on 0

We wlll need the following InequaUty:

1
2"mIn(nu ,I) ~ l-(l-U)n ~ mln(nu ,I) , all n ~ I, u E [0,1].

ThIs follows from 1-nu ~ (l-U)n ~ e- nll ; e-I < ~

- 2
for t >
-
1; and

e -I <
-
1-1. for t E[O,l].
2
Thus, If an = Inf(x:G (x) ~ ~)
n
and n Is so large that
an > 0, we have

00 00

~ ~ J1-(1-G(t))ndt / (an +nJ G(t)dt ) ~ 1.

o a.

Thus, we need only show that

00 00

n JG(t) dt / (an +nJ G(t)dt)-+ 0, all 0 > O.

on a.
48 CHAPTER 2

00 00

By our assumption In (vi), we have JG (t )dt I JG (t )dt -+ 00 when an In r--+ 0

a. on
00

(and this In turn follows of course from the fact that JG (t )dt < 00 Implies
o
tG (t) -+ 0 as t -+ 00). This concludes the proof of (VI).
We will now prove (vII) and (vIII) for R + and 11m sup (or 11m Inr)
x -+00 X --+00
xG (x) > O. The extension of the result to Rn Is left as an exercise. For
E E (0,6) we have the following chains of Inequalities:

on €n On
1..E (mln(R + ,8n)) = 1.. J l-(l-G (t))n dt = 1..( J + J )
n non 0 En

on
:S 1.. (w + n J G (t )dt ) :S € + 8nG (w) = € + i w G (w) ;
n En €

and

on
.!..E(mln(R + ,8n)) 2:.!.. J l_e-nG(t) dt 2: (l_e-nG(on»).
n n 0

This proves that 11m sup xG (x) > 0 Is equivalent to

Z --+00
11m sup E (mln(R + ,8n ))1 n > 0 for all 8 > 0 or for some 8 > 0; and that similar
n --+00
statements are true for the limit Infimum. This concludes the proof of Lemma
2.1.

We are left with the proof of Theorem 2.1. This wlll be taken care of In
small steps. From the observation that conditional on Mn, Mn *, the Ni 's are
binomially distributed with parameters n -2, Pi Ip , we deduce the following:
CHAPTER 2 49

Lemma 2.2.
(I) T < n 2.

m
(III) E (T IMn ,Mn *) ~ (n _2)2 :E p; 2 .
;=1

Proof of Lemma 2.2.

Part (I) Is obviously true. Parts (II) and (III) follow from

m
= n-2 + [(n-2)2-(n-2)) :E (p;/p)2
;=1

and the fact that

m m ~B ~.
:E p; 2 = :E (J f I(Rn 1m ))2(Rn 1m? ~ (Rn 1m) J f 2 .
;=1 ;=1 XI M"

Proof of Theorem 2.1 (i)

We start from Lemma 2.2 (Ill). Let {) > 0 be a sufficiently small number.
Then

m m
:E p; 2 = :E (X; +CX; )( J f I(x; +I-X; )?
;=1 ;=1 x,
50 CHAPTER 2

II
(where f (a ,x) = Inf Jf IIY-z D
z ~:.:ill z

M. '
= 1.... R n J f 2(Rn 1m ,x).
m M.

A(6) 8 00 8
Find values A (8) and A * (8) such that Jf 2 = - f 2 J f 2 = - f 2, and a
-00 3 A '(6) 3
value B (8) such that

Jf 2(a ,x) > (l-"£')J f

3
2 , all 0 < a ~ B (8).

Thus, If A Is the event [Mn < A (8), Mn * > A * (8)] and B Is the event
[Rnlm ~B(8)],wehaveonA nB,fora=Rnlm,

M. ' 00

J f 2(a ,x) ~ Jf 2(a ,x) - I f 2(a ,x) - I J2(a ,x)

M. M.' 00

Thus,

We also have

m
E Pi 2 ~ IAnB , G(c)
i=1

where

II
G (8) = sup
A (6)::;z <II <A '(6)
(Jz f )2 .
J-I..l/~
CHAPTER 2 51

Note that as 6! 0, we have B (6) -+ °and thus C (6) -+ 0. Combining these

bounds gives

EPi
i=1
2 ~ IA mln((l-6)..!...R n f
m
J 2 , C (6» = IA Z (Rn )

where Z (Rn) Is an Increasing function of Rn. By Gurland·s Inequalities (Gur-

land, 1Q68) we have E (IA Z (Rn» ~ P (A ) E (Z (Rn». We also know that
P (A ) -+ 1 for all 6 E (0,1). Thus, with a little extra manipulation we obtain the
following bound:

E(T)/n ~ (1+0(1»(1+E(mln(~Rn(1-6)Jf2, nC(6))))

n

~ (1+ 0 (1»(1+(1-6)E (mln(_l- Rn J f 2, nC (6)))) .

cn

This concludes the proof of Theorem 2.1 (I).

Proof of Theorem 2.1 (ii).

From Lemma 2.2, we have

E(TIMn,Mn *)/n ~ mln(n, 1+(R n /(c n p 2»Jf2)

~ 1 + mln(n ,(R n /(c n p2»J f 2) ,

Let us take expectations on both sides of this Inequality. For arbitrary E > Owe
have

~ E (mln(n ,(R n /cn)J f 2» + -!- J f

n
2 + n P (p < 1/v'1+€) + 1,
52 CHAPTER 2

The proof Is complete If we can show that the last probability Is 0 (1) for every
€ > o. Let U l ' U 2 be Independent uniform [0,1] random variables, and note that
p Is distributed as U ll/n U //(n-l). Thus,

~ 2 (1+€)-(n-l)/4 ,

and we are done.

2.3. A SUPERLINEAR NUMBER OF BUCKETS.

For many Infinite-tailed distributions, we know precisely how E (T) varies
asymptotically. For example, for densities covered by Theorem 2.3,

when m ~ en. We also have In those cases, by the proof of Theorem 2.1 (II),

E (T) < n (1 + 2 1 + _€- Jf 2 + E (Rn) Jf 2) ,

- (1+€)(n-l)/4 en en

for arbitrary € > o. Here en = min. When we sort, there Is an additional cost
of the form Am for some constant A > 0 due to the time needed to Initialize
and concatenate the buckets. If E (Rn ) -- 00, It Is easy to see that In the upper
bound,

E(R )
E (T) ~ n n Jf 2 (Ho (1))
en

provided that E (Rn )1 en -- 00. If we balance the two contributions to the cost
of searching with respect to m , then we will find that It Is best to let m Increase
at a faster-than-llnear pace. For example, consider the mInImIzatIon of the cost
functIon
CHAPTER 2 53

Am + n E (Rn) Jf 2 .

(.!!::)
n

The minimum Is attained at

m = n J E ~n) Jf 2 ,

and the minimal value of the cost function Is

If we had picked m ~ cn , then the main contribution to the sorting time would
have come from the selection sort, and It would have Increased as a constant
times n E (Rn). The balancing act reduces this to about n viE (Rn ), albeit at
some cost: the space requirements Increase at a superllnear rate too. Futhermore,
for the balancing to be useful, one has to have a priori Information about E (Rn ).
Let us consider a few examples. For the normal distribution, we would
optimally need

and obtain

Am ~ E (T) ~ n J \I! A log n

For the exponential distribution, we have

m ~ n .
V
1_1_
2A
log n ,

Am ~ E (T) ~ n J~ log n
54 CHAPTER 2

Similarly, for all distributions with finite fix Ir f (x)dx, f f 2(X )dx, we can
choose m such that

H...!...
Am "'-'E(T) ~ en 2r

for some constant C .

The Idea of a superllnear number of buckets to reduce the expected time can
f
also be used advantageously when f 2 = 00 and f has compact support.
When preprocessing Is allowed, as In the case of searching, and space require-
ments are no obstacle, we could choose m so large that E (Ds) and E (Du) are
both 0 (1). To Illustrate this point, we use the bound for E (T) used In the
proof of Theorem 2.1 (II), and the fact that

T 1
Ds =-+-.
2n 2

Thus, when ff 2 < 00, E (Rn ) -+ 00, we can choose

and conclude that

11m sup E (T) < 2 ,

n-+oo n -

3
11m sup E (Ds) ~ - .
n-+()() 2

We stress again that the Idea of a superllnear number of buckets seems more use-
ful In problems In which a lot of preprocessing Is allowed, such as In ordinary
searching and In data base query problems.
CHAPTER 3 55

Chapter 3

MULTIDIMENSIONAL BUCKETING.

3.1. MAIN THEOREM.

Several algorithms In computer science operate on points In R d by first stor-
Ing the points In equal-sized cells, and then traveling from cell to cell, to obtain
some solution. Often these algorithms have good expected time behavior when
the points are sufficiently smoothly distributed over Rd. This will be Illustrated
here by exhibiting necessary and sufficient conditions on the distribution of the
points for linear expected time behavior.
Our model Is as follows: X IJ • • • J Xn are Independent random vectors from
R d with common density f. We let Cn be the smallest closed rectangle cover-
Ing X IJ • • • J X n • Each side of Cn Is divided Into n' = Ln lid J equal-length
Intervals of the type [a ,b ); the rightmost Intervals are of the type [a ,b]. Let A
be the collection of all rectangles (cells) generated by taking d-fold products of
Intervals. Clearly, A has m cells where

The cells will be called A 1J . . . Am' and N j will denote the number of X j ' s
J

In cell A j • Thus, to determine all the cell memberships takes time proportional
to n. Within each cell, the data are stored In a linked list for the time being.
56 CHAPTER 3

.
. "
8 by 8 grid

. . • ,
, , , ' 64 po ints

,
I
I I ,
, •

.,
I I

It:
I I
C ell A. has N. =3 points

.
'I I
, I
I I
I I
I I
I
I

I
I I
I
I
I
I

I
I
I
I

I I

Figure 3.1.

The cell structure has been used with some success In computational
geometry (see for example, Shamos (1978), Weide (1978), Bentley, Weide and Yao
(1980), and Asano, Edahlro, Imal, Irl and Murota (1985)). Often It suIDces to
travel to each cell once and to do some work In the I-th cell that takes time
g (N; ) for some function g (or at least, Is bounded from above by ag (N; ) and
from below by bg (N;) for some appropriate constants a ,b: this sllghtly more
general formulation wlll not be pursued here for the sake of slmpllclty).
For example, one heuristic for the travellng salesman problem would be as
follows: sort the points within each cell according to their y-coordlnate, join
these points, then join all the cells that have the same x-coordinate, and finally
join all the long strips at the ends to obtain a traveling salesman path (see e.g.
Chrlstofides (1976) or Papadlmltrlou and Stelglltz (1976)). It Is clear that the
m
work here Is 0 (n) + 1: g (N;) for g (u )=u 2 or g (u )=u log(u +1) depending
;=1
CHAPTER 3 57

upon the type of sorting algorithm that Is used. The same serpentine path con-
struction Is of use In minimum-weight perfect planar matching heuristics (see e.g.
Irl, Murota, and Matsui 1981, 1983).
If we need to find the two closest points among X v ... ) Xn In [O,l]d, It
clearly sumces to consider all pairwise distances d (Xi ,Xj ) for Xi and Xj at most
ad (a constant depending upon d only) cells apart, provided that the grid Is con-
structed by cutting each side of lO,l]d Into n' =Ln 1/ d equal pieces. USing the J
Inequality (u I+U 2+ ... +Uk)2 :::; 2 -1(U /+ ..• +Uk 2), It Is not hard to see that the
m
total work here Is bounded from above by 0 (n ) plus a constant times E Ni 2.
i=1

I
"
8 by 8 grid
I
I
I
I I 64 points
I I
I

I
I I I
I I
I I
I
I I
I

I I
II I I I I
I I
I I I
I

.....
I I
I
I

""
I
I I j

.
I
I I
I
B
I I
I .....
.....
I
I I

.4~ A.j ~
Figure 3.2.
Range search problem: report all points in the
intersection of A and B. Grid to be used in solution is also shown.

For multidimensional sorting and searching, we refer to section 3.2. In sec-

tion 3.2, a few brief remarks about the point-location and point enclosure prob-
lems will be Included. The point enclosure problem can be considered as a special
case of range searching, I.e. the problem of retrieving all points satisfying certain
58 CHAPTER 3

characteristics. If for example we want to retrieve all points for which the coordi-
nates are between certain threshold values, then we can speak of an orthogonal
range query. In the survey articles of Bentley and Friedman (1g7g) and 'Asano,
Edahlro, Imal, Irl and Murota (1gS5), some comparisons between cell structures
and other structures for the range search problem are made. The range search
problem has one additional parameter, namely the number of points retrieved.
Query time Is usually measured In terms of the number of retrieved points plus a
function of n. If most queries are large, then It makes sense to consider large
cells. In other words, the cell size should not only depend upon nand f , but
also on the expected size of the query rectangle (see e.g. Bentley, Stanat and Wll-
llams, 1977). In addition, new distributions must be Introduced for the location
and size of the query rectangle, thus compllcatlng matters even further. For
these reasons, the range search problem wlll not be dealt with any further In this
collection of notes. The travellng salesman problem Is briefly dealt with In sec-
tion 3.3, and In section 3.4, we wlll look at some closest point problems In compu-
tational geometry. The latter problems differ In that the time taken by the algo-
rithm Is no longer a simple sum of an univariate function of cell cardlnalltles, but
a sum of a multivariate function of cell cardlnalltles (usually of the cardlnallty of
a central cell and the cardlnalltles of some neighboring cells). In the entire
chapter, we wlll deal with a work function g. Initially, the time of an algorithm
Is given by

m
T ~ g(Nj )
;=1

for some function g satisfYing:

(1) g Is nonnegative and g (u )/u i 00 as u i 00.

(11) g (u )/u" ! 0 as u -+ 00 for some finite constant k;

(lll) g Is convex on [0,00); g (0) = o.

Remark.
We would llke to point out that (1) and (11) Imply that g Is continuous and
that g (0)=0. Examples of functions g (.) satisfYing the llsted conditions are
g (u) = U r , some r ~ 1, and g (u) = u log(u +1).
CHAPTER 3 59

Theorem 3.1.
Let 1 be an arbitrary density on Rd. Then are equivalent:

(I) 11m Inf E (T )/n < 00 ;

n-+oo

(II) 11m sup E (T )/n < 00 ;

n-+oo

(III) Jg (f (x)) dx < 00 .

Proof of Theorem 3.I.

The proof Is In three parts:

A. 1 compact support, Jg (f ) = 00 => 11m Inf E (T )/ n

n-+oo
= 00.

B. 1 compact support, Jg (I ) < 00 => 11m Inf E (T )/n

n-+oo
< 00.

c. 1 does not have compa~t support, 11m Inf E (T )/ n = 00.

n-+oo

m
In the proof, we will use the symbols Pi J1 , C = U Ai, P = JI. The
A, i=l C
following fact will be needed a few times: given C,

Yi < N j < w,. + 2d ,1 ~ < m ,n > 2d ,

where Y j Is a binomial (n -2d ,Pi) random variable, w,. Is a binomial (n ,Pi /p)
random variable, and" <" denotes "Is stochastically smaller than", I.e.

P (Yi ~ x) ~ P (Ni ~ x) ~ P (Wi + 2d ~ x) , all x.

60
CHAPTER 3

Proof of A.
Let Co be the smallest closed rectangle covering the support of f , and let
f n (x) be the function defined by the relations: f n (x) = 0, x f. C,
fn(x) = (n-2d)p;, x E A; . We have

m m
E(T) = ~ E(g(N;)) = ~ E(E(g(N;)IC))
;=1 ;=1

m
> ~ E (E (g (Y; )1 C))
;=1

m 1
2: .~ E("2 g ((n-2d)p;-v'(n-2d)p;))
.=1
(by Lemma 5.4, If we agree to let 9 (u )=0 for u :SO)

= E (f 2A~C) 9 (f n -JT:)). (A denotes Lebesgue measure)

Shaded area is support of f.

C :smallest rectangle covering

support of f

C: smallest rectangle covering

all points

Data point

Figure 3.3.
CHAPTERS 61

Thus, by Fatou's lemma,

!lm Inf E (T )/n

n -+00
~ E <I !lm
n
Inf (~C g (f n -ffn)))
-+00 21\( )

where the Inner limit Infimum Is with respect to a.e. convergence. Now, for
almost all wEn (where (n,F ,P ) Is our probability space with probability ele-
ment w ), we have C -+ Co and thus )"(C) -+ ),,(C o)' But then, by Lemma 5.11,
for almost all (x ,w) E R d X n, we have f n (x ) -+ f (x). Thus, the Fatou lower
bound given above Is

I (2)..( C 0»-1 g (f -VI)

~ I (2)..( C 0»-1 g (f /2) ~ I (2)..( C 0»-1 g (f )/2 k = 00
I ;?:4 I ;?:4

when Ig(f) = 00 (for I g(f) ~ g(4» .. (C o) < 00).

I$.4

ProofofB.

m m
E(T) ~ ~ E(E(g(Wi +2d!C» ~ ~ E(E(g(2Wi )!C) +g(4d»
i=1 i=1

m
~ mg(4d) + 2k ~ E(E(g(Wi)!C»
;=1

m
~ mg (4d) + 2k ~ (aE (g (np;/p »+ag (1»
i=1

where a Is the constant of Lemma 5.4 (and depends upon k only). Thus, to
show that E (T) = 0 (n), we need only show that ~ E
m
(g (np";p »= 0 (n).
;=1
Now,

E (g (np;/p» ~ E (g (2nPi» + g (n)P (p <1/2).

62 CHAPTERS

The last term Is uniformly bounded In n as we wlll now prove. First, we have
g (n) ~ n kg (1). We wlll show that P (p <1/2) ~ 2d exp(-n /(4d» for all n.
Because the function 11 k e -u, 11 > 0, Is uniformly bounded, we see that
sup g (n )P (p < 1/2) < 00. Indeed,
n

d
[p < 1/2) ~ .U [p / < 1-1/(2d»)
J=1

where· Pj Is the Integral of / over all x ' s whose j-th component Ues between
I

the minimal and maximal j-th components of all the X;' s. But by the proba-
blllty Integral transform, when U l' . . . , Un are Independent uniform [0,1) ran-
dom variables,

P (p / < 1-1/(2d» ::::; 2P (mln( U 11 .•. , Un > 1/(4d» = 2(1-1/(4d»n

::::; 2 exp(-n /(4d» .

Finally, by Jensen's Inequallty,

m
~ E (g (2np;
;=1
»= m
~ E (g (2n )"(A; )
;=1 A.
J/ /)"(A;)))

m
::::; ~ E cJ g (2n )"(A; )/ ) /)"(A; »
;=1 A.

::::; E ( )..(";;) {g (f ) max(2n )..( C)/ m , (2n )..( C)/ m )k »

and B follows since m ....., n.

CHAPTER 3 63

Proof of C.
By a bound derIved In the proof of A and by the second Inequality of
Lemma 5.4, we need only show that

1 ~ E (g <l(n -2d )p;j ) = 00 .

m i=1

when f does not have compact support. By our assumptIons on g, (n -2d) can
be replaced by n. We may assume wIthout loss of generality that the first com-
ponent of XI has unbounded support. Let (a l'b 1), ... , (ad ,b d ) be € and 1-€
quantlles of all the margInal dIstrIbutIons where € E (0,1/2) Is chosen such that
d
B = .X
)=1
(a j ,b j) satIsfies JB f =.!..
2
Let Q be the collectIon of A j I 8 Intersect-

Ing wIth B, and let q be the cardInality of Q. Set Pj , J f, and let Z be

AjnB
the IndIcator of the event B ~ C. Clearly,

q -Z
~ E(- E g LnPi 'J ) ~ E(-Zg(-
q 1 E LnPi 'J ))
m q AJEQ m q AjEQ

q n 1 q n n
~ E(-Z g(--1)) ~ E(Z (- - - ) g(--1)/(--1)).
m 2q 2 n 2q 2q

where we used Jensen's Inequality. Since g (u )/u i 00, we need only show that
for any constant M, however large,

11m Inf P(Z=I, n/2q -1 ~ M) > 0.

n-+oo

Now, let U, V be the mInImum and the maxImum of the first components of
X11 • . . ,Xn • When Z = 1, we have

d -I
<
I ~ + 2) ,
q m-d-
V-U
mild
64 CHAPTER 3

and thus

P (Z =I,n ~2q (M +1))

d-l
~ P (Z =1,( b Ca l)m I( V -U )+2m -d- :S n 1(2(M +1)))

d-l
_P(2m-d- > n )
4(M +1) .

The second term of the last expressIon Is 0 (1) for obvIous reasons. The thIrd
term Is 0 (1) sInce m ~ n and V -U ---+ 00 In probability as n ---+ 00. The last
term Is 0 (1) sInce m ~ n. ThIs concludes the proof of C.

3.2. SORTING AND SEARCHING.

When d =1, and elements wIthIn each bucket Ai are sorted by an n 2 sort-
Ing algorIthm (such as selectIon sort, or InsertIon sort), Theorem 3.1 applies with
g (u )=u 2. The data can be sorted In expected tIme 0 (n) If and only If f has
compact support and

If however we employ an expected tIme n log n sortIng algorIthm based upon

comparIsons only (such as heapsort, quIcksort or tree InsertIon sort), the data can
be sorted In expected tIme 0 (n ) If and only If f has compact support and

Jf log+ f < 00 .

The latter condItIon Is only vIolated for all but the most peaked densItIes. These
results generalize those of Devroye and KlIncsek (Ill81). We should mentIon here
that If we first transform arbItrary data by a mappIng h : R 1 ---+ [0,1] that Is
contInuous and monotone, construct buckets on [0,1], and then carry out a subse-
quent sort withIn each bucket as descrIbed above, then often E (T) = 0 (n): In
other words, wIth little extra effort, we gaIn a lot In expected tIme. The Ideal
CHAPTER 3 65

transformation h unlformlzes, I.e. we should try to use F (x ) where F Is the dis-

tribution function of the data. In general, we can take h In such a way that It Is
equal to F ( x -I-' ) where F Is a fixed distribution function, I-' Is a sample estimate
a
of location (mean, median, etc.) and a Is a sample estimate of scale (standard
deviation, etc.). This should In many cases give satisfactory results. It Is prob-
ably advantageous to take robust estimates of location and scale, i.e. estimates
that are based upon the sample quantlles. MeUer and Akl (1980) and Weide
(1978) give variations of a slmllar Idea. For example, In the former reference, F
Is piecewise linear with cut-points at the extrema and a few sample quantlles.
One should of course Investigate If the theoretical results remain valid for
transformations F that are data-dependent.

Distribution function F

PROPERTY: When X has distribution function

F, then U=F(X) is uniformly distributed on [0, 1].

Figure 3.4.

The conditions on f mentioned above are satisfied for all bounded densities
f . It is nice exercise to verify that If a transformation

h (x) = x /(Hlx D

Is used and f (x) :S a exp(-b Ix Ie) for some a ,b ,c > 0, then the density of the
transformed density remains bounded. Thus, for the large class of densities with
exponentially dominated tall, we can sort the transformed data In average time
o (n) by any of the bucket-based methods discussed above.
 66 CHAPTER 3

~
Uniform interval widhts ---.
/
/
7
7

l
Nonuniform inte rval widths

J
1I~

V
~
Figure 3.5.
A nonlinear transformation useful for distribution
with unbounded support.

For the expected number of comparisons In a successful or unsuccessful

search of linked list based buckets, we obtain without effort from Theorem 3.1
the value 0 (1) (even when d;F1) when f has compact support and
J f 2 < 00. These conditions are necessary too. If within each bucket the X j ' 8
are ordered according to their first component, and are stored In a binary search
J
tree or a balanced binary tree such as a 2-3 tree, condition f 2 < 00 can be
J
replaced by f log+ f < 00. Just apply the Theorem with g (u )=u log(u +1),
J J
and note that f log(j +1) < 00 Is equivalent to f log+ f < 00 because
log+u ~ log(l+u) ~ log+u +log2. For a more detailed analysis, the quantity
m m
T = ~ Nj 2 of chapter 1 must be replaced now by T = ~ N j log(Nj +1).
j =1 j =1
Most of chapters 1 and 2 can be repeated for this new quantity. We leave It as
CHAPTER 3 67

an exercise to show that

E(T) :::; a + bn + n E(Iog (1 + mln(n ,Rn )))

:::; a + bn +n log (1 + E (Rn ))

J
for some constants a ,b > 0 when f log(f +1) < 00. Hence, If f Is any den-
sity with a finite moment generating function In a small neighborhood of the ori-
gin, we obtain E (T) = 0 (n log log n). Examples of such densities are the
exponential and normal densities. This extends an Interesting observation
reported In Akl and Meijer (1982).

Figure 3.6.
The planar graph pOint location problem:
return the set in the partition to which the query point belongs.
68 CHAPTER 3

Remark. [Point location problems.)

In the planar point-locatIon problem, a stralght-llne planar graph wIth J)
vertIces Is gIven, and one Is asked to find the set In the partItIon of the plane to
whIch a query poInt x belongs. In many appllcatlons, a large number of querIes
are raIsed Cor one fixed planar partItIon. We won't be concerned here wIth worst-
case complexItIes. It sumces to mentIon that each query can be answered In
worst-case tIme 0 (Iog(n» provIded that up to 0 (n log(n » tIme Is spent In set-
tIng up an approprIate data structure (Lipton and Tarjan, 1977 ; KIrkpatrIck,
19S3). See also Lee and Preparata (lg77) Cor an algorIthm wIth 0 ({Iog(n »2)
worst-case search tIme, and Shamos and Bentley (lg77) Cor the poInt-locatIon
problem when the space Is partitIoned Into nonoverlap pIng rectangles. It was
poInted out In Asano, Edahlro, Imal, Irl and Murota (lgS5) that these algorIthms
can be very slow In practice. In partIcular, they compare Infavorably wIth a
bucket-based algorithm oC Edahlro, Kokubo and Asano (lgS3).

Query paint

Figure 3.7.
The rectangular point location prob lem.

Assume for example the Collowlng probabilistIc model : the n points

X l' . . . ,Xn and the query poInt are lid random vectors unIformly dIstrIbuted In
the unIt square, and the graph Is constructed by connectIng poInts In an as yet
unspecIfied manner. In first Instance, we will be Interested In the expected worst-
case tIme, where "worst-case" Is wIth respect to all possible planar graphs given
the data. Let us construct an m -grId where for each bucket the followIng Infor-
matIon Is stored : the llst of vertIces sorted by y -coordInates, the collectIons of
CHAPTER 3 69

edges Intersecting the north, south east and west boundaries (sorted), and the
region of the partition containing the north-west corner vertex of the bucket.
This assumes that all regions are numbered beforehand, and that we are to
return a region number. Partition each bucket In a number of horizontal slabs,
where the slab boundaries are defined by the locations of the vertices and the
pOints where the edges cut the east and west boundaries. For each slab, set up a
linked list of conditions and region numbers, corresponding to the regions visited
when the slab Is traversed from left to right. (Note that no two edges cross In our
graph.) It Is Important to recall that the number of edges In a planar graph Is
o (n ) , and that the number of regions In the partition Is thus also 0 (n ) . One
can verify that the data structure described above can be set up In worst case
time 0 (n 3/2) when m ~cn for some constant c. The expected set-up time Is
o (n) In many cases. This statement uses techniques similar to those needed to
analyze the expected search time. We are of course mainly Interested In the
expected search time. It should come as no surprise that the expected search time
decreases with Increasing values of m . If m Increases linearly In n , the expected
search time Is 0 (1) for many distributions. Those are the cases of Interest to us.
If m Increases faster than n, the expected search time ,while still 0 (1), has a
smaller constant. Unfortunately, the space requirements become Inacceptable
because O(max(m ,n )) space Is needed for the given data structure. On the posi-
tive side, note that the space requirements are 0 (n ) when m Increases at most
as O(n).

Figure 3.8.

The slab method described above Is due to Dobkin and Lipton (1976), and
differs slightly from the method described In Edahlro, Kokubo and Asano (19S3).
The time taken to find the region number for a query point X In a given bucket
Is bounded by the number of slabs. To see this, note that we need to find the
slab first, and then travel through the slab from left to right. Thus, the expected
70 CHAPTER 3

m
time Is bounded by ~ Pj 8 j , where 8 j denotes the number of slabs In the i -th
j=1
bucket, Pj Is the probability that X belongs to the i -th bucket, and the expected
time Is with respect to the distribution of X, but Is conditional on the data. But
E (8j )~ npj +E (aj), where a j Is the number of edges crossing the boundary of
the i -th bucket. Without further assumptions about the distribution of the data
points and the edges, any further analysis seems dim cult, because E (aj ) Is not
necessarily a quantity with properties determined by the behavior of f In or near
the i -th bucket. Assume next that X Is uniformly distributed. Then, the
expected time Is bounded by

=~+E(a)
m m

where E (a) Is the expected value of the overall number of edge-bucket boundary
crossings. E (a) can grow much faster than m : Just consider a uniform density
on [0,1]2. Sort the points from left to right, and connect consecutive pOints by
edges. This yields about n edges of expected length close to 1/3 each. E (a)
should be close to a constant times n Vm. Also, for any planar graph,
a ~"fn Vm where "f Is a universal constant. Thus, It Is not hard to check that
the conditional expected search time Is In the worst-case bounded by

n n
-+"f--.
m Vm

This Is 0 {1) when m Increases as O( n 2). Often, we cannot afford this because of
space or set-up time limitations. Nevertheless, It Is true that even If m Increases
linearly with n , then the expected search time Is 0 (1) for certain probabilistic
models for putting In the edges. Help can be obtained If we observe that an edge
of length £ cuts at most 2(2+£ Vm) buckets, and thus leads to at most twice
that number of edge-boundary crossings. Thus, the expected time Is bounded by
CHAPTER 3 71

where e Is the total number of edges and L j Is the length of the j -th edge. SInce
e =0 (n ). and m "'-'en (by assumption). this gives 0 (1) provIded that

e
~ E(L j ) = O(vm).
j=l

In other words. we have obtained a condition which depends upon the expected
lengths of the edges only. For example. the condItion Is satisfied If the data
points have an arbitrary density f on [0.1]2 • and each point Is connected to Its
nearest neighbor: this Is because the expected lengths of the edges grow roughly
as l/Vn. The condition Is also satisfied If the points are all connected to points
that are close to It In the ordinary sense. such as for example In a road map.

r--
r--

I I-
I
I - "- I
"-
"~
Que ry point
'--

Figure 3.9.
The point enclosure problem:report
all rectangles to which query point belongs.
72 CHAPTER 3

Remark. [Point enclosure problems.J

In point-enclosure problems, one Is given n rectangles In Rd. For one query
point X, one Is then asked to report all the rectangles to which X belongs. Since
a rectangle can be considered as a point In R 2d , It Is clear that this problem Is
equivalent to an orthogonal range search query In R 2d • Thus, orthogonal range
search algorithms can be used to solve this problem. There have been several
direct attempts at solving the problem too, based mainly on the segment or Inter-
val tree (Bentley (lQ77), Bentley and Wood (lQSO), Valshnavl and Wood (lQSO),
Valshnavl (lQS2». For example, on the real line, the algorithm of Bentley and
Wood (lQSO) takes preprocessing time 0 (n log(n», space 0 (n log(n », and
worst-case query time 0 (log(n )+k) where k Is the number of segments (I.e.,
one-dimensional rectangles) reported. We will briefly look Into the properties of
the bucket structure for the one-dimensional point-enclosure problem.
First, we need a good probablllstic model. To this end, assume that (L ,R ),
the endpoints of a segment form a random vector with a density f on the
north-west triangle of [0,lJ2 (this Is because L '5:.R In all cases). The n Intervals
are lid, and the query point has a density g on [O,lJ. The segment [O,lJ Is parti-
tioned Into m buckets, where typically m """"cn for some constant c (which we
assume from here onwards). For each bucket, keep two linked lists: one linked
list of segments completely covering the bucket, and one of Intervals only par-
tially covering the bucket. Note that the entire structure can be set up In time
proportional to n plus n times the total length of the segments (because a seg-
ment of length l can be found In at least 1 and about nl linked lists). The space
requirements are formally similar. Under the probablllstic model considered here,
It Is easy to see that the expected space and expected preprocessing time are both
proportional to n times the expected value of the total length. Since the expected
value of the total length Is n times the expected value of the length of the flrst
segment, and since this Is a constant, the expected space and preprocessing
requirements Increase quadratically In n. The expected search time Is small.
Indeed, we first report all segments of the first linked list In the bucket of X.
Then, we traverse the second linked list, and report those segments that contain
X. Thus, the search time Is equal to k +1 plus the cardinality of the second
linked list, I.e. the number of endpOints In the bucket. With the standard nota-
tion for buckets and bucket probabllltles, we observe that the latter contribution
to the expected search time Is

:E P (X EA; )n (P (L EA; )+P (R EAj ».

m

;=1

In particular, If X Is uniformly distributed, then this expression Is simply 2n / m .

This can be made as small as desired by the appropriate choice of m. If, how-
ever, X Is with equal probability distributed as Land R respectively, which
seems to be a more realistic model, then the expression Is
CHAPTER 3 73

where h Is the density of X (I.e. It Is the average of the densities of Land R ),

J
and Pi = h . Here we used Lemma 1.1.
A.
There are other probablllstic models with totally dUferent results. For exam-
ple, In the car parking model, we assume that the midpoints of the segments
have density f on [0,1), and that the lengths of the segments are random and
Independent of the location of the segment : the distribution of the lengths how-
ever Is allowed to vary with n to allow for the fact that as more segments are
avallable, the segments are more likely to be smaller. For example, If the lengths
are all the same and equal to r ll where r ll tends to 0 at the rate lIn, the overlap
among Intervals Is quIte small. In fact, the preprocessIng and set-up times are
both 0 (n) In the worst case. If X has densIty f as well, then the expected
J
search tIme Is 0 (1) when f 2<00.

3.3. THE TRAVELING SALESMAN PROBLEM.

The traveling salesman problem Is perhaps the most celebrated of all discrete
optimization problems. A traveling salesman tour of X 11 . . • ,XII Is a permuta-
tion 0'11 ••• ,0'11 of I, ... , n: this permutation formally represents the path
formed by the edges (X O'I'X 0',), (X 0'.,)( O'.), .•. ,(X0'. ,X0'1)' The cost of a traveling
salesman tour Is the sum of the lengths of the edges. The travellng salesman
problem Is to find a minimum cost tour. When the lengths of the edges are the
Euclldean distances between the endpoints, the problem Is also called the
Euclldean travellng salesman problem, or ETSP.
74 CHAPTER 3

A traveling salesman path

Figure 3.10.
The Euclidean traveling salesman problem:
find the shortest path through all cities.

The ETSP Is an NP-hard problem (Papadlmltrlou (1977), Papadlmltrlou and

Stelglltz (1982)), and there has been considerable Interest In developing fast
heuristic algorithms (see PapadlmItrlou and Stelglltz (1982) and Parker and Rar-
dIn (1983) for surveys). It should be stressed that these algorithms are nonexact.
Nevertheless, they can lead to excellent tours: for example, a heuristic based upon
the minimal spanning tree for X 11 • . . Xn developed by Chrlstofldes (1976)
J

yields a tour which Is at worst 3/2 times the length of the optimal tour. Other
heuristics can be found In Karp (1977) (with additional analysis In Steele (1981))
and Supowlt, Reingold and Plaisted (1983). We are not concerned here with the
costs of these heuristic tours as compared, for example, to the cost of the optimal
tours, but rather with the time needed to construct the tours. For lid points In
[0,1]2, the expected value of the cost of the optimal tour Is asymptotic to
f3Vn Jff where 13 > 0 Is a universal constant (Steele, 1981). For the uniform
distribution, this result goes back to Beardwood, Halton and Hammersley (1959),
where It Is shown that 0.61 :s :s13 0.92.
For the ETSP In [0,1]2, we can capture many bucket-based heuristics In the
following general form. PartitIon [0,1]2 Into m equal cubes of side l/Vm each.
Typically, m Increases In proportIon to n for sImple heurIstics, and m =0 (n)
when the expected cost of the heuristic tour Is to be optimal In some sense (see
Karp (1977)and Supowlt, Reingold and Plaisted (1983)). The bucket data struc-
ture Is set up (In time 0 (n +m)). The cells are traversed In serpentine fashion,
CHAPTER 3 75

starting with the leftmost column, the second column, etcetera, without ever lift-
Ing the pen or skipping cells. The points within the buckets are all connected by
a tour which Is of one of three possible types:

A. Random tour. The points connected as they are stored In the linked lists.
B. Sorted tour. All points are sorted according to y coordinates, and then
linked up.
c. Optimal tour. The optimal Euclidean traveling salesman tour Is found.

I, I, (
,
r\

J '- ../
J
'- ,/
J
, ~

Figure 3.11.
Serpentine cell traversal.
76 CHAPTER 3

II"'"
r\ .,..
) 7~
K~
~2 J
/
;1
"" "-
"-

<
~
•
~I r; ~ ~
\. I~ ~ r J
"-~ / \
v
\ )

~\
-"
\ J
>
V v
I

Figure 3.12.
A sorted tour.

The time costs of A ,B,C for a bucket with N points are bounded respectively
by

CN,

CN log(N +1),

and

for constants C. For the optimal tour, a dynamic programming algorithm Is used
(Bellman, 1962). The m tours are then linked up by traversing the cells In ser-
pentine order. We are not concerned here with just how the Individual tours are
linked up. It should for example be obvious that two sorted tours are linked up
CHAPTER 3 77

by connecting the northernmost point of one tour with the southernmost point of
the adjacent tour, except when an east-west connection Is made at the U-turns In
the serpentine. It Is easy to see that the total cost of the between-cell connec-
tlons Is 0 (Vm), and that the total cost of the tours Is 0 (n /Vm) for all three
schemes. For schemes A and B therefore, It seems Important to make m pro-
portlonal to n so that the total cost Is 0 (rn), just as for the optimal tour. In
scheme C, as pointed out In Karp (lg77) and Supowlt, Reingold and Plaisted
(lQS3), If m Increases at a rate that Is sllghtly subllnear (0 (n)), then we can
come very close to the globally optimal tour cost because within the buckets
small optimal tours are constructed. The expected time taken by the algorithm
Is bounded by

m
o (n +m ) + E ('E CN;) ,
;=1

m
o (n +m ) + E ('E CN; 10g(N; +1)) ,
;=1

and

o (n +m ) + E (~ CN; 2 N ,)
;=1

respectively.

Theorem 3.2.
For the methods A ,B ,C for constructing traveling salesman tours, the
expected time required Is bounded by 0 (n +m ) plus, respectively

(A) Cn

(B) Cn I f log(2+~ f ) ::; Cn

m
I f log(2+ f ) + Cn 10g(H~)
m
;

where 'l/J( u) Is the functional generating function for the density f on [0,1]2.
78 CHAPTER 3

Remark.
The functional generating function for a denslty f on [0,1]2 ls defined
by

By Taylor's serles expanslon, It ls seen that

whlch explalns the name, Note that the Taylor serles ls not necessarlly conver-
gent, and that 1/J ls not necessarlly finlte: It ls finlte for all bounded densltles wlth
compact support, and for a few unbounded densltles wlth compact support. For
example, If f :$ f * on [0,1)2, then 'ljJ(u) :$ J7"e
1 u," ,u > 0. Thus, the bound

In (C) becomes

1 (l+..!!..)'"
2Cn--e m
f *

.!:..r
(In fact, by a dlrect argument, we can obtaln the better bound 2Cne m .) Note
that In the paper of Supowlt et al. (1983), m ls allowed to be plcked arbitrarily
close to n (e.g. m =n flog log log n). As a result, the algorlthm based on (C)
has nearly llnear expected tlme. SupowIt et al. (1983) provlde a further
modlficatlon of algorlthm (C) whlch guarantees that the algorlthm runs In nearly
llnear tlme In the worst case.

Proof of Theorem 3.2.

To show (B), we conslder

n n
= E (E B j log( E B j + 1»
j=l j=l
CHAPTER 3 79

n
= n E(Bllog(BI + L: B j + 1))
j=2

n
= nPi E (iog(2 + L: B j ))
j=2

~ nPi log(2 + (n -1)Pi) (Jensen's Inequallty).

where B l' . . . ,Bn are lid Bernoulli (Pi) random variables. Also, since
Pi log(2+(n -1)Pi ) Is a convex function of Pi. another appllcatlon of Jensen's Ine-
quallty yields the upper bound

n Jf log(2 + n-1
m
f ).
A;

which Is all that is needed to prove the statement for (B). For (C). we argue
slmllarly. and note that

E(Ni 2 N.)

= E((
j=1
EB IT 2Bl))
j )(
j=1

= nE (B 12Bl IT 2Bl)
j=2

= 2nPi (2Pi + (1-Pi ))n-l

= 2nPi(1 + Pi )n-l
<
-
2np.
I
e(n-l)l'.

n-l I
~ 2n Jf e m (Jensen's Inequallty).
A.

This concludes the proof of Theorem 3.2.

80 CHAPTER 3

Remark. (ETSP In higher dimensions.]

Halton and Terada (lgS2) descrlbe a heuristic for the ETSP In d dimensions
which Is slmllar to the heuristic given above In which within each cell an optimal
tour Is found. In particular, for points uniformly distributed on the unit hyper-
cube, It Is shown that the tour length divided by the optimal tour length tends
with probablilty one to one as n -400. Also, the time taken by the algorithm Is In
probablilty equal to 0 (n <p(n » where <p ls an arbitrary diverging function picked
beforehand and <p Is used to determine at which rate m / n tends to O. The
divergence of <p Is again needed to Insure asymptotic optlmallty of the tour's
length. The only technical problem In d dimensions Is related to the connection
of cells to form a travellng salesman tour.

3.4. CLOSEST POINT PROBLEMS.

Local algorithms are algorithms which perform operations on points In given
buckets and In neighboring buckets to construct a solution. Among these, we
have algorithms for the following problems:

(1) the close pairs problem: Identify all pairs of points within distance r of
each other;
(11) the isolated points problem: Identify all points at least distance r away
of all other pOints;
(lU) the Euclidean minimal spanning tree problem;
(1v) the all-nearest-neighbor problem: for each pOint, find Its nearest neigh-
bor;
(v) the closest pair problem: find the minimum distance between any two
pOints.
CHAPTER 3 81

, • •
••
Distance r in definition

..-e • • • of close pairs .

PROPERTY: Vertices with degree 0

are isolated points for distance r.

I • ~ ~
•
-' •
. ..... • •
•
• •
....... '- •
• •Figure 3.13. \
Close pairs graph.

These problems are sometimes called closest point problems (Shamos and
Hoey. lQ75; Bentley. Weide and Yao. IQSO). What compllcates matters here Is
the fact that the time needed to find a solution Is not merely a function of the
form

m
E g(N;)
;=1

as In the case of one-dimensional sorting. Usually. the time needed to solve these
problems Is of the form

m
E 9 (Nj • N j *)
j=1

where N j * Is the number of points In the neighboring buckets; the definition of a

82 CHAPTER 3

neighbor bucket depends upon the problem of course. It Is quite Impossible to

give a detailed analysis that would cover most Interesting closest point problems.
As our prototype problems, we will pick (I) and (II). Our goal Is not just to find
upper bounds for the expected time that are of the correct order but possibly of
the wrong constant: these can be obtained by first bounding the time by a func-
tion of the form

m
~ g(Nj + Nj ")
j=1

where g Is another function. The overlap between buckets Implicit In the terms
N j +Nj " does not matter because the expected value of a sum Is the sum of
expected values. Our goal here Is to obtain the correct asymptotic order and
constant. Throughout this section too, X l' . . . ,Xn are Independent random
vectors with density f on [O,lJd .
CHAPTER 3 83

Figure 3.14.
All nearest neighbor graph at left. This graph is a subgraph
of the minimal spanning tree, shown at right.

Remark. [Isolated points. Single-linkage clustering.]

If X 11 • • • 1 Xn are d-dlmenslonal data points, and r > 0 Is a number
depending upon n only, then Xi Is said to be isolated point If the closed sphere
of radius r around Xi contains no Xi' j ~i .
Isolated points are Important In statistics. They can often be considered as
"outliers" to be discarded In order not to destabilize certain computations. In
the theory of clustering, the following algorithm Is well-known: construct a
graph In which Xi and Xi are joined when they are within distance r of each
other. The connected components In the graph are the clusters. When r grows,
there are fewer and fewer connected components of course. Thus, If we can find
all pairs (Xi ,Xi) within distance r of one another very quickly, then the cluster-
Ing algorithm will be fast too, since the connected components can be grown by
the union-find parent pointer tree algorithm (see e.g. Aho, Hopcroft and Ullman
(1983, pp. 184-189)). This clustering method Is equivalent to the single linkage
clustering method (see e.g. Hartigan (1975, chapter 11)). The Isolated points
algorithms discussed below will all give an exhaustive listing of the pairs (Xi ,Xj )
84 CHAPTER 3

that satisfy IIXi-Xj II :S r, and can thus be used for clustering too. The prob-
lem of the Identification of these pairs Is called the close pairs problem.

There are two bucket-based solutions to the close-pairs problem. First, we

can define a grid of hypercubes (buckets) with sides dependent upon r. The
disadvantage of this Is that when r changes, the bucket structure needs to be
redefined. The advantage Is that when n changes, no such adjustment Is
needed. In the second approach, the bucket size depends upon n only: It Is
Independent of r .

Neighboring buckets
...-4--4-....:

Central cell

r/--.J2

Figure 3.15.

In the r -dependent grid, It Is useful to make the sides equal to r In

because any pair of pOints within the same bucket Is within distance r of each
other. Furthermore, points that are not In neighboring buckets cannot be within
distance r of each other. By neighboring bucket, we do not mean a touching
bucket, but merely one which has a vertex at distance r or less of a vertex of the
CHAPTER 3 85

original bucket. A conservative upper bound for the number of neighboring

buckets Is (2Vd +3)d. In any case, the number depends upon d only, and wlll
be denoted by 'Yd' To Identify Isolated pOints, we first mark single point buck-
ets, I.e. buckets with N j =1, and check for each marked point all 'Yd neighboring
buckets. The sum of distance computations Involved Is

j :N. =1 j :A J neighbor of A.

= E,Nj E, 1
j j :N. =1, and A; neighbor of A J

= 'Yd n.

The grid Inltlallzatlon takes time n(r- d ) and 0 (mln(r- d ,1». In particular, the
entire algorithm Is 0 (n) In the worst-case whenever rn lid 2: c > a for some
constant c. For r much smaller than n -lid, the algorithm Is not recommended
because nearly all the points are Isolated points - the bucket size should be made
dependent upon n Instead.

'1;MaXimal gap .I
t j j 1 l l l kjXj

Figure 3.16.
Finding the maximal gap in a sequence of
n points by dividing the range into n+1 intervals.

Remark. [The maximal gap.)

The maximal gap In a sequence of pOints x 11 . • • ,xn taking values on [0,1)
Is the maximal Interval Induced by these pOints on [0,1). As In the case of Iso-
lated points, the maximal gap can be found In worst-case time 0 (n). For exam-
1
pie, this can be done by observing that the maximal gap Is at least - - . . Thus,
n+1
86 CHAPTER 3

If we organize the data Into a bucket structure with n +1 Intervals, no two points
within the same bucket can define the maximal gap. Therefore, It Is not neces-
sary to store more than two points for each bucket, namely the maximum and
the minimum. To find the maximal gap, we travel from left to right through the
buckets, and select the maximum of all dUferences between the minimum of the
current bucket and the last maximum seen untU now. This algorithm Is due to
Gonzalez (lQ75).

Let us turn now to the close-pairs problem. The time needed for reporting
all close pairs Is of the order of

E
j :A, neighbor of A,

where the first term accounts for llstlng all pairs that share the same bucket, and
the second term accounts for all distance computations between points In neigh-
boring buckets.
For this problem, let us consider a grid of m buckets. This at least guaran-
tees that the Inltlallzatlon or set-up-tlme Is 0 (n +m). The expected value of our
performance measure V Is

E
j :A, neighbor of A,

and It Is the last term which causes some problems because we do not have a full
double sum. Also, when Pi = J f Is large, Pj Is lIkely to be large too since Ai
A,
and A j are neighboring buckets. The asymptotlcs for E (V) are obtained In the
next theorem. There are 3 situations when m =n :

A. nr d -+ 00 as n -+ 00: the expected number of close pairs Increases roughly

speaking faster than n .
B. nr d -+ 0 as n -+ 00: the expected number of close pairs Is 0 (n), and the
probablllty that any given point Is an Isolated point tends to 1.
c. nr d -+ (1 E (0,00) as n -+ 00: the expected number of close pairs Increases
as a constant times n. This Is the critical case.
CHAPTER 3 87

The upper bound In the theorem Is valid In all three cases. In fact, Theorem 3.3
also covers the situation that m ~n: m and / or r are allowed to vay with n
In an arbitrary manner.

Theorem 3.3.
Let "( = "(r , d, m) be the number of neighboring buckets of a particular
bucket In a grid of size m defined on [O,ljd, where r Is used In the definition of
neighbor. Then

2
E (V) ~ n + ..!:...("( + l)I f 2 .
m

If m -+ 00, n -+ 00, r -+ 0,

Note that If mr d -+ 00 r -+ 0 , "(r ,d ,m ) ~ mr d Vd where Vd Is the value of

the unit sphere In Rd. Thus,

If mr d -+ f3 E (0,00), then "( oscillates but remains bounded away from 0 and 00
In the tall. In that case,

E(V)=O(n)

when If 2 < 00, m ~ en. Notethat E (V) = !l(n ) In all cases.

Finally, If mr d -+ 0, such that r > 0 for all n, m, then "( -+ 3 d -1, and

2
E (V) = n + ..!:... 3 d If 2(1 + 0 (1)) .
m
88 CHAPTER 3

Proof of Theorem 3.3.

We will use the notatIon A (X) for the bucket Ai to whIch x belongs.
Furthermore, B (x) Is the collectIon of neIghborIng buckets of A (x). Define' the
densItIes

f n (x ) = IA 1()I J f , x E [O,IJ d
x A(z)

1
gn(x) = -IB()I J f, x E [O,IJ d .
x B(z)

Note that by the Lebesgue densIty theorem, If m -+ 00, r -+ a (and thus

IA (x)1 -+ 0, IB (x)1 -+ 0), f n (x ) -+ f (x) and gn (x) -+ g (x) for almost all x.
ThIs result can be obtaIned wIthout trouble from Lemmas 5.10, 5.11, and the fact
that the definItIon of neIghborIng bucket Is data Independent and depends upon
rand m only.
The upper bound will be derIved first. The sum V Is split Into V 1 + V 2'
m n2
Only V 2 causes some problems sInce E (V 1) ~ n 2 ~ Pi 2 + n ~ -J f 2 + n
i=1 m
by Lemma 1.1. Note also for future reference that
n
+ (1+0 (1»-J
2
E (V 1) >
-
n
m
f 2 when m -+ 00 If we apply the Fatou lower
bound argument of the proof of Lemma 1.1. TurnIng to V 2' we have, by Lemma
5.1,

PiPj n(n-l)
i=1 j:Aj neighbor of A,

m
~ n 2:E )'(Ai »'(Bi ) f n (Xi )gn (Xi) (for any x 1 E A 11 . . . } xm E Am)
i=1

m
= n2:E J),(B i ) f n (x )gn (x )dx
i=1 A,

SInce f nand gn' are probably very close to each other, the Integral In the last
J
expressIon Is probably very close to f n 2. Therefore, little will be lost If the
Integral Is bounded from above by the Cauchy-Schwartz Inequality:
CHAPTER 3 89

< J A(A1( X
)) (
A(x)
J f 2)dx J A(B\ x )) ( J B(x)
f 2)dx
(Jensen's Inequality)

m m
~ Jf 2 ~ Jf 2
;=1A, ;=1A,

To treat Jgn 2 we have argued as follows:

= J [ A(B\x)) sIx) f (y )dY ) 2 dx

(where B (x) now refers to an Infinite grid)

:s J 1
A(B (x))
( J f 2) dx
B(x)
(Jensen' s Inequality)

= E J ,,/A(A;)
;=1 A,
1 ~
j:A J neighborofA,A J
Jf 2(y )dy dx

= E ,,/A(A
;=1
1
1) A J
Jf 2(y ) [ ~
;:A, neighbor of A J A,
JdX) dy

;=1 A J

=J[2.

Note also that

11m Inf Jf n gn ~ J 11m Inf f n gn = Jf 2

90 CHAPTER 3

when m -+ 00, r -+ O. This concludes the proof of the first two statements of
the theorem. The remainder of the theorem Is concerned with the size of 1 as a
function of rand m , and follows from elementary geometric principles.

We note for example that when m -+ 00, mr d -+ 0, the optimal choice for
m would be a constant times n #Ji2 - at least, this would minimize
Cm + E (V) asymptotically, where C Is a given constant. The minimizing
value Is a constant times n #Ji2. The only situation In which E (V) Is not
o (n) for m "-' en Is when nr. a -+ 00, I.e. each bucket has very many data
pOints. It can be shown that the expected number of close pairs grows as a con-
stant times n 2r d, and this provides a lower bound for E (V). Thus, the
expected time for E (V) obtained In Theorem 3.3 has an optimal asymptotic rate.

Remark. [The all-nearest-nelghbor problem.]

All nearest neighbor pairs can be found In 0 (n log n) worst-case time
(Shamos and Hoey, 1975). Weide (1978) proposed a bucketing algorithm In
which for a given Xi' a "spiral search" Is started In the bucket of Xi, and con-
tinues In neighboring cells, In a spiraling fashion, until no data pOint outside the
buckets already checked can be closer to Xi than the closest data point already
found. Bentley, Weide and Yao (1980) showed that Weide's algorithm halts In
average time 0 (n ) when there exists a bounded open convex region B such that
the density f of X1 Is 0 outside B and satisfies
o < Inf f (x) < sup f (x) < 00. (This condition will be called the BWY condl-
B - B
tlon.)
CHAPTER 3

-
"
~

I
/
., ........
,
,
/ ~
( j

\\ I
v
\.

"""'"
~
" i-o- ./
1
".~

Figure 3.17.
Spiral search for nearest neighbor.

Remark. [The closest pair problem.]

To find the closest pair In [a,l]d, one can argue geometrically and deduce an
absolute upper bound of the form Cd In d for the smallest distance between any
two points among X 11 . . . ,Xn In [a,l]d. Here Cd Is a constant depending upon
d only . . If we construct a grid with buckets having sides Cd In d, then we can
hope to "catch" the closest pair In the same bucket. Unfortunately, the closest
pair can be separated by a bucket boundary. This case can be elegantly covered
by shifting the grid appropriately a number of times so that for one of the shifted
grids there Is a bucket which contains the closest pair (Yuval , 1976). Ignoring
the dependence upon d, we see that with this strategy, the time complexity Is of
n
the form c 1 n + c 2 :E No' 2 where the square accounts for the computations of all
;=1
pairwise distances within the same bucket, and C l' C 2 > a are constants. It Is
easy to see that If X l' . . . ,Xn are lid random vectors with density f on [a,l]d ,
92 CHAPTERS

J
then the shifted grid method takes expected time 0 (n ) If and only If f 2 < 00.
Rabin (lg76) chooses a small subset for which the closest pair Is found. The
corresponding minimal distance IS then used to obtain the overall closest pair In
llnear expected time. It Is perhaps Interesting to note that not much Is gained
over worst-case time under our computational model, since there exist algorithms
which can find the closest pair In worst case time 0 (n loglogn) (Fortune and
Hopcroft, Ig7g).

Remark. [The Euclldean minimal spanning tree.)

For a graph (V ,E)) Yao (lg75) and Cheriton and Tarjan (lg76) give algo-
rithms for finding the minimal spanning tree (MST) In worst-case time
o ~IE Ilog logl V i). The Euclldean minimal spanning tree (EMST) of n points In
R can therefore be obtained In 0 (n log log n) time If we can find a super-
graph of the EMST with 0 (n) edges In 0 (n log log n) time. Yao (lgS2) sug-
gested to find the nearest neighbor of each pOint In a critical number of direc-
tions; the resulting graph has 0 (n) edges and contains the MST. ThiS nearest
neighbor search can be done by a sllght modification of spiral search (Weide
(lg78)). Hence, the EMST can be found In expected time 0 (n log log n) for
any d and for all distributions satisfying the BWY condition. The situation Is a
bit better In R 2. We can find a planar supergraph of the EMST In expected time
o (n) (such as the Delaunay triangulation (the dual of the Voronol diagram), the
Gabriel graph, etc.) and then apply Cheriton and Tarjan's (lg76) 0 (n) algo-
rithm for finding the MST of a planar graph. For a llnear expected time Voronol
diagram algorithm, see Bentley, Weide and Yao (lgSO). Thus, In R 2 and for the
class of BWY distributions, we can find the EMST In llnear expected tlme.
CHAPTER 4 93

Chapter 4

THE MAXIMAL CARDINALITY

The expected value of the worst possible search time for an element In a
bucket data structure Is equal to the expected value of Mn = max N j times a
l:S;j :S;m
constant. This quantity differs from the worst-case search time, which Is the
largest possible value of max N j over all possible data sets, I.e. n. In a sense,
l:S;j :S;m
the maximal cardinality has taken over the role of the height In tree structures.
Its main Importance Is with respect to searching. Throughout the chapter, It Is
crucial to note the dependence of the maximal cardinality upon the density f of
the data points X 11 • • • X n , which for the sake of simplicity are assumed to
J

take values on [O,l)d. The grid has m ~ cn cells for some constant c > 0,
unless we specify otherwise.
In section 4.1, we look at the properties of Mn ' and In particular of E (Mn )
following analysis given In Devroye (1985). This Is then generalized to E (g (Mn ))
where g Is a nonlinear work function (see section 4.3). Such nonlinear functions
of Mn are Important when one particular bucket Is selected for further work, as
for example In a bucket-based selection algorithm (section 4.2). Occasionally, the
maximal cardinality can be useful In the analysis of bucket algorithms In which
certain operations are performed on a few buckets, where buckets are selected by
the data pOints themselves. In section 4.4, we will Illustrate this .on extremal
point problems In computational geometry.

4.1. EXPECTED VALUE AND INEQUALITIES.

For the uniform distribution on [0,1), Gonnet (1981) has shown that when
m=n,
94 CHAPTER 4

where r Is the gamma functIon. For example, when n = 40320, E (Mn ) Is near
7.35 (Gonnet, 1981, table V). In other words, E (Mn ) Is very small for all practI-
cal values of n. AddItIonal InformatIon Is gIven In Larson (1982). The sItuatIon
studIed by Gonnet pertaIns maInly to hashIng wIth separate chaInIng when a per-
fect hash functIon Is available. As we know, order-preservIng hash functIons lead
to non-unIform dIstrIbutIons over the locatIons, and we will see here how E (Mn )
depends upon f. ThIs Is done In two steps. FIrst we will handle the case of
bounded f , and then that of unbounded f .

Theorem 4.1.
Assume that f * = ess sup f < 00 (note: A{X: f (x) > f *}
= 0 ; A{X : f (x) > f *- ~} > 0 for all ~ > 0). Then, If m ~ en for some
e > 0,

E(Mn) ~ log n
log log n

and, In partIcular,

E (Mn ) =
log n
+ log n
(log log log n
f *e +
+ log(--) 0 (1)) .
log log n (log log n )2 e

Proof of Theorem 4.1.

We will use a Polssonlzatlon device. Assume first that we have shown the
statement of the theorem for Mn * where Mn * = m~xNj * and N j * Is the
I
number of Xj' 8 In Xl' ... ,XN belonging to A j , where N Is a Poisson (n)
random variable Independent of X l'X 2"" • Now, for all ~ > 0, we have

and

where I Is the IndIcator functlon, and where n (1+~) and n (1-~) should be read
as "the smallest Integer at least equal to .. :'. By Lemma 5.8,
CHAPTER 4 95

4
nP (IN -n I 2: n t) :S -4 "
nE

Define b (n )=1 + log(f * / C ) + log log n + log log log n,

(log log n)2
C (n ) = ~==---=---:­ Thus, by assumption,
log n

0(1)=E(Mn*)c(n)-b(n):S E(Mn(lH))c(n) + 4c(n) -ben)

n E4

c (n)
:S E (Mn(l+f))C (n (Ht))
C
((
n I+E
))

1
+ 0 (-) -b (n (Ht)) + (b (n (Ht)) -b (n )).
n

Now, b (n (I+E))-b (n )=0 (1), and, for n large enough, C (n )2: C (n (Ht))
log n
2:c (n) 2: C (n )/(Ht/log n).
log(n (Ht))

Thus,

b (n (Ht))+o (1)
E (Mn(l+f)) 2: C (n (I+t))(I+t/log n)

b (n (HE))+O (1)
C (n (HE))

Simllarly, It can be shown that E (Mn ) :S (b (n )+0 (1))/ C (n), and combIning
this glv"es us our theorem.

Lower bounds for Mn * .

Let 71 > 0 be an arbitrary number, and let t > 0 be the solution of
2
71 = -2 log(I---t) (this wIll turn out to be a convenient choice for E). Let A
/ *
be the set {x:/ (x) > / * -E}, and let {j = I dx (which Is positive by definition
A
of / *). Finally, let h = hn be the Integer part of b (n )-71. We let Pi keep Its
C (n )
meaning from the Introduction, and note that the function / n on [0,1] defined by
96 CHAPTER 4

f n (X) = mPi ,x E Ai '

Is a density. Because N 1 * ,N 2 *, ... , N m * are Independent Poisson random

variables with parameters np l'np 2' . . . , nPm respectively, we have the following
chain of Inequalities:

m
P (Mn * < h) = II P (Ni * < h)
i=1

m
< II (1-P (Ni * =h))
i=1

m
~ exp(- E P (Ni * =h ))
i=1

By Lemmas 5.10 and 5.12,

h
'!!:"'f -....!!..f.
m n
JA e m
.!!:...(f * -2€)
m

-....!!..f·
> e m J dx
A, 1/.-1 1::5:'

[ 8- J dX)
A,I/.-/I>'
CHAPTER 4 97

-'!!:"f'
> em (6-0(1».

I
Thus,

n f •
P(Mn * < h) :::; exp [ - ~ [ : ) (f
h
* _2E)h e -iii (6--0 (1» .

Using Stirling's approximation for h !, we see that the exponent Is -e 8 where

1
8 = log m - h log h + h - -log(21Th )
2

+ 0 (1) + h loge
f * -2E ) - n
- f * + log 6
c m

b (n )-1J f * -2E
= log n +
c (n)
(1 + loge
c
))

_ b (n )-1J loge b (n )-1J _ .!.log(_b-,-(n-:-,-:)-...!..1J + t

c(n) c(n) 2 c(n)

(where t = 1
log b - -log(21T) - - - + log c +
f * 0 (1))
2 c

= log n +
b (n )-1J
log n (l+log(
f * -2E )
(log log n )2 c

- log log n + log log log n + 0 (1))

1 1
- - log log n + - log log log n + t + 0 (1)
2 2

= logn [0(1)+ [1+ logloglogn + 1+10g (f*/C)-1J)

log log n log log n log log n

[ 1 + loge f *c- 2 € ) - log log n + log log log n + 0 (1)) + log log n)

= logn
log log n
(IOg( f*-2€)
c
-IOg(L)
c
+1J+ 0 (1»)
98 CHAPTER 4

log n
> -", (all n large enough)
3 log log n

because log(f * -2€) - log(f *) = _!l..

2

Thus, for all n large enough,

E(Mn *) ~ h P(Mn * ~ h) = h(1-P(Mn * < h))

", log n
~ h (1-exp(-exp(- ))) ~ h (1-exp(-exp(log log n )))
3 log log n

= h (1-..!:..) ~ (b (n )-", 1)(1-..!:..) = b (n )-rJ-o (1)

n c (n ) n c (n )

This concludes the proof of the lower bound, since", > 0 Is arbitrary.

Upper bounds for Mn * .

Again, we let", be an arbitrary positive number, and choose h = hn as the
Integer part of b (n )+"'. Let k > h be some Integer. Then, for h ~ c, by
c (n)
Lemma 5.Q,

n . e-c
P (Mn * ~ k) ~ ~ P (N; * ~ k) ~ n ~ c J - .-
;=1 i?:.k J!

k+1
~ nc k
k +1-c

Thus,

00 00
k+1
E (Mn *) ~ h + ~ P (Mn * ~ k) ~ h + ~ nc k
k =h k =h k+1-c

~ h + nc h ~ ( h +1 )2.
h! h +1-c
CHAPTER 4 99

By some straIghtforward analysIs, one can show that

log(ne h ~) ~ -(17+ 0 (1)) log n

h! log log n

and that

( h +1 )2 = 1 + ~ + 0 ( 1.) .
h+1-e h +1 h

Therefore,

E (Mn * ~ h +(1+ 2he +0 ( hI )) exp(-(17+0 (1)) log n )

log log n

~ h + ( 1+0 (1) )(1+~+O (1.)) ~ b (n )+17 + 1+0 (1)

log n h h e (n ) log n

b (n )+17+0 (1)
e (n)

But 17 was arbItrary. ThIs concludes the proof of the theorem.

For all bounded f , we have

log n
E(Mn)~ -...:::...-
log log n

whenever m ~ en. In flrst approxImation, the densIty does not Influence

E (Mn). The explanation Is due to the fact that the expected value of the max-
Imum of n Independent Poisson ()..) random variables Is asymptotiC to
log n / log log n for any constant)... The Influence of f * on E (Mn) Is In the
third largest asymptotiC expansion term only. The proof of Theorem 4.1 Is long
and tedIous because we want to obtain rather reflned InformatIon. From here
onwards, we will content ourselves with main asymptotiC terms only.
Theorem 4.1 remains valid when the minimum and the maximum of the
Xi' s are used to determine an Initial Interval, and the buckets are deflned by
dividing this Interval Into n equal SUb-Intervals. The density f Is assumed to
100 CHAPTER 4

have support contained In [0,1] but not In [0, 1-~] or [~,1] for any ~ > o.
When f Is unbounded, the theorem gives very little Information about
E (Mn)' Actually, the behavior of E (Mn ) depends upon a number of quantities
that make a general statement all but Impossible. In fact, any slow rate of con-
vergence that Is 0 (n) Is achievable for E (Mn)' Since Ni Is binomial (n ,Pi)
where Pi Is the Integral of f over the I-th bucket, we have

max nPi ~ E (m?-x N i ) = E (Mn ) .

I I

When f Is monotone nonlncreaslng, the left-hand-slde of this Inequality Is equal

1
to nF (-) where F Is the distribution function corresponding to f. Thus, since
n
any slow rate of decrease to 0 Is possible for F, when n -+ 00, any slow rate
o (n) Is achievable for E (Mn). The rate log n flog log n, achieved by all
bounded densities, Is also a lower bound for E (Mn ) for all densities.
This note would not be complete If we did not mention how E (Mn ) varies
when max nPi diverges. Most of this Information can be deduced from the Ine-
i
qualities given In Theorem 4.2 below. For example, we will see that
E (Mn ) ~ log n flog log n (the optimal rate achievable) when q diverges very
n
slowly, and that E (Mn ) ~ - q when q diverges rapidly.
m

Theorem 4.2.
Let q max mpj. Then
l~j~m

n
-;:; q ~ E(Mn ) ~ -;:; q
n
+ t1 (log m
n
+ -;:; t
q(e -t-l))

t
:- IOgt m + : q (e t- 1 ), all t > 0, m > 3.

Proof of Theorem 4.2.

The lower bound follows directly from Jensen's Inequality. To derive the
upper bound, we let Uj = Ni -nPi' U = max U j • Note that U Is a nonnega-
j
tlve random variable. We have

n
Mn ~ m?-X npj
I
+ m~x I
Ui = -
m
q + U.
CHAPTER 4 101

For r > 1, we can apply Jensen's Inequality again:

E r (U) :s E (U r ) = E (m?-x Ui r ) (u r Is considered sign-preserving)

I

< m m?-x E «Ui r )+)

l i e
:s m max E «_r_)' e tU') , all t
t
> o.

Here we used the Inequality u+ r :s (e r t)r e tu , t > 0, where u+=max(u ,0).

Also,

E (e tU') = E (e- tnp , e tN,) = e -tnp'(e t Pi +l-Pi t < e np,(e'-t-l)

~ q(e'-t-l)
<em

Thus,

E(Mn):S :q 1

+ :tm-; ex p ( : -;(e t -t-1)).

This bound Is minimal with respect to r when r = log m

(just+ ..'!!:...q (e t -t -1)
m
set the derivation of the logarithm of the second term In the bound equal to 0).
Resubstltutlon give the desired result. The restriction r > 1 forces us to choose
m > 3.

Theorem 4.2 shows that there are many possible cases to be considered with
respect to the rates of Increase of q and m. Assume that m ~ en , which Is the
standard case. Then

when q flog n --+ 00. To see this, observe that

102 CHAPTER 4

so that

E(Mn )< - n q -1 log m +...!!:..q .let , t > O.

- m t m 2

Take t = - / 2m log m (thIs mInImIzes the upper bound when e t Is neglected

V nq
), and note that

In thIs case, the bound of Theorem 4.2 Is tIght.

ConsIder a second case at the other end of the spectrum, the very small
q : q = (log n )0 (1) (or: log q = 0 (log log n)). Then the upper bound Is

log m log n
E (Mn) ~ (1+0 (1)) -...:..;..::;:...=.-
log log n
log [ 10: ; ]

whOn w, tak, t ~ log [ 10: : ]- log log [ 10: : ]

(note that thIs choIce of t almost mInImIzes the upper bound). Thus, Theorem
4.2 provIdes a consIderable short-cut over Theorem 4.1 If one Is only Interested In
first terms.
A thIrd case occurs when q = 0 (log n), but q Is not necessarily very small.
In that case, for the same choice of t suggested above, we have

E (Mn ) ~ (1+0 (1)) _~l.::::;og::..:..:.m::....-_ log n

CHAPTER 4 103

The only case not covered yet Is when q ~ a log n for some constant a > O. It
Is easy to see that by taking t constant, both the upper and lower bounq..for
E (Mn ) vary In proportion to q. Since obviously the bounds Implicit In Theorem
4.1 remain valid when q -+ 00, we see that the only case In which there might be
a discrepancy between the rate of Increase of upper and lower bounds Is our
"third" case.

Remark 4.1. [The behavior of max mPi.J

l:-S:i :-S:m
The behavior of Mn for unbounded densities depends rather heavlly on the
behavior of q = max mPi' It Is useful to relate this maximum to f . In par-
l<i<m
tlcular, we need to-be able to bound the maximum In terms of f. One possible
polynomial bound Is obtained as follows: for any set Ai, and any r 2: 1,

[ If
A~~i
I)
r
~ A(~')
I
If r
A,
(Jensen's Inequality).

Thus,

1.. 1..
q = max mPi ~ m
l:-S:i :-S:m
r cJ f r ) r .

The less outspoken the peakedness of f Is (I.e. the smaller f r ), the smaller the I
bound. For densities f with extremely small Infinite peaks, the functional gen-
erating function Is finite: "p( u ) = Ie ttf < 00, some u > O. For such densities,
even better bounds are obtainable as follows:

< _1_ I exp(u f )

A(Ai) A,

~ m "p(u).

Thus,

< log m + log "p( u )

max mpi
l:-S:i:-S:m U

The value of u for which the upper bound Is minimal Is typically unknown. If
104 CHAPTER 4

we keep u fixed, then the upper bound Is 0 (log( m)), and we are almost In the
domaIn In whIch E (Mn ) ~ log n flog log n. If 'IjJ( u ) < 00 for all u > 0 then
we can find a subsequence um i 00 such that 'IjJ( um ) :S m for all m. It Is easy
to see that the maxImum of the mp;' s Is 0 (log m ), so that
E(Mn):S logn (1+0(1)). If 'IjJ(loglogm):SmO(l), then
10g((Iog n )/ q )
log n
E (Mn ) = 0 ( ). Thus, the functIonal generatIng functIon aIds In
log log log n
the establishment of sImple verIfiable condItIons for dIfferent domaIns of behavIor
of E (Mn).

Remark 4.2. [Double bucketIng.]

It Is a rather straIghtforward exercise to show that for bounded f on [O,l]d ,
If all buckets are further subdIvIded Into grIds of sIzes N l ' . . . , N m , as Is done
In sectIon 1.5 for example, then, when m ~ en ,

E(Mn) ~ log log n

log log log n

Here Mn Is the maxImal cardInality In any of the buckets In the small grIds.
IntuItIvely, thIs can be seen as follows: for the orIgInal grid, Mn Is very close to
log n flog log n. For the buckets contaInIng about log n flog log n elements, we
obtaIn an estImate of E (Mn ) for the maxImal cardInality In Its sub-buckets by
applyIng the results of thIs sectIon after replacement of n by log n /Iog log n.
Thus, as a tool for reducIng the maxImal cardInality In the bucket data structure,
double bucketIng Is quIte efficIent although not perfect (because E (Mn ) -+ 00).

Remark 4.3. [Polssonlzatlon.]

The proof of Theorem 4.1 Is based upon Polssonlzatlon of the sample sIze.
The technIcal advantage Is that Mn ' a maxImum of dependent bInomIal random
varIables, Is replaced by Mn *, a maxImum of Independent PoIsson random varI-
ables. In fact, we can do wIthout the Polssonlzatlon by usIng specIal propertIes
of the multInomIal dIstrIbutIon. To lllustrate thIs, we could have used Mallows'
Inequallty:

m m
P( max N; :SX) < II peN; :Sx):Sexp(-~ peN; >x)),x 2:0
1::;;::;m ;=1 ;=1

(Mallows, 1968), from whIch one deduces wIthout work that

CHAPTER 4 105

E (max N j ) > E (max N j *)

l~j~m - 19~m

where, N 1 *, ... , N m * are Independent binomial random variables, distributed

Individually as N l ' . . . , N m • This can be used as a starting point for develop-
Ing a lower bound.

Remark 4.4. [HIstorical remark.]

Kolchln, Sevast'yanov and Chlstyakov (lg78, pp. g4-111) have studied In
some detall how Mn behaves asymptotically for different rates of Increase of m,
and for the uniform density on [0,1]. Their results can be summarized quite sim-
ply. A critical parameter Is ~, the average occupancy of a cell. There are three
m
cases:

Case 1. If n
m logm
-+ °as n -+ 00, then

Urn P(Mn = r-l) = e-X ,

n-+oo

Urn P(Mn = r) = 1 - e -x ,
n-+oo

where A Is a positive constant, and r = rn Is chosen In such a way that

n
e-"in(~r
n m
r > -, m------- -+ A. (Thus, asymptotically, Mn puts all Its mass on two
m r!
points.)

n
Case 2. ----'"-- -+ x E (0,00).
m log m
n n
Case 3. If ----'"-- -+ 00, then Mn /( - ) -+ 1 In probablilty.
m log m m

Case 1 Is by far the most Important case because usually m r-..J en. In cases 2
and 3, the asymptotic distribution of Mn Is no longer bl-atomlc because Mn
spreads Its mass more out. In fact, In case 3, Mn Is with high probablilty equal
to the value of the maximal cardlnallty If we were to distribute the n points
n

J 2:
evenly (not randomly!) over the m buckets! The difference Mn - - Is
m
r-..J log m In probablllty provided that m > nf for some f > 0.
106 CHAPTER 4

4.2. AN EXAMPLE: THE SELECTION PROBLEM.

Assume that a bucket structure Is used to find the k-th smallest of
X X n , Independent random varIables wIth densIty f on [0,1]. The m
l' . . . ,
1
buckets are of sIze - each, but what will be said below remaIns valid If the m
m
buckets are defined on [mIn X j ,max X j ]. In the algorIthm, we keep a count for
each bucket, so that In one addItIonal pass, It Is possIble to determIne In whIch
bucket the k-th smallest poInt lies. WithIn the bucket, thIs element can be found
In several ways, e.g. vIa a linear worst-case comparIson-based algorIthm
(Schonhage, Paterson and PIppenger, 1976; Blum, Floyd, Pratt, RIvest and Tar-
jan, 1973), via a linear expected tIme comparison-based algorIthm (Floyd and
RIvest, 1975; Hoare, 1961), or vIa a comparIson-based sortIng method. In the
former two cases, we obtaIn linear worst-case tIme and linear expected tIme
respectIvely, regardless of how large or small m Is - we mIght as well choose
m = 1. The constant In the tIme complexity mIght be smaller though for
m > 1. If the buckets have cardInalitIes N l' . . . , N m , then the tIme taken by
the linear worst-case algorIthm Is bounded by

v= an + /3 max N j
::;m
l::;j
+ '1m

where a, /3, 'I > °

are constants, and the mIddle term describes the contributIon
of the linear worst-case comparison-based selectIon algorIthm. While we can
obvIously bound all of thIs by (a+/3)n +'Im (whIch would lead us to the choIce
m=l), It Is InstructIve to mInImIze E(V). As we will see, It will be to our
advantage to take m proportIonal to Vn, so that E (V) = an + o( Vn) as
n -+ 00.
The suggestIon to take m proportIonal to Vn was also made by Allison and
Noga (1980), but theIr algorIthm Is dIfferent, In that wIthIn a selected bucket, the
algorithm Is applied recursIvely. Note that the algorIthm suggested here Is more
space efficient (sInce It Is not recursIve) but far less elegant (since It Is a hybrid of
a bucket algorithm and a fairly complicated linear comparison-based selection
algorithm).
We note here that max N j Is used In the definition of V because we do not
know beforehand which order statistic Is needed. For example, the situatIon
would be quite different If we were to ask for an average time, where the average
Is taken over all n possIble values for k - In that case, the mIddle term would
have to be replaced by /3ENj 2, and we can apply some of the analysIs of chapter
1.
If sortIng Is used wIthIn a bucket, then the total tIme for selectIon Is
bounded by
CHAPTER 4 107

v = an + (3 max N j log(Nj
1~j~m
+ 1) + ,m •

or

v = an + (3 max
1~j~m
Nj 2 + ,m .
dependIng upon whether an n log n or a quadratIc sort Is used. To obtaIn a
good estImate for E(V), we need good estImates for E(Mn log (Mn+1)) and
E (Mn 2), I.e. for expected values of nonlinear functIons of Mn. ThIs provIdes
some of the motIvatIon for the analysIs of sectIon 4.3. In thIs sectIon. we will
merely apply Theorem 4.2 In the desIgn of a fast selectIon algorIthm when a
linear worst-case algorIthm Is used wIthIn buckets. The maIn result Is gIven In
Theorem 4.3: thIs theorem applies to all bounded densItIes on [0,1] wIthout
exceptIon. It Is for thIs reason that we have to appeal, once agaIn, to the Lebes-
gue densIty theorem In the proof.

Theorem 4.3.
Define for posItIve a, (3. ,.

v= an + (3 max N j
1~j~m
+ ,m ,
where X l' . . . ,Xn are lid random varIables wIth bounded densIty f on [0,1]
f (x) ~ f * < 00 for all x. Then, for any q ,m :

n
an + ,m + (3-q
m
~ E(V)

where s

If we choose
108 CHAPTER 4

then

1 1

E{V) ~ an + 2V/hnl * + O{n 4" log2 n)

and, In fact

E (V) = an + 2V /hnl * {1+0 (I».

Proof of Theorem 4.3.

The proof of Theorem 4.3 Is based upon a crucial lemma.

Lemma 4.1.
For any bounded density I on [O,ljd, and for any sequence m -+ 00,
q = max mPi -+ I * = ess sup I·
19~m

Proof of Lemma 4.1.

We wlll use the fact that for such I, 11m {III Ir )1/r = I* {see Whee den
r-+oo
and Zygmund (1977, pp. 125-126)). Defining the density

I m (x ) = mPi ,x E Ai ,

on [O,ljd , we note that

I* ~ q = max
z
I m (x ) = ess sup Im ~ <f 1m r )1/r (any r) ,

and thus

11m Inf q r ~
m~oo
J 11m Inf 1m r
m-+oo
(Fatou's lemma)

= Jf r (Lemma 5.10)
CHAPTER 4 109

~(f*r-E

by choice of r = r (E), for arbitrary E > O. This concludes the proof of the
Lemma.

We continue now with the proof of Theorem 4.3. The starting point Is the
bound given Immediately following the proof of Theorem 4.2. The choice of t Is
asymptotically optimal when nq 1m log m -+ 00. Since q ~ 1 In all cases, this
follows If n 1m log m -+ 00, which Is for example satisfied when m ~ Vn, a
choice that will be convenient In this proof. The upper and lower bounds for
+ ,m + (J'!!:"'q.
J~
E (V), Ignoring lower order terms, are thus roughly em Because
m
q -+ f * (Lemma 4.1), the choice m = L nf *J Is again asymptotically
optimal. Resubtltutlon of this choice for m gives us our result.

Remark 4.5. [Choice of m .J

With the optimal choice for m, we notice that E (V) ~ an, I.e. the
expected value of the time taken by the algorithm has only one main contributor
- the set-up of the data structure. The other components, I.e. the traversal of the
buckets. and the selection within one particular bucket, take expected time
~ ..;(J,nf * each. Since f * Is unknown, one could use m ~ Vn Instead,
without upsetting the expected time structure: we will still have
E(V) = an + O(vn).
When f Is not bounded, and I or m Is not of the order of vn ,
the upper
bound of Theorem 4.3 should still be useful In the majority of the cases. Recall
the Inequalities for q obtained In Remark 4.1.

4.3. NONLINEAR FUNCTIONS OF THE MAXIMAL CAR-

DINALITY.
As we have seen In the study of the selection problem, and as we will see In
section 4.4 (extremal point problems), It Is Important to derive the asymptotic
behavior of
110 CHAPTER 4

where an l Is a gIven sequence of posItIve Integers (most often an I), =

Mn = max N i , and 9 (.) Is a work function satIsfyIng some regularIty· condI-
l<i<m
tlons. The followIng conditIons wlll be assumed throughout thIs sectIon:

(I) 9 Is nonnegatIve and nondecreasing on [0,00).

(ll) 9 (x) > 0 for x > 0
(m) g' (x) ~ a + bx B for some a ,b ,8 >0, all x 2:0.
(Iv) 11m 9 (x) = 00
%-+00

(v) 9 Is convex.
(vI) 9 Is regularly varyIng at InflnIty, I.e. there exIsts a constant p 2: 0 such that
for all u E R ,

11m g(ux) = up.

9 (x)
% -+00

Examples of such functIons Include

9 (x ) = Xr ,r 2: 1;

9 (x) = 1 + x log(I+X).

For the propertIes of regularly varyIng functIons, see Seneta (lg76) and Dehaan
(lg75) for example.
The maIn result of thIs sectIon Is:

Theorem 4.4.
Let 9 be a work functIon satIsfyIng (l-lv, vI), let X l' . . . ,Xn be lId random
vectors WIth bounded densIty f on [O,l]d , and let the grId have m '" en buck-
ets as n -+ 00 for some constant e > O. Then, for an as gIven above,
CHAPTER 4 111

If In addition, g (u ) ~ b * U 8 +1 for some b * > 0, and all u > 0, then

as n -+ 00.

If the work function satlfies (I-ll, lv-vI), then

log n )
E (g (an Mn)) ~ g ( an (Ho (1)).
log log n

If g satIsfies (I-vI), g (u ) ~ b * u s +1, some b * > 0, all u > 0, then

E (g (an Mn)) ~ g ( an log n ) .

log log n

If the work function satisfies (I-vi), then

E (g (M )) ~ g[ log n ).
n log log n

Proof of Theorem 4.4.

Let us define

where t > °Is arbitrary. We always have

112 CHAPTER 4

00

E(g(OI n Mn» ~ g(U)+ J P(g(OI n Mn) > t) dt

g(u)

00

= g(U) + J P(OI n Mn > V) g' (v) dv

u

00

= g (U) + J P(Mn > V) g' (Ol n V) OI n dv

u /a.
00

~g(U)+ J (a+bOl:VB)P(Mn >V)OI n dv

II /a.

00 _.!:...q -v log(...!!!!..)
~g(U)+ J (a+bOl;vB)me me enqOlndv
u /a.

by Lemma 5.5. If we can show that the Integral Is 0 (1), then we have

by conditions (Iv) and (vi) on g. Since € was arbitrary, we have shown the upper
bound In the theorem. By convexity of g, the lower bound follows easily from
theorem 4.1, Jensen·s Inequality and (vi):

log n ).
'" g (Ol n --=--
log log n

This leaves us with the proof of the statement that the second term Is 0 (1).
Note that q ~ f * , and that the bound of Lemma 5.5 remains valid If q Is for-
mally replaced by f *. It suffices to show that

u /a.
CHAPTER 4 113

because u / an i 00. But the Integral can be viewed as a tall-of-the gamma

Integral with respect to dv. Use v 8 ~ 2 8 -1(( ~)8 + (v -( ~ ))8), and
an an
u
v = -- + (v - -u- ) to obtain an upper bound of the form
an an

n
--q
-....!...log(~) e m
an mu 8 2 8 -1 e a. ea. nq
um
loge )
e an nq

The first of these two terms Is asymptotically dominant. It Is easlly seen that the
first term Is

( log(ma: +1 )+8 l o gII( - )n- - q -

tI tim)
-Iog(--)
o e a. m £r" e a" nq .

Note that ..!!!:.. remains bounded away from 0 and 00. Trivial calculations show
nq
that for our choice of u, the last expression Is 0 (1).
Consider finally all the statements InvolvIng the condItIon g (u) 2:: b· u 8 +1.
It Is clear that If the upper bounds for the Integral are 0 (g (u )) Instead of 0 (1),
then we are done. Thus, It sumces that the Integrals are 0 (u 8 +1), or 0 (an 8 +1).
This follows If

log m +s log(~) - ..!!:..q - ~ loge um ) ---+ -00 ,

an m an e an nq

log n
whIch Is satIsfied for u = (l+E) ---:;.--
log log n
114 CHAPTER 4

Theorem 4.4 Is useful because we can basIcally take the expected value
InsIde g. Recall that by Jensen's Inequality E (g (Mn» 2: g (E (Mn )) whenever g
Is convex. The opposIte Inequality Is provIded In Theorem 4.4. I.e. E (g (Mn» Is
1+0 (1) tImes larger than g (E (Mn)). maInly because Mn concentrates Its proba-
bility mas near E (Mn ) as n -+ 00.
The condItIons on g may appear to be a bIt restrIctIve. Note however that
all condItIons are satIsfied for most work functIons found In practIce. Further-
more. If g Is sufficIently smooth. then g I (x):s a + bx 8 and g (x) 2: b * X HI
can both be satIsfied sImultaneously.
A last word about Theorem 4.4. We have only treated bounded densIties
and grids of size m ~ en. The reader should have no difficulty at all to general-
Ize the techniques for use In other cases. For lower bounds. apply Jensen's Ine-
quality and lower bounds for E (Mn). and for upper bounds. use the Inequalities
given In the proof of Theorem 4.4.

4.4. EXTREMAL POINT PROBLEMS.

Extremal poInt problems are problems that are concerned wIth the
IdentificatIon of a subset of X l' . . . ,Xn which In some sense defines the outer
boundary of the "cloud" of points. The outer boundary Is Important In many
applicatIon. such as:
(I) pattern recognition: discrimination rules can be based upon the relative
posItion of a point with respect to the outer boundaries of the dIfferent
classes (see e.g. ToussaInt (1980. 1982».
(II) image processing and computer vision: objects are often characterized
(stored) vIa the outer boundary.
(III) statistics: poInts on the outer boundary of a collectIon of d-dlmenslonal
poInts can be considered as outllers. which need to be dIscarded before
further analysIs Is carried out on the data.
(Iv) computational geometry: The convex hull. one partIcularly sImple outer
boundary. plays a key role In varIous contexts In computatIonal geometry.
Often. InformatIon about the poInts can be derIved from the convex hull
(such as the dIameter of the collectIon of poInts).
CHAPTER 4 115

Outer layer

G e
e lID
e
•
• €I CI

• CI

•
•
Figure 4.1.
The convex hull and the outer layer of a cloud of points.

We will refer In this short section to only two outer boundaries: the convex hull
(the collection of all Xi I 8 having the property that at least one hyperplane
through Xi puts all n -1 remaining points at the same side of the hyperplane),
and the outer layer, also called the set of maximal vectors (the collection of
all Xi I 8 having the property that at least one quadrant centered at Xi contains
no X j , j ~i). Once again, we will assume that X 11 . . . ,Xn have a common
density f on [O,ljd. A grid of size m Is constructed In one of two ways, either
by partitioning [O,l]d or by partitioning the smallest closed rectangle covering
X 11 ." . . , X n • The second grid Is of course a data-dependent grid. We will go
through the mechanics of reducing the analysis for the second grid to that of the
first grid. The reduction Is that given In Devroye (1981). For SimpliCity, we will
consider only d =2.
116 CHAPTER 4

Figure 4.2.
Cell marking procedure.

For the outer layer In R 2, we find the leftmost nonempty column of rectan-
gles, and mark the northernmost occupied rectangle In this column. Let Its row
number be j (row numbers Increase when we go north). Having marked one or
more cells In column i, we mark one or more cells In column i +1 as follows: (I)
mark the cell at row number j, the highest row number marked up to that
point; (II) mark all rectangles between row number j and the northernmost occu-
pied rectangle In column i +1 provided that Its row number Is at least j +1. In
this manner a "staircase" of at most 2Vm rectangles Is marked. Also, any point
that Is a maximal vector for the north-west quadrant must be In a marked rec-
tangle. We repeat this procedure for the three other quadrants so that eventually
at most 8Vm cells are marked. Collect all pOints In the marked cells, and find
the outer layer by using standard algorithms. The naive method for example
takes quadratic time (compare each point with all other points). One can do
better by first sortIng accordIng to y-coordlnates. In an extra pass through the
sorted array, the outer layer Is found by keeping only partial extrema In the x-
direction. If heapsort or mergesort Is used, the time taken to find the outer layer
of n elements Is 0 (n log n) In the worst-case.
CHAPTER 4 117

e Outer layer point

Figure 4.3.
Finding the outer layer points for the north-west quadrant.

Thus, returning to the data-Independent grid, we see that the outer layer can be
found In time bounded by

com + c1n + c 3 L-Nj log(L-Nj + 1)

jEB jEB

where co' c I' C 2' C 3 > 0 are constants and B Is the collection of Indices of
marked cells. The random component does not exceed c 2 (8VmMn)2 and
c 3 8VmMn log(1+8VmMn ) respectively. Clearly, these bounds are extremely
crude. From Theorem 4.4, we recall that when m cn, f Is bounded,
r-..J

log n
E (Mn 2) r-..J ( )2, and E (Mn log(l+Mn )) r-..Jlog n. Thus, the expected
log log n
log n 2 r-
time Is O(n( )) In the form er case, and com + c1n + O(vn log n)
log log n
118 CHAPTER 4

In the latter case. In the latter case, we observe that the contribution of the
outer layer algorithm Is asymptotically negligible compared to the contribution of
the bucket data structure set-up. When we try to get rid of the boundedn~s
condition on f , we could argue as follows: fi.rst of all, not much Is lost by
replacing log( ~ N j + 1) by log(n +1) because ~ N j = O( Vm) and m ~ cn.
jEB jEB
Thus,

E( ~ N j log( ~Nj + 1»
jEB jEB

~ E(~Ni) log(n+1)
iEB

~ 8Vm log(n +1) E (Mn )

< r-
8ym log(n +1)
[-
log m n e t -1 )
- + - q ( - - ) (all t > 0)
- t m t

where q = max(mp 11 mPm) (Theorem 4.2). For constant t, we see that

••• ,
q e t_l
the upper bound Is 0 (n) + 8n log(n +1) r- - - . This Is 0 (n) for exam-
.;n ym t
pie when q = 0 (--), m ~ cn. This Is the case when
log n

Jf He < 00

for SOme! > 0 (Remark 4.1). See however the Important remark below.

Remark 4.6 [Optimization with respect fo m.J

We can once again tailor our grid to the problem by choosing m. Recall
that an upper bound for the expected time complexity Is
r- log m n e t -1
c1n + c 2m + c 3 ym log(n+l)(-- + - q ( - - » where c l ' c 2' c 3' t > 0
t m t
are constants. We can fi.rst choose t to approximately minimize the bound: for
example, minimization of
CHAPTER 4 119

suggests the value t = y' 2m nq log m, and we obtain

cin +c 2 m +c 3 rm IOg(n+1)[(2+0(I)Jnq~:m + ~q)

If m : : m -+ O. If we now minimize c 2m + c 3 rm q log(n +1), we obtain

the recipe

2/3
m [~.nqIOg(n+1»)
2c
.
2

Plugging this back Into our condition for the use of the bound, we note that It Is
satlsfl.ed In all cases since nq -+ 00. The bound becomes

+ c3 [.Jf + 0(1») log n Jnq log(nq).

Which term Is asymptotically dominant depends upon the density f. If f Is

bounded, then the upper bound Is c In + (K +0 (1» f *2/3(n log n ?/3 where K
does not depend upon f and f * Is the bound for f. We can also design the
grid for a particular class of densities. For bounded densities, we can take

* log n)
2/3
m [~nf
2c 2

and for densities with J.'r = cJ f r )l/r < 00, we can take
120 CHAPTER 4

m m
1.
r /J r log n I2/3

or, solving for m :

2r
C3 ) 3r-2
m [ - - n /J log n .
2c 2 r

This yields useful choices for r > 2. Using q ~ /J r m l/r , we obtain the further
bound

2r
C 1n +0 «n log n ) 3r-2) •

The main conclusion Is that If m Is growing slower than n , then for certain large
classes of densities, the asymptotically most Important component In the
expected time complexity Is c 1 n. For example, when Jf 4 < 00, we have
c In + o((n log n )4/5).

Of course, the same algorithm and discussion can be used for finding the
convex hull of X 1> ••• ,Xn because for arbitrary pOints there exist simple
o (n log n) and 0 (n 2) worst-case algorithms (see Graham (lg72) , Shamos
(lQ78), Preparata and Hong (lQ77) and Jarvis (lg73)) and all convex hull points
are outer layer points. In this form, the algorithm was suggested by Shamos
(lQ7Q).

Remark 4.7. [Bucket structure In polar coordinates.]

CHAPTER 4 121

Figure 4.4.
Points are ordered according to angular coordinates
for use in Graham's convex hull algorithm , bucket algorithm.

The bucket data structure can be employed In unexpected ways. For exam-
ple, to find the convex hulls In R 2, It suffices to transform X eX , ... ,Xn -x Into
polar coordinates where x Is a point known to belong to the Interior of convex
hull of X l' . . . ,Xn (note: we can always take X =X 1)' The points are sorted
according to polar angles by a bucket sort as described In chapter 2. This yields
a polygon P. All vertices of P are visited In clockwise fashion and pushed on a
stack. The stack Is popped when a non-convex-hull pOint Is Identified. In this
manner, we can construct the convex hull from P In linear time. The stack algo-
rithm Is based upon Ideas first developed by Graham (Hl72). It Is clear that the
expected tIme of the convex hull algorithm Is 0 (n) If J g 2 < 00 or
J g log+g < 00 where g Is the density of the polar angle of Xi -x, i 2: 1. For
example, when X l' . . . ,Xn have a radially symmetrIc densIty f , and x Is
taken to be the orIgin, the g Is the uniform density on [0,211"J, and the algorIthm
takes 0 (n ) expected time. When x Itself Is a random vector, one must be care-
J
ful before concluding anything about the finiteness of g 2. In any case, g Is
bounded whenever f Is bounded and has compact support.
122 CHAPTER 4

The results about E (Mn), albeit very helpful, lead sometimes to rather
crude upper bounds. Some Improvement Is possible along the lines of Theorem
4.5 (Devroye, 1985).

Theorem 4.5.
Let X l' . . . ,Xn be Independent random vectors with common density f
on [6,1]2, let the grid have m cells, and let q = max(mp v ... , mPm). Then, If
B Is the collection of Indices of marked cell In the extremal cell marking algo-
rithm,

n
-q
m
E(ENj ) ~ 8Vm n
jEB --q
1-e m

In particular, If m ~ en (for some constant e > 0),

E(ENj ) ~ (8+0(1))
1
jEB
1-e c

and

(8+0 (1)) .!..+.!.. .!..+.!.. .!..

E(ENj)~ n 2 re r 2cJfr)r
jEB 1
l-e c

for all r 2: 1.

Proof of Theorem 4.5.

We note that each N j Is stochastically smaller than a binomial (n ,Pj ) ran-
dom variable conditioned on the variable being at least 1. Thus,

n
npj npj -q
E(Nj ) ~ ____ < _~_ < __
m_ _
l-(l-pj)n l_e-npi _..!!...q
l-e m
CHAPTER 4 123

The first Inequality follows trIvIally from thIs. The second Inequality Is obvIous,
Cf
and the thIrd Inequallty Is based upon the fact that q ~ m l/r f r )1/r •

In the proof of Theorem 4.5, we have not used the obvIous Inequality
E Ni ~ sVmMn • If we find the outer layer or the convex hull by an
iEB
o (n log n) worst-case time method, then under the conditions of Theorem 4.5,
wIth m ~ cn , the expected tIme Is bounded by

o (n ) + 0 (In q) log n

and thIs does not Improve over the bound obtained when the crude Inequality
was used. For example, we cannot guarantee linear expected tIme behavIor when
J J
f 2 < 00, but only when a stronger condItIon such as f HE < 00 (some
f. > 0) holds. (We can of course always work on m, see remark 4.6).

There Is, however, a further possIble Improvement along the llnes of an outer
layer algorIthm of Machll and IgarashI (1gS4). Here we eIther find the outer
layers In all cells Ai' i E B, or sort all poInts In the Individual cells. Then, In
another step, the outer layer can be found In tIme llnear In the number of poInts
to be processed. Thus, there are three components In the time complexity: n +m
(set-up), E Ni log(Ni +1) (or E Ni 2) (sortIng), and E N j (final outer layer).
iEB iEB iEB
It should be clear that a sImilar strategy works too for the convex hull. The
principle Is well-known: dlvlde-and-conquer. It Is better to delegate the work to
the Individual buckets, In other words. For example, we always have

E (E Ni log(Nj + 1»
iEB

~ sVm E (Mn log(Mn + 1»

~ sVm log(n + 1) E (Mn ) ,

and, If we use a more refined bound from the proof of Theorem 4.5 combined
wIth Lemma 5.6,

E (E Ni log(Ni + 1»
iEB
124 CHAPTER 4

.!!:....q log(2 + .!!:....q )

~ 8Vm m
n
m
--q
1-e m

For example, when m ---> 00, n /m ---> 00, f ~ f * < 00, the bound Is

~ ~ f * log(.!!:....) .
Vm m

The optimal choice for m Is proportional to (f * n log n ?/3, so that the

expected time complexity for the algorithm Is e 1 n (for the set-up)
+ 0 «n log n )2/3). In another example, If m ~ en, q ---> 00, the upper bound
Is

~ Vn _8_ q log q ,
JC

which In turn Is 0 (n) when q = 0 (Vn /log n).

We turn now to the problem of data-dependent grids, and In particular grids

of size m formed by partitioning the smallest closed rectangle covering all the
pOints. For the convex hull and outer layer algorithms considered here, the ran-
dom terms are either

~ N j 10g(Nj + 1)
iEB

or

If dlvlde-and-conquer Is used, and

(~ Ni ) 10g(~Ni + 1)
iEB jEB
CHAPTER 4 125

or

otherwIse. All these terms are bounded from above by g (an Mn ) where an Is an
Integer, g Is a work functIon and Mn = max N j • Unfortunately, our analysIs
1<j<m
of Mn and g (Mn) does not apply here because the grId Is data-dependent. The
dependence Is very weak though, and nearly all the results gIven In thIs sectIon
remaIn valid If f has rectangular support [0,1]2. (Note: the rectangular support
J
of f Is the smallest rectangle R wIth the property that R f = 1.) To keep
thIngs sImple, we will only be concerned wIth an upper bound for E (g (an Mn ))
that Is of the correct order of Increase In n - In other words, we will not be con-
cerned wIth the asymptotIc constant. ThIs case can easily be dealt wIth vIa a
"shlfted grId" argument (Devroye, 1981). PartItIon [0,1]2 (or [0,1]d for that
matter) Into a grId of sIze m /2 d wIth member cells B j • Then consIder for each
U l' " " " , jd) E {0,1}d the shIfted grId wIth member cells B j (j v " " " , J"d),
1 <i <.!!!:.., where the shIft vector Is
- - 2d
128 CHAPTER 4

Shifted grid

Original grid
Figure 4.5.
Illustration of the shifted grid argument.

The key observation Is that every Aj In the original data-dependent grid Is con-
tained In some Bk (j l' . . . , jd)' Thus,

Mn < max Mn * (j l' ... , jd)

h .. ·, j,

where Mn * (j v ... , j d) Is the maximal cardinality for the (j v ... , jd) grid.
Thus,

j 1, . . . , ia

Each Individual term on the right hand side Is for a data-Independent grid, for
which we can derive several types of Inequalltles. Thus, typically, the expected
value of the right hand side Is about 2d times the expected value of one term.
For example, If f Is bounded and m ~ en, then for an ,9 as In Theorem 4.4,
d log n
the expected value of the right hand side Is ~ (1+0 (1))2 9 (an ).
log log n
CHAPTER 4 127

Chapter 5

AUXILIARY RESULTS FROM

PROBABILITY THEORY

5.1. PROPERTIES OF THE MULTINOMIAL DISTRIBU-

TION.
A random vector (Y 11 ... , Yk ) Is multinomial (n;p l' . . . , Pk) when

k
where ~ Pj = 1 and all Pi' 8 are nonnegative. Y 1 Is said to be binomial
j=1
(n,p 1)'

Lemma 5.1. [Moments of the multinomial distribution; see e.g. Johnson and
Kotz,11J61J]
For Integer r,8 2: 1:

= pjrp/n(n-1) ...(n-r-8+1), i=/oj.

Thus,
128 CHAPTERS

E(yj 3 ) = npj + 3n(n-l)pj 2 + n(n-l)(n-2)pj 3,

E(Yj4) = npj + 7n(n-l)pj 2 + 6n(n-l)(n-2)pj 2 + n(n-l)(n-2)(n-3)pj 4,

and for i =/:j,

E(Yj Y j ) = n(n-l)pjPj ,

E(Yj (Yj-l)Yj(Yj-l)) = n (n-l)(n-2)(n-3)pj 2 p/ '

and

E (Yj 2y/) = n (n -l)(n -2)(n -3)pj 2p/

+ n (n -l)(n -2)(pj Pj 2+pj 2pj ) + n (n -l)pj Pj

Lemma 5.2. [Moment generating function of the multinomial distribution.]

The random vector Y 11 ••• J Yk has moment generating function

k k
E (exp( ~ tj Y j )) = ( ~ Pj exp(tj ))n
j=l j=l

Lemma 5.3. [Uniform bounds for the moments of a binomial random vari-
able.]
If Y Is binomial (n ,p) and r > 0 Is a constant, then there exist a ,b > 0
only depending upon r such that

E (yr) :s a (np Y + b .
CHAPTERS 129

Proof of Lemma 5.3.

When r :S 1, we have E (yr) :S (np)r, by Jensen's Inequality. We will
thus assume that r > 1. Anderson and Samuels (1965) have shown that for all
k ~ np +1, P (Y ~ k) :S P (z ~ k) where Z Is a Poisson (np) random vari-
able. Thus,

E (yr) :S (np +1)' + E (yr ly~np +1) :S (np +1)' + E (zr I z ~np+l)

:S (np +1)' + (np +r)r +

k r (np )k e -np
k >np +r k!

Because (u +v y :S 2 r -1( U r +v r), the first two terms In the last sum are not
greater than a (np )' + b for some constants a, b only depending upon r. The
last sum can be bounded from above by

~
L.J
(_k_)r (np )k-r
~~- e- np (np y .
k>np+r k-r (k-r)!

Assume that np > 1. Then this Is not greater than

(np)r (1 +...!:....)' :S (1+r)'(np)'.

np

For np :S 1, we have E (yr) :S 2 r + E (zr) where Z Is Poisson (1). This con-

cludes the proof of Lemma 5.3.

Lemma 5.4.
Let 9 (u) be a nonnegative nondecreaslng function on [0,00), and let Y be
binomial (n ,p). Then If 9 (u) = 0 on (-00,0),

E(g(Y)) >
-
..!..g(np-v'TiP).
2

1 1
If P E [O''4 J, we have E(g(Y)) ~ '2
g (Ln p J ).
130 CHAPTER 5

If also 9 (u )/ u" ! as u -+ oo4'or some finite constant k , then

E(g(Y)):::; a max(g (np),g (1))

for some finite constant a depending upon k only.

Proof of Lemma 5.4.

For any t ~ 0, we have E (g (Y)) ~ 9 (t ) P (Y ~ t). Now, by the
Chebyshev-Cantelll Inequality,

P(Y <_ np-v'np(l-p)) = P( Y-np < -1) < _1_ =..!..

v'np (l-p) - - HI 2

Thus,

E(g(Y)) >
-
..!..g (np-v'np (l-p)) > ..!..g(np-vnp).
2 - 2

The second Inequality follows directly from Theorem 2.1 In Slud (1977). Next,

E (g (Y)) = E (g (Y)Iy $ np) +E (g (Y )Iy >np )

:::; 9 (np) + E ((g (y)/Y") yk IY>np)

:::; g(np) + g(np) E(yk)

(np )"

:'S 9 (np ) + 9 (np)a + bg (np )/(np )k

where a ,b are the constants of Lemma 5.3. If np ~ I, the last sum Is not
greater than 9 (np )(Ha +b). If np :::; I, we have E (g (Y)) :::; E (g (Z))
1
:::; 9 (1)(l+a +b) where Z Is a binomial (n ,-) random variable. This concludes
n
the proof of Lemma 5.4.
CHAPTER 5 131

Lemma 5.5. [Maximum of a multinomial random vector.]

Let B be a binomial (n ,p ) random variable. Then, for arbitrary x > 0,

PCB 2': x) :S e -np ( enp )X

x

If N v ... J Nm Is mutlnomlal and x 2': q

= max(mp IJ ... J mPm)' then

n
P( max Ni 2': xl < me --;:q (enq)X
l::;i::;m mx

Proof of Lemma 5.5.

For the first part, we use Chernoff's bounding method (Chernoff, 1952) and
note that for any t > 0:

< e-tx + np(e'-l)

-Iog(..!..)x + np (..!..-l)
= e np np

where we took e t = -=-, since this choice minimizes the upper bound.
np
Note
that the upper bound remains valid when B Is binomial (n ,p I ), p' :S p. For
the multinomial distribution, we apply Bonferronl's Inequality.

Lemma 5.6. [Logarithmic moment of the binomial distribution.]

Let Y be binomial (n ,p ). Then

E (Y log(H Y)) :S np log(2+np).

132 CHAPTERS

Proof of Lemma 5.6.

Let Z be a binomial (n -l,p ) random variable. Then, by Jensen's Inequal-
Ity,

E(Y log(HY))= i~1l:) i log(i+1)pi(l-p)n-i

.t
1=1
(np) r:~:) p i-1(1_p i n - 1)-{i-1) log(i +1)
= np E (iog(Z +2))

:S np log(E (Z )+2)

:S np log(np +2).

5.2. PROPERTIES OF THE POISSON DISTRIBUTION.

Lemma 5.7. [Exponential Inequality for the Poisson tall.]

If Y Is Poisson (A) distributed, then

Proof of Lemma 5.7.

By Chernoff's bounding technique, we have

:S 2 e A(e'-1-t-tf)

where we used the fact that e -t :S e t -2t. The exponent e t -l-t (l+E) Is
CHAPTER 5 133

mlnlmallC we take t = log(l+e), and this gives the bound

2 exp(A(e-(1+e)log(l+e))) :::; 2 exp(-Ae 2 /2(1+e)) .

Here we used the Taylor's series with remainder term to obtain the last inequal-
Ity.

Lemma 5.S. [Fourth moment inequality Cor the Poisson tall.]

If Y is Poisson (A) distributed, then

4
PCI Y-AI>>'e)
-
< - , all e > O.
- >.2 f 4

Proof of Lemma 5.8.

By Chebyshev's inequality,

PCI Y -AI> Af) < E <I Y _AI4)

- - (Ae)4

= A+3A 2 < _4_ .

A4e4 - A2e 4

Lemma 5.9. [precise estimates oC the Poisson tai1.]

Let Y be a Poisson (>.) random variable, and let k be a fixed integer. Then,
Cor k+l > A,

1 :::; P(Y>k) < k+1

P(Y =k) k +1-A
134 CHAPTERS

Proof of Lemma 5.g.

Observe that

5.3. THE LEBESGUE DENSITY THEOREM.

In this section we give several forms of the Lebesgue density theorem, that
will enable us to obtain theorems without continuity conditions on f. For
proofs and additional details, we refer to Wheeden and Zygmund (1977) and to
de Guzman (1975, 1981).

Lemma 5.10.
Let A be the class of all rectangles containing the origin of R d, and with
sides8 11 " , 1 8d satisfying aj 8j:s :s
bj for some fixed positive numbers
aj :s
bj , 1 :s :s
i d.
There exists a set D ~ R d such that "A(D C ) = 0 (D C Is the complement of
D) and

sup J f /"A(x +rA ) - f (x) I ~ 0 as r ~ 0, all xED.

A EA x+rA

Proof of Lemma 5.10.

See Wheeden and Zygmund (1977) or de Guzman (1975, 1981).

Lemma 5.11.
Let C be a fixed rectangle of R d with sides C 11 . . • 1 cd. Let {An} be a
sequence of rectangles tending to C as n ~ 00. Let A n be the collection of all
translates of An that cover the origin. Then, for any sequence of positive
numbers rn ! 0,
CHAPTERS 13S

11m sup 1 J f /)..(x +rn A ) - f (x)1 = 0 , almost all x.

n-+ooAEA" z+r.A

The set on which the convergence takes place does not depend upon the choice of
the sequences An and rn •

Lemma 5.12. [Scheffe's theorem (1947).]

Let f n be a sequence of densities converging almost everywhere to a density
f on Rd. Then

as n -+ 00.

Proof of Lemma 5.12 ..

Note that

II J
f n - f 1 = 2 (f - f n )+ -+ 0 ,
where we used the almost everywhere convergence of f n to f and the Lebesgue
dominated convergence theorem.
136 REFERENCES

REFERENCES

A.V. Aho, J.E. Hopcroft, and J.D. Ullman, Data Structures and Algorithms,
Addison-Wesley, Reading, Mass. (HI83).
S.G. Akl and H. Meijer, "Recent advances In hybrid sortIng algorIthms,"
Utilitas Mathematica 21 pp. 325-343 (HI82).
S.G. Akl and H. Meijer, "On the average-case complexity of bucketing algo-
rIthms,"' Journal of Algorithms 3 pp. 9-13 (1982).
D.C.S. Allison and M.T. Noga, "SelectIon by dIstrIbutIve partItIonIng," In-
formation Processing Letters 11 pp. 7-8 (1980).
T.W. Anderson and S.M. Samuels, "Some Inequalltles among bInomIal and
PoIsson probabilitIes," pp. 1-12 In Proceedings of the Fifth Berkeley Sympo-
sium on Mathematical Statistics and Probability, UnIversIty of CalifornIa
Press (1965).
T. Asano, M. Edahlro, H. Imal, M. Irl, and K. Murota, "PractIcal use of
bucketing technIques In computational geometry," pp. 0-0 In Computational
Geometry, ed. G.T. Toussalnt,North-Holland (1985).
J. Beardwood, J.H. Halton, and J.M. Hammersley, "The shortest path
through many poInts,"' Proceedings of the Cambridge Philosophical Society
55 pp. 299-327 (1959).
R. Bellman, "Dynamlc programmIng treatment of the travelling salesman
problem," Journal of the ACM 9 pp. 61-63 (1962).
J.L. Bentley, "Solutions to Klee's rectangle problems," Technical Report,
Department of Computer SCience, Carnegie-Mellon University, Pittsburgh,
PA. (1977).
J.L. Bentley, D.F. Stanat, and E.H. Wlillams, "The complexity of ftxed-
radius near neighbor searching,"' Information Processing Letters 6 pp. 209-
212 (1977).
J.L. Bentley and J.H. Friedman, "Data structures for range searching,"'
ACM Computing Surveys 11 pp. 397-409 (1979).
J.L. Bentley, B.W. WeIde, and A.C. Yao, "OptImal expected-time algorithms
for closest point problems,"' ACM Transactions on Mathematical Software
6 pp. 563-580 (1980),
REFERENCES 137

J.L. Bentley and D. Wood, "An optimal worst-case algorithm for reporting
Intersections of rectangles;' IEEE Transactions on Computers C-29 pp.
571-577 (1980).
M. Blum, R.W. Floyd, V. Pratt, R.L. Rivest, and R.E. Tarjan, "Time
bounds for selection," Journal of Computers and System Sciences 7 pp. 448-
461 (1973).
D. Cheriton and R.E. Tarjan, "Finding minimum spanning trees," SIAM
Journal on Computing 5 pp. 724-742 (1976).
H. Chernoff, "A measure of asymptotic efficiency for tests of a hypothesis
based on the sum of observations," Annals of Mathematical Statistics 23 pp.
493-507 (1952).
Y.S. Chow and H. Teicher, Probability Theory, Springer-Verlag, New York,
N.Y. (1978).
N. Chrlstofides, "Worst-case analysis of a new heuristic for the traveling
salesman problem," Symposium on Algorithms and Complexity, Department
of Computer Science, Carnegie-Mellon University (1976).
L. Dehaan, "On Regular Variation and Its Application to the Weak Conver-
gence of Sample Extremes," Mathematical Centre Tracts 32, Mathematlsch
Centrum, Amsterdam (1975).
L. Devroye and T. KlIncsek, "Average time behavior of distributive sorting
algorithms," Computing 26 pp. 1-7 (1981).
L. Devroye, "On the average complexity of some bucketing algorithms,"
Computers and Mathematics with Applications 7 pp. 407-412 (1981).
L. Devroye, "On the expected time required to construct the outer layer of a
set of points," Information Processing Letters 0 pp. 0-0 (1985).
L. Devroye, "Expected time analysis of algorithms In computational
geometry : a survey," pp. 0-0 In Computational Geometry, ed. G.T.
Toussalnt,North-Holland (1985).
L. Devroye, "The expected length of the longest probe sequence when the
distribution Is not uniform," Journal of Algorithms 6 pp. 1-9 (1985).
L. Devroye and F. Machell, "Data structures In kernel density estimatIon,"
IEEE Transactions on Pattern Analysis and Machine Intelligence 7 pp. 360-
366 (1985).
D. DobkIn and R.J. LIpton, "MultIdImensIonal searchIng problems," SIAM
Journal on Computing 5 pp. 181-186 (1976).
W. Doboslewlcz, "SortIng by dIstrIbutIve partItionIng," Information Process-
ing Letters 7 pp. 1-6 (1978).
138 REFERENQES

M. Edahlro, I. Kokubo, and T. Asano, "A new point-location algorithm and

Its practical efficiency --- comparison with existing algorithms," Research
memorandum RMI 83-04, Department of Mathematical Engineering and In-
strumentation Physics, University of Tokyo (IQ83).
G. Ehrlich, "Searching and sorting real numbers," Journal of Algorithms
2 pp. 1-12 (IQ82).
J.D. Esary, F. Proschan, and D.W. Walkup, "Association of random vari-
ables, with applications," Annals of Mathematical Statistics 38 pp. 1466-
1474 (IQ67).
R. Fagin, J. Nlevergelt, N. Pippenger, and H.R. Strong, "Extendible hashing
- a fast access method for dynamic files," ACM Transactions on Database
Systems 4 pp. 315-344 (IQ7Q).
P. FlaJolet, "On the performance evaluation of extendible hashing and trle
search," Acta Informatica 20 pp. 345-36Q (IQ83).
R.W. Floyd and R.L. Rivest, "Expected time bounds for selection," Com-
munications of the ACM 18 pp. 165-172 (lQ75).
S. Fortune and J. Hopcroft, "A note on Rabin's nearest-neighbor algorithm,"
Information Processing Letters 8 pp. 20-23 (IQ7Q).
J. Galambos, The Asymptotic Theory of Extreme Order Statistics, John WI-
ley, New York, N.Y. (IQ78).
J. Geffroy, "Contributions ala theorle des valeurs extremes," Publications de
l'ISUP 7 pp. 37-185 (IQ58).
B.V. Gnedenko, "Sur la distribution du terme maximum d'une serle
aleatolre," Annals of Mathematics 44 pp. 423-453 (IQ43).
G.H. Gonnet, "Expected length of the longest probe sequence In hash code
searching," Journal of the ACM 28 pp. 28Q-304 (IQ81).
G.H. Gonnet, A Handbook of Algorithms and Data Structures, Addlson-
Wesley, Reading, Mass. (IQ84).
T. Gonzalez, "Algorithms on sets and related problems," Technical Report,
Department of Computer Science, University of Oklahoma (IQ75).
R. Graham, "An efficient algorithm for determining the convex hull of a
finite planar set," Information Processing Letters 1 pp. 132-133 (1972).
J. Gurland, "Inequalities for expectations of random variables derived by
monotonlcltyor convexity," The American Statistician 22 pp. 26-27 (IQ68).
M. de Guzman, "Differentiation of Integrals In R n ," Springer Lecture Notes
In Mathematics # 481, Springer-Verlag, Berlin (IQ75).
REFERENCES 139

M. de Guzman, "Real Variable Methods In Fourier Analysis;' North-Holland

Mathematical Studies #46, North-Holland, Amsterdam (1981).
J.H. Halton and R. Terada, "A fast algorithm for the Euclidean traveling
salesman problem, optimal with probability one;' SIAM Journal on Comput-
ing 11 pp. 28-46 (1982).
J.A. Hartigan, Clustering Algorithms, John Wiley, New York, N.Y. (1975).
C.A.R. Hoare, "Find (algorithm 65)," Communications of the ACM 4 pp.
321-322 (1961).
M. Irl, K. Murota, and S. MatSUi, "Linear-time approximation algorithms for
finding the minimum-weight perfect matching on a plane," Information Pro-
cessing Letters 12 pp. 206-209 (1981).
M. Irl, K. Murota, and S. MatSUi, "Heuristics for planar minimum-weight
perfect matching," Networks 13 pp. 67-92 (1983).
R.A. Jarvis, "On the Identification of the convex hull of a finite set of points
In the plane;' Information Processing Letters 2 pp. 18-21 (1973).
N.L. Johnson and S. Kotz, Urn Models and Their Application, John WHey,
New York, N.Y. (1977).
R.M. Karp, "Probabilistic analysis of partitioning algorithms for the
traveling-salesman problem In the plane," Mathematics of Operations
Research 2 pp. 209-224 (1977).
D. Kirkpatrick, "Optimal search In planar subdivisions," SIAM Journal on
Computing 12 pp. 28-35 (1983).
D.E. Knuth, The Art 0/ Computer Programming, Vol. 1: Fundamental Algo-
rithms, Addison-Wesley, Reading, Mass. (1973). 2nd Ed.
D.E. Knuth, The Art 0/ Computer Programming, Vol. 9 : Sorting and
Searching, Addison-Wesley, Reading, Mass. (1973).
V.F. Kolchln, B.A. Sevastyanov, and V.P. Chlstyakov, Random Allocations,
V.H. Winston and Sons, Washington, D.C. (1978).
P .-A. Larson, "Expected worst-case performance of hash files," The Comput-
er Journal 25 pp. 347-352 (1982).
D.T. Lee and F.P. Preparata, "Location of a point In a planar subdivision
and Its applications;' SIAM Journal on Computing 6 pp. 594-606 (1977).
E.L. Lehmann, "Some concepts of dependence," Annals of Mathematical
Statistics 37 pp. 137-1153 (1966).
R.J. Lipton and R.E. Tarjan, "Applications of a planar separator theorem;'
pp. 162-170 In Proceedings of the 18th IEEE Symposium on the Foundations
of Computer Science, (1977).
140 REFERENCES

M. Loeve, Probability Theory, Van Nostrand, Princeton, New Jersey (1963).

M. Machll and Y. Igarashi, "A hashing method of finding the maxima of a
set of vectors," Technical Report CS-84-2, Department of Computer Science,
Gunma University, Gunma, Japan (1984).
M.D. Maclaren, "Internal sorting by radix plus sifting," Journal of the ACM
13 pp. 404-411 (1966).
C.L. Mallows, "An Inequallty Involving multinomial probabllltles," Biometri-
ka 55 pp. 422-424 (1968).
H. Meijer and S.G. Akl, "The design and analysis of a new hybrid sorting al-
gorithm," Information Processing Letters 10 pp. 213-218 (1980).
G. Nagy and S. Wagle, "Geographic data processing," Computing Surveys
11 pp. 139-181 (1979).
C.H. Papadlmltrlou and K. Stelglltz, "Some complexity results for the trav-
ellng salesman problem," pp. 1-9 In Eighth Annual ACM SIGACT Symposi-
um, (1976).
C.H. Papadlmltrlou, "The Euclldean travellng salesman problem Is NP-
complete," Theoretical Computer Science 4 pp. 237-244 (1977).
R.G. Parker and R.L. Rardin, "The travellng salesman problem: an update
of research," Naval Research Logistics Quarterly 30 pp. 69-96 (1983).
J. PlckandsIII, "Moment convergence of sample extremes," Annals of
Mathematical Statistics 39 pp. 881-889 (1968).
F.P. Preparata and S.J. Hong, "Convex hulls of finite sets of points In two
and three dimensions," Communications of the ACM 20 pp. 87-93 (1977).
M. Rabin, "Probablllstic algorithms," In Algorithms and Complexity, ed. J.
Traub,Academlc Press, New York, N.Y. (1976).
H.L. Royden, Real Analysis, Macmlllan, London (1968).
H. Samet, "T~e quadtree and related hierarchical data structures," Comput-
ing Surveys 16 pp. 187-260 (1984).
H. Scheife, "A useful convergence theorem for probablllty distributions," An-
nals of Mathematical Statistics 18 pp. 434-458 (1947).
A. Schonhage, M. Paterson, and N. Pippenger, "Finding the median," Jour-
nal of Computers and System Sciences 13 pp. 184-199 (1976).
E. Seneta, "Regularly Varying Functions," Springer Lecture Notes In
Mathematics #508, Springer-Verlag, Berlln (1976).
M.l. Shamos and D. Hoey, "Closest-point problems," Proceedings of the 16th
IEEE Symposium on the Foundations of Computer Science, pp. 151-162
REFERENCES 141

(1g75).
M.1. Shamos and J.L. Bentley, "Optimal algorithms for structuring geo-
graphic data;' Proceedings of the Symposium on Topological Data Structures
for Geographic Information Systems, pp. 43-51 (lg77).
M.1. Shamos, "Computational Geometry," Ph.D. Dissertation, Yale Universi-
ty, New Haven, Connecticut (lg7S).
M.1. Shamos, Ig7g.

G. Simons and N.L. Johnson, "On the convergence of binomial to Poisson

distributions," Annals of Mathematical Statistics 42 pp. 1735-1736 (lg71).
E.V. Slud, "Distribution Inequalities for the binomial law," Annals of Proba-
bility 5 pp. 404-412 (lg77).
J.M. Steele, "Subaddltlve Euclidean runctlonals and nonlinear growth In
geometric probability," Annals of Probability 9 pp. 365-376 (lgSI).
K.J. Supowlt, E.M. Reingold, and D.A. Plaisted, "The traveling salesman
problem and minimum matching In the unit square," SIAM Journal on Com-
puting 12 pp. 144-156 (lgS3).
M. Tammlnen, "Order preserving extendible hashing and bucket tries," BIT
21 pp. 41g-435 (lgSI).
M. Tammlnen, "Analysis of recursive bucket methods," Report HTKK-
TKO-B44, Helsinki University of Technology, Laboratory for Information
Processing Science, Otakaarl 1, SF-02150 Espoo 15, Finland (lgS2).
M. Tammlnen, "The extendible cell method for closest point problems," BIT
22 pp. 27-41 (lgS2).
M. Tammlnen, "Analysis of N-trees," Information Processing Letters 16 pp.
131-137 (lg83).
M. Tammlnen, "Two levels are as good as any," Journal of Algorithms 6 pp.
13S-144 (lgS5).
M. Tammlnen, "On search by address computation," BIT 25 pp. 135-147
(lg85).
G.T. Toussaint, "Pattern recognition and geometrical complexity," pp.
1324-1347 In Fifth International Conference on Pattern Recognition, (IgSO).
G.T. Toussaint, "Computational geometric problems In pattern recogni-
tion," pp. 73-g1 In Pattern Recognition Theory and Applications, ed. J.
KIttler, K.S. Fu and L.F. Pau,D. Reidel (lgS2).
V.K. Valshnavl and D. Wood, "Data structures for the rectangle contain-
ment and enclosure problems," Computer Graphics and Image Processing
13 pp. 372-384 (lgSO).
142 REFERENCES

V.K. Valshnavl, "Computing point enclosures," IEEE Transactions on Com-

puters C-31 pp. 22-29 (1982).
W.B. VanDam, J.B.G. Frenk, and A.H.G. Rlnnooy Kan, "The asymptotic
behaviour of a distributive sorting method," Computing 31 pp. 287-303
(1983).
B.W. Weide, "Statistical Methods In Algorithm Design and Analysis,"
Ph.D.Dlssertatlon, Carnegie-Mellon University, Pittsburgh, Pennsylvania
(1978).
R.L. Wheeden and A. Zygmund, Measure and Integral, Marcel Dekker, New
York, N.Y. (1977) .
. A.C. Yao, "An 0 ( E I I log log I V I ) algorithm for finding minimum span-
ning trees;· Information Processing Letters 4 pp. 21-23 (1975).
A.C. Yao, "On constructing minimum spanning trees In k-dlmenslonal space
and related problems," SIAM Journal on Computing 11 pp. 721-736 (1982).
G. Yuval, "Finding nearest neighbors," Information Processing Letters 5 pp.
63-65 (1976).
INDEX
INDEX
absolute contInuIty 14
all nearest neIghbor graph 83
all-nearest-nelghbor problem 80 90
Anderson-Samuels Inequality 129
assocIatIon Inequality 19 57
atomIc dIstrIbutIon 3
average successful search tIme 5 12
average unsuccessful search tIme 12
balanced bInary tree 66
Bernoulli random varIables 79
bInary search tree 66
bInary trle 8
bInomIal dIstrIbutIon 59 78-79 100-101
105 122 128-132
Bonferron\'s Inequality 131
bounded densIty 107-108
bucket searchIng 10-36
bucket sortIng 10-36
BWY condItIon 90-92
car parkIng model 73
Cauchy densIty 45
Cauchy-Schwartz Inequality 88-89
cell 55 105
cell markIng algorIthm 116 122
chaIn 3
Chebyshev-Cantelll Inequality 6 27130
Chebyshev's Inequality 20 133
Chernoff's boundIng technIque 4 28-29
131 132
143
chI-square test 1
close paIrs graph 81
close paIrs problem 80 84-85 86-90
closest paIr problem 80 91-92
closest poInt problems 58 80-92
clusterIng 83
collisIon 2-3
compact support 7
comparIsons 10
computatIonal geometry 1 5 5693 114
computer graphIcs 1
computer vIsIon 114
convergence In probability 17-23
convex hull 114 115 120-121
convex hull algorIthm 7 121
convex work functIon 110 112
data transformatIons 65
database 1
data-dependent grId 124-126
Delaunay trIangulatIon 92
densIty 5
dImensIon 9
dIscrete optImIzatIon 73
dIstrIbutIon functIon 65
dlvlde-and-conquer 123-124
double bucketIng 6 30-36 104
dynamIc bucket structure 8
dynamIc programmIng 76
144
economic surveys 1
edge length 70-71
empty cell test 1
essential supremum 94 108
estimate of locatIon 65
estimate of scale 65
ETSP 73-80
Euclidean mInImal spannIng tree prob-
lem 80 92
expected search time 69 72
expected time 1-2
expected value 10-17
exploratory data analysis 1
exponential distribution 42 53 67
exponential Inequality 28
exponential tall Inequality 132
exponentially dominated tall 65
extendible hashing 8
extremal point problems 114-126
Fatou's lemma 11 61 88 89 108
functional generating function 77-78
103-104
Gabriel graph 92
gamma function 93
geographic data processing 1
Graham's convex hull algorithm 121
graph 74
grid 3 55-56
grid size 8 9
Gurland's Inequality 51
hash functIon 1 2
hashing 1 14 93-94
INDEX
heapsort 64 116
heuristics 56-57 74-75 80
histogram 1 15
histogram approximatIon 15-16
Image processing 114
Infinite tails 37-54
Insertion sort 64
Internal point 121
Interval tree 72
Isolated point 83
Isolated pOints problem 80 83-85
Jensen's Inequality 4 13 14 19 23 25 32
43 62 63 79 89 100 101 103 112
114 129 132
large deviation Inequalities 26-30
Lebesgue density theorem 4 15 35 88
107 108 134-135
Lebesgue dominated convergence
theorem 20 34 35 135
Lebesgue measure 60
linear selection algorithm 106
linked list 3 72
logarithmic moment 131-132
Mallows' Inequality 104
management science 1
marginal distribution 63
Markov's Inequality 27
maximal cardinality 7 75-104
maximal gap 85-86
maxImal vectors 115
mergesort 116
minImal spanning tree 74 83 92
INDEX 145

minimum weight perfect planar planar point location problem 67-69

matching heuristics 57 point enclosure problem 57 72-73
moderate deviation Inequallty 30 point location problem 57 67-71
moment generating function 43 67 128 Poisson distribution 18-19 20 30 35 94
moment Inequallty 133 99 104 129 132-134
moments 127-128 Poisson random variable 18-19 21-23
monotone hash function 2 35
multidimensional bucketing 55-92 Polssonlzatlon 4 21-23 94-100 104-105
multidimensional grid 55-56 polar coordinates 120-121
multidimensional searching 57 64-73 polygon 100
multidimensional sorting 57 64-73 preprocessing time 54 72 109 118 124
multidimensional trie 8 probability Integral transform 65
multinomial distribution 4 24 28 104-
105127-132 query point 67-72
query rectangle 57-58
nearest neighbor 71 91-92 query time 58
neighbor bucket 81-82 84-85 quicksort 64
normal distribution 42 53 67
NP-hard 74 radial symmetry 121
N-trees 3(1 33 random tour 75
range of a sample 37
occupancy problems 105 range searching 58
optimal tour 74-75 80 rectangular point location problem 68
optimization of grid size 90 101-103 rectangular support 125
107-109 123-124 recursive bucketing 30
order statistics 6 106-109 recursive selection algorithm 106
order-preserving hash functions 1-2 regular variation 43 110
orthogonal range query 58 72 relative stability 41-42
outer layer 114-120 123-124 road map 71
robust estimate 65
parent pointer tree 83
pattern recognition 1 5 114 sample mean 65
peakedness 7 103 sample median 65
planar graph 67-70 92 sample quantlle 65
planar partition 67-69 satelllte data 1
146 INDEX

Scheffe's theorem 135 unbounded support 37-54

segment tree 72 uniform distribution 1 13 23 32
selection 106-109 uniform distribution 70 72 80 93-94
selection problem 106-109 uniform model 1
selection sort 64 unlformlzlng transformations 65
serpentine path 57 74-75 union-find algorithm 83
Shamos' convex hull algorithm 7 120
shifted grid 125-126 variance 23-26
shifted grid method 91 Voronol diagram 92
single-linkage clustering 83
slab method 69 weak convergence 17-23
Slud's Inequality 130 Weide's algorithm 90
sorted tour 75-76 work function 7 56-58 81-82 93 110-
spiral search 90 92 114 129-130
square Integral 13 worst-case search time 93
stack algorithm 121
standard deviation 65
statistics 1 114
Stirling's approximation 97
stochastic ordering 59 122
subllnear number of buckets 62 66
superllnear number of buckets 52-54
support of a density 38 60 121

Taylor's series expansion 78 133

transformations 65
traveling salesman problem 56-57 73-
80
traveling salesman tour 73
2-3 tree 66
tree Insertion sort 64
trle 8
truncation 1-2

unbounded density 103-104

PROGRESS IN COMPUTER SCIENCE
Already published

PCS 1 Mathematics for the Analysis of Algorithms

D. Greene, D. Knuth
ISBN 0-8176-3102-X, 127 pages, hardcover

PCS 2 Applied Probability - Computer Science: The Interface, Vol. I

R. Disney, T. Ott
ISBN 0-8176-3067-8,532 pages, hardcover

PCS 3 Applied Probability - Computer Science: The Interface, Vol. II

R. Disney, T. Ott
ISBN 0-8176-3093-7, 514 pages, hardcover

PCS 4 Notes on Introductory Combinatorics

G. P6lya, et al.
ISBN 0-8176-3123-2,202 pages, hardcover
ISBN 0-8176-3170-4, softcover

PCS 5 The Evolution of Programs

N. Dershowitz
ISBN 0-8176-3156-9,354 pages, hardcover
ISBN 0-8176-3171-2, softcover