Meshfree Chapter 3

MATH 590: Meshfree Methods
Chapter 3: Positive Definite Functions
Greg Fasshauer
Department of Applied Mathematics

Illinois Institute of Technology
Fall 2010
fasshauer@iit.edu MATH 590 Chapter 3 1

Outline
1 Positive Definite Matrices and Functions
2 Integral Characterizations for (Strictly) Positive Definite Functions
3 Positive Definite Radial Functions

We know that the solution of the scattered data interpolation
problem with RBFs and/or kernels amounts to solving a linear
system
Ac = y,
where Ajk = (kx j x k k2 ), j, k = 1, . . . , N.
Linear algebra tells us that this system will have a unique solution
whenever A is non-singular.
Necessary and sufficient conditions to characterize this general
non-singular case are still open.
We will focus on basic functions that generate positive definite
matrices.

Positive Definite Matrices and Functions
Definition
A real symmetric matrix A is called positive semi-definite if its
associated quadratic form is non-negative, i.e.,
N X
X N
cj ck Ajk 0 (1)
j=1 k =1
for c = [c1 , . . . , cN ]T RN .
If the quadratic form (1) is zero only for c 0, then A is called positive
definite.
Since the eigenvalues of a positive definite matrix are all positive a
positive definite matrix is non-singular.
We therefore look for basis functions Bk that generate a positive
definite interpolation matrix.
This leads to positive definite functions.

Definition
A complex-valued continuous function : Rs C is called positive
definite on Rs if
X N XN
cj ck (x j x k ) 0 (2)
j=1 k =1
for any N pairwise different points x 1 , . . . , x N Rs , and

c = [c1 , . . . , cN ]T CN .
The function is called strictly positive definite on Rs if the quadratic
form (2) is zero only for c 0.
Remark
For theoretical reasons the definition employs complex-valued
functions.
All of our applications will only be real-valued.
Because of the inequality in (2) we know that any (strictly) positive
definite function must have a real-valued quadratic form.
Remark
History (i.e., analysts in the 1920s and 30s) defined positive
definite functions in analogy to positive semi-definite matrices.
This concept is not strong enough to guarantee a non-singular
interpolation matrix and so we need to define strictly positive
definite functions as in [Micchelli (1986)].
This leads to an unfortunate difference in terminology used in the
context of matrices and functions.
Sometimes the literature is confusing about this and you might
find papers that use the term positive definite function in the strict
sense, and others that use it in the way we defined it above.
Hopefully, the authors are clear about their use of terminology.

Example
The function
B (x) = eixy , y Rs fixed,
is positive definite on Rs since its quadratic form is
N X
X N N X
X N
cj ck B (x j x k ) = cj ck ei(x j x k )y
j=1 k =1 j=1 k =1
N
X N
X
= cj eix j y ck eix k y
j=1 k =1
2
N
X
ix j y

= cj e 0.

j=1

Definition 2 suggests that we should use strictly positive definite

functions as basis functions for our scattered data interpolation
problem, i.e.,
N
X
Pf (x) = ck (x x k ), x Rs . (3)
k =1
Remark
At this point we dont require to be radial.
In fact, the interpolant obtained via Pf of (3) is translation invariant.
We will later specialize to strictly positive definite functions that are
also radial on Rs .
One can also generalize this setup by introducing (strictly) positive
definite kernels (a bit more later). Then we may even give up
translation invariance.

Theorem (Properties of (strictly) positive definite functions)
(1) Non-negative finite linear combinations of positive definite functions are positive
definite. If 1 , . . . , n are positive definite on Rs and cj 0, j = 1, . . . , n, then
n
X
(x) = cj j (x), x Rs ,
j=1
is also positive definite. Moreover, if at least one of the j is strictly positive

definite and the corresponding cj > 0, then is strictly positive definite.
(2) (0) 0.
(3) (x) = (x).
(4) Any positive definite function is bounded. In fact,
|(x)| (0).
(5) If is positive definite with (0) = 0 then 0.

(6) The product of (strictly) positive definite functions is (strictly) positive definite.

Proof.
Properties (1) and (2) follow immediately from Definition 2.
Property (5) follows immediately from (4).
Property (6) follows from Schurs theorem in linear algebra which
ensures that the elementwise (or Hadamard) product of positive
(semi-)definite matrices is positive (semi-)definite.
We will now give details for Properties (3) and (4).
For more details see [Cheney and Light (1999)] or
[Wendland (2005a)].

Proof (cont.)
To show (3) we let N = 2, x 1 = 0, x 2 = x, and choose c1 = 1 and
c2 = c.
Then we have the quadratic form
2 X
X 2
cj ck (x j x k ) = (1 + |c|2 )(0) + c(x) + c(x) 0
j=1 k =1
for every c C.
Now we can take c = 1 and c = i.
These, respectively, imply that both
(x) + (x) and i ((x) (x))
must be real.
This, however, is only possible if (x) = (x).

Proof (cont.)
For the proof of (4) we let N = 2, x 1 = 0, x 2 = x, and choose
c1 = |(x)| and c2 = (x).
Then we get the quadratic form
2 X
X 2
cj ck (x j x k ) = 2(0)|(x)|2 (x)(x)|(x)| 2 (x)|(x)|
j=1 k =1
0.
Since (x) = (x) by Property (3), this gives
2(0)|(x)|2 2|(x)|3 0.
If |(x)| > 0, we divide by |(x)|2 and get |(x)| (0) as claimed.

If |(x)| 0, then the statement holds trivially.

Example
The cosine function is positive definite on R.
First we remember that for any x R
1 ix
cos x = e + eix .
2
By our earlier example the function B (x) = eixy is positive definite for
any fixed y Rs .
Therefore, 1 (x) = eix and 2 (x) = eix are positive definite, and the
cosine function is positive definite by Property (1).

Property (3) shows that any real-valued (strictly) positive definite

function has to be even.
We even have
Theorem
A real-valued continuous function is positive definite on Rs if and
only if it is even and
N X
X N
cj ck (x j x k ) 0 (4)
j=1 k =1
for any N pairwise different points x 1 , . . . , x N Rs , and

c = [c1 , . . . , cN ]T RN .
The function is strictly positive definite on Rs if the quadratic form (4)
is zero only for c 0.
See [Wendland (2005a)] for a proof.

Integral Characterizations for (Strictly) Positive Definite Functions
We summarize some facts about integral characterizations of positive

definite functions.
They were established in the 1930s by Bochner and Schoenberg.
We will also mention more recent extensions to strictly positive definite
functions that are needed for the scattered data interpolation problem.
Many more details in [Wendland (2005a)].
We start with a very brief discussion of concepts from measure theory
and integral transforms that appear in the results later on.

Integral Characterizations for (Strictly) Positive Definite Functions Some Important Concepts from Measure Theory
Bochners theorem below is formulated in terms of Borel measures.

Definition
Let X be an arbitrary set and denote by P(x) the set of all subsets of
X . A subset A of P(X ) is called a -algebra on X if
(1) X A,
(2) A A implies that its complement (in X ) is also contained in A,
(3) Ai A, i N, implies that the union of these sets belongs to A.
Definition
Given an arbitrary set X and a -algebra A of subsets of X , a measure
on A is a function : A [0, ] such that
(1) () = 0,
(2) for any sequence {Ai } of disjoint sets in A we have

[
X
( Ai ) = (Ai ).
i=1 i=1
Integral Characterizations for (Strictly) Positive Definite Functions Some Important Concepts from Measure Theory
Definition
If X is a topological space, and O is the collection of open sets in X ,
then the -algebra generated by O is called the Borel -algebra and
denoted by B(X ).
If in addition X is a Hausdorff spacea , then a measure defined on
B(X ) that satisfies (K ) < for all compact sets K X is called a
Borel measure.
The carrier of a Borel measure is given by the set
X \ {O : O O and (O) = 0}.

a
X is a Hausdorff space if any two distinct points of X can be separated by
open sets

Integral Characterizations for (Strictly) Positive Definite Functions A few important integral transforms
Definition
The Fourier transform of f L1 (Rs ) is given by
Z
1
f () = p f (x)eix dx, Rs ,
(2)s Rs
and its inverse Fourier transform is given by

Z
1
f (x) = p f ()eix d, x Rs .
s
(2) R s
This definition from [Rudin (1973)] is based on angular frequency and

is used in [Wendland (2005a), Schlkopf and Smola (2002)].
Another, just as common, definition from [Stein and Weiss (1971)]
uses ordinary frequency, i.e.,
Z
f (x)e2ix dx = (2)s f (2).
p
fSW () =
Rs
It appears in [Buhmann (2003), Cheney and Light (1999)].
Similarly, we can define the Fourier transform of a finite (signed)

measure on Rs by
Z
1
() = p eix d(x), Rs .
(2)s Rs

Since we are mostly interested in positive definite radial functions, we

note that the Fourier transform of a radial function is again radial:
Theorem
Let L1 (Rs ) be continuous and radial, i.e., (x) = (kxk). Then its
Fourier transform is also radial, i.e., () = Fs (kk) with
Z
1 s
Fs (r ) = (t)t 2 J s2 (rt)dt,
r s2 0 2
s2
where J s2 is the classical Bessel function of the first kind of order 2 .
2
Proof in [Wendland (2005a)]. This transform is also called

Fourier-Bessel transform or Hankel transform.

Remark
The Hankel inversion theorem [Sneddon (1972)] ensures that the
Fourier transform for radial functions is its own inverse, i.e., for radial
functions we have
Fs [Fs ] = .

Integral Characterizations for (Strictly) Positive Definite Functions Bochners Theorem
One of the most celebrated results on positive definite functions is their

characterization in terms of Fourier transforms established by Bochner
in 1932 (for s = 1) and 1933 (for general s).
Theorem (Bochner)
A (complex-valued) function C(Rs ) is positive definite on Rs if and

only if it is the Fourier transform of a finite non-negative Borel measure
on Rs , i.e.,
Z
1
(x) = (x) = p eixy d(y), x Rs .
(2)s Rs

Proof.
There are many proofs of this theorem.
Bochners original proof can be found in [Bochner (1933)]. Other
proofs can be found, e.g., in [Cuppens (1975)] or
[Gelfand and Vilenkin (1964)].
A proof using the Riesz representation theorem to interpret the Borel
measure as a distribution, and then taking advantage of distributional
Fourier transforms can be found in [Wendland (2005a)].
We will prove only the one (easy) direction that is important for the
application to scattered data interpolation.

Proof (cont.)
We assume is the Fourier transform of a finite non-negative Borel
measure and show is positive definite.
Thus,
N
N X N
N X Z
X 1 X
i(x j x k )y
cj ck (x j x k ) = p cj ck e d(y)
j=1 k =1
(2)s j=1 k =1 Rs

Z N N
1 X X
= p cj eix j y ck eix k y d(y)
s
(2) R j=1
s
k =1
2
N
Z X
1 ix j y

= p cj e d(y) 0.

s
(2) Rs j=1

The last inequality holds because of the conditions imposed on the

measure .

Remark
Bochners theorem shows that for any fixed y Rs the function
B (x) = eixy , x Rs ,
of our earlier example can be considered as the fundamental positive

definite function since all other positive definite functions are obtained
as (infinite) linear combinations of this function.
While Property (1) of Theorem 4 implies that linear combinations of B
will again be positive definite, the remarkable content of Bochners
Theorem is the fact that indeed all positive definite functions are
generated by B .

Integral Characterizations for (Strictly) Positive Definite Functions Extensions to Strictly Positive Definite Functions
In order to guarantee a well-posed interpolation problem we have to

extend (if possible) Bochners characterization to strictly positive
definite functions.
We give a sufficient condition for a function to be strictly positive
definite on Rs .
Theorem
Let be a non-negative finite Borel measure on Rs whose carrier is a
set of nonzero Lebesgue measure. Then the Fourier transform of is
strictly positive definite on Rs .

Proof.
As in the proof of Bochners theorem we have
N
N X N X
N Z
X 1 X
i(x j x k )y
cj ck (x j x k ) = p cj ck e d(y)
j=1 k =1
(2)s j=1 k =1 Rs

Z N N
1 X X
= p cj eix j y ck eix k y d(y)
s
(2) Rs j=1 k =1
2
N
Z X
1 ix j y

= p cj e d(y) 0.

(2)s Rs j=1

Now let
N
X
g(y) = cj eix j y ,
j=1
and assume that the points x j are all distinct and c 6= 0.

Proof (cont.)
Since the x j are distinct the functions
y 7 eix j y
are linearly independent, and so g 6= 0.

Since g is an entire function its zero set, i.e., {y Rs : g(y) = 0} can
have no accumulation point and therefore it has Lebesgue measure
zero (see, e.g., [Cheney and Light (1999)]).
Now, the only remaining way to make the inequality
2
Z N Z
1 X ix y 1
p c j e j d(y) = p |g(y)|2 d(y) 0
(2)s s

R j=1

s
(2) R s
an equality is if the carrier of is contained in the zero set of g, i.e.,

has Lebesgue measure zero.
The hypothesis of the theorem does not allow this.
Remark
Another sufficient condition is given in [zu Castell et al. (2005)].
A complete integral characterization of functions that are strictly
positive definite on Rs is to my knowledge still missing. The case
s = 1 is covered in [Chang (1996)].
Complete characterizations of strictly positive definite functions on
compact spaces (such as spheres) do exist in the literature.

The following corollary gives us a way to construct strictly positive

definite functions.
Corollary
Let f be a continuous non-negative function in L1 (Rs ) which is not

identically zero. Then the Fourier transform of f is strictly positive
definite on Rs .
Proof.
This is a special case of the previous theorem in which the measure
has Lebesgue density f . Thus, we use the measure defined for any
Borel set B by Z
(B) = f (x)dx.
B
Then the carrier of is equal to the (closed) support of f . However,
since f is non-negative and not identically equal to zero, its support
has positive Lebesgue measure, and hence the Fourier transform of f
is strictly positive definite by the preceding theorem.
A very useful criterion to check whether a given function is strictly

positive definite is given by
Theorem
Let be a continuous function in L1 (Rs ). is strictly positive definite if
and only if is bounded and its Fourier transform is non-negative and
not identically equal to zero.
Proof in [Wendland (2005a)].

If 6 0 (which implies that then also 6 0) we need to ensure only
that be non-negative in order for to be strictly positive definite.

Positive Definite Radial Functions
We now focus on positive definite radial functions.

Earlier we characterized (strictly) positive definite functions in terms of
multivariate functions .
When working with radial functions (x) = (kxk) it is convenient to
call the univariate function a positive definite radial function.
This is a small abuse of our terminology for positive definite functions,
but commonly done in the literature.
If follows immediately that
Lemma
If = (k k) is (strictly) positive definite and radial on Rs then is
also (strictly) positive definite and radial on R for any s.

We now return to integral characterizations and begin with a theorem

due to Schoenberg (see, e.g., [Schoenberg (1938a), p.816], or
[Wells and Williams (1975), p.27]).
Theorem
A continuous function : [0, ) R is positive definite and radial on
Rs if and only if it is the Bessel transform of a finite non-negative Borel
measure on [0, ), i.e.,
Z
(r ) = s (rt)d(t).
0
Here
cos r for s = 1,
s (r ) = s2
s 2 2

2 r J s2 (r ) for s 2,
2
s2
and J s2 is the classical Bessel function of the first kind of order 2 .
2

Remark
Now we can view the function (x) = cos(x) as the fundamental
positive definite radial function on R.
We will see in the next chapter that the characterization of the
previous theorem immediately suggests a class of (even strictly)
positive definite radial functions.

A Fourier transform characterization of strictly positive definite radial

functions on Rs can be found in [Wendland (2005a)]. This theorem is
based on the Fourier transform formula of radial functions given earlier
and the check for strictly positive definite functions.
Theorem
A continuous function : [0, ) R such that
r 7 r s1 (r ) L1 [0, ) is strictly positive definite and radial on Rs if
and only if the s-dimensional Fourier transform
Z
1 s
Fs (r ) = (t)t 2 J(s2)/2 (rt)dt
r s2 0
is non-negative and not identically equal to zero.

Above we saw that any function that is (strictly) positive definite and
radial on Rs is also (strictly) positive definite and radial on R for any
s.
Therefore, we are interested in those functions which are (strictly)
positive definite and radial on Rs for all s.
The characterization for positive definite functions is from
[Schoenberg (1938a), pp. 817821] and the strictly positive definite
case from [Micchelli (1986)]:
Theorem (Schoenberg)
A continuous function : [0, ) R is strictly positive definite and

radial on Rs for all s if and only if it is of the form
Z
2 2
(r ) = er t d(t),
0
where is a finite non-negative Borel measure on [0, ) not

concentrated at the origin.
Letting be a point evaluation measure in Schoenbergs theorem

shows us that the Gaussian
2r 2
(r ) = e
is strictly positive definite and radial on Rs for all s.

Moreover, we can view the Gaussian as the fundamental member of
the family of functions that are strictly positive definite and radial on Rs
for all s.
By Schoenbergs theorem all such functions are given as infinite linear
combinations of Gaussians.

Schoenbergs theorem employs a finite non-negative Borel measure

on [0, ) such that Z
2t 2
(r ) = er d(t).
0
If we want to find a zero r0 of then we have to solve
Z
2 2
(r0 ) = er0 t d(t) = 0.
0
Since the exponential function is positive and the measure is

non-negative, it follows that must be the zero measure.
However, then is identically equal to zero.
Therefore, a non-trivial function that is positive definite and radial on
Rs for all s can have no zeros.

The discussion on the previous slide implies in particular that

Theorem
There are no oscillatory univariate continuous functions that are
strictly positive definite and radial on Rs for all s.
There are no compactly supported univariate continuous functions
that are strictly positive definite and radial on Rs for all s.

Appendix References
References I
Buhmann, M. D. (2003).
Radial Basis Functions: Theory and Implementations.
Cambridge University Press.
Cheney, E. W. and Light, W. A. (1999).
A Course in Approximation Theory.
Brooks/Cole (Pacific Grove, CA).
Cuppens, R. (1975).
Decomposition of Multivariate Probability.
Academic Press (New York).
Fasshauer, G. E. (2007).
Meshfree Approximation Methods with M ATLAB.
World Scientific Publishers.
Gelfand, I. M. and Vilenkin, N. Ya. (1964).
Generalized Functions Vol. 4.
Academic Press (New York).

Appendix References
References II
Iske, A. (2004).
Multiresolution Methods in Scattered Data Modelling.
Lecture Notes in Computational Science and Engineering 37, Springer Verlag
(Berlin).
Rudin, W. (1973).
Functional Analysis.
McGraw-Hill (New York).
Schlkopf, B. and Smola, A. J. (2002).
Learning with kernels: Support vector machines, regularization, optimization, and
beyond.
MIT Press, Cambridge, Massachussetts.
Sneddon, I. H. (1972).
The Use of Integral Transforms.
McGraw-Hill (New York).

Appendix References
References III
Stein, E. M. and Weiss, G. (1971).

Introduction to Fourier Analysis in Euclidean Spaces.
Princeton University Press (Princeton).
Wells, J. H. and Williams, R. L. (1975).
Embeddings and Extensions in Analysis.
Springer (Berlin).
Wendland, H. (2005a).
Scattered Data Approximation.
Cambridge University Press (Cambridge).
Bochner, S. (1933).
Monotone Funktionen, Stieltjes Integrale und harmonische Analyse.
Math. Ann. 108, pp. 378410.
zu Castell, W., Filbir, F. Szwarc, R. (2005).
Strictly positive definite functions in Rd .
J. Approx. Theory 137 2, pp. 277280.

Appendix References
References IV
Chang, K. F. (1996).
Strictly positive definite functions.
J. Approx. Theory 87, pp. 148158. Addendum: J. Approx. Theory 88 1997,
p. 384.
Micchelli, C. A. (1986).
Interpolation of scattered data: distance matrices and conditionally positive
definite functions.
Constr. Approx. 2, pp. 1122.
Schoenberg, I. J. (1938a).
Metric spaces and completely monotone functions.
Ann. of Math. 39, pp. 811841.

Meshfree Chapter 3

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Meshfree Chapter 3

Încărcat de

Drepturi de autor:

Formate disponibile

MATH 590: Meshfree Methods

Chapter 3: Positive Definite Functions

Department of Applied Mathematics

fasshauer@iit.edu MATH 590 Chapter 3 1

1 Positive Definite Matrices and Functions

2 Integral Characterizations for (Strictly) Positive Definite Functions

3 Positive Definite Radial Functions

fasshauer@iit.edu MATH 590 Chapter 3 2

fasshauer@iit.edu MATH 590 Chapter 3 3

fasshauer@iit.edu MATH 590 Chapter 3 5

for any N pairwise different points x 1 , . . . , x N Rs , and

fasshauer@iit.edu MATH 590 Chapter 3 7

fasshauer@iit.edu MATH 590 Chapter 3 8

Definition 2 suggests that we should use strictly positive definite

fasshauer@iit.edu MATH 590 Chapter 3 9

Theorem (Properties of (strictly) positive definite functions)

is also positive definite. Moreover, if at least one of the j is strictly positive

(5) If is positive definite with (0) = 0 then 0.

fasshauer@iit.edu MATH 590 Chapter 3 10

fasshauer@iit.edu MATH 590 Chapter 3 11

(x) + (x) and i ((x) (x))

fasshauer@iit.edu MATH 590 Chapter 3 12

Since (x) = (x) by Property (3), this gives

If |(x)| > 0, we divide by |(x)|2 and get |(x)| (0) as claimed.

fasshauer@iit.edu MATH 590 Chapter 3 13

The cosine function is positive definite on R.

First we remember that for any x R

fasshauer@iit.edu MATH 590 Chapter 3 14

Property (3) shows that any real-valued (strictly) positive definite

for any N pairwise different points x 1 , . . . , x N Rs , and

See [Wendland (2005a)] for a proof.

fasshauer@iit.edu MATH 590 Chapter 3 15

We summarize some facts about integral characterizations of positive

fasshauer@iit.edu MATH 590 Chapter 3 17

Bochners theorem below is formulated in terms of Borel measures.

X \ {O : O O and (O) = 0}.

fasshauer@iit.edu MATH 590 Chapter 3 19

and its inverse Fourier transform is given by

This definition from [Rudin (1973)] is based on angular frequency and

Similarly, we can define the Fourier transform of a finite (signed)

fasshauer@iit.edu MATH 590 Chapter 3 21

Since we are mostly interested in positive definite radial functions, we

Proof in [Wendland (2005a)]. This transform is also called

fasshauer@iit.edu MATH 590 Chapter 3 22

fasshauer@iit.edu MATH 590 Chapter 3 23

One of the most celebrated results on positive definite functions is their

A (complex-valued) function C(Rs ) is positive definite on Rs if and

fasshauer@iit.edu MATH 590 Chapter 3 24

fasshauer@iit.edu MATH 590 Chapter 3 25

The last inequality holds because of the conditions imposed on the

fasshauer@iit.edu MATH 590 Chapter 3 26

of our earlier example can be considered as the fundamental positive

fasshauer@iit.edu MATH 590 Chapter 3 27

In order to guarantee a well-posed interpolation problem we have to

fasshauer@iit.edu MATH 590 Chapter 3 28

and assume that the points x j are all distinct and c 6= 0.

are linearly independent, and so g 6= 0.

an equality is if the carrier of is contained in the zero set of g, i.e.,

fasshauer@iit.edu MATH 590 Chapter 3 31

The following corollary gives us a way to construct strictly positive

Let f be a continuous non-negative function in L1 (Rs ) which is not

A very useful criterion to check whether a given function is strictly

Proof in [Wendland (2005a)].

fasshauer@iit.edu MATH 590 Chapter 3 33