Documente Academic
Documente Profesional
Documente Cultură
Transformation
and Approximation
Revised Edition
JAMES P. KEENER
University of Utah
Salt Lake City, Utah
~ ew
I I I I
I
4.3.1 Domain of an Operator . . . . . . . . . . . . . . . . . . 151
4.3.2 Adjoint of an Operator . . . . . . . . . . . . . . . . . . . 152 6.5.4 Sine Functions . . . . . . . . . . . . . . . . . . . . . . . . 270
4.3.3 Inhomogeneous Boundary Data . . . . . . . . . . . . . . . 154 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
J Problems for Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . ~ . 274
4.3.4 The Fredholm Alternative . . . . . . . . . . . . . . . . . . 155 ~
~
4.4 Least Squares Solutions . . . . . . . . . . . . . . . . . . . . . . . 157
4.5 Eigenfunction Expansions . . . . . . . . . . . . . . . . . . . . 161 i
-~
7 Transform and Spectral Theory
7.1 Spectrum of an Operator . . . . . . . . . . . . . . . . . . . . . .
283
283
4.5.1 'Irigonometric Functions . . . . . . . . . . . . . . . . . .
4.5.2 Orthogonal Polynomials . . . . . . . . . . . . . . . . . .
.
.
164
167
I~ 7.2 Fourier Transforms . . . . . . . . , . . . . . . . . . . . . . . . . . 284
~ 7.2.1 Transform Pairs . . . . . . . . . . . . . . . . . . . . , .. 284
4.5.3 Special Functions . . . . . . . . . . . . . . . . . . . . . . . 169
4.5.4 Discretized Operators . . . . . . . . . . . . . . . . . . . . § 7.2.2 Completeness of Hermite and Laguerre Polynomials . . . 297
169
Further R.eading . . . . . . . . . . . . . . . . . · · · · · · · · · · · ·
Problems for Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . .
·
.
171
171
Is" 7.2.3 Sine Functions . . . . . . . . . . . . . . . . . . . . . . . .
7.2.4 Windowed Fourier Transforms . . . . . . . . . . . . . . .
299
300
7.2.5 Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
I 7.3 Related Integral Transforms . . . . . . . . . . . . . . . . . . . . . 307
5 Calculus of V~iations 177
5.1 The Euler-Lagrange Equations . . . . . . . . . . . . . . . . . . . 177 i•
~
7.3.1 Laplace Transform . . . . . . . . . . . . . . . . . . . . . . 307
5.1.1 Constrained Problems . . . . . . . . . . . . . . . . . . . . 180 7.3.2 Mellin Transform . . . . . . . . . . . . . . . . . . . . . . . 308
5.1.2 Several Unknown Functions . . . . . . . . . . . . . . . . . 181 I 7.3.3 Hankel Transform . . . . . . . . . . . . . . . . . . . . . . 309
5.1.3 Higher Order Derivatives . . . . . . . . . . . . . . . . . .
5.1.4 Variable Endpoints . . . . . . . . . . . . . . . . . . . . . .
184
184
I
li
7.4 Z Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.5 Scattering Theory . . . . . . . . . . . . . . . . . . . . . . . . . .
310
312
~
7.5.1 Scattering Examples . . . . . . . . . . . . . . . . . . . . . 318
I
5.1.5 Several Independent Variables . . . . . . . . . . . . . . . . 185
5.2 Hamilton's Principle . . . . . . . . . . . . . . . . . . . . . . . . . 186 7.5.2 Spectral Representations . . . . . . . . . . . . . . . . . . . 325
5.2.1 The Swinging Pendulum . . . . . . . . . . . . . . . . . . . 188 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Problems for Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . , . . 328
5.2.2 The Vibrating String . . . . . . . . . . . . . . . . . . . . . 189 I Appendix: Fourier Transform Pairs . . . . . . . . . . . . . . . . . . . . 335
5.2.3 The Vibrating Rod . . . . . . . . . . . . . . . . . . . . . . 189 !
5.2.4 Nonlinear Deformations of a Thin Beam . . . . . . . . . . 193 i
5.2.5 A Vibrating Membrane . . . . . . . . . . . . . . . . . . . 194 Jiii 8 Partial Differential Equations 337
5.3 Approximate Methods . . . . . . . . . . . . . . . . . . . . . . . . 195 ~ 8.1 Poisson's Equation . . . . . . . . . . . . . . . . . . . . . . . . . . 339
5.4 Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . 198 8.1.1 Fundamental Solutions . . . . . . . . . . . . . . . . . . . . 339
5.4.1 Optimal Design of Structures . . . . . . . . . . . . . . . . 201 8.1.2 The Method of Images . . . . . . . . . . . . . . . . . . . . 343
Further R.eading . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 202 8.1.3 Transform Methods . . . . . . . . . . . . . . . . . . . . . 344
Problems for Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . 203 8.1.4 Hilbert Transforms . . . . . . . . . . . . . . . . . . . . . . 355
viii CONTENTS CONTENTS ix
Bibliography 559
9.3 Korteweg-deVries Equation . . . . . . . . . . . . . . . . . . . . . 421 ~
~
9.4 The Toda Lattice . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 I Selected Hints and Solutions 567
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 I
~I
Problems for Chapter 9 . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Index 596
10 Asymptotic Expansions 437
10.1 Definitions and Properties . . . . . . . . . . . . . . . . . . . . . 437
10.2 Integration by Parts . . . . . . . . . . . . . . . . . . . . . . . . . 440 I
10.3 Laplace's Method . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.4 Method of Steepest Descents . . . . . . . . . . . . . . . . . . . ;
442
449 I
10.5 Method of Stationary Phase . . . . . . . . . . . . . . . . . . . . .
Further Reading . . . . . . . . ~ . . . . . . . . . . . . . . . . . . . . .
Problems for Chapter 10 . . . . . . . . . . . . . . . . . . . . . . . . . .
456
463
463
'I
11 Regular Perturbation Theory 469 !
11.1 The Implicit Function Theorem . . . . . . . . . . . . . . . . . . 469 I
11.2 Perturbation of Eigenvalues . . . . . . . . . . . . . . . . . . . . . 475
11.3 Nonlinear Eigenvalue Problems . . . . . . . . . . . . . . . . . . . 478 I*
11.3.1 Lyapunov-Schmidt Method . . . . . . . . . . . . . . . . . 482 I
~
11.4 Oscillations and Periodic Solutions . . . . . . . . . . . . . . . . . 482
11.4.1 Advance of the Perihelion of Mercury . . . . . . . . . . . 483
11.4.2 Vander Pol Oscillator . . . . . . . . . . . . . . . . . . . . 485
11.4.3 Knotted Vortex Filaments . . . . . . . . . . . . . . . . . . 488
11.4.4 The Melnikov Function . . . . . . . . . . . . . . . . . . . 493
11.5 Hopf Bifurcations . . . . . . _. . . . . . . . . . . . . . . . . . . . . 494
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
Preface to First Edition
Applied mathematics should read like good mystery, with an intriguing begin-
ning, a clever but systematic middle, and a satisfying resolution at the end.
Often, however, the resolution of one mystery opens up a whole new problem,
and the process starts all over. For the applied mathematical scientist, there
is the goal to explain or predict the behavior of some physical situation. One
begins by constructing a mathematical model which captures the essential fea-
tures of the problem without masking its content with overwhelming detail.
Then comes the analysis of the model where every possible tool is tried, and
some new tools developed, in order to understand the behavior of the model as
thoroughly as possible. Finally, one must interpret and compare these results
with real world facts. Sometimes this comparison is quite satisfactory, but most
1
1
often one discovers that important features of the problem are not adequately
accounted for, and the process begins again.
Although every problem has its own distinctive features, through the years
I
it has become apparent that there are a group of tools that are essential to the
analysis of problems in many disciplines. This book is about those tools. But
more than being just a grab bag of tools and techniques, the purpose of this
book is to show that there is a systematic, even esthetic, explanation of how
~
-?£ these classical tools work and fit together as a unit.
I•
(1
of water, one on the counter, one on a cold burner, while anot_her burner was
already quite hot, but empty.
In order to test their cooking aptitudes, the instructor first asked the engineer
to demonstrate to the rest of the class how to boil the pot of water that was
already sitting on the stove. He naturally moved the pot carefully from the cold
burner to the hot burner, to the appreciation of his classmates. To be sure the
class understood the process, the pot was moved back to its original spot on the
cold burner at which time the mathematician was asked to demonstrate how to
heat the water in the pot sitting on the counter. He promptly and confidently
xi
xii PREFACE TO FIRST EDITION PREFACE TO FIRST EDITION xiii
exchanged the position of the two pots, placing the pot from the counter onto might try to represent u and I as polynomials or infinite power series in x, but
the cold burner and the pot from the burner onto the counter. He then stepped we quickly learn that this guess does not simplify the solution process much.
back from the stove, expecting an appreciative response from his mates. Baffled Instead, the ''natural" choice is to represent u and I as trigonometric series,
by his actions, the instructor asked him to explain what he had done, and he 00 00
replied naturally, that he had simply reduced his problem to one that everyone
already knew how to solve.
u(x) =L u~,: sinkx, l(x) =L 1~.: sin kx.
k=l k=l
We shall also make it our goal to reduce problems to those which we already
know how to solve. Using this representation (i.e., coordinate system) we find that the original dif-
We can illustrate this underlying idea with simple examples, we know that ferential equation reduces to the infinite number of separated algebraic equations
to solve the algebraic equation 3x = 2, we multiply both sides of the equation (k 2 + 2)u~,: = -1~.:· Since these equations are separated (the kth equation de-
with the inverse of the "operator" 3, namely 1/3, to obtain x = 2/3. The same pends only on the unknown u~.:), we can solve them just as before, even though
is true if we wish to solve the matrix equation Ax = b where there are an infinite number of equations. We have managed to simplify this
problem by transforming into the correctly chosen coordinate system.
A=(~!). b=(~)· For many of the problems we encounter in the sciences, there is a natural way
to represent the solution that transforms the problem ip.to a substantially easier
one. All of the well-known special functions, including Legendre polynomials,
Namely, if we know A-t, the inverse of the matrix operator A, we multiply both Bessel functions, Fourier series, Fourier integrals, etc., have as their common
sides of the equation by A-t to obtain x : A - 1 b. For this problem, motivation that they are natural for certain problems, and perfectly ridiculous
in others. It is important to know when to choose one transform over another.
A-t=! (
8 -1
3 -1 )
3 ' x=~(!)· Not all problems can be solved exactly, and it is a mistake to always look for
exact solutions. The second basic technique of applied mathematics is to reduce
hard problems to easier problems by ignoring small terms. For example, to find
H we make it our goal to invert many kinds of linear operators, including
matrix, integral, and differential operators, we will certainly be able to solve the roots of the polynomial x 2 + x + .0001 = 0, we notice that the equation is
many types of problems. However, there is an approach to calculating the =
very close to the equation x 2 + x = 0 which has roots x -1, and x 0, and =
inverse operator that also gives us geometrical insight. We try to transform the we suspect that the roots of the original polynomial are not too much different
original problem into a simpler problem which we already know how to solve. from these. Finding how changes in parameters affect the solution is the goal
Fbr example, if we rewrite the equation Ax= bas T- 1 AT(T- 1 x) = T- 1 b and of perturbation theory and asymptotic analysis, and in this example we have
choose a regular perturbation problem, since the solution is a regular (i.e., analytic)
function of the "parameter" 0.0001.
- ( 1 -1 ) -1- 1 ( 1 1 )
T- 1 1 I T - 2 -1 1 I Not all reductions lead to such obvious conclusions. For example, the poly-
nomial 0.0001x 2 + x + 1 = 0 is close to the polynomial x + 1 = 0, but the first
we find that
has two roots while the second has only one. Where did the second root go?
1
T- AT = ( ~ -~ ) . We know, of course, that there is a very large root that "goes to infinity" as
"0.0001 goes to zero," and this example shows that our naive idea of setting all
With the change of variables y = T- 1 x, g = T- 1b, the new problem looks small parameters to zero must be done with care. As we see, not all problems
like two of the easy algebraic equations we already know how to solve, namely with small parameters are regular, but some have a singular behavior. We need
4yt = 3/2, -2y2 = 1/2. This process of separating the coupled equations to know how to distinguish between regular and singular approximations, and
into uncoupled equations works only if T is carefully chosen, and exactly how what to do in each case.
this is done is still a mystery. Suffice it to say, the original problem has been This book is written for beginning graduate students in applied mathemat-
transformed, by a carefully chosen change of coordinate system, into a problem ics, science, and engineering, and is appropriate as a one-year course in applied
we already know how to solve. mathematical techniques (although I have never been able to cover all of this
This process of changing coordinate systems is very useful in many other material in one year). We assume that the students have studied at an intro-
problems. For example, suppose we wish to solve the boundary value problem ductory undergraduate level material on linear algebra, ordinary and partial
u"- 2u = l(x) with u(O) = =
u(1r) 0. As we do not yet know how to write
down an inverse operator, we look for an alternative approach. The single most
differential equations, and complex variables. The emphasis of the book is a
working, systematic understanding of classical techniques in a modern context.
important step is deciding how to represent the solution u(x). For example, we Along the way, students are exposed to models from a variety of disciplines.
xiv PREFACE TO FIRST EDITION PREFACE TO FIRST EDITION XV
It is hoped that this course will prepare students for further study of modem entiate analytic functions at a reasonably sophisticated level is indispensable to
techniques and in-depth modeling in their own specific discipline. the remaining text. Section 6.3 (Applications to Fluid Flow) is included because
One book cannot do justice to all of applied mathematics, and in an effort it is a classically important and very lovely subject, but it plays no role in the
to keep the amount of material at a manageable size, many important topics remaining development of the book, and could be skipped if time constraints
were not included. In fact, each of the twelve chapters could easily be expanded demand.
into an entire book. The topics included here have been selected, not only for Chapter seven continues the development of transform theory and we show
their scientific importance, but also because they allow a logical flow to the that eigenvalues and eigenfunctions are not always sufficient to build a trans-
development of the ideas and techniques of transform theory and asymptotic form, and that operators having continuous spectrum require a generaliZed con-
analysis. The theme of transform theory is introduced for matrices in Chapter struction. It is in this context that Fourier, Mellin, Hankel, and Z transforms,
one, and revisited for integral equations in Chapter three, for ordinary differ- as well as scattering theory for the SchrOdinger operator are studied.
ential equations in Chapters four and seven, for partial differential equations in In Chapter eight we show how to solve linear partial differential and differ-
Chapter eight, and for certain nonlinear evolution equations in Chapter nine. ence equations, with special emphasis on (as you guessed) transform theory. In
Once we know how to solve a wide variety of problema via transform theory, this chapter we are able to make specific use of all the techniques introduced so
it becomes appropriate to see what harder problems we can reduce to those we- far and solve some problems with interesting applications.
know how to solve. Thus, in Chapters ten, eleven, and twelve we give a survey Although much of transform theory is rather old, it is by no means dead.
of the three basic areas of asymptotic analysis, namely asymptotic analysis of In Chapter nine we show how transform theory has recently been used to solve
integrals, regular perturbation theory and singular perturbation theory. certain nonlinear evolution equations. We illustrate the inverse scattering trans-
Here is a summary of the text, chapter by chapter. In Chapter one, we form on the Korteweg-deVries equation and the Toda lattice.
review the basics of spectral theory for matrices with the goal of understanding In Chapter ten we show how asymptotic methods can be used to approx-
not just the mechanics of how to solve matrix equations, but more importantly, imate the horrendous integral expressions that so often result from transform
the geometry of the solution process, and the crucial role played by eigenvalues techniques.
and eigenvectors in finding useful changes of coordinate systems. This usefulness In Chapter eleven we show how perturbation theory and especially the study
extends to pseudo-inverse operators as well as operators in HUbert space, and of nonlinear eigenvalue problems uses knowledge of the spectrum of a linear
so is a particularly important piece of background information. operator in a fundamental way. The nonlinear problems in this chapter all have
In Chapter two, we extend many of the notions of finite dimensional vector the feature that their solutions are close to the solutions of a nearby linear
spaces to function spaces. The main goal is to show how to represent objects problem.
in a function space. Thus, Hilbert spaces and representation of functions in a Singular perturbation problema fail to have this property, but have solutions
Hilbert space are studied. In this context we meet classical sets of functions that differ markedly from the naive simplified problem. In Chapter twelve, we
such as Fourier series and Legendre polynomials, as well as less well-known sets give a survey of the three basic singular perturbation problema (slowly varying
such as the Walsh functions, Sine functions, and finite element bases. oscillations, initial value problems with vastly different time scales, and bound-
In Chapter three, we explore the strong analogy between integral equations ary value problems with boundary layers).
and matrix equations, and examine again the consequences of spectral theory. This book has the lofty goal of reaching students of mathematics, science,
This chapter is more abstract than others as it is an introduction to functional and engineering. To keep the attention of mathematicians, one must be system-
analysis and compact operator theory, given under the guise of Fredholm integral atic, and include theorems and proofs, but not too many explicit calculations.
equations. The added generality is important as a framework for things to come. To interest an engineer or scientist, one must give specific examples of how to
In Chapter four, we develop the tools necessary to use spectral decompo- solve meaningful problema but not too many proofs or too much abstraction.
sitions to solve differential equations. In particular, distributions and Green's In other words, there is always someone who is unhappy.
functions are used as the means by which the theory of compact operators can In an effort to minimize the total displeasure and not scare off too many
be applied to differential operators. With these tools in place, the completeness members of either camp, there are some proofs and some computations. Early
of eigenfunctions of Sturm-Liouville operators follows directly. in the text, there are disproportionately more proofs than computations because
Chapter five is devoted to showing how many classical differential equations as the foundations are being laid, the proofs often give important insights. Later,
can be derived from a variational principle, and how the eigenvalues of a differ- after the bases are established, they are invoked in the problem solving process,
ential operator vary as the operator is changed. but proofs are deemphasized. (For example, there are no proofs in Chapters
Chapter six is a pivotal chapter, since all chapters following it require a sub- eleven and twelve.) Experience has shown that this approach is, if not optimal,
stantial understanding of analytic function theory, and the chapters preceding it at least palatable to both sides.
require no such knowledge. In particUlar, knowing how to integrate and differ- This is intended to be an applied mathematics book and yet there is very
xvi PREFACE TO FIRST EDITION
little mention of numerical methods. How can this be? The answer is simple
and, I hope, satisfactory. This book is primarily about the principles that one
uses to solve problems and since these principles often have consequence in
numerical algorithms, mention of numerical routines is made when appropriate.
However, FORTRAN codes or other specific implementations can be found in
many other good resources and so (except for a code for fast Walsh transforms)
are omitted. On the other hand, many of the calculations in this text should
Preface to the Second
be done routinely using symbolic manipulation languages such as REDUCE or
MACSYMA. Since these languages are not generally familiar to many, included Edition
here are a number of short programs written in REDUCE in order to encourage
readers to learn how to use these remarkable tools. (Some of the problems at
the end of several chapters are intended to be so tedious that they force the
reader to learn one of these languages.) When the first edition of this book was published, l thought that it was an
This text would not have been possible were it not for the hard work and up-to-date look at classical methods of applied mathematics. I did not expect
encouragement of many other people. There were numerous students who strug- the basic tools of applied mathematicians to change dramatically in the near
gled through this material without the benefit of written notes while the course future. Indeed the basic ideas remain the same, but there are many ways in
was evolving. More recently, Prof. Calvin Wilcox, Prof. Frank Hoppensteadt, which those basic ideas have found new applications and extensions.
Gary deYoung, Fred Phelps, and Paul Arner have been very influential in the Some of the most significant new techniques of the last decade are wavelet
shaping of this presentation. Finally, the patience of Annetta Cochran and analysis, multigrid methods, and homogenization theory, all of which are exten-
Shannon Ferguson while typing from my illegible scrawling was exemplary. sions of methods already discussed in the first edition. Additionally, software
In spite of all the best efforts of everyone involved, I am sure there are still tools have become much more sophisticated and reliable, and it is not possible
typographical errors in the text. It is disturbing that I can read and reread to ignore these developments in the training of an applied mathematician.
a section of text and still not catch all the erors. My hope is that those that So the first reason to revise this book is to bring it up to date, by adding
remain are both few and obvius and will not lead to undue confusion. material describing these new and important methods of applied mathematics.
The second reason for this revision is to make it easier to read and use. To
that end, the text has been thoroughly edited, with emphasis on clarity and
filling in the gaps. I have tried to eliminate those annoying places where a step
was described as obvious, but is far from obvious to the reader. I have also tried
to eliminate many of the (mostly typographical, but nonetheless noisome) errors
that were in the first edition. I have added equation numbers to those equations
that are referenced in the text, and many figures have been added to aid the
geometrical interpretation and understanding. Finally, I have added a section
of hints and solutions for the exercises. Many of the exercises are difficult and
both students and instructors find such a section to be most helpful.
Many new exercises have been added, mostly those which are intended to use
modern software for symbolic computation and graphical interpretation. The
first edition of this book referred to REDUCE, which, sadly, disappeared from
widespread use almost immediately after the publication of that edition. The
good news is that the new software tools are much better. My personal pref-
erences are Maple and Matlab, but other choices include Mathematica, Math-
Cad, Derive, etc. Exercises that are marked with .II!, are designed to encourage
computer usage with one of these software packages. I have mentioned Maple
throughout the text, but the intent is "generic software package with symbolic
computation and graphics capability".
xvii
PREFACE TO SECOND EDITION xix
xviii PREFACE TO SECOND EDITION
folks at Calvin College for providing me with a quiet place to work for several
While the basic philosophy of the text remains unchanged, there are impor-
months, and to the University of Utah for a sabbatical leave during which this
tant ways in which most chapters have been modified. In particular, here are
revision was finished.
some of the major modifications and additions by chapter:
Further corrections or modifications can be found at the Internet site
Chapter 1. A section describing the problem of Procrustes (who devised an http://www.math.utah.edu/-keener/AppliedMathBook.html.
unusual method for coordinate transformations) has been added.
Also, a section on applications of eigenvalues and eigenfunctions James P. Keener
of matrices has been added, wherein matrix iterative methods and University of Utah
the ranking of sports teams are discussed. June 1999
Chapter 2. A section introducing wavelets has been added, replacing the dis-
cussion of Walsh functions.
Chapter 4. The discussion of extended operators, which I always found to be
awkWard, is replaced by a much more transparent and straight-
forward discussion of inhomogeneous boundary conditions using
integration by parts.
The content of Chapters 3, 6, 9 and 10 has changed little from the first
edition.
As with any project of this size, I owe a debt of gratitude to many people who
helped. Eric Cytrynbaum, Young Seon Lee, Todd Shaw and Peter Bates found
many of the errors that I missed and made numerous helpful suggestions about
the presentation, while David Eyre and Andrej Cherkaev made many helpful
suggestions for new material to include. Steve Worcester did the initial 'J.EX-ing
of the manuscript in record time. Nelson Beebe is a 'J.EX-pert, whose wizardry
never ceases to astound me and who8e help was indispensable. Thanks to the
Chapter 1
1
2 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.1. LINEAR VECTOR SPACES 3
Definition 1.1 A set of objects Sis called a linear vector space if two prop- 2. A linear combination of two vectors in m?, Clt(Xt,Yl) + Cl2(:Z:2,Y2), is the
erties hold: zero vector whenever :z:1 = -/3:z:2 and Yt = -f3y2 where f3 = a2/a1. Geo-
metrically, this means that the vector (:z:t, yl) is collinear with the vector
1. If x, y E S, then x +y E S (called closure under vector addition), (:z::~, y2), i.e., they are parallel vectors in 1R2.
2. If a E IR (or a E ([;') and x E S, then ax E S (called closure under scalar Important examples of linearly independent vectors are the monomials. That
multiplication). is, the powers of x, {1, x, x 2 , ••• , xn} form a linearly independent set. An easy
proof of linear independence is to define f(x) = ao + alx + a2x 2 + ... + anxn
If the scalars are real, S is a real vector space, while if the scalars are complex, and to determine when f(x) is identically zero for all x. Clearly, /(0) = ao and
S is a complex vector space. if f(x) = = =
0 for all x, then /(0) 0 implies a0 0. The kth derivative of f(x)
at x = 0,
Examples:
1. The set of real ordered pairs (:z:,y) (denoted m?) is a linear vector space
dk f(x)
d,xk
I
z=O
= k!ak
if addition is defined by (:z:1, yt) + (:z:2, y2) = (:z:1 + x2, Yt + y2), and scalar
multiplication is defined by a(:z:,y) = (a:z:,ay). is zero if and only if ak = 0. Therefore, f(x) =
0 if and only if ai = 0 for
j = 0, 1, 2, ... , n.
2. The set of real n-tuples (:z:t, x2, ... , :Z:n) (denoted lRn) forms a linear vector It is often convenient to represent an element of the vector space as a linear
space if addition and scalar multiplication are defined component-wise as combination of a predetermined set of elements of S. For example, we might
in the above example. Similarly the set of complex n-tuples (denoted <IJR)
want to represent the continuous functions as a linear combination of certain
forms a linear vector space with the complex scalars.
polynomials or of certain trigonometric functions. We ask the question, given
3. The set of all polynomials of degree n forms a linear vector space, with a subset of S, when can any element of S be written as a linear combination of
"vectors" Pn(:z:) = :E.i=o ai:z:i, with Clj E 1R (or Clj E IV), if addition and elements of the specified subset?
scalar multiplication are defined in the usual way.
4. The set of all continuous functions defined on some interval [a, b] forms a Definition 1.2 A set of vectors T C S is a spanning set if every x E S can
linear vector space. be written as a linear combination of the elements ofT.
Definition 1.3 The span of a set of vectors T is the collection of all vectors
There are many sets that are not linear vector spaces. For example, the
that are linear combinations of the vectors in T.
set of all ordered pairs of the form (x, 1) is not closed under addition or scalar
multiplication and so does not form a vector space. On the other hand, ordered Definition 1.4 If a spanning set T c S is linearly independent then T is a
pairs of the form (x, 3x) do comprise a linear vector space. Similarly, the set of basis for S. If the number of elements ofT is finite, then S is a finite dimensional
continuous functions /(x) defined on the interval [a,b) with f(a) =
1 does not vector space whose dimension is the number of elements ofT.
form a vector space, while if instead the constraint /(a) = 0 is imposed, the
properties of closure hold. One can easily show (see Problem 1.1.1) that every basis of S has the same num-
In a vector space, we can express one element of S in terms of additions ber of elements. For a given basis T, a vector inS has a unique representation
and scalar multiplications of other elements. For example, if Xt. x2, ... , Xn, are in terms of the elements of T.
elements of S, then x = a1x 1 + a2x 2 + · · · + anXn is also inS, and xis said to
be a linear combination of Xt, x2, ... , Xn· If there is some linear combination Examples:
=
with a1x 1 + a 2x 2 + · · · + anXn 0, and not all of the scalars ai are zero, then
1. In 1R2 , any three vectors are linearly dependent, but any two noncollinear
the set {x 1 , x 2 , ••• , Xn} is said to be linearly dependent. On the other hand, vectors form a basis.
if the only linear combination of Xt, x2, ... , Xn which is zero has all ai equal to
2. In 1R 3 the vectors { (1, 0, 0), (0, 1, 0), (0, 0, 1), (1, 1, 0), (0, 1, 1)} form a span-
zero, the set {x 1 , x 2 , •.• , Xn} is said to be linearly independent.
ning set, but the last two vectors are not necessary to form a spanning set.
From this set, there are eight different ways to choose a subset of 3 linearly
Examples: independent elements which is a basis. In lRn, the so-called natural basis
is the set of vectors {et, e2, ... , en} where the jth entry of e~c is Okj where
1. The single nontrivial vector x E lRn, :z: ::j: 0 forms a linearly independent o,i = 0 if j ::j: k and O~ok = 1. The object o"i is called the Kronecker
set because the only way to have a:z: = 0 is if a = 0. delta.
CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.1. LINEAR VECTOR SPACES 5
4
3. The set {1,z,z2 ,z2 -1} is linearly dependent. However, any of the subsets similarly we discretize g(x) to define G = (g(zl),g(z2), ... ,g(xn)), where
{1,z,z2}, {1,z,z2 - 1}, or {z,z2,z 2 - 1} forms a basis for the set of :z:,. = kfn,
and it makes sense to define
quadratic polynomials. 1 n
4.· The set of all polynomials does not have finite dimension, since any linear (F, G)n = - 2: f(x~c)g(xlc)·
n k=l
combination of polynomials of degree n or less cannot be a polynomial of
degree n + 1. The set of all continuous functions is also infinite since the (While this is an inner product on JR", this is not ap inner product for the
polynomials are a subset of this set. We will show later that the polynomials continuous functions. Why?) Taking the limit n -+ oo, we obtain
form a "basis" for the continuous functions, but this requires more technical
information than we have available now. (/,g} = 1 1
f(x)g(x)dz. (1.2)
When you first learned about vectors in an undergraduate Physics or Math- It is an easy matter to verify that (1.2) defines an inner product on the
ematics course, you were probably told that vectors have direction and magni- vector space of continuous functions.
tude. This is certainly true in JR3 , but in more complicated vector spaces the
3. There are other interesting ways to define an inner product for functions.
concept of direction is hard to visualize. The direction of a vector is always given For example, if the complex valued functions I and g are continuously
relative to some other reference vector by the angle between the two vectors. differentiable on the interval [0, 1], we might define
To get a concept of direction and angles in general vector spaces we introduce
the notion of an inner product. (/,g)= 1 1
(!(z)g(x) + /'(z)gl(z)) dz.
Definition 1.5 For x, y E S, the inner product (also called scalar product More generally, if I and g are n times continuously differentiable on [0, 1]
or dot product) of x andy, denoted (x,lf), is a function{-,·): S x S ~ IR, we might define
(or (U if Sis a complex vector space) with the properties:
2. (ax,y) = a(x,y}, as an inner product. Indeed, one can show that these satisfy the required
properties of inner products.
3. (x+y,z)=(x,z)+(y,z),
The magnitude of a vector can also be defined for a general vector space.
4. (x,x) > 0 if x :/: 0, (x,x) = 0 if and only if x = 0.
Definition 1.6 For x E S, the norm (also called amplitude, magnitude, or
A linear vector space with an inner product is called an inner product length) of x, denoted llxll, is a function 11·11: S-+ [O,oo) with the properties:
space.
1. llxll > 0 if x :/: 0 and llxll = 0 implies x = 0,
Examples: 2. llaxll = lal·llxll for a E IR (or a E <U if Sis a complex vector space),
1. In JR.", suppose z = (zt, :1:2, ... , Zn) and Sl =(Sit, f12, ... , Sin) then 3. llx + Yll $ llxll + IIYII (triangle inequality).
n
<x, 11> = E x,y, In JRn, we can take llxll = (L:~= 1 Ixkl 2 ) 1 1 2 or more generally, llxll =
lo•l (L:~= 1 1xkiP) 1 1P, 1 < p < oo. With p = 2, this is the Euclidean norm or
"2"-norm, however, for differing values of p the meaning of size changes. For
is the usual Euclidean inner product. H x, Sl e (U" I then = =
example, in JR2 the "unit sphere" llxll 1 with p 2 is the circle x~ + x~ 1 =
n while with p = 1 the "unit sphere" lx1l + lx2l = 1 is a diamond (a square with
(:~:, y) = E :l:loYio (1.1) vertices on the axes). H we let p ~ oo, then llxll = maxk lxkl llxlloo is called =
lo=l the "infinity" or "sup" (for supremum) norm, and the "unit sphere" llxll = 1 is
is an inner product. a square (Fig. 1.1).
A linear vector space with a norm is called a normed vector space. Al-
2. For two real continuous functions l(x) and g(z) defined for :1: e [0,1], we
though there are many ways to define norms, one direct way to find a norm is
discretize l(x) to define a vector in mn
IF= (f(xt), l(z2), ... ,/(xn)) and
CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.1. LINEAR VECTOR SPACES 7
6
If y =f
0, we pick a= fMtt,
(thereby minimizing llx- ay!l 2) from which the
Schwarz inequality is immediate. 1
Using the Schwarz inequality, we see that the triangle inequality holds for
any induced norm, since
t; lx~~:l )1/2
n product spaces, such as the space of continuous functions, we take this to be
2 {1.4)
llxll = (
, the definition of cosO. The law of cosines and the Pythagorean theorem are
immediate. For example,
is induced by the Euclidean inner product {1.1), and for real continuous func- llx + vW = llxll 2 + 2llxll . IIYII cos(} + IIYW
tions with inner product (/,g)= J:
f(x)g(x)dx, the induced norm is
is the law of cosines, and if cos() = 0 then
11/11 =
b
L1f(x)l dx
)1/2
2 llx + Yll 2 = 2
llxll + IIYW (Pythagorean Theorem).
(
Following this definition of cos (), we say that the vectors x and y are or-
For n times continuously differentiable complex valued functions, the inner prod- thogonal if (x, y) = 0.
r
uct (1.3) induces the norm
Examples:
2. With inner product (!,g)= J;.,. f(x)g(x)dx, sinx ap.d sin2x are orthogonal
For any induced norm, llxll > 0 if x :/: 0, and llaxll = lal· llxll for scalars a. ·
smce r"J1r smxsm
Jo · · 2x dx = 0.
To see that the triangle inequality also holds, we must first prove an important
3. With inner product (!,g)= J01 f(x)g(x)dx, the angle 0 between the func-
result:
tions f(x) = 1 and g(x) = x on [0, 1) is 30° since cosO= ~ = .;'3/2.
Theorem 1.1 (Schwarz 1 inequality) For x,y in an inner product space,
Orthogonal vectors are nice for several reasons. If the vectors {¢1, ¢2, ... ,
l(x, v>l ~ llxii·IIYII· {1.5) ¢n} E S, cfoi =f 0, are mutually orthogonal, that is, (¢i,¢;) = 0 fori =f j,
then they form a linearly independent set. To see this suppose there are scalars
a1, a2, ... , an so that
=
so that ai 0 for j = 1, 2, ... , n, provided t/J; is nontrivial.
H the set {1/Jt. t/J2, ... , t/Jn} forms a basis for S, we can represent any element
f E S as a linear combination of 1/J;, .
~\),
n \
I= Lf3itP3· \, q, 2
\
j=l
To determine the coefficients /3;, we take the inner product of f with tPi and
find a matrix equation B/3 =
'7 where B = (bi;), btj = (1/Jj,tPi}, {3 = ({3i), Xz
'1i = (/, t/Jt}. This matrix problem is always uniquely solvable since the tPt 's
are linearly independent. However, the solution is simplified enormously when Figure 1.2: Graphical representation of the Gram-Schmidt orthogonalization
the tPi 's are mutually orthogonal, since then B is a diagonal matrix and easily procedure to produce tP2 from Xt = c/>1 and x2.
inverted to yield
(!, tPi}
{3 Example:
i = llt/Jill
2 •
Consider the powers of x, {1, x, x 2 , ••• , xn}, which on any interval [a, b] (a<
The coefficient f3t carries with it an appealing geometrical interpretation in b) form a linearly independent set.
m,n. H we want to qrthogonally "project" a vector f onto ¢ we might imag- With [a,b] = [-1, 1] and the inner product
ine shining a light onto tjJ and measuring the shadow cast by f. From simple
1
trigonometry, the length of this shadow is II/II cosO where 8 is the angle in the (!,g) =[ f(x)g(x)dx,
plane defined by the vectors f and 1/J. This length is exactly ~ and the vector 1
of the shadow, ~1/J, is called the projection off onto 1/J. Notice that the the Gram-Schmidt procedure produces
quantity llx- 'i'YII is minimized by picking 'i'Y to be the projection of x onto y, if>o = 1,
. ~
1.e., 'i' = TTUfP"Y· l/>1 =x,
The Gram-Schmidt orthogonalization procedure is an inductive tech- l/>2 = x 2 -1/3,
l/>3 = x3 - 3x/5,
nique used to generate a mutually orthogonal set from any linearly independent
=
set of vectors. Given the vectors Xt, x2, ... , Xn, we set t/J1 Xt. To find a vector and so on. The functions thus generated are called the Legendre poly-
t/J2 that is orthogonal to t/J1, we set t/J2 = x2 - a¢1, and use the requirement nomials. Other sets of orthogonal polynomials are found by changing the
=
(t/J2, t/J1) 0 to determine that underlying interval or the definition of the inner product. We will see more
of these in Chapter 2.
The same vector x would have different coordinates if it were expressed relative Definition 1. 7 An eigenpair of A is a pair (A, :z:), A E <V, :z: E <lin satisfying
to a different basis.
Suppose the matrix equation Ax = b is expressed in coordinates relative
Ax = A:z:, :z: i- 0.
to the basis {t/J1 , '1/J?., ••• , t/Jn} and we wish tore-express the problem relative to The vector :z: is called the eigenvector and A is called the eigenvalue of A.
some other basis {tPb tP2, ... , t/>n}. What does this change of basis do to the A number.>. is an eigenvalue of A if and only if the equation (A- >.I)x = 0
original representation of the matrix problem? has a nontrivial solution, that is, if and only if the matrix A- >.I is singular. It
Since {t/Ji} forms a basis, for each vector t/>;, there are numbers c~i), i = follows that A is an eigenvalue of A if and only if it is a root of the nth order
1, 2, ... , n for which t/>; = c~i) tPi. The numbers c~i) are the coordinates of
E;=t polynomial
the vector t/>; relative to the basis {t/JJ}· In the case that {tPJ} is the natural PA(A) = det(A- >.I), (1.6)
basis, the number c~i) is the jth element of the vector tj>;. called the characteristic polynomial for A.
H $; and :z:~ are coordinates of a vector x relative to the bases {t/J;} and {tPi}, There are always n roots, counting multiplicity, of an nth order polynomial.
respectively, then The order of any particular root is called its algebraic multiplicity while the
number of linearly independent eigenvectors for a given eigenvalue is called its
geometric multiplicity. The geometric multiplicity can be no greater than
x = I;x~:t/J~c
n
= 'Ex~tj>;
n
= I;x~ I;c~')t/Ji = 2:; 'Ec~'>x~ t/JJ,
n (n ) n (n )
the algebraic multiplicity for an eigenvalue A.
A:=l i=t i=t j=l i=l i=l Our motivation for finding eigenvectors is that they provide a way to repre-
sent a matrix operator as a diagonal operator. Another important interpretation
so that, since the representation of a vector in terms of a given basis is unique,
of eigenvectors is geometrical. If we view the matrix A as a transformation that
ZJ = E~ 1 c~')x~. Written in vector notation,$= C:z:' where Cis the matrix transforms one vector into another, then the equation Ax = .>.:z: expresses the
of coefficients that expresses the basis {t/>;} in terms of the basis {t/J;}, with the fact that some vector, when it is transformed by A, has its direction unchanged,
number c~') in the jth row and ith column of C. even though the length of the vector may have changed. For example, a rigid
Now the original problem A$= b becomes A'x' = c-tACx' = b', where body rotation (in three dimensions) always has an axis about which the rotation
A' = c-tAC, z, bare the coordinates of x,b relative to the original basis takes place, which therefore is the invariant direction.
{t/J;}, and x', b' are the coordinates of :z:, b relative to the new basis {1{1;}. The
Theorem 1.2 Suppose A is an n x n matri:z:.
transformation A' = c- 1 AC is called a similarity transformation, and we
see that all similarity transformations. are equivalent to a change of basis and 1. If the matri:z: A has n linearly independent real (or complex} eigenvectors,
vice versa. Two matrices are said to be equivalent or similar if there is a there is a real (or complex) change of basis in Jl(l (or <lin) so that relative
similarity transformation between them. to the new basis A is diagonal.
This transformation can be made a bit clearer by examining what happens
2. lfT is the matri:z: whose columns are the eigenvectors of A, then T- 1 AT=
when the original basis is the natural basis. H AE is the representation of A
A is the diagonal matri:z: of eigenvalues.
with respect to the natural basis and {t/>;} is the new basis, then 0 1 is the matrix
whose columns are the vectors tj>;. The representation of A with respect to {t/>;}
is At = 01t AECt. Similarly, if C~ has as its columns the basis vectors {t/Ji}, The factorization A= TAT- 1 is called the spectral representation of A.
then the representation of A with respect to {t/J;} is A~ = C:J 1AEC~. Thus, if Proof: Suppose :z:1, X2, ... , :Z:n are linearly independent eigenvectors of A, with
we are given A~, the representation of A with respect to {t/J;}, and wish to find Axi = Ai:z:;. LetT be the matrix with columns of vectors Xi. Then we have
At, the representation of A with respect to {t/>1}, we find that
Xn)[ ~ {A:z:1 A:z:2 ... Ax]n) = (A1:z:1 A2:z:2 ... Ana:n)
At= 01t AECt = C1 1 (C~A~c;t)Ct = (C;tCt)- 1 A~(c;tCt)·
AT = A(:z:1 :z:2 ...
1
.>.2 0
In other words, the matrix C that transforms A~ to At is C = c;tct where = (:z:1 :z:2 ... :z:n) = TA,
Ct has t/>; in its ith column and C~ has t/J; in its ith column.
0 An
Under what conditions is there a change of basis (equivalently, a similarity
transformation) that renders the matrix A diagonal? Apparently we must find where A is diagonal, or
a matrix C so that AC = C A where A is diagonal, that is, the column vectors T- 1 AT=A.
of C, say :z:;, must satisfy A:z:, = ~;:z:;, where A; is the ith diagonal element of
A. I
12 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.2. SPECTRAL THEORY FOR MATRICES 13
Examples: Proof: The proof of this statement is by induction. For each eigenvalue >..~:,
we let x.~: be the corresponding eigenvector. For k = 1, the single vector
1. The matrix x1 =F 0 forms a linearly independent set. For the induction step, we suppose
A=(~!) x1,x:l,· .. ,x.~:-1 are linearly independent. We try to find scalars a1, a2, ... ,a.~:
has eigenpairs ~~ = 4, =
XI (1,1)T, and ~2 x2=2, =
(1,-1)T 2 • The
vectors X1, x2 are real and linearly independent. The matrix
so that
a1x1 + a2x2 + · · · + a~cx1c = 0.
If we multiply this expression by A and use that Axi = AiXi, we learn that
T= ( 1 1)
1 -1 a1>.1x1 + a2>.2x2 + · · · + a~c>.~cx~c = 0.
gives the required change of basis and
If we multiply instead by >..~:, we obtain
T- 1 AT= ( 0 4 0) 2 . >.~ca1x1 + >.1ca2x2 + · · · + >.~ca~cx~c = 0.
2. The matrix Subtracting the second of these expressions from the first, we find that
A= ( c?s(} -sin(} )
smO cos(} a1(>.1- >..~:)xl + · · · + a.~:-1(>..~:-1- >..~:)x.~:-1 = 0.
transforms the vector (cos¢, sin c/J)T into the vector (cos(¢+0),sin(¢+0))T
and so is called a rotational matrix. This matrix has eigenvalues ~ = It follows that a1 = = ··· =
a2 a1c-1 =
0, since xl,X2,···•XA:-l are assumed
sin(}± icos(} and eigenvectors (=Fi, l)T. There is no real change of basis to be linearly independent and Ale is distinct from >.1, >.2, ... , Aie-l, and finally
which diagonalizes A, although it can be diagonalized using complex basis a.~:= 0 since x~c is assumed to be nonzero.
vectors. I
3. The matrix There is another important class of matrices for which diagonalization is
A=(~~) always possible, namely the self-adjoint matrices, but before we can discuss
these, we must define the adjoint of a matrix.
has characteristic polynomial ~ 2 = 0, so there is one eigenvalue .>. = 0
with algebraic multiplicity two. However, there is only one eigenvector Definition 1.8 For any matrix A, the adjoint of A is defined as the matrix
x 1 = (1, O)T, so the geometric multiplicity is one. There is no change of
A* where (Ax,y) = (x,A*y), for all x, yin (i;ffl.
basis which diagonalizes A.
We need to determine when there are n linearly independent eigenvectors. As Definition 1.9 If A= (a;J), the transpose of A is AT= (a;,).
we see from the last example, the dilemma is that even though the characteristic To find the adjoint matrix A • explicitly, note that (using the Euclidean inner
polynomial (1.6) always has n roots, counting multiplicity, there need not be an product)
equal number of linearly independent eigenvectors. It is this possible deficiency
that is of concern. For any eigenvalue >., the geometric multiplicity is at least (Ax, y)
n,n
= L L aiJXJfii = L Xj
n (nL aiJili
)
one, since det(A- >.I) = 0 implies there is at least one nontrivial vector x for i=l j=l j=l i=l
which (A- >.I)x = 0. In fact, every square matrix A has at least one eigenpair
(although it may be complex).
so that A • = AT. For example, if
Theorem 1.3 If A has n distinct eigenvalues, then it has n linearly independent au a1:l )
eigenvectors. A= a21 a22 ,
(
aa1 aa2
2 Remark about notation: When dealing with vectors and matrices, it is important to then
distinguish between row vectors and column vectors. For example, (1, 2) is a row vector and A• =( au ~~ aa1 ) .
( ~ ) is a column vector. However, it is often convenient to write column vectors as the Cii2 a:l2 aa:l
transpose of a row vector, for example, ( ~ ) = (1, 2)T. Definition 1.10 A matrix A is self-adjoint if A • = A.
14 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.2. SPECTRAL THEORY FOR MATRICES 15
H A is self-adjoint and complex, then A is called a Hermitian matrix We observe that if M is an invariant manifold over the complex scalar field
whereas, if A is self-adjoint and real, it is symmetric. Notice that by definition, for some matrix A, there is at least one vector x E M with Ax = ..\x for
a self-adjoint matrix is square. some ..\ E a::'. To verify this, notice that since M lies in a k-dimensional space,
it has a basis, say {xi. x2, ... , xk}· For any x E M, Ax E M so x and Ax
Theorem 1.4 If A is self-adjoint the following statements are true: have representations relative to the basis {xi, x2, ... , Xk}· In particular, take
1. (Ax, x) is real for all x, x == I:~=l aixi and Axi = I:7=l f3jiXj· To solve Ax- ..\x = 0, we must have
Observing that {XI, x2, ... , Xk} is a linearly independent set, we have
The proof of this theorem is as follows:
k
1. H A= A*, then (Ax,x) = (x,A*x) = (x,Ax) = (Ax,x), so (Ax,x) is real. L(f3ii- Mii)ai = 0, j = 1,2, ... ,k,
2. H Ax= Ax, then (Ax,x) = (Ax,x} = ,\(x,x). Since (Ax,x} and (x,x} are i=I
real, ,\ must be real. which in matrix notation is (B- ,\l)a = 0 where B = ((3ji) is a k x k matrix,
3. Consider the eigenpairs ( ..\, x) and (#-', y). Then and I is the k x k identity matrix. Of course, as we noted earlier, every square
matrix has at least one eigenpair, which concludes the verification .
..\(x, y) = (.\x, y) = (Ax, y) = (x, Ay) = (x, 1-'Y} = J.l(x, y) We are now ready to prove item 4, namely that the eigenvectors of an n x n
self-adjoint matrix form an n-dimensional orthogonal basis. Suppose A = A*.
so that Then there is at least one eigenpair (..\IJ x 1 ) with Axi = ..\1xi and ..\1 real and
= 0.
(,\- J.l){X, y) the eigenvector x 1 forms a linearly independent set (of one element). For the
H ,\and 1-' are distinct(,\ f: J.l), then (x, y) = 0, i.e., x andy are orthogonal. induction step, suppose we have found k - 1 mutually orthogonal eigenvectors
Xi, Axi = AiXi with Ai real fori= 1, 2, ... , k- 1. We form the linear manifold
The proof of item 4 requires more background on linear manifolds.
Mk = {xi (x, Xj) = 0, j = 1, 2, ... , k - 1},
Definition 1.11
called the orthogonal complement of the k- 1 orthogonal eigenvectors x1,
1. A linear manifold M C S is a subset of S which is closed under vector
x2, ... , Xk-l· This manifold is invariant for A since, ifx E Mk, then (x,xj) = 0
addition and scalar multiplication.
and
2. An invariant manifold M for the matrix A is a linear manifold M C S (Ax,xj) = (x,Axj) = ,\j(x,x3) = 0
for which x E M implies that Ax E M. for j = 1, 2, ... , k- 1. Therefore, Mk contains (at least) one eigenvector Xk
corresponding to a real eigenvalue ..\k and clearly (xk, Xj) = 0 for j < k, since
Examples: Xk E Mk. The eigenvalue Ak is real since all eigenvalues of A are real. In
1. N(A), the null space of A, is the set of all x for which Ax= 0. If x andy summary, we can state the main result of this section:
satisfy Ax= 0 and Ay = 0, then A( ax+ {3y) = 0 so that N(A) is a linear
manifold. If x e N(A), then Ax= 0 so since A(Ax) = AO = 0, Ax is in Theorem 1.5 (Spectral Decomposition Theorem) If A is ann x n self-
N(A) as well, hence N(A) is an invariant manifold. adjoint matrix, there is an orthogonal basis {x1, x2, ... , Xn} for which
2. R(A), the range of A, is the set of all x for which Ay = x for some y. 1. Axi = AiXi with Ai real.
Clearly R(A) is a linear manifold and it is invariant since if x E R(A) then
surely Axe R(A). 2. (xi,Xj) = Dij (orthogonality).
1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 17
16 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES
,.""'""''
9. The matrix Q with x; as its jth column vector is unitary, that is, ' I
I
I
q-1 = Q*. ;;
,'
I
I
I
I
I -~ b
X
4. Q* AQ =A where A is a diagonal matrix with real entries Ai· I
I
-
I I
I
I
I
I
to the basis of eigenvectors, this is a diagonal problem. That is, setting x = Qx',
I
=
I
I
I
=
b Qb' we find that Q* AQx' Ax' =b'. H all the eigenvalues are nonzero, I
I
,I
the diagonal matrix A is easily inverted for x' = A- 1 b'. Rewriting this in terms
I
I
of the original basis we find that x =QA- 1Q*b. These operations can be
I
I
I
I
I
A-l I
Ax=b ~ X= A- 1 b I
I
;
b=Qb' .t. t =
b' Q*b
I
~-
;;
A-1
Ax'= b' ~ x' = A- 1b'
The solution of Ax = b can be found directly by applying the inverse of A Figure 1.3: Graphical solution of the 2 x 2 matrix equation Ax = b with A given
(the top of the diagram) or indirectly in three steps by making a change of coor- by (1.7), using the eigenvectors x 1 and x2 of the matrix A as coordinates.
dinate system to a diagonal problem (the leftmost vertical descent), solving the
diagonal problem (the bottom of the diagram), and then changing coordinates
back to the original coordinate system (the rightmost vertical ascent).
1.3 Geometrical Significance of Eigenvalues
The geometry of diagonalization can be illustrated on a piece of graph paper. Aside from providing a natural basis in JR" in which to represent linear prob-
Suppose we wish to solve Ax = b where lems, eigenvectors and eigenvalues have significance relating to the geometry of
A= ( -~
-1 1)
1 .
(1.7) the linear transformation A.
For a real symmetric matrix A, the quadratic form q(x) = (Ax, x) produces a
real number for every vector x in JR". Since q( x) is a continuous function in IR",
We first calculate the eigenvalues and eigenvectors of A and find At = !, with it attains a maximum on the closed, bounded set of vectors x with llxll = 1.
eigenvector Suppose the maximum is attained at x = x 1 • Then for every unit vector x
Xt = ( ~) orthogonal to x 1, q(x) :::; q(xt). But, on the subset (x,x1) = 0, llxll = 1, q(x)
again attains a maximum at, say, x = x2. Continuing inductively in this way we
and A2 = -1, with eigenvector produce a set of mutually orthogonal unit vectors, x1, X2, ••• , Xn at which the
local extremal values of q(x) are attained on the set llxll = 1. We will show that
X2 = ( i)' x,
the are the eigenvectors of A and that the values q(xi) are the corresponding
eigenvalues.
We plot these eigenvectors on a piece of rectangular graph paper. Any vector
b can be represented· uniquely as a linear combination of the two basis vectorE Theorem 1.6 (Maximum Principle) If A is a real symmetric matrix and
XlJ x2, say b = a1x1 + a2x2. To find the solution x of Ax =
b (since x = q(x) = (Ax,x}, the following statements hold:
2a1x1 - a2x2) we take twice the first coordinate of b added to the negative
of the second coordinate of b with respect to the eigenvector basis. These arE 1. At = maxllzll=l q(x) = q(x1) is the largest eigenvalue of the matrix A and
depicted in Fig. 1.3. x1 is the eigenvector corresponding to eigenvalue Al·
As was suggested in the preface, this diagonalization procedure is the basif
on which most transform methods work, and is reiterated throughout this text. 2. (inductive statement) Let Ak = maxq(x) subject to the constraints
Hopefully, this commuting diagram will become emblazoned in your mind beforE
the end of this text.
a. (x,x;} = 0, j = 1, 2, ... 'k - 1,
CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 19
18
changed by reversing the sign of h, which is not permitted. Thus, we must have
b. llxll = 1.
(Az1 - J.'Zl, h) = 0 for all h. Taking h = Azt - J.'Xt, we conclude that
Then .X~c = q(x~c) is the kth eigenvalue of A, .Xt 2: .X2 2: · · · 2: .X~~: and x~c is
the corresponding eigenvector of A.
Azt - J.'Zl = 0,
so that Xt is the eigenvector of A. Furthermore, q(xt) = (Axt, Xt) = p(xlt Xt) =
· p, since llxtll = 1 so that max q(x) = p is the eigenvalue and the vector that
Examples: maximizes q(x) is an eigenvector of A.
To verify the inductive step, we again use Lagrange multipliers and maximize
1. Consider the matrix
A=(~ !)• P~c(x) = (.Az,x)- p((x,x) -1)-
k-1
L qJ(X,xj),
then q(x) = (Az,z) = 3vl + 2111112 + 3vt where J=l
plane is minimized. The slice that minimizes the principal axis contains the two Proof: According to the minimax principle, for any quadratic form q, the
smaller eigenvectors of A, but not the largest, so we find (x2, ..X2). relative maxima are given by
Proof: To prove the minimax principle we note that since A is symmetric, there
is a unitary matrix Q that diagonalizes A, so that A = Q AQ* where A is the
..Xj =min {max q(x)},
diagonal matrix of eigenvalues ..X1 ? . X2 • • • ? ..Xn. It follows that 0 llzll=l
O:z:=O
n
where Cx = 0 represents j - 1linear constraints on x. If Xi, i = 1, 2, ... , j - 1,
(Ax,x} = (QAQ*x,x} = (Ay,y} = 'E..Xiy~,
are the vectors for which the first j - 1 relative maxima of (j(x) are attained,
i=l
then the minimum for q(x) is attained by taking the rows of C to be xi, and
where y = Q*x and Cx = 0 if and only if CQy =By= 0, where B = CQ. We (restricting x so that llxll = 1)
define p. by
~i =min {max q(x)} = Il_!ax q(x) = max q(x) ~ ..Xi+k•
0 Oz=O (z,z;)=O (z,x;)=O
+ k- 1 constraints on
q(x). Similarly, (restricting x so that llxll = 1)
If we pick the matrix B so that By = 0 implies that Y1 = Y2 = · · · = Yk-1 = 0,
then p. :::; max11 11 11=1l:~=k ..Xiy[ = ..Xk. Now pick a different matrix B. Since B ~i =min
1/i
{ max q(x)}
(z,l/;)=0
$ max q(x) $
(z,x;}=O
max q(x) = ..Xj,
(x,z;)=O
has rank $ k - 1, the solution of By = 0 is not unique, and we can satisfy the
equation By = 0 with then- k additional restrictions YH1 = Yk+2 = · · · = where XiJ i = 1, 2, ... , j - 1 are those vectors at which the first j - 1 relative
Yn = 0. With these n- k restrictions, the augmented system, call it BaY= 0, maxima of q(x) are attained. 1
has rank$ k- 1 + (n- k) = n- 1 so always has nontrivial solutions. For this For a geometrical illustration of these results, consider a 3 x 3 positive definite
restricted set of vectors (i.e., with YH1 = Yk+2 = · · · = Yn = 0) the maximum matrix A and its associated quadratic form q(x) = (Ax,x}. If we view q(x) as
for each B might be reduced. Therefore, (restricting y so that IIYII = 1) the "radial" map of the unit sphere llxll = 1, x -t q(x)x, the surface thus
constructed is football shaped with three principal axes, the major axis, the
p. = min {max
B B11=0
t
i=l
..Xiy[ ~~
J . min { m~ ti=1 ..Xiy[}
B B.,l/=0
semi-major axis, and the minor axis with lengths . X 1 ~ . X2 ? ..Xa, respectively.
If we slice this football-shaped surface with any plane through the origin, the
intersection of q(x) with the plane is "elliptical" with a major and minor axis
? min
B
{~ax ..xk
B.,l/=0
tyr}
.
=1
= ..xk.
with lengths :X1 ?. :X2, respectively. It is an immediate consequence of the
preceding theorem that these are interlaced,
1 2 3 )
A= 2 4 6
( 3 6 9
22 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 23
which is of rank 1 and therefore has two zero eigenvalues. The range of A is
spanned by the vector (1, 2, 3)T and A has the positive eigenvalue ..\1 = 14. It
follows (see Problem 1.3.2) that the eigenvalues of A satisfy ..\1 ~ 14, ..\2 ~ 0,
..\a~ 0.
To use the minimax principle further we impose the single constraint :Ct = 0.
-
uj-1 ui uf+l
__..,
With this constraint the extremal values of q(:c) are the eigenvalues of the mj-1 mi mf+J
diagonal su bmatrix
~
A= ( 36 6)
8 . Figure 1.4: One dimensional lattice of balls connected by springs.
a3.,3. -
- - J. -- 1, .•. , n,
kj+ki-1
mi ,
·=_,b(!)· aJJ-1
aJ,J+l
2
= km j , J = , ... ,n,
kj-1 •
= .;;;, j = 1. ... , n- 1,
We try the nearby unit vector
and all other entries are zero. Of course, one could envision a more complicated
•=;.. 0)
lattice for which connections are not restricted to nearest neighbor interactions
in which case the matrix A would not be tridiagonal. Something to try with
the system (1.8) is to assume that the solution has the form u = 4Jeiwt in which
case we must have
and find that q(:c) = ¥/ =13.07 so that 13.07 ~ ..\t ~ 14. A4J = -w2 4J.
An improved estimate of ..\2 is found by imposing the single constraint :c2 = 0
to get the restricted quadratic form q(:c) = :c~ + 6:ct:ca + 9:c~. For this Thus, the eigenvalues >. = -w 2 of the matrix A correspond to the natural
quadratic form the extremal values are ~· It follows that -0.1098 ~ frequencies of vibration of this lattice, and the eigenvectors 4J determine the
shape of the vibrating modes, and are called the natural modes or normal
.tfB ~ ..\2 ~ 0. These estimates are actually quite good since the exact
modes of vibration for the system.
eigenvalues are ..\ = -1, -0.076, and 13.076.
The matrix A for the simple lattice here is symmetric if and only if the
To understand further the physical significance of eigenvalues, it is useful to masses m; are identical. H the masses are identical we can use the minimax
think about vibrations in a lattice. By a lattice we mean a collection of objects principle to draw conclusions about the vibrations of the lattice.
(balls or molecules, for example) which are connected by some restoring force H mi = m for j = 1, 2, ... n, we calculate that
(such as springs or molecular forces). As a simple example consider a one- n n
dimensional lattice of balls connected by linear springs (Fig. 1.4). Let m; and m(Au, u} = m L L aiJUiUJ
u; denote the mass and displacement from equilibrium, respectively, of the jth i=l j=1
n-1
ball. The equation of motion of the system is given by
= -kou~- knu;- L kj(Uj- 2
UJ+1) .
cflui j=l
ffij dt2 =k;(Uj+l-Uj)+kj-1(Uj-1-Uj)•
Since the constants ki are positive, A is a negative definite matrix (a negative
Here we have assumed that the restoring force of the jth spring is linearly pro- definite matrix is one for which (Ax,x} < 0 for x ::f: 0). Notice further that
portional to its displacement from equilibrium with constant of proportionality increasing any kj decreases the quantity {Au, u} and hence, the eigenvalues of
ki > 0 (i.e., Hooke's law). A are decreased. From the minimax principle we have some immediate conse-
Suppose that the masses at the ends of the lattice are constrained to remain quences:
=
fixed so that u 0 Un+l = 0. This system of equations can be written as 1. H the mass m is increased, all of the eigenvalues of A are increased, that
is, made less negative, so that the natural frequencies of vibration are
rPu =Au, (1.8) decreased.
dt2
24 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 25
2. If any of the spring constants k; is increased then the eigenvalues of A The first half of the Fredholm alternative (Theorem 1.9) is easily proven. If
are decreased (made more negative) so that the natural frequencies of A*v = 0 and xo satisfies Axo = b then
vibration are increased.
(b, v) = (Axo, v) = (xo, A"v) = (xo, 0) = 0.
The natural frequencies of vibration of a lattice are also called the resonant
frequencies because, if the system is forced with a forcing function vibrating To prove the last half, suppose (v, b) = 0 for all v with A*v = 0. We write
at that frequency, the system resonates, that is, experiences large amplitude b = br + bo where br is the component of b in the range of A and bp is the
oscillations which grow (theoretically without bound) as time proceeds. This component of b orthogonal to the range of A. Then 0 = (bo, Ax) = (A*b0 , x)
observation is the main idea behind a variety of techniques used in chemistry for all x, so that A"bo = 0. Since (b, v) = 0 for all v in the null space of A*,
(such as infrared spectroscopy, electron spin resonance spectroscopy and nu- and since bo is in the null space of A*, we conclude that (b0 , b) = 0. Expanding
clear magnetic resonance spectroscopy) to identify the structure of molecules in b = br+bo we find 0 = (bo,br+bo) = (bo,br)+(bo,bo). Since (b0 ,br) = 0 it must
liquids, solids, and gases. be that bo = 0 so that b = br is in the range of A, i.e., Ax= b has a solution. 1
It is also the idea behind microwave ovens. If an object containing a water
molecule is forced with microwaves at one of its resonant frequencies, the water
molecule vibrates at larger and larger amplitudes, increasing the temperature
Example:
of the object. As you may know, microwave ovens do not have much effect on
objects whose resonant frequencies are much different than that of water, but To illustrate these two theorems in a trivial way, consider the matrix
on objects containing water, a microwave oven is extremely efficient. Microwave
ovens are set at frequencies in the 10 11 hz range to resonate with the rotational
frequencies of a water molecule. A=(!~)·
The null space of A is spanned by the vector (2, -l)T so that solutions of
1.4 Fredholm Alternative Theorem Ax = b, if they exist, are not unique. Since the null space of A • is spanned
by the vector (3, -l)T, solutions of Ax= b exist only if b is orthogonal to v
The Fredholm Alternative Theorem is arguably the most important theo- so of the form
rem used in applied mathematics (and we will see it many times in this text) as
it gives specific criteria for when solutions of linear equations exist.
b=a(!),
=
Suppose we wish to solve the matrix equation Ax b where A is an n x m which is no surprise since (1, 3)T spans the column space of A.
matrix (not necessarily square). We want to know if there is a solution, and if
so, how many solutions are possible. ·
One useful restatement of the Fredholm alternative is that the null space of
Theorem 1.9 (Fredholm Alternative Theorem) The equation Ax b has = A* is the orthogonal complement of the range of A and that together they span
= =
a solution if and only if (b, v) 0 for every vector v satisfying A • v 0. IR.m, so that IRm = R(A) EB N(A*). Stated another way, any vector b E JRm
can be written uniquely as b = br + bo where br is in the range of A and b0 is in
the null space of A•, and br is orthogonal to b0 (see Fig. 1.5).
It is interesting to note that this theorem is true for any inner product on
mn I although if one changes the inner product, the adjoint matrix A. changes
as do the vectors v for which A*v 0.= 1.5 Least Squares Solutions-Pseudo Inverses
Theorem 1.10 A solution of Ax = b (if it emts) ~ unique if and only if x = 0
Armed with only the Fredholm alternative, a mathematician is actually rather
~the only solution of Ax= 0.
dangerous. In many situations the Fredholm alternative may tell us that a
system of .equations Ax = b has no solution. But to the engineer or scientist
Proof: We prove Theorem 1.10 first. Suppose Ax = 0 for some x I- 0. If who spent lots of time and money collecting data, this is a most unsatisfactory
Axo = b then Xt = xo +ax also satisfies Ax 1 = b for any choice of a so that the answer. Surely there must be a way to "almost" solve a system.
solution is not unique. Conversely, if solutions of Ax = b are not unique then Typical examples are overdetermined systems from curve fitting. Suppose
there are vectors Xt and x2 with x = x 1 - x2 I- 0 satisfying Axt = Ax2 =b. the data points (ti, Yi), i = 1, 2, ... , n, are thought to be linearly related so that
=
Clearly Ax A(xt - x2) = Axt _: Ax2 0. = Yi = a(ti -l) + /3, where t = ~ L:~=l ti. This is a system of n linear equations
26 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 27
We have control only over the selection of x, so the minimum is attained when
we minimize II Ax - brW. Since br is in the range of A, Ax = br always has a
solution and minx IIAx- bll = llboW· We know that br is the projection of b
onto the range of A and bo is in the orthogonal complement of the range of A.
Therefore, 0 = (bo,Ax) = (A•bo,x) for all x so that A•bo = 0. Now Ax= br is
equivalent to Ax = b - bo so that Ax - b = -bo must be in the null space of
A•, which is true if and only if A• Ax= A•b. Thus the least squares solution of
Ax = b is any vector x satisfying A • Ax = A • b. One such x always exists. The
equation
A•Ax = A•b (1.10)
is called the normal equation.
Another derivation of the equation (1.10) uses ideas of calculus. Since we
wish to minimize IIAx- bll 2 =(Ax- b,Ax- b}, we let x = xo + ay, where xo
is the minimizing vector and y is an arbitrary perturbation. Since xo renders
IIAx- bll 2 a minimum, (A(xo + ay)- b,A(xo + ay)- b)~ (Axo- b,Axo- b)
for ay nonzero. Expanding this expression, we find that we must require
Figure 1.5: Graphical interpretation of the Fredholm alternative theorem. a(y, A•(Axo- b)}+ a(A.(Axo- b), y} + a 2 (Ay, Ay} ~ 0.
in the two unknowns a and /3 of the form This last expression is quadratic in a, and unless the linear terms vanish, for
sufficiently small a, a change of sign of a produces a change of sign of the entire
expression, which is not permitted. Thus we require (y, A*(Axo- b)} = 0 for
t 1 ) ( Yl ) ally. That is, we require A• Axo = A*b, which is the normal equation.
A(~)= ( t.~: ~ (~)= ~ .
t1 -
(1.9) We know from our first derivation that the normal equation always has a
solution. Another check of this is to apply the Fredholm alternative theorem
tn-t 1 Yn directly to (1.10). We require (v,A"'b} = 0 for all v with A• Av = 0. (Notice that
If there are more than two data points this system of equations does not A• A is a square, self-adjoint matrix). If A* Av = 0 then Avis simultaneously
in general have an exact solution, if only because collected data always contain in the null space of A • and in the range of A. Since these two subspaces are
error and the fit is not exact. orthogonal, Av must be 0, the only element common to both subspaces. Thus
Since we cannot solve Ax = b exactly, our goal is to find an x that minimizes (v, A•b) = (Av, b) = 0, as required.
As an example of the least squares solution, consider the curve fitting prob-
IlAx- bll. lem described earlier. We seek a least squares solution of (1.9). Premultiplying
The minimal solution is norm dependent. One natural, and common, choice
of norm is the Euclidean norm (1.4) while other choices such as llxlloo or llxll = by A*, we find (use that E~= 1 (t,- f)= 0)
E~= 1 lxsl =llxlh are useful as well, but lead to different approximation schemes. n n
Definition 1.12 For an m x n matrix A and vector b E <lim, the least squares
a '.L:(t,- f) 2
= '.L:Yi(ti- f),
i=l i=l
problem is to find x E ~ for which the Euclidean norm of Ax- b is minimized. n
A vector x (not necessarily unique) that minimizes the Euclidean norm of Ax-b n/3 = LYi!
i=l
is the least squares solution.
To find the least squares solution, recall from the Fredholm alternative that b which is always solvable for a and /3.
can always be written as b = br + bo, where br is in the range of A and bo is Other types of curve fitting are possible by changing the underlying as-
orthogonal to the range of A. Since Ax - br is in the range of A then (by the sumptions. For example, one could try a quadratic fit Yi = at~+ j3t, + 'Y· Other
Pythagorean theorem) choices of basis functions (such as are discussed in Chapter 2) can also be used,
but in all cases we obtain a linear least squares problem. For the exponential
IIAx- bll2;, IIAx- brll2 + llboll2· fit y1 = aef3t,, the parameters a, /3 occur nonlinearly. However, the equation
28 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 29
A=(!!)· (1.11)
so that
For this matrix, 2 1 ) ( 1 )
X = ( Sbl + Sb2 1 =( 2/5
2/5
1/5 ) ( bl )
1/5 b2
R(A) = span { ( l )}• R(A*) =span { ( ~ )}' and that the pseudo-inverse of A is
I ( 2/5 1/5 )
N(A*) =span{ ( _; ) } ·
A = 2/5 1/5 .
There are a number of different numerical algorithms one can use to calcu-
The pseudo-inverse of A must project the vector late .the pseudo-inverse A'. In most situations it is not a good idea to directly
calculate (A* A)- 1 since, even if A has linearly independent columns, the calcu-
·b=(:~) lation of (A* A)- 1 is numerically poorly posed. The algorithms that we describe
30 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONs-PSEUDO INVERSES 31
below are included because they illustrate geometrical concepts, and also, in the the k vectors {uj} form an orthonormal basis for the null space of A•. Thus,
case of the last two, because of their usefulness for numerical calculations. the projection matrix P is given by P = I- 2::=
1 u,uf. Notice that if the null
space of A• is empty, then k = 0 and P =I. If the null space of A is empty we
Method 1: Gaussian Elimination row reduce the augmented matrix [A, P], but if not, we append the additional
The first idea is to mimic Gaussian elimination and find A' via ele-
constraint (w,, x} = 0 for each linearly independent w,
E N(A), and row reduce
the augmented matrix
mentary row operations. Recall that Gaussian elimination is the process of
[fr ~]·
reducing the augmented matrix [A, I] to [J,A- 1] by means of elementary row
operations. The goal of reduction is to subtract appropriate multiples of one row
from lower rows of the matrix so that only zeros occur in the column directly
below the diagonal element. For example, one step of the row reduction of the wT
I
0
matrix (equivalent to the system of equations Ax = Pb, {w 1, x) = 0, i = 1, ... , l),
au a12 B1n discarding all zero rows, which leads to [I, A'].
a22 B2n To see that this can always be done, notice that the projection P is guaran-
teed by the Fredholm alternative to project onto the span of the columns of A.
Furthermore, the rows of A span the range of A • and the vectors {Wi} span the
A= BA:A: BA:n null space of A, so that, from the Fredholm alternative JRn = R(A•) E9 N(A),
BA:+1,A: BA:+1,n the rows of A augmented with {wf} span mn
and hence can be row reduced to
an identity matrix (with possibly some leftover zero rows).
0
BnA: ann Example:
is accomplished by premultiplying A by the matrix (only nonzero elements are Consider the matrix
displayed)
1 A= ( 21 42 6)
3 .
Since A has linearly dependent columns, A has no Moore-Penrose pseudo
inverse. The null space of A is spanned by the two vectors
M.~:= I
1
-O'A:+l
-0'1:+2
1
1
w,=(_D· --=( -0·
and the null space of A • is spanned by the single unit vector
-O'n 1
u= ~ ( -~ )·
where O'J = BJA:/a,~:,~:. Specifically, the nonzero entries of M.~: are ones on the
AB a result,
diagonal and -ui in the kth column, below the diagonal. The numbers ui
are called multipliers for the row reduction. The result of a sequence of such p = I- 51 ( 1
-2
-2 ) 1 ( 4
4 = 5 2
2 )
1 .
row operations is to triangularize A, so that MnMn-1 · · · M1A = U is an upper We row reduce the augmented matrix
triangular matrix. The product of elementary row operations MnMn-1 · · · M1 =
L- 1 is a lower triangular matrix which when inverted gives A = LU. This
representation of A is called an LU decomposition. wr
A p)0
(2 4 6-
4/5
2/5 1 2 3
2/5)
1/5
0 .
By analogy, our goal is to find some way to augment the matrix A with ( wf 0 - 1 1 -1 0
2 -1 0 0 0
another matrix, say P, so that the row reduction of [A, P] yields the identity
and the pseudo-inverse [I, A']. The least squares solution we seek must satisfy Elementary row operations reduce this matrix to
Ax= br, subject to the conditions {w,,x) = 0 where {wi} spans the null space
1 0 0 2/70 1/70 )
of A, and the vector br is the projection of the vector b onto the range of A. We 0 1 0 4/70 2/70
.
can represent br as br = Pb = b.- Li=t A: {u,, b)u, = ( I- Li=t T)
A: u,u, b, where ( 0 0 1 6/70 3/70
1
32 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 33
(we have discarded one identically zero row) so that Eliminating the last row of U and last column of L (which are superfluous)
we determine that
A' = _!_
70
( ~6 3~ ) . A = LU = ( ~5 2~ ) ( 1 2 ) .
0 -2
It is easy to check that A • AA' = A •. Notice that since L has linearly independent columns, the null space of A is
the same as the null space of U and similarly, since the rows of U are linearly
independent, the null space of A* is the null space of L*. The range of A is
Method 2: L U Decomposition spanned by the columns of L and the range of A • is spanned by the columns of
U*. If A = A •, then U = o:L * for some scalar o:.
Any matrix A can be written as A = LU where L is lower triangular and
Once the LU decomposition is known, the pseudo-inverse of A is easily found.
U is upper triangular (with the proviso that some row interchanges may be
necessary). In practice, A is usually not reduced to a diagonal matrix, but Theorem 1.11 If A = LU, then
simply to triangular form. The decomposition is unique if we require L to
have ones on the diagonal and U to have linearly independent rows, except for A'= U*(UU*)- 1 (L*L)- 1 U.
possible zero rows.
To see why the LU decomposition is desirable, suppose we want to solve
Ax= band we know that A= LU. We let Ux =
y so that Ly =b. Solving This statement can be directly verified by substituting x:::: A'b into A* Ax=
Ly = b involves only forward substitution, and solving U x =
y involves only A*b and using that A= LU. Since U has linearly independent rows and L has
backward substitution. Thus, if both L and U are full rank, the solution vector linearly independent columns, UU* and L* L are both invertible. Notice also
x is easily determined. that if Aw = 0, then Uw = 0 and
The LU decomposition is found by direct elimination techniques. The idea
is to reduce the augmented matrix [A, I] to the form (U, L -l] by lower triangu-
{w, A' b) = {w, u•wu•)- 1 (L* L)- 1L*b)
lar elementary row operations. When implementing this on a computer, to save
= (Uw, (UU*)- 1 (L* L)- 1L*b) = 0,
memory, one can simply store the multipliers from the row reduction process so that x is the smallest possible least squares solution. The usefulness of this
in the sub diagonal element that is zeroed by the row operation (see Prob- representation of A' is primarily its· geometrical interpretation.
lem 1.5.12). The sub-diagonal half of the resulting matrix is the sub-diagonal
part of L and the diagonal elements of L are all ones. If A has linearly in- Method 3: Orthogonal Transformations
dependent rows, then zero rows of U and corresponding columns of L can be
harmlessly eliminated. Another important way to solve least squares problems is to transform the
matrix A to upper triangular form using carefully chosen transformations. The
advantage of transforming to a triangular system is that if the diagonal elements
Example: are nonzero, it can be solved by backward substitution, which is not much more
difficult than solving a diagonal system. However, there is an additional bonus
n
Consider the matrix
if the matrix transformations we use are orthogonal matrices.
A=u
By row reduction, storing multipliers below the diagonal, we find that A row
Definition 1.14 A square complex matrix Q is unitary (also called orthog-
onal) if Q* = Q- 1 , i.e., Q*Q = I.
(Recall that unitary matrices first appeared in Thm. 1.5.) Unitary matri-
reduces to
ces have two important geometrical properties, namely they preserve angles
(: -~) and lengths, that is, for any vectors x and y, (Qx, Qy) = (x, Q*Qy) = (x, y),
and IIQxll = llxll. In other words, unitary transformations are rotations and
reflections.
so that
1 0 0) The transformation we seek represents A as A= QR where Q is unitary and
U=O -D· L=
( 3
5
1
2
0
1
. R is upper triangular, and this is called the QR decomposition of A. This de-
composition can always be found because this is exactly what the Gram-Schmidt
34 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 35
I
I
I
I
I
X u = x+ llxlle
u = x -llxlle I 1
I
____ ... -
I
llxll e 1 I
I llxlle e
I I
----------- Uv I
I
-Pv
Figure 1.8: Sketch of the geometry of a Householder transformation applied to
Figure 1.7: Sketch of the geometry of an elementary reflector, where Pv = f1:ii~ u the vector u = x±llxllel. The two choices of a= ±llxll give different reflections.
is the projection of v onto u.
Proof:
procedure does, namely, it represents a vector as a linear combination of pre- Since x f. -cre1, u f. 0 and U is an elementary reflector. Since cr 2 = llxll 2 ,
viously determined orthogonal vectors plus a new mutually orthogonal vector.
Here, the columns of Q are the orthogonal basis that results from applying the
llull 2 = 2a 2 + 2crxl.
Gram-Schmidt procedure to the columns of A. The matrix R indicates how to where X1 = (x, e1) is the first component of x, and
express the columns of A as linear combinations of the orthogonal basis Q.
To carry this out we could use the Gram-Schmidt procedure, however, it uu*x (x + cre1,x)u
UX = X - = X - = X - U = -CTel.
turns out that this is a numerically unstable process. Instead, we use a method U
2
+ O"Xl CT 2 + CTX1
that looks similar to Gaussian elimination, which, instead of using elementary
row operations as in Gaussian elimination, uses an object called an elementary It is wise (for numerical reasons, to avoid cancellation errors) to choose a to
reflector, or more commonly, Householder transformation. have the same sign as Xt, so that x f. -cre1 unless x = 0 already, in which case,
the transformation is not needed. I
Definition 1.15 An elementary reflector (Householder transformation) is a
The geometry of Theorem 1.12 is depicted in Fig. 1.8.
matrix of the form
We now see that we can place zeros in the first column of A, below the diag-
2uu• onal, by premultiplying A by the appropriate Householder transformation. To
U =I -llull2' provided u ':/: 0.
carry out the transformation on the entire matrix, we do not actually calculate
the matrix U but rather note that for any column vector y
It is easy to verify that U = =
u• and u•u I. The identification of the
Householder transformation as a reflector comes from the fact that if v is parallel Uy = Y _ 2uu•y = y _ 2(u,y) u,
= =
to u, then Uv -v, while if vis orthogonal to u, then Uv v. Thus, if Pv is u*u (u,u)
= =
the projection of v onto u, then v Pv +(I- P)v, and Uv -Pv +(I- P)v.
so that U y can be viewed as a linear combination of two column vectors with
In other words, U "reflects" the vector v through the plane that is orthogonal
to u. A sketch of this is shown in Fig. 1.7. a multiplier related to the inner product of the two vectors. Premultiplying
A useful property of elementary reflectors is that, as is also true of elementary by a sequence of Householder transformations, we sequentially triangularize the
row operations, they can be chosen to introduce zeros into a column vector. matrix A. We first apply the Householder transfor:rp.ation that places zeros
in the first column below the diagonal. Then we apply a second Householder
Theorem 1.12 For z E IEr', let u =
±llzll, and suppo1e z ':/: -ue1. If u = transformation that leaves the first row and first column undisturbed and places
z + ue1, then U = I - ~ u an elementary reflector whose action on z u zeros in the second column below the diagonal, and so on sequentially. Note
=
Ux -ue1 (e1 u the first natural basis vector). that if
2UkUk
uk = h- llukll 2
36 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 37
is a k x k Householder transformation that places zeros in the first column below this cannot happen, since the transformation that diagonalizes Mo is not or-
the diagonal of a k x k matrix, then the n x n matrix thogonal. However, if Mo is not symmetric, then Mn converges to a "nice"
matrix whose eigenvalues are easy to find.
In-k 0 ) Householder transformations provide a very efficient way to find the QR
V~c = ( 0 U~c decomposition of a matrix. Good software packages to do these calculations on
a computer are readily available, and a good description of the corresponding
is the Householder transformation for an n x n matrix that transforms only the theory is found in [103).
lower diagonal k x k submatrix and leaves the remaining matrix unscathed.
H Vi, V:l,· · ·, Vn are the chosen sequence of transformation matrices, then Method 4: Singular Value Decomposition {SVD)
with Q* = Vn Vn-1 · · · V1 we have
The final method by which A' can be calculated is the singular value
IIAx- bll = IIQ* Ax- Q*bll = II&- Q*bll, decomposition. This method extends the idea of the spectral decomposition
of a matrix to nonsquare matrices and is often used in applications.
where As we will see, the singular value decomposition of a matrix A is similar to
the spectral representation of a square matrix A = TAT- 1 • Once the spectral
R= ( ~)' decomposition is known (if it exists), the inverse is easily found to be A- 1 =
TA- 1T- 1 , provided the diagonal matrix A has only nonzero diagonal entries.
with R upper triangular. H the columns of A are linearly independent (the This does not, however, work to find a pseudo-inverse.
same condition as required for the Moore-Penrose pseudo-inverse) the matrix R If A is self-adjoint, then the spectral decomposition of A is A= QAQ* where
is square, has nonzero diagonal elements, and is therefore invertible. Q is orthogonal. To find the pseudo-inverse of A, we want to minimize
H we denote the vector
IIAx- bll:l = IIQAQ*x- bll = IIAQ*x- Q*bll =
Q*b= (~)I IIAy- Q*bll, (1.12)
where y = Q*x. It is an easy matter to verify (see Problem 1.5.2) that the
where the vector b1 has the same number of elements as R has rows, then pseudo-inverse of an m x n diagonal matrix D of the form
the least squares solution is found by solving Rx =b1 and for such an x,
IIAx- bll = 11~11, and this error cannot be made smaller. It follows that the
pseudo-inverse of A is given by A'= R'Q*, where R' is the pseudo-inverse of R.
D=(D0 0)
0 '
This method is most useful in the case that the matrix A has linearly inde-
pendent columns so that R is invertible and the solution of Rx = b1 is found by where D is a (square) diagonal matrix with nonzero diagonal elements is the
back substitution. H, however, R is not invertible, one must find the smallest n x m diagonal matrix
solution of 1ix = i1]., which, unfortunately, cannot be found by back substitution.
There is another important use for Householder transformations that should
D' = ( Dt
0
0)
0 .
be mentioned here. As is well known, there is no way to find the eigenvalues It follows that the pseudo-inverse of the n x n diagonal matrix A is
of a matrix larger than 4 x 4 with a finite step algorithm. The only hope for
larger matrices is an iterative procedure, and one of the best is based on the
Q R decomposition.
A'= ( x-l0 o)
0 '
Two matrices have the same eigenvalues if they are related through a simi-
larity transformation. The goal of this algorithm is to find an infinite sequence where A is the nonzero part of A. Therefore we take
of similarity transformations that converts a matrix Mo into its diagonal matrix
of eigenvalues. y = Q*x = A'Q*b
The method is to use Householder transformations to decompose the matrix andy= Q*x is the smallest minimizer of (1.12). However,
Mn into the product of an orthogonal matrix and an upper triangular matrix
Mn = QnRn· To make this into a similarity transformation we reverse the order X= QA'Q*b
of matrix multiplication and form Mn+l = RnQn so that Mn+l = Q;; 1 MnQn.
Now the amazing fact is that if M 0 is a real symmetric matrix, then Mn con- has the same norm as Q*x and so is the smallest least squares solution of (1.12).
verges to the diagonal matrix of eigenvalues. Of course, if Mo is not symmetric, It follows that the pseudo-inverse of A is A'= QA'Q*.
38 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 39
The geometrical interpretation of this pseudo-inverse is as follows. H A = Suppose A* A bas k positive eigenvalues A1, A2 1 ••• , Ak > 0, and Ak+l =
QAQ*, then Q represents the basis relative to which A has a diagonal represen- ).k+ 2= · · · =An = 0. Let Yi = $t.
i = 1, 2, ... , k, and notice that
tation. H one of the diagonal elements of A is zero, then of course, the inverse
of A does not exist. To find A'b, we form Q*b which expresses b relative to the (y;,yi} = .JX~\f>;;"(Ax;,Axi} = .JXIJ>:;(xi,A'"Axi}
new coordinate system. The expression A'Q*b projects Q*b onto the range of
A and determines the smallest element whose image under A is Q* b. Finally = .:§f(x;,xi} = 5ii·
Q(A'Q*b) expresses this vector in terms of the original coordinate system (see
Problem 1.5.6). Since y; E mm I this guarantees that k :::; m. We can use the Gram-Schmidt
The change of coordinates Q identifies both the range and null space of A. procedure to extend the y;'s to form an orthonormal basis for !Rm, {y;}~ 1 . Let
What we have done here is to project the vector b onto the range of A and U be the matrix with columns Yl, Y2, ... , Ym. The vectors y; are the orthonormal
then find another vector in the range of A whose image is this projection. Since eigenvectors of AA •. We now examine the entries of U* AV. For i ::; k, the ijth
A* =A, the matrix Q is appropriate for both of these transformations. entry of u• AV is
This construction of A' can work only for symmetric matrices where R( A •) =
R(A). Unfortunately, this construction fails for nonsymmetric matrices for two
(y;,AxJ}= -it-(Ax;,Axj}
'S.~'
= +(x;,A*Axi}
v Ai
important reasons (see Problem 1.5.9}. First, since A* :f. A, R(A*) :f. R(A) so = ~(Ax;,Axi) = .J>:iaiJ·
projecting onto R(A) does not also put us on R(A*). Furthermore, if A* :f. A,
Fori> k, note that AA*y; = 0 so that A*y; is simultaneously in the null space
the basis provided by the diagonalization of A is not an orthogonal basis and
of A and the range of A*. Therefore (the Fredholm alternative again}, A*y; = 0
hence projections are not orthogonal.
and (y;,Axi} = (A*y;,xi} = 0 fori> k. It follows that U*AV = E where
What is needed is an analogous construction for nonsymmetric matrices.
This construction is provided by the singular value decomposition, which we
= =
!: (O';j), O'ij = .J>:i5;i, i 1,2, ... , m, j =
1, 2, ... ,n, and this is equivalent
to (1.13).
now state.
It is noteworthy that the orthogonal vectors {x 1 , x 2 , ••• , xk} span the range
Theorem 1.13 Suppose A is any m x n matrix. of A • and the vectors {x k+l, ... , Xn} span the null space of A. Similarly, the
vectors {y1, Y2, ... Yk} span the range of A and {Yk+l• ... , Ym} span the null
1. A can be factored as space of A*. Thus the orthogonal matrices U and V provide an orthogonal
A= UEV*, (1.13} decomposition of !Rm and !Rn into !Rm = R(A) EB N(A*) and !Rn = R(A"') EB
where U (m x m) and V (n x n) are unitary matrices that diagonalize AA• N(A), respectively. Furthermore, the singular value decomposition A= UEV*
and A • A respectively, and E is an m x n diagonal matrix {called the shows the geometry of the transformation of X by A. In words, v· X decomposes
= =
matrix of singular values) with diagonal entries O'ii ...;>:i1 where Ai 1 i x into its orthogonal projections onto R(A•) and N(A), then EV*x rescales the
1,2, ... ,min(m,n), are the eigenvalues of AA* {see Problem 1.B.3}. projection of x onto R(A*), discarding the projection of x onto N(A). Finally,
multiplication by U places !:V*x back onto the range of A.
B. The pseudo-inverse of A is It is now clear that to minimize
A'= V!:'U., (1.14) IlAx- bll 2 = IIU!:V*x- bll 2 ,
where !:' 1 the pseudo-inverse of !: 1 is an n x m diagonal matrix with we take
diagonal entries o11 = 1II =-it-
-(1'
V AI
provided O'ii :f. 0 and 0 otherwise. X= V!:'U*b
so that the pseudo-inverse of A is as stated in (1.14). I
The factorization A = U!:V* is called the singular value decomposition of The singular value decomposition gives a nice geometrical interpretation for
A. the action of A. That is, first there is a rotation by V*, followed by rescaling
Proof: To show that this decomposition always exists, notice that the matrix by E and finally a second rotation by U. Similarly, to find the least squares
A• A is a symmetric n x n matrix with non-negative eigenvalues A1, A2 1 ••• , An· solution of Ax = b, we first use U to decompose b into its projection onto R(A)
Let V be the n x n matrix that diagonalizes A• A. The columns of V are the and N(A*). The component on R(A) is rescaled by E' and then transformed
orthogonal eigenvectors x1,x2, .•. ,xn of A* A, normalized to have length one. onto R(A*) by V.
Then This again illustrates nicely the importance of transform methods. As we
A* A= VAV*. know, the least squares solution of Ax = b can be found by direct techniques.
40 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 41
However, the change of coordinates provided by the orthogonal matrices U, V in the direction orthogonal to the nearly parallel straight lines of the original
transforms the problem Ax= b into ~y = b where y = V'"x, b = U'"b, which is problem. In other words, the singular value decomposition identified the stable
a diagonal (separated) problem. This least squares problem is easily solved and and unstable directions for this problem, and by purposely replacing € by zero,
transforming back we find, of course, x =
V~'U'"b. This transform process is we were able to keep the unstable direction from contaminating the solution.
illustrated in the commuting diagram In a more general setting, one can use the singular value decomposition to
filter noise (roundoff error) and stabilize inherently unstable calculations. This
A'
Ax=b --t x=A'b is done by examining the singular values of A and determining which of the
y=V'"x x=Vy singular values are undesirable, and then setting these to zero. The resulting
b=U'"b
.t. t b=Ub pseudo-inverse does not carry with it the destabilizing effects of the noise and
E' roundoff error. It is this feature of the singular value decomposition that makes
~y=b ~ y= ~'b
it the method of choice whenever there is a chance that a system of equations
where the top line represents the direct solution process, while the two verti- has some potential instability.
cal lines represent transformation into and out of the appropriate coordinate The singular value decomposition is always recommended for curve fitting.
representation, and the bottom line represents solution of a diagonal system. The SVD compensates for noise and roundoff error prppagation as well as the
The singular value decomposition also has the important feature that it fact that solutions may be either overdetermined or underdetermined. In a
allows one to stabilize the inversion of unstable (or ill-conditioned) matrices. least squares fitting problem it is often the case that there is simply not enough
To illustrate this fact, consider the matrix experimental evidence to distinguish between certain basis functions and the
resulting least squares matrix is (nearly) underdetermined. The SVD signals
A=(1+-Jfo
2-710 2+-fto
1-1fo), this deficiency by having correspondingly small singular values. The user then
has the option of setting to zero small singular values. The decision of how
small is small is always up to the user. However, with a little experimentation
where € is a small number. We can visualize what is happening by viewing the one can usually learn what works well.
equation Ax = b as describing the intersection of two straight lines in JR 2 by
noting that for € small the lines are nearly parallel. Certainly small changes in
b give large changes to the location of the solution. Thus, it is apparent that 1.5.1 The Problem of Procrustes
the inversion of A is sensitive to numerical error. The Householder transformation solves the problem of finding an orthogonal
We calculate the singular value decomposition of A to be transformation that transforms one vector into another, provided the lengths of
A' _ _
1 ( 1 1) ( 'f0to 0) ( 1 2 ) _ 1:_ ( 1 2 )
IIQA- Bll~ = Tr(AA'") + Tr(BB'")- 2Tr(QAB*).
- v'iO 1 -1 0 2 -1 - 10 1 2 ' 3
Procrustes was a legendary Greek outlaw, living in Athens, who would offer his hospitality
to travelers, and then force them to lie on an iron bed, stretching them on a rack if they were
the solution is now stable. That is, small changes in the vector b translate into too short or lopping off limbs if they were too long for the bed - an unusual method of
only small changes in the solution vector x =
A'b and, in fact, x varies only coordinate transformation,--to be sure.
42 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES ).6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 43
Thus, it is equivalent to maximizing Tr(QAB*). where eAt is the matrix whose only nonzero components are the diagonal ele-
Now suppose that the matrix AB* has the singular value decomposition ments e).it, j = 1, 2, ... , n. It follows that
1.6.1 Exponentiation of Matrices Proof: We suppose that At is the largest eigenvalue of A and that t/11 is the
corresponding eigenvector of A •, (l/Jt. tPt) = 1. We represent the initial vector
=
The solution of the scalar ordinary differential equation ~ au with u(O) Uo = as xo = a1l/Jt + r1 with a1 = (xo, t/11) and note that rt is orthogonal to tPI.
=
is u(t) uae'". It would be nice if we could make a similar statement for the (rt, tPt) = 0. The orthogonal complement of tPt is an invariant subspace of A
vector differential equation = = = =
since (Ar, t/Jt) (r, A*t/Jt) (r, At tPt) 0 if (r, tPt) 0. Furthermore,
dx=Ax (1.15)
dt IIArll $ Kllrll
where A is ann x n matrix and x(t) is a vector in m.n.
The problem, however, for some constant K < IAtl, whenever (r,t/Jt) = 0.
is that we do not know how to interpret the matrix eAt. Now we calculate that
The answer is readily found if the matrix A can be diagonalized. If there is
a similarity transformation so that A= T-tAT, then (1.15) becomes Akxo = a1Atl/J1 + r~;
dx where
T-
dt
=T AT-1Tx = ATx llr~;ll = IIAAiroll $ Kkllroll,
so that
so that
dy
-=Ay
lim ,1
k-+oo "l
~;A"xo-+ atlPl·
dt
Thus, if a1:f: 0, our result is established. Furthermore, the rate of convergence
where Tx = y. Since A is a diagonal matrix, the solution of the system !!If = Ay is approximately f, which is a measure of the separation between the largest
is readily written as eigenvalue At and tillother eigenvalues of A. 1
y(t) = eAty(O),
44 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 45
Example: The Ranking of Teams are all strictly positive. FUrthermore, the largest eigenvalue of the matrix A+ p.I
is ,\ + p., where tjJ is the positive eigenvector and ,\ the corresponding positive
Every Fall a controversy rages throughout the United States trying to decide
which is the best collegiate football team {division I-A). The dilemma is that eigenvalue of A. Hence, if we power the matrix A + JLl, convergence to tjJ is
there are over a hundred teams vying for the title, but no team plays more assured (provided A is irreducible), and tjJ is the positive ranking vector we
than 12 or 13 opponents, so there are often good teams that have not been seek.
matched up head to head. It is certain that this same controversy occurs in
many other sports and in many other countries. 1.6.3 Iteration Methods
A possible resolution to this controversy is to devise a ranking scheme that
assigns a rank to each team based on its performance. One idea (there are The power method is one of the many iteration methods that allow rapid extrac-
several) for how to calculate a team's rank is as follows: We assume that tion of important information about a matrix. In fact, matrix iteration is the
team i has rank r;, a number between 0 and 1. The team earns a score based only practical method by which to solve many important problems associated
on its games with other teams with partial differential equations. Direct methods, such as Gaussian eliminar-
tion, are certain to work if the problem is small enough, but direct methods
8; = _!_ L a;Jrh
n; J'f.i
have serious difficulties when matrices are large and sparse (a sparse matrix is
one which has mostly zero entries), as they often are with the simulation of
where a;J is a number that reflects the result of the head to head contest partial differential equations.
between teams i and j. The score depends on the product of a;; and r; in The main idea of many iteration methods is to split the matrix A into two
order to give some weight to the strength of opposition. It is reasonable to parts, A = A1 + A2, where A1 is invertible and easily solved, and then to write
take a;i + ai; = 1 with a;J > aii if team i beat team j in their head to head
the equation Ax = b as
contest. The division by n;, the total number of games played by team i, is
necessary to normalize the score 8;. The entry a;; is zero if team i has not
A1x = b- A2x.
played team j, and of course, a;t = 0. We then define an iterative procedure by
It is now reasonable to suppose that the score and the rank of a team should
be related by ..\r, = s;, so that the determination of rank reduces to an AlXHl = b- A2Xk· (1.17)
eigenvalue problem Ar = ..\r, where the entries of A are the weighted elements
a;; /n;. The question is to determine if this equation has a useful solution. The convergence of this iteration is easily seen to depend on the eigenvalues
of the matrix A1 1 A2 . That is, if xis a solution of the problem Ax= b, then
This leads us to consider an important class of matrices called positive ma-
trices. A positive matrix is a matrix A all of whose entries are positive. Sim- Xk+l - x = A1 1 A2(xk - x).
ilarly, a nonnegative matrix is one all of whose elements are nonnegative. The If A1 1 A2 has eigenvectors t/J; with corresponding eigenvalues J.l.j, and if Xo -x =
matrix that we constructed to rank football teams is nonnegative. The classi-
Ej=l a;t/J;, then
cally important statement about these matrices is called the Perron Frobenius
Theorem. Xk- X=~ OjJ.l.Nj·
j
Theorem 1.15 (Perron Frobenius) A nonnegative matrix A has a nonneg- The iteration converges provided the largest eigenvalue of A1 1 A2 is smaller than
ative eigenfunction with corresponding positive eigenvalue. Furthermore, if the 1 in absolute value.
matrix is irreducible, the eigenfunction is unique and strictly positive, and the Now the trick is to split A in such a way as to make the largest eigenvalue
corresponding eigenvalue is the largest of all eigenvalues of the matrix. of A1 1 A2 as small as possible, thereby making convergence as fast as possible.
The easiest split of A is perhaps
We leave the proof of this theorem to the interested reader, which can be
A=D+L+U
, found in several sources, including [56]. There are several equivalent ways to de-
scribe irreducibility. Perhaps the most geometrical is that a matrix is irreducible where D is the diagonal part of A, and L and U are lower and upper triangular
if for every pair of integers i, j, there is a sequence k1 , k2 , ••• , km for which the (off-diagonal) parts of A, respectively. Then, if we choose A1 = D, and A2 =
product Oik 1 OA: 1 A: 2 ••• akmi is nonzero. In practical terms for football teams, this L + U, iterates are called Jacobi iterates. These iterates are certainly easy to
means that between any two teams there is a path of common competitors. calculate, since they only involve solving the system
The implementation of the Perron Frobenius theorem uses the power method.
There is a positive number, say p.,_ so that the eigenvalues of the matrix A+ p.I Dxn+l = b- (L + U)xn,
46 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES 1.6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 47
which, if the diagonal elements are all nonzero, is easy and fast. However, it is Now suppose that !Jmsn and /Jmaz are the smallest and largest eigenvalues of
not much harder to take At= D + U and A 2 = L, in which case the iterates A1 1 A 2 ,
and that -1 < l'min < l'maz < 1. Then the optimal relaxation pa-
rameter is found by balancing the numbers 1- w(JJj + 1) between -1 and 1,
(D + L)xn+l = b- Uxn taking
2
require solution of a lower triangular system of equations at each step. This Wopt = 2 + Jlomln + Jloma3l 1
The convergence of the SOR method is assured if the eigenvalues of • R. Courant and D. Hilbert, Methods of Mathematical Physics, Volume I,
Wiley-Interscience, New York, 1953.
A} 1 ( -wA2 + {1 ·~ w)A1 ) = -wA} 1 A2 + (1- w)I
• J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University
1 Preas, London, 1965.
are less than one in magnitude. However, if tPi is an eigenvector of Ai A2, then
Numerical aspects of linear algebra are discussed, for example, in
( -wAi1 A2 + (1- w)I)t!JJ = (1- w(JJJ + 1))tPi·
48 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES PROBLEMS FOR CHAPTER 1 49
9. A Starting with the set {1, x, x 2 , ••• , xk, ... }, use the Gram-Schmidt pro- (c) Is the converse true?
cedure and the inner product
3. (a) Show that if A is ann x m matrix and B is an m x n matrix, then
(f,g} = ib f(x)g(x)w(x)dx, w(x) > 0
AB and BA have the same nonzero eigenvalues.
(b) Show that the eigenvalues of AA • are real and non-negative.
to find the first five orthogonal polynomials when 4. Show that the eigenvalues of a real skew-symmetric (A = -AT) matrix
are imaginary.
(a) a= -1, b = 1, w(x) = 1 (Legendre polynomials)
(b) a= -1, b = 1, w(x) = (1- x 2 )- 112 (Chebyshev polynomials) 5. Find a basis for the range and null space of the following matrices:
(c) a= 0, b = oo, w(x) = e-~ (Laguerre polynomials) (a)
a= -oo, b = oo, w(x) = e-:~~ (Hermite polynomials)
2
(d)
Remark: All of these polynomials are known by Maple.
10. ,!;!. Starting with the set {1, x, x2 , ••• , xn, ... } use the Gram-Schmidt pro-
cedure and the inner product
A~o D·
(b)
1
(f,g} = /_ (f(x)g(x) + f'(x)g'(x))dx 1 2 3)
1
relative to the basis {(1, 1, O)T, (0, 1, l)T, (1, 0, l)T}. (b)
(b) The representation of a transformation with respect to the basis
{(1,1,2}T,(1,2,3)T,(3,4,1)T} is ( -~ ~)
c00)
1 1 1) (c)
A= 2 1 3 .
( 1 0 1
1 2 0
Find the representation of this transformation with respect to the
2 1 3
basis {(1, 0, O)T, (0, 1, -l)T, (0, 1, 1)T}.
c10)
2. (a) Prove that two symmetric matrices are equivalent if and only if they (d)
have the same eigenvalues {with the same multiplicities).
(b) Show that if A and B are equivalent, then 0 1 0
detA = detB. 0 1 1
52 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES pROBLEMS FOR CHAPTER 1 53
(c) Express (b) in matrix form; that is, show that is not positive.
n
4. The moment of inertia of any solid object about an axis along the unit
x= LP,x vector x is defined by
i=t
2. Suppose P projects vectors in JRn (orthogonally) onto a linear manifold 3. (a) For any two vectors x,y E mn with llxll = IIYII find the Householder
M. What is the solvability condition for the equation Px = b? (orthogonal) transformation U that satisfies Ux = y.
3. Show that the matrix A= (lliJ) where a,i = (</Jit</JJ) is invertible if and (b) Verify that a Householder transformation U satisfies U*U =I.
only if the vectors <Pi are linearly independent.
4. Use the Gram-Schmidt procedure (even though Householder transforma-
4. A square matrix A (with real entries) is positive-definite if (Ax, x) > 0 for tions are generally preferable) to find the QR representation of the matrix
all x :/: 0. Use the Fredholm alternative to prove that a positive definite
matrix is invertible.
A= 2 l 3)
Problem Section 1.5
(9 1 2
4 2 1 .
1. Use any of the algorithms in the text to find the least squares pseudo- 5. For the matrix
( 5 -3)
inverse for the following matrices:
A=
c00)
(a) 0 4 I
the range, null space, etc. of a matrix and to augment a matrix with
another. Use these features of Maple to develop a program that uses illustrate on a piece of graph paper how the least squares pseudo-inverse
exact computations to find A' using the Gaussian elimination method A' = VE'U* transforms a vector b onto the least squares solution of
(Method 1), and use this program to find A' for each of the above Ax = b. For the second of these matrices, show how setting the smallest
matrices. singular value to zero stabilizes the inversion process.
2. Verify that the least squares pseudo-inverse of an m x n diagonal matrix 9. For a nonsymmetric matrix A= T- 1 AT, with A a diagonal matrix, it is
D with ~i = t1i6i; is the n x m diagonal matrix D' with ~i = ; 1 6,; not true in general that A' = T- 1 A'T is the pseudo-inverse. Find a 2 x 2
whenever q' :/: 0 and d~i = 0 otherwise. example which illustrates geometrically what goes wrong.
56 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES PROBLEMS FOR CHAPTER 1 57
10. Find the singular value decomposition and pseudo-inverse of Problem Section 1.6
1. A Write a computer program (in MATLAB) to find the largest eigen-
2v'S -2v'5) value of a matrix using the power method. Use this program to solve the
A= 3 3 .
( 6 6
following problems:
(a) Find the largest eigenvalue of the matrix in Problem 1.3.1.
11. Find the least squares solution of then linear equations (b) Find the smallest eigenvalue of the matrix in Problem 1.3.1. (Shift
the eigenvalues by subtracting an appropriate constant from the di-
a;x + b;y = Ci = 1,2, ... ,n,
i agonal.)
where a;b1 - a1b, :f: 0 for i :f: j. H r1, j = 1, 2, ... , k = n(n- 1)/2 are 2. ~ Develop a ranking scheme for your favorite sport. First collect the
the solutions of all possible pairs of such equations, show that the least data for all the games played between teams (or individuals).
squares solution
(a) Take a;i = 1 if team i beat team j and zero otherwise. Does this
r = (:) give a reasonable ranking?
is a convex linear combination of ri, specifically (b) Take aii to be the percentage of points earned by team i in the contest
between team i and j. Does this give a reasonable ranking?
k (c) Propose your own method to assign values to the matrix A.
r = LPJTj,
J=l 3. Prove that a diagonally dominant matrix has a nonnegative eigenvector.
where
D2
P - i
i - """
L..ti=l
D2'
i
fflij =1 if i = j
m,k = -aile if i > k
m;1 =0 otherwise.
14. (a) Show that IIAIIF = Tr(A* A) is a norm on the space of n x k matrices.
(b) Show that if A and B are related through an orthogonal similarity
transformation, then IIAIIF = IIBIIF·
(c) Show that IIAIIF = IIA*IIF, even though A and A* may have different
dimensions.
I
·~
i
:iJ
~
•'ffi
t
•
-~
I
:}§
-~
J Bibliography
j
·~
1t
~ [1] M. J. Ablowitz, P. A. Clarkson, and M. A. Ablowitz. Solitons, Nonlin-
I
ear Evolution Equations and Inverse Scattering, volume 149 of London
Mathematical Society Lecture Notes. Cambridge University Press, 1991.
{432}
·~::1
~ [2] M. J. Ablowitz and A. S. Fokas. Complex Variables: Introduction and
Applications. Number 16 in Cambridge Texts in Applied Mathematics.
.~
t
};~ Cambridge University Press, Cambridge, 1997. {273}
.~
~ [3] M. J. Ablowitz and H. Segur. Solitons and the Inverse Scattering Thins-
~
•';!
,
'i form. SIAM, Philadelphia, 1981. {432}
-_'4
.~ [4] M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions.
I Dover, New York, 1974. {402}
I [5] K. Aki and P. G. Richards. Quantitative Seismology, Theory and Methods:
I1" Vol. JJ. Freeman, 1980. {132}
'$
[6) R. Askey. Orthogonal Polynomials and Special Functions. SIAM, Philadel-
!
'$1
phia, 1975. {93}
[9] M.S. Bauer and D. S. Chapman. Thermal regime at the Upper Stillwater
dam site, Uinta Mountains Utah. Tectonophysics, 128:1-20, 1986. {408}
559
560 BIBLIOGRAPHY
~j
[35] G. E. Forsythe, M.A. Malcolm, and C. B. Moler. Computer Methods for
1991. {48}
~~
Mathematical Computations. Prentice-Hall, 1977. {48}
;j
[19] A. Cherkaev. Variational Methods for Structuml Optimization. Springer,
New York, 1999. {203} ~ [36] A. C. Fowler: Mathematical Models in the Applied Sciences. Cambridge
:l Texts in Applied Mathematics. Cambridge University Press, Cambridge,
1997. {551}
[20) Y. Choquet-Bruhat and C. DeWitt-Moretti. Analysis, Manifolds and
Phusics. North-Holland, Amsterdam, 1982. {497}
[21] S. N. Chow and J. K. Hale. Methods of Bifurcation Theory. Springer-
j [37) B. Friedman. Principles and Techniques of Applied Mathematics. Dover,
New York, 1990. {171, 328, 401}
Verlag, New York, 1982. {496} .; [38] P. R. Garabedian. Partial Differential-Equations. Chelsea Publishing Co.,
~
,')
[22] S. N. Chow, J. K. Hale, and J. Mallet-Paret. An example of bifurcation New York, 1998. {401}
to homoclinic orbits. J. Diff. Eqns., 37:351-373, 1980. {497}
I
~;j (39) I. M. Gel'fand and S. V. Fomin. Calculus of Variations. Prentice-Hall,
[23] C. K. Chui. An Introduction to Wavelets. Academic Press, 1992. {93, ~ Englewood Cliffs, 1963. {202}
~
307, 328}
[28] S. J. Cox and M. L. Overton. On the optimal design of columns against [44) D. Gottlieb and S. A. Orszag. Numerical Analysis of Spectral Methods;
buckling. SIAM J. Appl. Math., .23:287-325, 1992. {203} Theory and Applications. SIAM, Philadelphia, 1977. {401}
562 BIBLIOGRAPHY BIBLIOGRAPHY 563
(45] J. Guckenheimer and P. Holmes. Nonlinear Oscillations, Dynamical Sys- [62] 0. Kappelmeyer and R. Haenel. Geothermics with Special Reference to Ap-
tems, and Bifurcations of Vector Fields, volume 42 of Applied Mathemat- plication. Number No 4 in Geoexploration Monographer, Ser. 1. Gebriider
ical Sciences. Springer-Verlag, New York, revised edition, 1990. {496} Borntraeger, Berlin, 1974. {401}
(46] R. Haberman. Elementary Partial Differential Equations. Prentice-Hall, [63] J. Keener and J. Sneyd. Mathematical Physiology. Springer-Verlag, New
3rd edition, 1997. {401} York, 1998. {551}
(47] J. K. Hale. Ordinary Differential Equations. Wiley-Interscience, New (64] J.P. Keener. Analog circuitry for the van der Pol and Fitzhugh-Nagumo
York, 1969. {513, 551} equations. IEEE 7rans on Sys., Man, Cyber., pages 101Q-1014, 1983.
(48] P.R. Halmos. Finite Dimensional Vector Spaces. Van Nostrand Reinhold, SMG-13. {497}
New York, 1958. {47}
(65] J. P. Keener. The Perron Frobenius theorem and the ranking of football
(49] D. Hanselman and B. Littlefield. Mastering Matlab. Prentice-Hall, Upper teams. SIAM Rev., 35:8Q-93, 1993. {48}
Saddle River, NJ, 1996. {48}
[66] J. B. Keller. Inverse problems. Am. Math. Monthly, 83:107-118, 1976.
[50] A. Heck. Introduction to Maple. Springer-Verlag, New York, 2nd edition, {432}
1996. {48}
[67] J. B. Keller and S. Antman. Bifurcation Theory and Nonlinear Eigenvalue
[51] P. Henrici. Applied and Computational Complex Analysis. Wiley and Problems. Benjamin, New York, 1969. {496}
Sons, 1986. {273}
[68] J. Kevorkian and J. D. Cole. Multiple Scale and Singular Perturbation
[52] H. Hochstadt. Integral Equations. Wiley-Interscience, New York, 1973. Methods, volume 114 of Applied Mathematical Sciences. Springer-Verlag,
{127} Berlin, 1996. {546, 551}
[53] H. Hochstadt. The FUnctions of Mathematical Physics. Dover, New York, (69] S. Kida. A vortex filament moving without change of form. J. Fluid Mech.,
1986. {93} 112:397-409, 1981. {497} .
[54] M. H. Holmes. Introduction to Perturbation Methods. Springer-Verlag, (70] D. J. Korteweg and G. deVries. On the change of form of long waves
New York, 1995. {551} advancing in a rectangular canal, and on a new type of long stationary
waves. Phil. Mag., 39:422-443, 1895. {432}
(55] F. Hoppensteadt. Mathematical Theories of Populations: Demographics,
Genetics and Epidemics. SIAM, Philadelphia, 1975. {127} [71] Y. Kuznetsov. Elements of Applied Bifurcation Theory. Springer-Verlag,
New York, 2nd edition, 1998. {496}
[56] R. A. Hom and C. R. Johnson. Matrix Analysis. Cambridge University
Press, Cambridge, 1985. {44, 47} (72] G. L. Lamb, Jr. Elements of Soliton Theory. John Wiley and Sons, New
York, 1980. {328, 432, 497}
[57] P. Horowitz and W. Hill. The Art of Electronics. Cambridge Press, Cam-
bridge, MA, 2nd edition, 1989. {497} [73] L. D. Landau and E. M. Lifshitz. Mechanics. Butterworth-Heineman, 3rd
edition, 1976. {203}
[58] C. M. Hutchins. The acoustics of violin plates. Scientific American,
245:17Q-187, 1981. {401} [74] C. L. Lawson and R. J. Hanson. Solving Least Squares Problems. SIAM,
Philadelphia, 1995. {48}
[59] F. M. Phelps Ill, F. M. Phelps IV, B. Zorn, and J. Gormley. An experi-
mental study of the brachistochrome. Eur. J. Phys., 3:1-4, 1984. {204} [75] P. Lax. Integrals of nonlinear equations of evolution and solitary waves.
Comm. Pure Appl. Math., 21:407-490, 1968. {432}
(60] G. Iooss and D. Joseph. Elementary Stability and Bifurcation Theory.
Springer-Verlag, New York, 1989. {496} (76] G. A. Lion. A simple proof of the Dirichlet-Jordan convergence theorem.
Am. Math. Monthly, 93:281-282, 1986. {165}
(61] E. Jahnke and F. Emde. Tables of FUnctions. Dover, New York, 1945.
{402} (77] W. V. Lovitt. Linear Integral Equations. Dover, New York, 1950. {127}
564 BIBLIOGRAPHY 565
BIBLIOGRAPHY
[78] J. Lund and K. L. Bowers. Sine Methods for Quadrature and Differential [94] T. D. Rossing. The physics of kettledrums. Scientific American, 247:172-
Equatiom. SIAM, Philadelphia, PA, 1992. {93, 274} 179, Nov. 1982. {401}
[79] W. S. Massey. Algebraic Topology: An Introduction. Springer-Verlag, New [95] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill, New York,
York, 1990. {497} 3rd edition, 1976. {92, 165}
[80] S. F. McCormick, editor. Multigrid Methods. Frontiers in Applied Math- [96] W. Rudin. Real and Complex Analysis. McGraw-Hill, New York, 3rd
ematics. SIAM, Philadelphia, 1987. {402} edition, 1987. {164}
[81] L. M. Milne-Thomson. Theoretical AerodJmamics. Dover, New York, 1973. [97] D. R. Smith. Singular Perturbation Theory. Cambridge University Press,
{274} Cambridge, 1985. {551}
[82) J. D. Murray. Asymptotic Analysis. Springer-Verlag, New York, 1984. [98] I. N. Sneddon. Fourier Transforms. Dover, New York, 1995. {328}
{463}
[99] I. Stakgold. Branching of solutions of nonlinear equations. SIAM Rev.,
[83) J. D. Murray. Mathematical Biology. Springer-Verlag, New York, 1989. 13:289-289, 1971. {496}
{402}
[100] I. Stakgold. Boundary Value Problems of Mathematical Physics. Macmil-
[84) F. Natterer. The Mathematics of Computerized Tomography. Wiley, New lan, New York, 2nd edition, 1998. {171, 328}
York, 1986. {128}
[101] F. Stenger. Numerical methods based on Whittaker cardinal, or sine
[85] A. W. Naylor and G. R. Sell. Linear Operator Theory in Engineering and functions. SIAM Rev., 23:165-224, 1981. {274}
Science. Holt, Rinehart and Winston, New York, 1971. {127} ·
[102] F. Stenger. Numerical Methods Based on Sine and Analytic F'u.nctions,
[86) F. W. J. Olver. Asymptotics and Special F'u.nctions. A. K. Peters, Ltd., volume 20 of Springer Series in Computational Mathematics. Springer-
London, 1997. {463} Verlag, New York, 1993. {93, 274}
[87] R. E. O'Malley, Jr. Introduction to Singular Perturbations. Academic [103) G. W. Stewart. Introduction to Matrix Computations. Academic Press,
Press, New York, 1974. {551} New York, 1980. {48}
[88) M. Pickering. An Introduction to Fast Fourier Transform Methods for [104] J. J. Stoker. Nonlinear Elasticity. Gordon and Breach, New York, 1968.
Partial Differential Equations, with Applicatiom. Wiley and Sons, New
York, 1986. {401}
University Press, Cambridge, 1992. {48, 77, 402} [107) G. Strang and G. Fix. An Analysis of the Finite Element Method. Prentice-
don and Breach, Langhorne, PA, 1994. {497} [108] G. Strang and T. Nguyen. Wavelets and Filter Banks. Wellesley-
~~
1 Cambridge Press, Wellesley, MA, 1997. {93, 328}
[92] R. H. Rand and D. Armbruster. Perturbation Methods, Bifurcation The-
ory and Computer Algebra, Applied Mathematical Sciencu, volume 65. ~
~~
(109] A. F. Timin. Theory of Approximation of Functions of a Real Variable.
Springer-Verlag, New York, 1987. {497} :1 Macmillan, New York, 1963. {328}
:1J'l
(93] G. F. Roach. Green's FUnctions: Introductory Theory with Applications. [110) E. C. Titchmarsh. Theory of F'u.nctions. Oxford University Press, 1939.
Van Nostrand Reinhold, London, 1970. {171} J~ {273}
566 BIBLIOGRAPHY
567
566 BIDLIOGRAPHY
[126) E. Zauderer. Partial Differential Equations of Applied Mathematics. Wiley llx- ayll 2 = llx- f3Yll 2 + ll(o:- f3)yjj 2 ,
and Sons, New York, 2nd edition, 1998. {401}
which is minimized when a= {3. Clearly, if x = ay, then
[127] A. H. Zemanian. Distribution Theory and Thlnsform Anal1Jais: An Intro- 2
duction to Generalized FUnctions, with Applications. Dover, New York, l(x,y)j 2 = llxii IIYW·
1987. {171}
If so, then we calculate directly that llx- {3yjj 2 = 0, so that x = {3y.
567
568 APPENDIX A. SELECTED HINTS AND SOLUTIONS 569
1.1.9; (a) ¢o(x) = 1, c/J1(x) = x, c/J2(x) = x 2 - ~, ¢a(x) = x3 - ~x, ¢4(x) = (d) T does not exist.
/x4- li.x2
7
+ .1...
35.
1 1
(b) ¢o(x) = 1,¢1(x) = x,¢2(x) = x 2 - ~ = ~cos(2cos- 1 x),¢3 (x) = 1
(e) T = -1 1 } ) , T- 1 AT = ( 0
0 0
~ 0
~).
x - ~x =-! cos(3cos- 1 x), ¢4(x) = x 4 - x 2 + k = kcos(4cos- 1 x).
3
( 0 -v'3 ~ 0 0 3
(c) ¢o(x) = 1,¢1(x) = x -1,¢2(x) = x 2 - 4x + 2,¢3 (x) = x 3 - 9x 2 +
18x- 6, ¢4(x) = x 4 - 48x 3 + 72x 2 - 96x + 24.
1.2.7; T= ( _1 _2 ) ,T-1 AT= ( 3 0) .
O
(d) ¢o(x) = 1,¢1(x) = x,¢2(x) = x 2 - !,¢a(x) = x 3 - ~x,¢4(x) = 2 1 6
x4- 3x2 + i·
1.2.8; If x is in M, then Px = x, and if x is in the orthogonal complement of
1.1.10; ¢o(x)
33 2
= 1,¢1(x) = x,¢2(x) = x - ~,¢a(x)
27 ... ( ) 5 1930 3 4411
2
= x3 - t x,¢4(x) =
0
x4 - M, the Px = 0. Therefore, P has two eigenvalues, A= 0, 1.
28 X + 140' '1'5 X = X - 'i359X + 1057X.
~ ~
n
1.2.10; (d) Examine the eigenvectors of A and A* where A= ( ).
2 3
1.2.1; (a) Relative to the new basis, A= 1 1
(
0 0 1.3.1; Hint: Minimize (Ax,x) with a vector of the form xT = (1, -1,z,O).
(b) Set C = 11 21 43 ) , and D = ( 01 1 0 0) 1 . Then the repre-
1.3.2; (a) Prove that if the diagonal elements of a symmetric matrix are in-
creased, then the eigenvalues are increased (or, not decreased) as
( 2 3 1 0 -1 1 well.
sentation of A in the new basis is
4 4 )
(b) Find the eigenvalues and eigenvecto"' of B = ( : 8 -4 , and
4 ) 1"
A'= v-'cAc-'v = (
53
~ ~'~
19
=! . ¥,
1 23)
1.2.2; (c) A = ( 01o)
1 , and B
are not equivalent.
(lo)
0 2 have the same determinant but
=
l 1.3.3; The matrix
Apply 1.3.2a.
(3 2 2 4
4 7
has a positive, zero, and negative eigenvalue.
1.2.5; (a) R(A) = {(1,1,2)T,(2,3,5)T},N(A) = O,R(A*) = m?,N(A*) = 1.4.2; b must be in the range of P, namely M.
{(1, 1, -1)T}.
1.4.3; ( =>) Suppose A is invertible, and try to solve the equation ~i ai</Ji =
cn
0.
(b) R(A) = R(A*) = IR 3 , N(A) = N(A*) = 0. Taking the inner product with <Pi, we find 0 = Aa, so that a = 0, since
(c) T= ( ~1 -10
1 00) , T-1AT=
1 .
coo) 0 2 0
0 0 3
.
must be empty. Hence, (b, x) = 0 for all x in N(A*) so that Ax = b has
a solution. Similarly, the solution is unique since the null space of A is
empty.
570 APPENDIX A. SELECTED HINTS AND SOLUTIONS 571
0 0 0
(c) A' = 1\ ( :
0
~
-4
)
2.1.5; maxt lfn(t) - fm(t)l
form and n large. However,
= !(I- '!!;) if m > n, which is not uniformly small
J; lfn(t)- fm(t)!Zdt = (~2:':~ 2
< 1 ~n.
(d) A' = i ( -6 2 2)
-5 4 1
2.1.6; (a) Hint: Show that l(lul-lvl)l ~ ju- vi.
-4 2 2 2.1.8; J; X(t) = 0.
1.5.3; (a) Take u = :c -v.
2.1.11; J; (I; f(:c,y)dx) dy = - J; (I; f(:c,y)dy) dx = -~. Fubini's theorem
1.6.4; Q = (~,.P,,.P,), where~ = ;ky (:) ,¢>. = }.;. ( ~0 ) ,.P, = fails to apply because J; (I; lf(x, y)ldx) dy = f; (I; a:'!v' dx) dy =
J;; tan- ;dy does not exist.
-(v'ffi *
1
1(
75
2
~1 ) , and R- ¥) .
1Ji5 1f
~ 2.2.1; With w(x) = l,p(z) = \:x 2 + 136 , with w(:c) = v'f='?,p(x)
1), and with w(x) = ~,p(:c) = 3~(4x2 + 1).
= 1:1T (6x2 +
1.5.5; Use that
2.2.2; Define the relative error e~,: by e~ =1- ~· where /A:(x) is the kth
1 ( 1 -2 ) ( ViO 0 ) ( 1 -1 ) partial sum, f~.:(x) =~!::a ant/>n(x). Then
A = ViQ 2 1 0 2v'iQ . 1 1
or (a) H(:c) - 2H(x -?r) = ~ ~::0 2n~l sin(2n + l)x, e5 = 0.201,
A= ( i ~ ) ( ~ ~ ) ( ~ !a ) · (b) :c -?r = -2~~ 1 ~ sinnx, e5 = 0.332,
< X < 11" 4 ~oo
(c) { 211" X- x 11"0 < x < 2?r = 2 - 1T ~n=o
1r 1 (
cos 2n +
1) -
1.5.6; Use that A = i ( ~ ~2 ) ( ~ ~ ) ( !2 ~ ),so that (2n+t)!l x, e5 -
0.006,
AI = 1( -2} -2)
5 4 . l (d) x2 (2?r- x) 2 = 81~· -48 ~:: 1 ~ cosnx, es = 0.002.
~
1.5.8; For A = ( ~:: ~::~ ) , singular values are ViO and eVlO with E =
1 . f:t'n I
0.00 v LV, A =
( 0.} +
0.1 -
Jw
:Jrn =
0.2 -
0.2 +
~
:Jrn ) ( 200.1
-199.8
-99.8 )
100.2 .
'
j
~'
D.J
0.0
:1 .... ·1
l
~
·1.0
:u 100 2.2.15; Suppose the transforms of j,g, and hare fJ_, and respectively. Use F•
:u 10 direct substitution and the fact that
1
-k
L,i=~ e 21riJk/N = 1 if k is an
1.0 10 integer multiple of N (including 0), and = 0 otherwise to show that hj =
u
40
1.0 h9i·
10
o.a 2.2.16; Rewrite the definition of the discrete Fourier transform as a matrix mul-
0.0
0.0
. o.a 1.0
.til
1.1 1.0 CLO CL6 1.0
.til
... 1.0
tiplication. Show that the matrix is orthogonal.
2.2.19; See Fig. A.4.
Figure A.2: Left: Partial sum fk(x) for Problem 2.2.2c with k = 5; llight:
Partial sum /k(x) for Problem 2.2.2d with k = 3. 0.4
0.2
2.2.4; With the additional assumption that the function f(x) to be fit intersects
o.o-1---~
the straight line at the two points x = x 1 and x = x2, x1 < x 2, we must
have that x2- Xt =!,and x2 + x1 = 1, so that a=£/(})- !f(~),p = .0.2
2.2.6; (a) You should have a procedure that generates the Legendre polynomi- -2 ·1 0 3
X
als from Problem 1.1.9, or use Maple, which knows about Legendre
polynomials.
Figure A.4: The wavelet generated by the B-spline N 3 (x), for Problem 2.2.19.
I.Q I.Q
....
0.0
.... A=
(
21 0)
1 4 1 ,
/o-
P= -3 ( /o -h
h) ·
0 1 2 h-h
.... 0.0 CL6 1.0
•1.0
·1.0 .... 0.0 CL6 1.0
X X
The solution is
Figure A.3: Left: Legendre polynomials P 1 (x),P2(x) and P3 (x); llight: Legen-
dre polynomials P4 (x),P5 (x).
a=
(
_2
_112 !0 -~1! ) ( h lo) .
i -! i h
(b) g(x) = ax+bx3 +cx5 , where a= ~(1r4 -15311"2 + 1485) = 3.10346, 2.2.26; (a) Solve Aa = P where
b= -m<11"
4 2
- 12511" + 1155) = -4.814388, c = ~(11" _ 10511" +
4 2
1¢~'1//jdx, 11 t/J~'t/ljdx.
945) = 1.7269. N
=
(c) g(x) 3.074024x-4.676347x 3 + 1.602323x5 • A plot of g(x) is barely
distinguishable from sin 'II"X on the interval -1 :$ x :$ 1.
aij =
1 0
Pi=- ~h
i=O 0
Use Problem 2.2.23 to evaluate the coefficients, and find that the ith
2.2. 7; Use integration by parts to show that the Fourier coefficients for the two equation is at-1 + 4at + ai+l = *(fi+l- /i-t), fori ::fi 0, N.
representations are exactly the same for any function which is sufficiently {b) Solve Aa = P where
smooth.
2.2.9; (b) Write tPn+l - AnXtPn = r:,:=o
PktPk, and evaluate the coefficients by
taking inner products with tPb and using part (a), and the fact that llt/Jkll =
aij = 1¢~¢jdx,
1
0
·
PJ = -
N
L:
~0
/i 1t/J~t/Jjdx.
0
1
1. Show that Bn = -An(XtPn 1 tPn) 1 On= An/An-1· Use Problem 2.2.23 to evaluate the coefficients, and find that the ith
2.2.14; Direct substitution and integration yields h(t) = 211" r:,:._ fkgkeikt. equation is -ai-1 + 8ai- at+l = lUi+l- /i-t), fori ::fi O,N.
00
574 APPENDIX A. SELECTED HINTS AND SOLUTIONS 575
(c) Require ai-l+ 4ai + Gi+t = *Ui+t -/i-t), fori =f 0, N. 3.4.3; An = n281r2 is a double eigenvalue with ¢n(x) =sin n;x, '1/Jn(x) =cos n~x
(d) Solve Aa = f3 where for n odd.
= k(y,x):~~~·
Uii = 1'1/J~'(x)'l/J'j(x)dx,
1 3.4.4; k*(x,y)
II>.Kr+
1
3.2.1; Find a sequence of functions whose £ 2 norm is uniformly bounded but 3.6.1; (a) Show that llu- unll < "["
1 _ j>.KII ·
whose value at zero is unbounded. There are plenty of examples. (b) Use that IIKII :::; ! to
conclude that n 2: 2. The exact solution is
2
u(x) = 3~~ and some iterates are u 0 (x) = 0, u1(x) = 1, u2(x) = {,
3.2.2; The proof is the same for all bounded linear operators; see page 107. ua(x) = 9 .
3.2.3; (a) The null space is spanned by u = 1 when A= 2, therefore solutions 3.6.2; If f(x"') = 0, for convergence to x*, require II- f'(x"')l < 1 or 0 < f'(x"') <
exist and are unique if A =f 2, and solutions exist (but are not unique) 2.
112
if A= 2 and I 0 f(t)dt = 0. 1f(x·v"<x·) 1 < 1, wh'ICh 1'f f( x "') = o, and
3 •6 . 3 ; ror
D
convergence to x • , reqmre•
1 fl , . \'1 2
(b) The null space is spanned by u = x when A= 3, therefore solutions
f'(x*) =f 0 is always satisfied.
exist and are unique if A =f 3, and solutions exist, but are not unique,
1
if A= 3 and J0 tf(t)dt = 0. 3.6.4; (a) Let Xn+l = D- 1 b- D- 1 Rxn-
*
(c) The null space is spanned by ¢(x) = cosjx if A =
if A =f
Therefore, *· 3.6.5; Show that Un(x) = 2:::~=0 ~ ·
* for j = 1, · · ·, n, the solution exists and is unique, while if
2
A = for some j, then a solution exists only if J0 '/r f (x) cos j xdx = 0. 3.7.1; y = y(x) where a 2x = sin- y'y + yJl- y2, a=~·
1
3.4.1; (a) Eigenfunctions are ¢n(x) = sinn1l'x for An= n 2~ 2 . (b) Let y = ljc 2(z), then use Abel's technique to show that z(c) =
(b) f(x) = -;!-r E~o (~~~\)2 sin(2n- 1)11'x. IJl/c ~
-:; 1/co yc2p2-t'
3.4.2; (b) </11 (x) = sinx, At = ~' ¢2(x) = cosx, A2 = ~'/r. (c) T(x) = ~lnJl + y 2 - y, where y = ~~·
(c) There are no eigenvalues or eigenfunctions. Remark: The existence tsin(k+~)n d
of eigenfunctions is only guaranteed for self-adjoint operators.
4.1.1; ( a) Use that Sk ( x) = 2 sin if for -1 < x < 1, an then observe that
4.1.4; Observe that X= f~oo 1/l(x)dx is a test function, x(O) = 0, so that X= x</J
1i 4.2.12; g(x,y) = -!e-lv-xl, u(x) = ~ J.:O e-lv-xlu(y)dy- f~oo !e-lv-xlf(y)dy.
for some test function </J. Hence, 1/J(x) = Jx (x<P(x)). ' 4.2.13; u(x) = cosx +).. J; p(e) sin(x- e)u(e)~.
4.1.5; u(x) = c1 + c2H(x) + ca5(x). Show that a test function 1/J is of the form 1
1/J = .Jx(x2</J) if and only if f~oo 1/Jdx = f 0 1/Jdx = 1/J(O) = 0.
00 j
l
4.2.14; u(x) = 1 + J; eln(Vu<e)~.
4.1.6; u(x) = 5(x) + c1x + c2. ~ 4.3.1; L*v = v"- (a(x)v)' + b(x)v, with boundary conditions v(O) + v'(l)-
! a(l)v(l) = 0, v(l) + v'(O)- a(O)v(O) = 0.
4.1.7; Set u = xv, so that x 2v' = 0, and then u(x) = c1x + c2xH(x) (using that
x5(x) = 0). 4.3.2; L*v = -(p(x)v')' + q(x)v with p(O)v(O) = p(1)v(1), and p(O)v'(O) =
p(l)v' (1).
4.1.8; u(x) = c1 + c2H(x).
4.3.3; L*v = v"- 4v'- 3v with v'(O) = O,v'(1) = 0.
4.1.9; 5(x2 - a 2) = 2lal(5(x- a)+ 5(x +a)).
4.3.4; (a) g*(y,x)w(y) = (g*(e,x),<>(e- y))e = (g*(e,x),Len(e,v))e
4.1.10; In the sense of distribution, x'(x) = 5(x)- 5(x- 1), since (x'(x)<P(x)) = = (Leg*(e, x), g(e, y))e =(<>(e-x), g(e, y))e = g(x, y)w(x).
1
-(x(x), <P'(x)) =-I: x(x)<P'(x)dx =- f 0 <P'(x)dx = <P(o)- </1(1). (b) u(x)w(x) =(<>(e-x, u(e))e = (Leg*(e, x), u(e))e = (g*(e, x).J(e))e
4.1.11; (a) For distributions f and g, define (! * g, <P) = (g, 1/J), where 1/J(t) = = w(x) J: g(y, x)f(x)dx.
(f(x), </J(x + t)).
4.3.6; This follows from Problem 4.3.2.
{b) 5*5=5(x).
4.2.1; g(x,y) = -x for 0 ~ x ~ y,g(x, y) = g(y,x). 4.3.7; M*y =!fit+ AT(t)y, with the vector ( ~~~~~ ) E N..L([L, R]).
4.2.2; U(x) =xis a solution of the homogeneous problem. There is no Green's 4.3.8; (a) The operator is formally self-adjoint, but not self-adjoint.
function.
(b) Require u'(O) = u'(l) = 0, for example.
4.2.3; g (x, y ) = cosa(;-lx-v\)
111n f prov1'ded a .,...
..J..
2mr.
4.3.9; Require J:1r f(x) sinxdx =a, and J:1r f(x) cosxdx = {3.
for 0 $ x < y ~ 1
-(1- y)2x
4.2.4; g(x, Y) = { (2 _ y)yx- y for 0 ~ y < x ~ 1 1
~.3.10; Require f 0 f(x)dx = -{3.
4.2.5; g(x,y) = lny for x < y, g(y,x) = g(x,y). 4.3.11; Require J0! f(x) sin 1rxdx = {3 + 1ra.
_x(3y5/2+2) for 0 ~ x < y 4.3.12; Require Jg f(x)dx =a- {3.
4.2.6; g(x, y) ={ 3tf( -a/ )
- 11 5 3x+2x 2 for x > y
4.4.1; g(x,y) = yx+~(y-x)+l2 -~(x2 +y2 ),for0 ~ x < y ~ l,g(x,y) = g(y,x).
4.2.7; u(x) = -£:. f~oo e-alx-eiJ(e)cJe.
4.2.8; The functions sin 2x and cos 2x are elements of the null space of the op-
4.4.2; g(x, y) = (~ + y2 (4y- 3))(x- ~) + t- ~-X: - xH(y- x).
erator. 4.4.3; g(x, y) = -~ cos21r(x- y)- ~sin 21r(x- y)- 4~ sin 21r(y- x).
4.2.9; u(x) = J; g(x, y)f(y)dy-).. J01 g(x, y)u(y)dy+a(1-x)+{3x, where g(x, y) = 4.4.4; g(x, y) = ~ cosxsiny+~ sinxcosy- ~ sinxsin y-2H(y-x) sinxcos y-
x(y- 1) for 0 ~ x<y ~ 1, g(x, y) = g(y, x). 2H(x - y) cos x sin y.
4.2.10; u(x) = fg g(x,y)f(y)dy-)..jg g(x,y)u(y)dywhereg(x,y) = i(x+1)(y-2) 4.4.5; g(x,y) = ~xy- x- -T(x2 + y2) for x < y,g(x,y) = g(y,x).
for 0 ~ x < y, and g(x, y) = g(y, x).
4.4.6; g(x, y) = ~ ln(l-x)+ ~ ln(l +y)+ ~for -1 ::; x < y S 1, g(x, y) = g(y, x).
1
J;
4.2.11; u(x) = f 0 g(x, y)f(y)dy- ).. g(x, y)u(y)dt where g(x, y) = 2~ xn(yn -
y-n) for 0 ~ x < y ~ 1,g(x,y) =g(y,x). 4.4.7; u(x) = l cos2x + ({3- a)~;+ ax-~~ (a+~).
578 APPENDIX A. SELECTED HINTS AND SOLUTIONS 579
4.4.8; u(x) = -~xcosx + cosx + f2 sin3x- /; sinx. 5.1.9; The Euler-Lagrange equations are y" = z, z" = y.
4.4.9; u(x) = 0.
5.1.10; The Euler-Lagrange equation is ~ - y = 0.
4.5.1; u(x) = aiJ3 - f +ax + 0;a x 2 + E:.:
1 n~:-2 cos mrx, where
J:
bn = -2 (f(x) +a - /3) cos mrxdx.
5.1.12; y(x) = 2sink1rx.
4.5.3; Use Fourier series on [0, 1]. 5.2.2; Uzz = 0 and f..l.tUzzz- f..1.2Uz = 0 at X= 0, 1.
4.5.4; No eigenfunction expansion solution exists. 5.2.3; H u is the vertical displacement of the string, then T = J~ pu~dx + !
4.5.5; u(x) =ax+ /3- a1r + E:., muHO, t) + mu~{l, t), U = ~ J~( .;(1 + u!)- 1)dx + fu 2(0, t) + ~u 2 (l, t).
1 Cln cos(2n- 1)~, Then, require puu = f..l./z ~ subject to the boundary conditions
where yl+u;:
_ 8 1-coe(:ln-1)!
Cln - ll'(:ln 1)2 (:ln-1)2/:1-1 • mUtt + ku = f..I.--J!.&...;
yl+u;:
at x = 0, and mUtt + ku = -f..I.--J!.&...; at x = l.
yl+u;:
4.5.6; u(x) = -~(Jx:l- !) - !x. Solution is exact if a+! = 0. 5.2.4; puu = f..I.Uzz on 0 <X< 4and!< X< 1, and mUtt= f..I.Uz(~,t)l;~~~.
4.5.7; u(x) = -~(x) + (b + 4c)L1(x), where L 1(x) = 1- x, and ~(x) =
i(x:l - 4x + 2) are Laguerre polynomials. 5.2.5; Use that T = !ml202 + To:~z 2 sin2 8, U = mgl(1 -cos 8).
5.1.1; (a) (~ )' = 0. 5.3.2; (a) u(x) = a(1+ ~~~~x+ 2:1~l x2 + ~::!x ) = a(l.O+ 1.013x+0.4255x2 +
9 3
3
0.2797:~: ).
(b) fl-y=- cosx.
(c) y' -y = -ez. (b) u(x) = a(U:i; + :~jWx + ~:~~~x:l + ::6~7 :~: 3 ) = a(0.99997 + 1.013:~: +
0.4255x2 + 0.2797x ).
5.1.2; y(x) = l{x:l- ax+ 1).
W O<x<l
5.1.4; Find the increase in energy due to rotation and determine how this affects 5.3.3; u' = { Ui' 1 < <I .
199' 2 X
the moving velocity. Then determine how this affects the fastest path.
Show that the rotational kinetic energy of a rolling bead is 2/5 that of the 5.3.4; (a) u(x) = 1- x -112x(1- x) = 1- x- 0.230x(1- x).
• T = Jo
1'ts t ranslat'1onal kine t'1c energy. Minimize' J
7 (l+V' 2 > -~-
rzt iog(IIO-II) u;c;.
(b) u(x) = 1-x-f2x(1-x) = 1-x-0.227x(1-x). The exact solution
is u(x) _ elnh(z-1)
1 1 - alnhm •
5.1.6; Maximize f y(x)dx subject to f0
0 vfl + y12 dx =l-a-b, y(O) = a,y(1) =
b. 5.3.5; Using (non-orthogonal) basis functions rp 1 (x) = x, r/J:I(x) = x 2, r/Ja(x) = x 3,
u(x) = a(l + ;r/Jt(x) + /&r/J:I(x) + /forp3(x)). The Sobolev inner product
5.1. 7; The arc of a circle. does not give a useful answer.
581
580 APPENDIX A. SELECTED HINTS AND SOLUTIONS
5.4.1; Use the functional D(¢) = J01 p(x)¢'2 (x)dx + ap(1)¢2 (1), a 2:: 0, subject (c) iz- il < 2, iz- il > 2.
to H(¢) = J 0
1
w(x)¢ 2 (x)dx = 1 and (d) iz- il < 1,1 < iz- il < v'2, iz- il > J2.
(e) iz- il < 1, 1 < iz- il < v!z, v'2 < iz- il < 2, iz- il > v!z.
(a) ¢(0) = ¢(1) = 0,
(b) ¢(0) = 0, 6.2.6; ~zl=l/2 z21z~l dz = 0.
(c) ¢(0) = 0, and a= 0. 6.2.7; ~zl=l/ exp[z 2 ln(1
2 + z)]dz = 0 (There is a branch point at z = -1).
Show that >.(a) 2:: >.(b)~ >.(c).
6.2.8; ~zl=l/ 2 arcsin zdz = 0 (There are branch points at z = ±1).
5.4.2; Minimize J0 (x¢' 2 (x) + ~ ¢ 2 (x))dx, subject to J; x¢(x)dx = 1, with
1
/
(c) No such values exist. 6.3.7; The upper half.; plane.
6.1.4; ii = e-(n/ 2+ 2n?r)
for all integer n; ln(l + i}i1r = -7r 2 (t + 2n) + i; ln2; 6.3.8; Flow around a corner with angle(}= {31r. This makes sense only for (3 < 2.
arctanh 1 has no value.
. F ·p - 2 U2 . 2 ia
6 •3 •9 , x - t y - - 1r p a sm ae .
6.1.6; It is not true. For example, consider z 1 = e1ril 2 , z2 = e31ri/ 2 •
6.3.11; f(z) = UJZ2+ a 2 •
6.1.7; The two regions are lzl < 1 and izi > 1; There are branch points at
w = ±1. 6.3.12; Show that 1; = iJ (e-wf 2k + vfl + e-w/k ).
6 • 2 • 1 '. f( Z ) -- 4(z-2)2(z-!)'
15-8i oo dx 271'
6.4.1; J-oo ax2+bx+c = -/4ac-bl'
roo ln(1+111 2) .J- 7.1.3; Use improper eigenfunctions to show (L 4 - A)- 1 is unbounded. Show that
6 •4 •17; Jo , ...L-2 u;.G = wIn 2• {Xn} with Xn = sin n9 is an improper eigenfunction.
6.4.18; /~00 co:: a:dz = '8
2
2
• 7.1.4; Show that f/>(x) = sinpz is ·an eigenfunction for all p. Notice that the
operator is not self-adjoint.
6.4.20; Evaluate f0 P~(z) on some contour C that contains all the roots.
7.2.1; (a) o(x-~)=2:E~=lsin( 2 n2-l1Tx)sin( 2 n2-l11'~).
6.4.22; The change of variables p2 = r 2 cos2 9+z2 sin2 9 converts this to an integral
{b) o(x- ~) = ~ fo coskxcosk~dk.
00
6.4.23; f0 00 1 2
=.[fir exp(iifi[r).
t- 1 e'"'dt
7.2.6; (a) ~2~42·
6.4.24; J:O tl/2ei"tdt = vrre::!' •.
(b) l!i~.
6.4.27; f~oo (..~:~)1! dz = 2e0111 e-wi/J/'J sin ~IPI/3-t r(1- /3)H( -p). - :1
(c) .j!e=if-.
7.2.7; H(p+1r) -H(p-1r).
6.4.28; Use that V'z+lat!v'z+id = Yif{fa:'fJf"', but be careful to apply Jordan's
1emma correct1y. Then, J_oo00 v'z+at+OI+Vl - ~e f"", -
../i -iw/4e- 110 -jJ
01
-e-""H( P)• 7 • 2 • 8·' ..!.. f 00
211" -oo
-
1
-(iuF(u))e-'":cdu-
l+iiS ,.. ,.. ,.. - J:c roo e:c-•J'(s)ds '
= fo~,.. 6(1-4 + asin 8)d8 = Ja;-p2 if II' I < lal and = 0 otherwise. 7.5.7; The gene~al solution is u(z) = aeikz(k 2 + 1 + 3iktanhx- 3tanh~ x) +
(b) Take the inverse Fourier transform of the answer to part (a). ,Be-ikz(k2 + 1- 3iktanhx- 3tanh 2 x). Tr(k) = ~!Z+~H!Z+!~. The bound
states are ¢1(x) = tanhx sechx fork= i 1 and ¢J2(x) = sech2x fork= 2i.
7.2.15; Show that Ne(1) = 1/120, N 6 (2) = 13/60, and N 8 (3) = 11/20. Then
Es( z ) -- wZ
_L ~
eoZ + 20
+ 1l 11 13 +
+ GoZ 1
120.: 2 '
(d) M = 1: , .
(b) u.n (Y) = aalnh(Df{u-11)) ~~~sinh~!: (e)t~e+ aalnh ~
nwalnh li£1 0 a n mrslnh • 11
t
sinh(!!!!:(e-
a
b))F(e)de 1 where Fn({) = !!!!:(!({) - ( -1)ng({)). In the special
(e) M[eilll] = i'r(8). a a
8.1.12; If f(O) = l::': 0 ancosn(8- <Pn), then u(r,O) = l::'=oan(~)ncosn(O 8.2.7; With appropriately scaled space and time variables, ~-~=sin¢>.
= 2~ Io ,. f(O)dO+ #2:::': 1 (~)n Itr f(<P)cosn(O- </J)d</J.
2
<Pn) This infinite
sum can be summed by converting the cosine to complex exponentials and 8.2.8; u(x, t) = L::'::0 (an sin cyt + bn cos¥) sin T where
using geometric series.
(a) bn = n~~2 sin~,., an = 0,
8.1.13; u(r, 8) = -~ Iooo F(J.L) sina~nlb<~:a~ sin(J.t In fi)dJ.L, where
F(J.L) = fo00
J(Re-t)sinj.ttdt.
(b) an = ni~a sin n211', bn = 0,
(c) bn = 9 ~~2 sin~~, an= 0,
8.1.14; u(r,8) = 2:::'=-ooan (ft)7 sin n:o, where an=~ j ° f(O)sin n:11 dO. (d) an=- c!~~2 (cos n511' -cos~~), bn = 0.
0
1
8.2.13; Construct the Green's function from H~ ) (>.jr - el) with A = ~, and
n_ ~-rlnleinll where a = .1.. f 2,. g(O)e-inll df)
00
8 ' 1 •21·' u(r' 0) = ""'
L..,n=-oo ...... ci+lnli3 ' n 2,. Jo • then the solution is proportional to (up to a scalar constant) 1/J(r, 0) =
1 ei>.r sin(>.asln 9) for large r.
8.1.22; -v(~')
_ 8u
= Inn . v~G(x, e)J(x)dVx - Ian n . veG(x, e)v(x)dSx, where 7r >.ainiJ
V - lfn·
8 ' 3 ' 1 .1 C8V _
8t -
1 82 V
R1
V
8z'I" - R2 .
8.1.24; Eigenfunctions are <Pnm(x, y) =sin n:x sin m; 11 , with eigenvalues Anm =
2(n~ m~)
-7r ~+v. 8.3.3; G(x, t; ~, r) = 2L:;:'=1(1- e-n 11'~(t-r)) cosn1rxcosn1r~, fort> r.
2
~
8.1.25; Eigenfunctions are Jn(J.'n!cfi)sinnfJ for n > 0. Thus, eigenvalues are the
same as for the full circle, with n = 0 excluded.
8.3.4; G(x, t) = 2J;te-tf-at.
8.1.26; Eigenfunctions are <Pnmk = Jn(J.'nkfi)einll sin m;z with eigenvalues A = 8.3.5; G(r, t) = (47rt)-nl2 e-r 2
/
4 t, where r = lxl.
I'~ m~
-(1ft-+ (iT). -a. 00 -a.J72 :z::2
8.3.7; T(x, t) =To+ ax+ (Tt- To)7 f_ 00 ~2+1 dTJ, where a= m·
8.1.27; Eigenfunctions are <Pnm(</J,fJ) = P~(cosfJ)sinn</J (or cosn</J) with Anm =
bm(m+l). 2
8.3.8·, X = ln 2 y-w
{ijj = 0.S2ro, t = Inw = 3.47 X 106 s = 40 days.
589
588 APPENDIX A. SELECTED HINTS AND SOLUTIONS
8.3.9; t = =
At~ In ~ 5.5 hours. At this time the temperature at the bottom
(b) 2Jt = e-Un-1 _ e-Un+l,
of the cup is 116° F. This seems like a long time. What might be wrong 9.2.3; qxt = sinh q.
with the model?
9.3.2; The bound state has J.L = -4· There is a single soliton with amplitude
8.3.10; (a) Require ut =
Di"iJ2u on regions i =
1,2 subject to the conditions !A2 and speed c = 4J.L 2 = A2 .
that u and n · kiVu be continuous at the interface.
9.3.3; Use that r(x) = 6e-x+Bt + 12e- 2x±64 t and find a solution of the form
(b) In a spherical domain, eigenfunctions are
(9.18) with~= x- 4t, 'f1 = 2x- 32t.
lain~
cp(r) = { /. r-h'
0 < r < rp
9.3.4; One soliton will emerge.
ram , rp < r < R
9 • 3 • 5 .' !l£u. i !ku - 0•
dt = - 2k cu' dt -
The tempera! behavior is cp(r)e-1h and the values I" must satisfy the 9.4.3; Verify (9.20) and (9.21).
transcendental equation ~tan~ = --jt; tan e~>.
9.4.4; (a) a= 1 ~~ 2 z- 2i, (3 = 1- a, R = ~·
(c) /-'2 j::::j 6.6 X 10-4 fa.
= _.! (4a5-l~f4a5zl-1) (3 _ 1 zl(16a~-4a5+1)-4afi Th t
(b) a 4 a5z z~-1) ' - 4 a5(zL1) . ere are wo
8.3.11; Solve the transcendental equation Jh:ie-~-ot =(J fort. The velocity is 2 4a
2
rootsof(3(z)=Oatz = ._, ?~ ...
h
t'
8.3.12; Propagation occurs provided h3 (J < /(l) = 0.076, where (c) a= _ ~~
2B (3 = zl±2Bz-1
z2-1 ·
f(y) =
(41ry)- 312 e- 1 / 4 v. The propagation velocity is v where /( lg.) = 9.4.5; anWn = a~z) (z- ~).
h 0, and the minimal velocity is i~·
3
9.4.6; With q0 > 0, set a 0 = !e-qo/2 , a 1 = !eqo/2 , and then use Problem 9.4.4b
8.3.13;
to show that two solitons moving in opposite directions emerge.
u{:c, t) = ei<.lt (1 + iw: ia:-y)eio(L-z) - (1 + iw _ ia:-y)eio(z-L)
9.4.7; Set b1 = -~ =F 0 and then use Problem 9.4.4c to show that one soliton
(1 + IW + ia:-y)eioL _ (1 + iw _ ia-y)e-iaL ,
emerges.
where a2 = 1 + iw. 2 2
9.4.8; Choose spring constants kn with 4k; = 1 + sinb wsech nw.
8.3.14; p(:c, t) = e"-'t P(:c), where P(:c) = exp( -q(w):c), q:l(w) = k+~""'.
10.2.1; En(x) = r(:) L~o(-1)krtltf>.
8.4.1; Un(t) = exp( -e•lniw/k) ) 2) sin(.ar). H we set n and h = we= llf, l, r 1 (cosm + t2)eixtdt = (-i.X
10•2 ' 2·' JO + ~2 + ~
2i )eix _ 2i + i(1 + eix) '\'oo (2!:)2k.
~ L.Jk=O
have in the limit k -+ oo, u(:c,t) = exp(-!jf.)sin(ar), which is the
X X
-
correct solution of the continuous heat equation with periodic initial data. 10.2.3; I; eixtt-lf2dt = Iooo eixtrtf2dt- It eixtt-ll2dt
8.4.2; Un(t) = Jn(-f). _ (;i + ei"' '\'oo (-
1)k-1 r(k±l/2)
- Vx 7/r L.Jk=O xk+t ·
8.4.5; Un(t) = J2n(f). 10.2.4; C(x) = ! A - P(x) cosx 2 + Q(x)sinx 2 , S(x) = h/I- P(x) sinx 2 -
8.4.6; k(w) =cos- 1 (1- ~~~~h 2
). Q(x)cosx 2 , where P(x) = ~ (~- ~ + · · ·),
Q( x) = 12 (lx - ~ 16x + .. ·) .
4x + ~
8.4. 7; For stability, require -2 < 6t.\ < 0, for all eigenvalues ,\ of A.
10.3.1; E1(x) =e-xl:~ 0 (-1)k Jb.
8.4.8; For stability, require that all eigenvalues of A be negative.
roo e-•t )k (4k)l
9.1.3; q(:c) = -2H(:c)sech2 :c. (See Problem 7.5.13.) 10.3.2; Jo l+t' dt = '\'00 (
L.Jk=O -1 z'41'+T.
10.3.6; f0 00
e:ctt-tclt = y'21jiell (1 - 2 11 ! -
6 ~3112 + · · ·), where 71 = ez- 1. 11.1.3; X1 = 1- E + 3E2 - 12c:3 + 0(E4 ), X2,3 = ±~ -! ± ~iE + !e2 + 0(~).
10.3. 7; I: F sin 1rtdt 2
= ~- 1 2
~~ + (50 - 81r2);; + .. ·. 11.1.4; ¢ + i.,P = U(z- a:)+ iEa(i) 20 + 0(E2).
"00
rff ezt2t-1/3 cos tdt -- 12 L.tlt=O <ct~;
10 • 3 • 8.1 Jo 2
r(lt+tfJ>
z'+t a •
11.1.5; ¢ + i.,P = z + ic:e1z + O(e).
11.1.6; u(x,y) = a(1-y)+by+E(a+(b-a)y)~ -w(x,y)+O(c:2), where V2v = 0
8
10.3.11; (a) I:~=O ( ~ ) kin-" =nIt en(ln(l+z)-z)dx ,.., ~for large n. 2 bz2
and Vz(O,y) = O,vz(1,y) = a/2 + (b- a)y/2, v(x, 0} = a; , v(x, 1) = T·
Use separation of variables to find v.
(b) 2:~=0 ( ~ ) kl..\" = Iooo e-z(1 + ..\x)ndx,.., (n..\)ne-n+! ex:1 >fii· 11.1.8; u(r,O} = rcosO + c:(£{1- r 4 ) + -fi(r2 - r 4 ) cos28) + O(e).
10.3.13; Iooo flee-t Intdt = v'21rxz+te-z lnx( 2zb - 2..;m + ·· ·). 11.1.9; u(r,O) = rcosO- !c:r2 sin29 + O(e2).
oo wdq,...,
-•'1 2
1 3
11.3.3; Bifurcation points occur at solutions of ~(1-/:4) = n 21r2 and there are
10.4.10; (a) f -co v. fl[
Q"(l- ~ + ici2' •• ·), for large a. a finite number of such solutions. The nth solution exists if ~ > n 21r2.
10.5.1; I: f(x)e'"'(:c)dx = f(a)r(~)("'<~ca) )2/3eiltg(a). 11.3.4; u(z, y) = c:sinn1rxsin m;v
+ 0(E2), ,\ = -(n2 + !m2)1r2 - E2a + 0(E4 ). !
6
10.5.4; Set u(x, t) = e-d/2w(x, t) so that Wtt = W:cz + ~w. 11.3.5; u = e¢ + O(c:4 ), ..\ = .\o - ~a4 f::: if a 4 f:. 0 whereas u = c:¢ + O(c:6 ),
11.1.1; X1 = -0.010101, X2,3 = -0.49495 ± 0.86315i. ,\ = Ao - c:4 a5 f<P:dz . a4 = 0, a5 f:. 0.
t/J dz if
= =
11.1.2; x1 0.98979. Since x 0 is a double root of x 3 - x 2 = 0, the implicit 11.3.6; The exact solution is ¢ = 1 + ..\a where a = (1 + .\a) 2. A plot of the
function theorem does not apply for r.oots near x = 0. amplitude a as a function of ,\ is shown in Fig. A.5.
592 APPENDIX A. SELECTED HINTS AND SOLUTIONS
593
. . 10
11.5.1; There are two Hopf bifurcation points, at A= ! ± ~-
11.5.3; Steady state solutions have s = >., x = (>. + a)(l- >.). A Hopf bifurcation
occurs at >. = !(a+ 1).
-o.• -o.2 0.0 0.2 o.•
),
11.5.4; Suppose '1/Jo satisfies AT '1/Jo = 0, and 'l/;1 satisfies AT '1/Jt = -iA'l/; 1 . Then a
small periodic solution takes the form x = e(a¢o + bei>.t¢1 + be-i>.t¢)1 ) +
Figure A.5: Plot of ifJ as a function of A for Problem 11.3.6.
O(e 2) provided 2lbi 2(Q(¢1, ¢}1), '1/Jo) + a(B¢o, '1/Jo) + a 2 (Q(¢o, ¢o), '1/Jo) = 0,
where 2a(Q(¢o, ¢1), 'l/;1) = -(B¢1, 'l/;1).
11.3.7; {a) u(x) = Aa(x), where A satisfies the quadratic equation
12.1.1; u(t, E)= a(1 + ~) cos((l- ~t- E)+ O(e 3).
A2 fol a3(y)dy- ~ +! a(y)dy = 0. fol
12.1.2; u(t) = A( Et) sin( (1 + e2w2)t) + eB( d) cos( (1 + e2w2)t) + 0( e2) where At =
(b) u(x) = Aa(x), where 1 = Asin(A) J; a (y)dy.
2
!A(1- ~A2 ), Bt = !B(1- ~') + {!,(A2 - 4)(6A 2 -1), and w2 = -l
6•
"" a.o 12.1.3; The solution is u(t) = eA(r) sin(t) + O(e 2 ) where AT= !A(1J(r)- ~A 2 ),
and f.L = E21J, T = d.
2.11
15
2.0
-.: 10
~ ...
1.0
c 12.1.4; (a) The Landau equation is AT= -iA 3
, with solution A(r) = ,; 3 ]t~,,,
o.o where A(O) = Ao, so that u(t) = V1+AfA oft sin(t + ¢o) + O(e).
4
01
1 I I I i I :p=:, I 0.0
o.o o.z 0.4 o.e o.a
),
1.0 1..2 1.4 1.1 ... .. -:z 0
),
(b) The Landau equation is AT = - 3~ A 2 , with solution A( r) = , , 1r_Ap~_
3
f~oo g(x)uti(x)dx = 0. behavior predicted by the leading order equation Rt = -\'-R3 is incorrect,
because it predicts algebraic rather than exponential, decay.
11.4.7; (a) Use the implicit function theorem to show that u = _l+fw~ coswt +
O(e2 ). 2.1.11; The exact solution is u(x) = aJ0 ( ~eax) + bYo( ~eax). The approximate
solution is (set E = %and employ adiabatic invariance, or approximate
(b) Require a= ~(ifJ) = 3
~wsech w21r sinifJ. the Bessel functions for large arguments) is u(x) "'e-ax/ 2 sin( ~eax + ¢).
595
594 APPENDIX A. SELECTED HINTS AND SOLUTIONS
12.1.12; Write the equation (12.16) as the system u 111 = r(~)v, Vz = J(x, ~),and (b) u(x) = 4::~:-:_13 - 2tanh(z;t + tanh- 1 (i))
then make the exact change of variables u = Uo + EW(~)z, v = z + EZ, u(x) = H(x- i> 51:+h + t(l- H(x- i)) ia:-:!3 - ~ tanh(lf(x -!))
1
(c)
where~= r(u)-r, and Z = J; J:
J(y,u)du-u f(y,u)du, with u = ~· where H(x) is the usual Heaviside function.
12.1.13; Set v = z + Eh(~)u where h' = g- g, and then !!if = z + eh(~)u, ~ = 12.3.8; u(x) = a+~-l + dn(cosh( 2z-<;!.B-l)) + O(e).
-gu- Eh(~)z -·f2h(~) 2 u.
L2.3.11; (a) T"' 2ln 2 ~a.
12.1.15; In dimensionless time, rc has units of length- 2 •
A good dimensionless
parameter is E = ~, r = r c + r m/ h. The function W is a piecewise linear (b) T(v) = -r! 1 ln !~ !+-r :, where c(v) = v':~v!i·
"sawtooth" function with slope - roh?frm •
12.1.16; Write the equation as the system u' = v,v' = u- (1 + g'(~))u8 + Eav.
Then the transformation v = z + Eg( ~) transforms this into a system
to which Melnikov's method can be applied. The requirement is that
=
af~oo U/?(t)clt J~00 9'(!.¥)U8(t)Ub(t)clt, where Uo(t) v'2sech t. =
12.2.2; The device is a hysteretic switch. There are two stable steady outputs,
Vo = V~ if V<V~ and Vo = VR_ if V_ > VR-.
12.3.3; u(t) = -tan-1 (t) + E113 exp( 2 ;ih) (l sin( 2~a)- cos(~))
1 1
+ i1 El/322/3 exp(2 / (t-l))
,_1/1
12.3.4; For all {3 there is a solution with u(x) ""'-1+ a boundary layer correction
=
at x 1. For {3 > 0.2753 (the real root of 3x 3 + 12x2 + llx- 4 0}, there =
is a solution with u(x) ""'2+ a boundary layer correction at x = 1.
·12.3.5; The solutions are u(x) = ;h - tanh( 111; [1 ) + O(E) and u(x) = ~ -
tanh( 111 2?) + O(E) with 1/J. and fb appropriately chosen.
12.3.6; The two solutions are u(x) = ::;:~ + 2e-z/e + O(E}, and u(x} = ::~ -
fe-
111
/'- + O(E).
596
598 INDEX INDEX 599
improper, 296 Fresnel integrals, 464, 466 heat equation, 338 Lebesgue, 63
nodal curves, 361 Frobenius norm, 41 Heaviside distribution, 139 line, 217
eigenpair, 11, 161 Fubini's theorem, 64 Heisenberg uncertainty principle, 334 Riemann, 63, 217
eigenvalue, 11, 161, 199, 283 function spaces, 60 Helmholtz equation, 360 interpolation, 78, 397
eigenvector, 11 functional, 108, 177 Hermite functions, 89 invariant manifold, 14
elastic membrane, 194 fundamental theorem of algebra, 276 Hermite polynomials, 50, 73, 169, inverse integral operator, 110
elastic rod, 189 fundamental theorem of calculus, 218 298 inviscid fluid, 228
elastica equation, 193 Hermitian matrix, 14 irrotational flow, 229
elementary reflector, 34 Gabor transform, 301 Hilbert space, 65, 283 isolated singularities, 226
elementary row operations, 30 Galerkin approximation, 92, 124, 131, Hilbert transform, 355 isoperimetric problem, 181
entire function, 226 176, 197 Hilbert-Schmidt kernel, 103, 112 isospectral flow, 417, 418
enzyme kinetics, 523 Galerkin method, 349 Hilbert-Schmidt operator, 112, 161 iterative solution technique, 121
equivalent matrices, 10 Galerkin projections, 399 hodograph transformation, 245
essential singularity, 227 gamma function, 259, 441 Hooke's constant, 189, 319 Jacobi iterates, 45
Euclidean inner product, 4 gauge function, 438 Hooke's law, 368 Jacobi polynomials, 73, 168
Euclidean norm, 5 Gauss-Seidel iterates, 46 Hopf bifurcation, 494, 496, 510 jet nozzle, 243, 246
Euler column, 193 Gaussian elimination, 30 Householder transformation, 34 Jordan's lemma, 253, 291, 292, 353,
Euler identity, 209 Gegenbauer polynomials, 73, 168 413, 450, 461
Euler-Lagrange equation, 178 Gel 'fand-Levitan-Marchenko ( GLM) ill-conditioned matrices, 40 Josephson junction, 406
exponential integral, 440 equation, 413, 415, 433 implicit function theorem, 470, 475, Joukowski transformation, 242, 311
generalized Green's function, 159 513 jump condition, 147, 159
fast Fourier transform, 77 generalized Laguerre functions, 168, improper eigenfunctions, 296
Fermat's problem, 179 281 improper eigenvalues, 296 Kaplan matching principle, 530, 541
Fick's law, 380 generating function indicia! equation, 263 kinetic energy, 186, 188, 204
filter, 78, 296 for Bessel functions, 266 induced norm, 6 Kirchhoff's laws, 365, 485, 523
finite element space, 88 for Legendre polynomials, 269, inequality Korteweg-deVries {KdV) equation,
finite elements, 88 280 Bessel, 68 418
FitzHugh-Nagumo equations, 503 geometric multiplicity, 11, 52, 476 Bunyakowsky, 6 Kronecker delta, 3
formally self-adjoint, 153 Gram-Schmidt procedure, 8, 34, 39, Cauchy, 6 Kutta condition, 242
Fourier coefficients, 68, 136, 162 65, 68, 114 Schwarz, 6
Fourier convergence theorem, 74 Green's formula, 338 Sobolev, 66 .L'Hopital's rule, 251, 264
Fourier cosine integral transform, 290 Green's functions, 146, 284, 339, 383 triangle, 5, 60 lady bugs, 366
Fourier integral transform, 290 generalized, 159 influence function (see also Green's Lagrange multiplier, 18, 180, 198
Fourier series, 68, 74, 164, 303 modified, 159, 175 function), 146 Laguerre polynomials, 50, 73, 168,
Fourier sine integral transform, 289 Green's theorem, 217 inner product, 4, 65, 108 298
Fourier transform theorem, 290 inner product space, 4 Landau equation, 509, 593
Fourier's law of cooling, 380 Haar functions, 83 integral equations Langrangian, 187
Frechet derivative, 178 Hamilton's equations, 188 Abel, 103 Laplace transform, 307, 331
Fredholm alternative, 24, 110, 156, Hamilton's principle, 186 Fredholm, 101, 102 Laplace's equation, 223, 229, 337
476, 493, 507, 514, 516 Hamiltonian, 187 Hammerstein, 500 Laplace's method, 442
Fredholm integral equation Hammerstein integral equations, 500 singular, 103 Laplacian operator, 337
of the first kind, 101 Hankel functions Volterra, 101 lattice, 22, 310, 420
of the second kind, 102 of the first kind, 265, 407 integration Laurent series, 225, 267
free boundary problems, 243 of the second kind, 265 by parts, 217, 338 least squares problem, 26
Frenet frame, 489 Hankel transform, 309, 352 complex, 217 least squares pseudo-inverse, 28
Frenet-Serret equations, 489 Hardy-Weinberg proportions, 527 contour, 220, 291, 353 least squares solution, 26
600 INDEX INDEX 601
Lebesgue dominated convergence the- moment of inertia, 53 Hermite, 50, 73, 169, 298 reflectionless potential, 323
orem, 64, 94, 134 Moore-Penrose least squares solution, Jacobi, 168 region of influence, 369
Lebesgue integration, 63, 94 28 Laguerre, 50, 73, 168, 298 Reisz representation theorem, 138
Legendre function Moore-Penrose pseudo-inverse, 28 Legendre, 9, 50, 72, 167 relaxation oscillations, 545
of the first kind, 269 mother wavelet, 80 orthogonality, 7 removable singularity, 226
of the second kind, 269 multigrid method, 4 7 renewal processes, 102
Legendre polynomials, 9, 50, 72, 167 multiplicity Parseval's equality, 69, 113, 271, 293 residue, 225
Legendre's equation, 262, 268 algebraic, 43, 52, 476 path independence, 219 residue theorem, 225, 285, 414
Lewis number, 540 geometric, 52, 476 pendulum, 148, 188, 510 resolvent kernel, 120, 128
lift, 232 multiscale method, 509 perihelion of Mercury, 183, 483 resolvent operator, 110, 118
linear combination, 2 periodic boundary conditions, 152 resolvent set, 283
linear functional, 108, 137 natural basis, 3 Perron Frobenius theorem, 44 resonant frequencies, 24
bounded, 108 natural modes, 23 Phragmen-Lindelof theorem, 222 Riemann mapping theorem, 237
linear manifold, 14, 107 natural norm, 6 pitchfork bifurcation, 482 Riesz representation theorejll, 108
linear operator, 105 near identity transformation, 512 Plancherel's equation, 293 Rodrigue's formula, 268
linear vector space, 2 Neumann boundary conditions, 338 Poisson equation, 337, 392 root cellar, 388
linearly dependent, 2 Neumann iterates, 122 Poisson kernel, 223 rotating top, 206
linearly independent, 2 Newton's law, 233 pole of order n, 226 rotational matrix, 12
Liouville's theorem, 221 Newton's method, 131 population dynamics, 101, 472
Lipschitz continuity, 136, 164 nodal curves, 361 populations genetics, 526 saddle point, 449
logistic equation, 471 normal equation, 27 positive definite saddle point method, 450
LU decomposition, 30, 32 normal modes, 23 matrices, 18 scalar product, 4
lung tissue, 409 normed vector space, 5 operator, 161, 338 scattering transform, 411
Lyapunov-Schmidt method, 482 norms,5,59 positive matrices, 44 SchrOdinger equation, 169, 313, 411
Euclidean, 5 potatoes, 386 nonlinear, 420
Maple, 48, 50, 453, 455, 492 Frobenius, 41 potential energy, 187-189, 194 SchrOdinger operator, 312
Mathematica, 48 induced, 6 potential function, 228 Schwarz inequality, 6
Mathieu's equation, 502, 552 £ 2 ,60 power method, 43 Schwarzschild metric, 182
Matlab, 48 LP,60 precompact, 111 secular terms, 506, 532
maximum modulus theorem, 222 IJ', 59 principal branch, 211 self-adjoint, 13, 153
maximum principle, 17, 115, 117 natural, 6 problem of Procrustes, 41 separable kernel, 112
mean value theorems, 222 Sobolev, 66 projection, 8, 98, 119, 124, 347, 397 separated boundary conditions, 147,
Mellin transform, 308, 354 supremum, 5, 60 pseudo-inverse, 28 152
Melnikov function, 494 uniform, 60 pseudo-resolvent operator, 120 separation of variables, 345
method of averaging, 511 null space, 106 Pythagorean theorem, 7 sequence spaces, 59
method of images, 343, 370 sequentially compact, 111
method of lines, 391 operational amplifier (op-amp}, 521 QR decomposition, 33 set of measure zero, 63, 94, 134, 163
Meyer functions, 304 orthogonal complement, 15, 108 quadratic form, 17, 186 shallow water waves, 421
Michaelis-Menten law, 525 orthogonal functions quasi-steady state solution, 523, 525 Shannon sampling function, 78
microwave ovens, 24 associated Legendre, 167 similar matrices, 10
midline strain, 191 Haar, 83 range, 14 similarity solution, 384
minimax principle, 19, 199 Laguerre, 168 range of an operator, 106 similarity transformation, 10
minimum principle, 199 orthogonal matrices, 33 ranking of teams, 44 similarity variable, 384
modified Bessel function, 265, 266 orthogonal polynomials, 9, 69 Rayleigh-Ritz technique, 92, 197 sine function, 78, 270, 302
modified Green's function, 159 Chebyshev, 50, 73, 168, 281 recurrence formulas, 266 sine integral transform, 289
modulus of continuity, 71 Gegenbauer, 73, 168 reflection coefficient, 316 sine-Gordon equation, 419
602 INDEX INDEX 603