APPLIED FUNCTIONAL ANALYSIS

Numerical Methods, Wavelet Methods, and Image Processing

ABUL HASAN SIDDIQI
Kmg Fahd University of Petroleum & Minerals Dhahran. Saudi Arabia

MARCEL

~
DEKKER

MARCEL DEKKER, INC.

NEW YORK BASEL

Although great care has been taken to provide accurate and current information, neither the author(s) nor the publisher, nor anyone else associated with this publication, shall be liable for any loss, damage, or liability directly or indirectly ca~ or alleged to be caused by this book. The material contained herein is not intended to provide specific advice or rccomlllcndatiorL'i for any specific situation.

Trademark notice: Product or corporate names may be trademarks or registcre<l trademarks and are used only for identification and explanation without intent to infringe.
Library of Congress Calaloging-in-Publication Data A catalog record forthis book is available from the Library ofCongres!>. ISBN: 0-8247-4ocn-1

This book is printed on acid-free paper.

lleadquaners Marcel Dekker, Inc., 270 Madison Avenue. New York. NY 10016, U.S.A. tel: 212-696-9000; fax: 212-685-4540 Distribution and Customer Service Marcel Dekker, Inc., Cimarron Rood, Monticello, New York 12701, U.SA tel: 800-228-1160; fax: 845-796-1772 Eastern Hemisphere Dislribudon Marcel Dekker AG, Hutgassc 4, Postfaeh 812. 0-1-4001 Basel. Swi17..crland tel: 41-61-260-6300; fax: 41-61-260-6333 Wor1d Wide Web
http://www.dekker.com

The publisher offers discounts on this book when ordered ill bulk qUllntitics. For more information, write to Special SalcslProfcssional Marketing at the headquarters address above
Copyright 2004 by Man:e1 Dekker, Inc. All Rights Resen'ed.

Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechaniad, including photocopying, microfilming, and recording, or by any information storage and retrieval system, \vithout pennission in writing from the publisher. Current printing (last digit): 1098765432
PRINTED IN THE UNITED Sf ATES OF AMERICA

PURE AND APPLIED MATHEMATICS

A Program of Monographs, Textbooks, and Lecture Notes

EXECUTIVE EDITORS

Earl 1. Taft
Rutgers University New Brunswick. New Jersey

Zuhair Nashed
University ofCentral Florida Orlando, Florida

EDITORIAL BOARD

M. S. Baouendi
University ofCalifornia. San Diego

Anil Nerode Cornell University

Jane Cronin Rutgers University

JackK. Hale Georgia Institute ofTechnology

Donald Passman University of Wisconsin, Madison

Fred S. Roberts

Rutgers University
David L. Russell

S. Kobayashi
University ofCalifornia. Berkeley

Virginia Polytechnic Institute

and State University Walter Schempp

ManJin Marcus University ofCalifornia, Santa Barbara

Universitiit Siegen
Mark Teply University of Wisconsin.

w. S. Massey Yale University

Milwaukee

MONOGRAPHS AND TEXTBOOKS IN PURE AND APPLIED MATHEMATICS

1. K. Yano, Integral Formulas in Riemannian Geometry (1970) 2. S. Kobayashi, Hyperbolic Manifolds and Holomorphic Mappings (1970) 3. V. S. Vladimirov, Equations of Mathematical Physics (A. Jeffrey, ed.: A. Uttlewood, lrans.) (1970) 4. B. N. Pshenichnyi, Necessary Conditioos for an Extremum (L. Neustadt, translation ed.: K. Makowski, lrans.) (1971) 5. L. Narid el al.. Functional Analysis and Valuation Theory (1971) 6. S. S. Passman, Infinite Group Rirgs (1971) 7. L. Domhoff, Group Representation Theory. Part A: Ordinary Representation Theory. Part B: Modular Representation Theory (1971,1972) 8. W. Boothby and G. L. Weiss, eds., Symmetric Spaces (1972) 9. Y. Malsushima, Differentiable ManWolds (E. T. Kobayashi, trans.) (1972) 10. L. E. Ward, Jr., Topology (1972) 11. A. Babakhanian, Cohomological Methods in Group Theory (1972) 12. R. Gilmer. Multiplicative Ideal Theory (1972) 13. J. Yeh, Stochastic Processes and the Wiener Integral (1973) 14 J. Banos-Nefo, Introduction 10 the Theory of Distributions (1973) 15. R. Larsen, Functional Analysis (1973) 16. K. Yanoand S. Ishihara, Tangenl and Cotargent Bundles (1973) 17. C. Proces;, Rings with Polynomial Identities (1973) 18. R. Hennann, Geometry, Physics, and Systems (1973) 19. N. R. Wallach, Harmonic Analysis on Homogeneous Spaces (1973) 20. J. Dieudonne, Introduction 10 the Theory of Formal Groups (1973) 21. I. Vaisman, Cohomology and Differential Forms (1973) 22. B.- Y. Chen, Geometry of Submanifolds (1973) 23. M. Marcus, Finite Dimensional Multilinear Algebra (in two parts) (1973,1975) 24. R. Larsen, Banach Algebras (1973) 25. R. O. Kujala and A. L. Vltter, eds., Value Distribution Theory: Part A: Part B: Defial and Bezout Estimates by Wilhelm Stoll (1973) 26. K. B. SloIal'Sky, Algebraic Numbers and Diophantine Approximation (1974) 27. A. R. Magid, The separable Galois Theory of Commutative Rings (1974) 28. B. R. McDonald, Finite Rings with Identity (1974) 29. J. Salake, Unear Algebra (S. Koh el al.. trans.) (1975) 30. J. S. Golan, Localization of Noncommutative Rings (1975) 31. G. Klambauer. Mathematical Analysis (1975) 32. M. K. Agoston, Algebraic Topology (1976) 33. K. R. Goodearl, Ring Theory (1976) 34. L. E. Mansfield, Unear Algebra with Geometric Applications (1976) 35. N. J. PuNman, Matrix Theory and Its Applications (1976) 36. B. R. McDonald, Geometric Algebra Over Local Rings (1976) 37. C. W. Groetsch, Generalized Inverses of Unear Operators (19n) 36. J. E. Kuczkowski and J. L. Gersting, Abslract Algebra (1977) 39. C. O. Christenson and W. L. Voxman, Aspects of Topology (1977) 40. M. Nagafa, Field Theory (1977) 41. R. L. Long, Algebraic Number Theory (1977) 42. W. F. Pfeffer, Integrals and Measures (1977) 43. R. L. Wheeden and A. Zygmund, Measure and Inte9ral (1977) 44. J. H. Curliss, Introduction to Functions of a Complex Variable (1978) 45. K. Hrbacek and T. Jech, Introduction to Set Theory (1978) 46. W. S. Massey, Homology and Cohomology Theory (1978) 47. M. Marcus, Introduction to Modem Algebra (1978) 48. E. C. Young, Vector and Tensor Analysis (1978) 49. S. B. Nadler, Jr., Hyperspaces of Sets (1978) SO. S. K. segal, Topics in Group Kings (1978) 51. A. C. M. van Rooij, Non-Archimedean Functional Analysis (1978) 52. L. Cotwin and R. Szczarba, Calculus in Vector Spaces (1979) 53. C. Sadosky, Interpolation of Operators and Singular Integrals (1979) 54. J. Cronin, Differential Equations (1980) 55. C. W. Groefsoh, Elements of Applicable Functional Analysis (1980)

56. I. Vaisman, Foundations of Three-Dimensional Eudidean Geometry (1980)

57. H. I. Freedan, Deterministic Mathematical Models in Population Ecology (1980)

58. 59. 60. 61. 62. 63. 64. 65. 66.

67.
68. 69. 70.
71.

72.
73.

74.
75.

76.
77. 78. 79. 80. 81.
82.

83.
84. 85. 86. 87. 88. 89. 90. 91. 92. 93. 94. 95. 98. 97. 98. 99.
100. 101.

102.
103.

104.
105.

106.
107. 108. 109.

110. 111. 112.

S. B. Chae, Lebesgue Integration (1980) C. S. Rees el al., Theory and Applications of Fourier Analysis (1981) L. Nachbin, Introduction to Functional Analysis (R. M. Aron, trans.) (1981) G. Orzech and M. Orzech, Plane Algebraic Curves (1981) R. Johnsonbaugh and E. Pfaffenberger, Foundations of Mathematical Analysis (1981 ) W. L Voxman and R. H. Goelschel, Advanced calculus (1981) L J. Corwin and R. H. Szczarba, Multivariable Calculus (1982) V. I. Istr~escu, Introduction to Linear Operator Theory (1981) R. D. JalVinen, Finite and Infinite Dimensional Unear Spaces (1981) J. K Boom and P. E. Ehrlich, Global Lorentzian Geometry (1981) D. L Annacost, The Structure of Locally Compact Abelian Groups (1981) J. W. Brewer and M. K Smith, eds.. Emmy Noether: A Tribule (1981) K H. Kim, Boolean Mallix Theory and Applications (1982) T. W. W",ling, The Mathematical Theory 01 Chromatic Plane Ornamenls (1982) D. B.Gauld, Differential Topology (1982) R. L Faber, Foundations of Eudidean and Non-Euclidean Geornelry (1983) M. carme/i, Statistical Theory and Random Mallices (1983) J. H. calTOlh ef al., The Theory 01 Topological Semi9roups (1983) R. L. Faber, Differentiat Geomelry and Relativity Theory (1983) S. Barnett, Polynomials and Unear Conlrol Systems (1983) G. Katpilovsky, Commutative Group Algebras (1983) F. Van Oystaeyen and A. Versc!loren, Relative Invariants of Rings (1983) I. Vaisman, A Firsl Course in Differential Geomelry (1984) G. W. Swan, Applications 01 Optimal Control Theory in Biomedicine (1984) T. Pelrie and J. D. Randall, Transformation Groups on Manifolds (1984) K. Goebel and S. Reich, Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings (1984) T. Albu and C. Nastasescu, Relative Finiteness in Module Theory (1984) K. HrlJacek and T. Jech, Introduction to Set Theory: Second Edition (1984) F. Van Oyslseyen and A. Verschoren, Relative Invarianls 01 Rings (1984) B. R. McDonald, Unear Algebra Over Commutative Rings (1984) M. Namba, Geometry of Projective Algebraic Curves (1984) G. F. Webb, Theory 01 NcnIinear Age-Dependent Population Dynamics (1985) M. R. Bremner et a/., Tables of Dominanl Weisj11 MultiPlicities for ReDresentations of Simple Ue Algebras (1985) A. E. Fekele, Real Unear Algebra (1985) S. B. Chae, Holomotphy and calculus in Ncrmed Spaces (1985) A. J. Jerri, Introduction to Integral Equations with Applications (1985) G. Katpilovsky, Projective Representations of Finite Groups (1985) L. Narid and E. Beckenstein, Topological Vector Spaces (1985) J. Weeks, The Shape 01 Space (1985) P. R. Gribik and K. O. Kortanek, Extremal Methods of Operations Research (1985) J.-A. Chao and W. A. Woyczynski, eds., Probability Theory and Harmonic AnalysiS (1986) G. D. Crown et aI., Abstract Algebra (1986) J. H. carruth ef aI., The Theory 01 Topological Semigroups, Volume 2 (1986) R. S. Doran and V. A. Belli, Characterizations 01 CO-Algebras (1986) M. W. Jeter, Mathematical Programming (1986) M. Altman, A Unified Theory of Nonlinear Operator and Evolution Equations with Applications (1986) A. Verschoren, Relative Invarianls 01 S/leaves (1987) R. A. Usmani, Applied Linear Algebra (1987) P. Blass and J. Lang, zariski Surfaces and Differential Equations in Characteristic p > 0(1987) J. A. Reneke el al.. Structured Hereditary Systems (1987) H. Busemann and B. B. Phadke, Spaces with Distinguished Geodesics (1987) R. Harle, Invertibility and Singularity for Bounded Unear Operators (1986) G. S. LOOele el al., Oscillation Theory of Differential Equations with Deviating Argumenls (1987) L. Dudkin et al., Iterative Aggregation Theory (1987) T. Okubo, Differential Geomelry(I987)

vv.

<:<.,.,tt"lJGolty"""'c.>J.;~I>IFo,tt>_

113. D. L. Stancl and M. L. Stancl, Real Analysis with Point-SetTopoiogy (1987) T. C. Garr!, Introductioo to Stochastic Differential Equatioos (1988) S. S. Abhyankar, Enumerative Combinatorics of Young Tableaux (1988) H. Strade and R. Farnsleinar, Modular Lie Algebras and Their Representations (1988) J. A. Huckaba, Commutative Rings with Zero Divisors (1988) W. D. Wallis, Combinatorial Designs (1988) W. Wil'slaw, Topological Fields (1988) G. Karpilovsky, Field Theory (1988) S. Gaenepeel and F. Van Clystaeyen, Brauer Groups and the Cohomology d Graded Rin9S (1989) 122. W. Kozlowski, ModUlar Functioo Spaces (1988) 123. E. Lowen-Colebunders, Function Classes of Cauchy Continuous Maps (1989) 124. M. Pavel, Fundamentals of Pattem Recognition (1989) 125. V. Lakshmikantham el al., Stability Analysis of Nonlinear Systems (1989) 126. R. Sivaramakrishnan, The Classical Theory of Arithmetic Functioos (1989) 127. N. A. Watson, Parabolic Equations 00 an Infinile Strip (1989) 128. K J. Hastings, Introductioo 10 the Mathematics of Operatioos Research (1989) 129. B. Fine, Algebraic Theory 01 the Bianchi Groups (1989) 130. D. N. Dikranjan et aI., Topological Groups (1989) 131. J. C. Morgan II, Poinl Sel Theory (1990) 132. P. Biler and A. Wilkowski, Problems in Mathematical Analysis (1990) 133. H. J. Sussmann, NoolinearControilability and Optimal Control (1990) 134. J.-P. Florens el a/., Elemenls of Bayesian Statistics (1990) 135. N. Shell, Topological Fields and Near Valuatioos (1990) 136. B. F. Doolin and C. F. Mattin, Introductioo to Differential Geometry lor Engineers (1990) 137. S. S. Holland, Jr., Applied Analysis by the Hilbert Space Method (1990) 138. J. Dkninski, Semigroup Algebras (1990) 139. K Zhu, Operator Theory in Functioo Spaces (1990) 140. G. B. Price, An Introductioo 10 Multicomplex Spaces and Functions (1991) 141. R. B. Darst, Introductioo to Linear Programming (1991) 142. P. L. sachdev, Nonlinear Ordinary Differential Equatioos and Their AppIicatioos (1991) 143. T. Husain, Orthogonal Schauder Bases (1991) 144. J. Foran, Fundamentals of Real Analysis (1991) 145. W. C. Brown, Matrices and Vector Spaces (1991) 146. M. M. Rao and Z. D. Ren, Theory of Orticz Spaces (1991) 147. J. S. Golan and T. Head, Modules and the Structures of Rings (1991) 148. C. SmaR, Arithmetic of Finite Fields (1991) 149. K Yang, Complex Algebraic Geometry (1991) 150. D. G. Hoffman el a/., Coding Theory (1991) 151. M. O. Gonzalez. Classical Complex Analysis (1992) 152. M. O. Gonzalez. Complex Analysis (1992) 153. L. W. Baggeff, Functiooal Analysis (1992) 154. M. Sniedovich, Dynamic Programming (1992) 155. R. P. Agarwal, Difference Equations and Inequalities (1992) 156. C. Brezinski, Biorthogooality and lis Applications 10 Numerical Analysis (1992) 157. C. Swartz, An Introduction 10 Functiooal Analysis (1992) 158. S. B. Nadler, Jr., Cootinuum Theory (1992) 159. M. A. AI-Gwaiz, Theory of Distributions (1992) 160. E. Peny, Geometry: Axiomatic Developmenls with Problem SoIvin9 (1992) 161. E. Castillo and M. R. Ruiz-Cobo. Functional Equations and Modelling in Science and Engineering (1992) 162. A. J. Jerri, Integral and Discrete Translorms with Applications and Error Analysis (1992) 163. A. Chartieret a/., Tensors and the Cliffool Algebra (1992) 164. P. Biler and T. Nadzieja, Problems and Examples in Differential Equations (1992) 165. E. Hansen, Global Optimizatioo Using Inlerval Analysis (1992) 166. S. Guerra-De/abriBre, Oassical Sequences in Banach Spaces (1992) 167. Y. C. Wong, Introductory Theory ofTopoiogical Vector Spaces (1992) 168. S. H. Kulkarni and B. V. Limaye, Real Functioo Algebras (1992) 169. W. C. Brown. Matrices Over Commutative Rings (1993) 170. J. Lous/au and M. Dillon. Linear Geometry with Computer Graphics (1993) 171. W. V. PetJyshyn, Approximation-Solvability of Nonlinear Functional and Differential Equatioos (1993)

114. 115. 116. 117. 118. 119. 120. 121.

E. C. Young, Vector and Tensor Analysis: Second Edition (1993) T. A. Bick, Elementary Boundary Value Problems (1993) M. Pavel, Fundamentals of Pattern Recognition: Second Edition (1993) S. A. AJbeverio et aI., Noncommutative Distributions (1993) W FulkS, Complex Variables (1993) 1n. M. M. Rao, Conditional Measures and Applications (1993) 178. A. Janicki and A. Weron, Simulation and Chaotic Behavior of -Stable Stochastic

172. 173. 174. 175. 176.

Processes (1994)

179. P. Neittaanmaki and D. Tiba. Optimal Control of Nonlinear Parabolic Systems (1994) 180. J. Cronin, Differential Equations: Introduction and Qualitative Theory, Second Edition
(1994)

181. S. Heikkila and V. Lakshmikantham. Monotone Iterative Ted"lniques for Discontinuous Nonlinear Differential Equations (1994) 182. X. Mao, Exponential Stability of Stochastic Differential Equations (1994) 183 B. S. Thomson, Symmetric Properties of Real Functions (1994) 184. J. E. Rubio. Optimization and Nonstandard Analysis (1994) 185. J. L. Bueso et al., Compatibility, Stability, and Sheaves (1995) 186. A. N. Michel and K. Wang, Qualitative Theory of Dynamical Systems (1995) 187 M. R Darnel, Theory of Laltice-Ordered Groups (1995) 188. Z. Naniewicz and P. D. PanagiotopouJos. Mathematical Theory of Hemivariational Inequalities and Applications (1995) 189. L. J. Corwin and R. H. Szczerba, Calculus in Vector Spaces: Second Edition (1995) 190. L. H. Erbe et al., Oscillation Theory tor Functional Differential Equations (1995) 191. S. Agaian et aI.. Binary Polynomial Transforms and Nonlinear Digital Filters (1995) 192. M. I. Gil', Norm Estimations for Operation-Valued Functions and Applications (1995) 193. P. A. Grillet, Semigroups: An Introduction 10 the Structure Theory (1995) 194. S. Kichenassamy, Nonlinear Wave Equations (1996) 195. V. F. Krotov, Global Methods in Optimal Control Theory (1996) 196. K. I. Beidaret aI., Rings with Generalized Identities (1996) 197. V. I. Amautov et al., Introduction 10 the Theory of Topological Rings and Modules (1996) 198 G. Sierksma, Unear and Integer Programming (1996) 199. R. Lasser, Introduction fo Fourier Series (1996) 200. V. Sima, Algorithms for Linear-Quadratic Optimization (1996) 201. D. Redmond, Number Theory (1998) 202. J. K. Seem et aI., Global Lorentzian Geometry: second Edition (1996) 203. M. Fontana et al., Prtifer Domains (1997) 204. H. Tanabe, Functional Analytic Methods tor Partial Differential Equations (1997) 205. C. Q. Zhang, Integer Flows and Cyde Covers of Graphs (1997) 206. E. Spiegel and C. J. O'Donnell, Inddence Algebras (1997) 207. B. Jakubczyk and W. Respondek, Geometry of Feedback and Optimal Conlrol (1998) 208. T. W. Haynes et aI.. Fundamentals of Domination in Graphs (1998) 209. T. W Haynes et aI., eds., Domination in Graphs: Advanced Topics (1998) 210. L A. D'Alotto et al., A Unified Signal Algebra Approach to Two-Dimensional Parallel 211. 212 213. 214
Digital Signal Processing (1998) F. HaUer-Koch, Ideal Syslems (1998) N. K. Govil et aI., eds., Approximalion Theory (1998) R. Cross, Mullivalued Linear Operalers (1998) A. A. Martynyuk, Stability by Liapunov's Matrix Function Method with Applications

(1998) 215. A. Favini and A. Vagi, Degenerate Differential Equations in Banach Spaces (1999) 216. A. lIIanes and S. Nadler. Jr., Hyperspaces: Fundamentals and Recent Advances
(1999)

217. G. Keto end D. Struppa, Fundamentals of Algebraic Microlocal Analysis (1999) 218. G. x.-Z. Yuan, KKM Theory and Applications in Nonlinear Analysis (1999) 219. D. Motreanu and N. H. Pavel, Tangency, Flow Invaliance for Differential Equations,
and Optimization Problems (1999)

220. K. Hroacek and T. Jech, Introduction to Set Theory, Third Edition (1999) 221. G. E. Kolosov, Optimal Design of Control Systems (1999) N. L. Johnson, Subplane Covered Nels (2000) 223. B. Fine and G. Rosenberger, Algebraic Generalizations of Discrete Groups (1999) 224. M. Viith, Volierra and Integral Equations of Vector Functions (2000) 225. S. S. Mfller and P. T. Mocanu, Differential Subordinations (2000)

=.

......

226. R. U et al., Generalized Difference Methods for Differential Equations: Numerical

Analysis of Finite Volume Methods (2000) 227. H. Li and F. Van Dystaeyen, A Primer of Algebraic Geometry (2000) 228. R. P. Agarwal. Difference Equations and Inequalities: Theory, Methods, and Applications. Second Edition (2000) 229. A. B. Kharazishvili. Strange Functions in Real Analysis (2000) 230. J. M. Awe" at al., Partiallnlegral Operalors and Inlegro-Differential Equations (2000) 231. A. I. Prilepko ef at., Melhods for SolVing Inverse Problems in Mathematical Physics (2000) 232. F. Van Oystaeyen. Algebraic Geometry for Associative Algebras (2000) 233. D. L. Jagennan, Difference Equations with Applications 10 Queues (2000) 234. D. R. Hankerson et al. Coding Theory and Cryplography: The Essentials. Second Edition, ReVised and Expanded (2000) 235. S. DasCillescu et at.. Hopf Algebras: An Inlroduction (2001) 236. R. Hagen et al.. C -Algebras and Numerical Analysis (2001) 237. Y. Talpaelt, Differential Geometry: With Applications to Mechanics and Physics (2001) 236. R. H. Villarreal. Monomial Algebras (2001) 239. A. N. Michel ef al., Qualitative Theory of Dynamical Systems: Second Edition (2001) 240. A. A. Samarskii. The Theory of Difference Schemes (2001) 241. J. Knopfmacher and W-a. Zhang. Number Theory Arising from Finite Fields (2001) 242. S. Leader, The Kurzwei~HenstoekIntegral and lis Differentials (2001) 243. M. Blliotti at at. Foundations of Translation Planes (2001)

244. A. N. Kochubei, Pseudo-Differential Equations and Stoehastics over NQo.Archimedean

Fields (2001) 245. G. Sierl<sma. Linear and Integer Programming: Second Edition (2002) 246. A. A. Maltynyuk. Qualitative Methods in Nonlinear Dynamics: Novel Approacnes to Liapunov's Matrix Functions (2002) 247. B. G. Pachpatte. Inequalities for Finite Difference Equations (2002) 248. A. N. Michel and D. Liu, Qualitative Analysis and Synthesis of Recurrent Neural NelWOl1<s (2002) 249. J. R. Weeks. The Shape of Space: Second Edition (2002) 250. M. M. Rae and Z, D. Ren, Applications of Orlicz Spaces (2002) 251. V. Lakshmikantham and D. Trigiante, Theory of Difference Equations: Numerical Methods and Applications, Second Edition (2002) 252 T. Albu, Cogalois Theory (2003) 253. A. Be2dek, Discrete Geometry (2003) 254. M. J. Corless and A. E. Frazho. Linear Systems ana Control: An Operalor Perspective (2003)

255. J. Graham and G. Kohr, Geometric Functioo Theory in One and Hisj1er Dimensions
(2003) 256. G. V. Demidenko and S. II. Uspenskii, Parllal Differential Equations and Systems Not Solvable with Respect to the Highest-<Jrder Derivative (2003) 257. A. Kelarev, Graph Algebras and Aulomata (2003) 258. A. H. Siddiqi, Applied Functional Analysis: Numerical Melhods, Wavelel Methods, and Image Processing (2004)

259. F. W. steutel and K. van Ham, Infinile Divisibility of Probability Dislributions on Ihe
Real Line (2004)

260. G. S. Ladde and M. SBmbandham, Stochastic Versus Delerministic Systems of Differential Equations (2004) 261. B. J. Gardner and R. Wleganclt, Radical Theory of Rings (2004) 262. J. Haluka. The Mathematical Theory of Tone Syslems (2004)

Additional Volumes in Preparation

E. Hansen and G. W. Walster, Gk>bal Optimization Using lnlerval Analysis: Second

Edition. Revised and Expanded (2004)

C. Menini and F. Van Oystaeyen, The ERASMUS Guide to Algebra: From Zero 10
One (2004)

To my wife, Azra

fi.._,... .~ ..".. ."...,."-'

..... ,

Preface
FUnctional analysis was illvcnt.cd and developed in the t\\"cntleth century. I3csidcs being an area of independent mathematical interest, it pro\'ides many fundamental notions essential for modeling, analysis, numerical approximation and computer simulation processes of real-world problems. As science and tcdmology are increasingly refined and interconnected, the demand for advanced matlwffiatics beyond the basic vector algebra and differential and integral calculus has greatly increusp(1. Therp is no dispute on the relevance of functional analysis; hO\\'c\'cl", there have been differences of opinion among experts about the le\'(~l and methodology of teaching functional analysis. In the recent past, its applied nature has been gaining ground. The main objective of this book is to present all those results of functional analysis which ha\'e been fre<luently applied in emerging areas of science and technology. FUnctional analysis provides basic tools and foundation for flreas of \'ital import<lncp sHch flS optimization. boundary value probkfllS, modeling real-world phenomena, finite and boundary ekment methods, variational C(luations and inequalities, and wawJets. \Va\'Clets, formally invcnted in the mid-eighties, have found significant applications in image processing and pmtial differential equations. The book comprises 11 chapters, 6 apPf'.Jldices and a eomprehensh'f' updat.ed hibliography ehaptel"\vise. Chapter I is devoted to basic results of metric spaces, especially an important fixed point theorem eall(od the Banach contraction mapping theorem, and its applications to matrix, integral and diffcrential equations. Chapter 2 d(~als with basic definitions and examples rclated to Banach spaccs and operators defined on such spaces. Sufficient number of examples are presented to make the ideas clear. Algebras of operators and properties of convex functionals nrc discussed. I-lilbert space, an infinite-dimensional analogue of Euclidean space of finite dimension, is int;roduccd and discussed in detail in Chapter 3. [n addition, important results such as projection theorem, Iliesz representation

..

'fi,..,...,.._........."".","-'!

theorem, propertie::; of self-adjoint, positive, nonnal and unitary operator::;, relatiOlI::;hip between bounded linear operator and bounded biline<tr form and Lax-fvliIgram lemma dealillg with the existence of solutions of abstract variational problems are presented. Applicatsions alld gCllCralizations of the La...x-tvlilgram lemma are discus~J in Chapter:> 7 and 8. Chapter 4 is devoted to the Hahn-Banach t.heorem, Banach-Alaogue thcorem, uniform bounded principle, open mapping ami c!oseO graph theorems along with the cOilcept of weak convergence and weak topologies. Chapter 5 provides an extellsioll of finite-dimellsiolml clas:>ical calculus to infinite-dimensional spaces which is essent.ial to U1lderstl:Uld and illterpret various current developmellts of sciellce and techllOlogy. More precisely, derivatives in the sense of Gat.eau, rr'echet, Clarke (subgradiellt) and Schwartz (dist.ributiOllil.1 derivative) alOllg with Sobolev spaces are the main theule::; of this chapter. In Chapter 6, fundamelltal rCl;ults cUllcernillg existence and U11iqueness of solutions and algorithms for filldillg ::;olutions of optimizatioll problems are described. Variational formulation alld existence of solutions of boundary value problems represelltillg physical phellomena are described in Chapter 7. Galerkill and Ritz approximation methods are also included. 1n Chapter 8, finite element and boundary element methods are introduced and several theorems conceming errol' estimatioll and convergence are proved Chapter 9 is devoted to variational inequalities. A comprehensive account of this elegalll mathematif'.'l.l model in temlS of operators is givrm. Apart from existence and uniqueness of solut.iOlIS, eITor estimation and finite element met.hod::; for approximate solutiolls atld parallel algorithms are discusse<1. The chapter is maillly based on the work of one of its inventors, J.L. lions and his co-workers and research students. Activities at Stampacchia Sc!lool of Mathematics, Italy arc providing impetus t.o researchers ill t.his field. Chapter 10 deals with wavelets. Broadly, wavelet analysis b a refillement of Fourier analysis and has .'l.ttracted the atteutioll of researchers ill mathematics, physics and ellgineerillg alike. Replacemellt of the classical Fourier met.hods, wherever they have been applied, by emerging wavelet methods has resulted in drastic improvements. Ln this chapter, a detaile<1 accoUilt of this exciting theory is presented. Chapter I] presellts an intro duction to applications of wavelet methO(ls to part.ial differential e<juations alld image processing. These are emergillg areas of current int.erest. There is st.ill wide scope for further research. tvlodels and algorithms for removal of uJnvanted COlllpOilellt (noise) of a signal are discussed in detail Error estimation of a given image with its wavelf1t representation in the Besov Ilorm is given. In appendices, we present a resume of the results of topology, real analysis, calculus and Fourier analysis which we often use in this book. Chapters 9, 10 and 1] contain recent results opelling up avenues for further work.

.n", "'_'_"_"""10"-"

The book is self-cOiltaitle<1 and provides examples, updated refereJlces, and applications ill diverse fields. Several problems are thought-provoking and may lead to new results (illd applications. The book is illt.ended to be a textbook for graduate or senior undergraduate student.~ in mat.hematics. It could also be used for an advance course in systems engiJwering, computer engineering and mallagf~ment sciences. The proofs of thoorems and other itelll:' uJarked wit.h all asterisk lIlay be umitted for a b~niur undergraduat,e course or a course in other disciplines. Those who are mainly interested in applications of wavelets may study Chapters 2, 3,10 and 11. Readers interested ill variatiOilal illequalities and its applicatiOils may pursue Chapters 3, 8 and 9. 1.n brief, this book is a handy manual of contemporary aualytic and numeric meth()(ls in illfinite-dil1lensional spaces, particularly Hilbert spaces. The book has beell written under a buok-writing project No. tvlSjFulH.:tionalj207 of the King Fahd University of PptrolcUlIJ & Minerals (KFUPM), Dhahran, Saudi Arabia. I would like to express my profound appreciation to KFUPM for providing generous grant which elwbled me to complete the book. I have used a major part of the material presellted in the book while teaching at various universities of the world. I have also illCorJlorated in this book the ide..'lS emerging after discussion with some senior mathematicians including Professor I\1.Z. Nashe<l, Delaware l)niversity, Professor P.L. Butzer, Aacheu Technical University, Professor Jochim Zowe, Professor Michel K~''ara, Erlangen Ulliversity, and Professor Martin Brokate, Technical University, MUllich. [would also like to express my thanks to th(' Abdus Salam International Centre for Theoretical Physics, T!ieste, Italy, especially its Director Professor A. Virasoro, for extending hospitality atld the use of the excellent facilit.ies of the Centre during t.he preparation of the book. I take this opportunity to thank 01'. P. l'vlallchanda, Chairperson, GurunaIlak Dev University, Amritsar, India for reading the manuscript and for her valuable suggestions. Finally, I would likf' to express my sincere gratitude to ~1s r.'1aria Allegra, Acquisitions &litor and ~'1anager, tvlarcel Dekker Inc., for her help and cooperation.

Abul Hasan Siddiqi

B.

......

, to<l'._..

....,..",,"l1>"-I

Contents
Preface List of Figures 1

!\.fetric Spac~l> and Banach .Fixed Point Theorem t.1 Lntroduct.ion 1.2 BWJUch Contraeuoll Fixl?d Poillt Theorem 1.3 Applicatioll of Banach Contraction Mapping Theorem 1.3. t Application to Real-valued Equatioll 1.3.2 Application to Matrix Equatioll 1.3.3 Application to Integral Equation 1.3.-1 Application to Differential Equation 1 A Problems

References

Banach Spaces 2.1 Introductiol1 2.2 Defillitiotls aJld Examples of Normed and Banach Spaces 2.2.1 Examples of Normed and Banach Spaces 2.3 Bask Properties-Closure, Denseness and Separability 2.3.1 Closed, Dense and Separable Sets 2.3.2 lliesz Theorem 311d Construction of a New Banach Space 2.3.3 DimelisioIJ uf Nunued Spcu.:es 2.3.4 Open a1Jd Closed Spheres 2.4 BOUlJded and Ullboullded Operators 2.4.] Definitions and Examples 2.4.2 Properties of Lineal' Operators 2.4.3 Unbounded Operators 2.5 Represclltation of BOllnded and Linear FUlJctionals 2.6 A1b'bra of Operators

B.

..... ,

, toOl'._..

....,..",,"l1>"-I

2.7

2.8

CO!l\'CX f\UlctiolJals 2.7.1 Convex Sets 2.7.2 AffilJe Operator 2.7.3 Lower Semicontinuous and Upper SemiCOIltinuo\ls FilllCtionals Problems: 2.8.1 Solved Problems 2.8.2 Unsolved Problem:"

References
3

Hilbert. Space 3.1 Introduction 3.2 Basic Definition and Properties 3.2. t DefinitiolJs. Examplps and Properties of hlller Product. Space 3.2.2 Parallelogram Law and Char<.lcterizatioll of Hilbert Space 3.3 Orthogol18l Complemellts atld ProjectiOiI Theorem 3.3. t OrthogOl181 Complements and Projections 3.4 OrtbugollClI Projectiom; and Projection Theorem 3.5 Projection on Convex Sets 3.6 OrthOilormal Syst.ems aud Fourier Expansion 3.7 Duality alld Reflexivity 3.7.1 Rjesz Represelltat,ioll Theurem 3.7.2 Reflexivity of Hilbert Spac:l:S 3.8 Operators in Hilbert Space 3.8.1 Adjoint of Bounded LlIlear Operators on a Hilbert Space 3.8.2 Self-Adjoint, Positive, Normal and Unitary Operators 3.8.3 Adjoint of an Ullbounded Linear Operator 3.9 BilillCar Forms wId Lax-!vlilgl'am Lemma 3.9.1 Basic Properties 3.10 Problems 3.10.1 Solved Problems 3.10.2 UIl~lJlv~ Problems

References 4 Fundamental TheOl'ems 4.1 Introduction 4.2 Hahu-Banach Theorem

.n",

"'_'_""""'''''''10''-''

4.3
4.4

4.5 4.6

4.7

4.8

4.2.1 ExtellsiOlI Form of Balm-Banach Theorem 4.2.2 Extension Form of the Hahll-Banach Theorem Topologies Oil Norme<1 Spaces 4.3.1 Strong and We1lk Topologies Weak Convergence 4.4.1 Weak COilvergellce ill Banach Spaces 4.4.2 Weak Convergence in Hilhert Spaces Banach-Alaoglu Theorem Prilwiple of Ulliform BOUJl(ledIlB;S awl Us Applir:ations 4.6.1 Principle of Uniform Boundedness Open Mapping and Closed Graph Theorems 4.7.1 Graph of a Lillear Operator and Closedness Property 4.7.'1 Opell \flapping Theorem 4.7.3 rhe Closed-Graph Theorem Problems 4.8.1 Solved Problems 4.8.2 Unsolved Prohlems

References

:;

Differential and Integral Calculus in Banach Spaces 5.1 Intro<luctiOlI 5.2 The Gateaux a1ld Frcchet Derivatives 5.2.1 The Giiteaux Derivative 5.2.2 The Prechet Derivative 5.3 Geueralized Gradient (Subdifferential) 5.4 Sonw Basic Results from Distribution Theory and Sobolev Spaces 5.4.1 DistributiOlls 5.4.2 Sobolev Space 5.4.3 The Sobolev Embedding Theorems 5.5 bltegratioll in Banach Spaces 5.6 Problems 5.6.1 Solved Problems 5.7 Unsolved Problems

References

Optimization Problems 6.1 hltroductioll 6.2 General Results on Optimi".ation 6.3 Special Classes of OplimizaT.lon Problems

.n",

"'_'_""""'''''''10''-''

6.-1.

6.5

Convex, Quadratic and Linear Programming Calculus of Variations and Euler-Lagrange Equatioll Minimization of Energy FUllctional (Quadratic FUllCtional) Algorithmic Optimization 6.4.1 Newton Algorithm and Its Generalization 6.4.2 Conjugate Gradient Method Problems

6.3.1 6.3.2 6.3.3

References
7 Opel'atol' Equations and VaIjational Methods 7.1 ItltroductiOlJ 7.2 BOlllldary Value Problems 7.3 Operator Equation!'> and Solv:thility COIJditiolls 7.3. J Equivalence of Operator Equation anci !'..,1iulluization Problem 7.3.2 Solvdbility Conditions 7.3.3 Existence Theorem for Nonlinear Operator~ 7.4 Existence of Solutions of Dirichlet and NeumaJlIl Boundary Value Problem:. 7.5 Approximation Method for Operator EquatiOlJs 7.5.1 Galerkill tl'1ethod 7.5.2 Rayleigh-Ritz-Galcrkin tvlct.hod 7.6 Eigenvalue Problems 7.6.1 Eigell\'alue of Bilinear Form 7.6.2 Existence and Uniqueness 7.7 BoulJ(lary Value Problems ill ScielJce aJld TecllIlology 7.8 Problems

References 8 Finite Element and Boundar)' Element Methods 8.1 IntroductiOiI 8.2 Fillite Element Mct.hod 8.2.1 Abstract Prohlpm fIlJd Error EstimatiOiI 8.2.2 Imernal Approximation of [fl (0) 8.2.3 Finite Elemellts 8.3 Applications of the Fillite lI-Icthod in SolvlIlf!: Boundary Value Problems 8.4 Basic Ingredients of Boundar}" Element Method 8.4.1 Weighted Residuals l'vlethod

...... "'_'_""""'''''''10''-''

8.5

8.4.2 Inverse Problem and Boundary Solutions 8.4.3 BOIlIldary Element Method Problems

References 9 Variational Inequalities and Applications 9.1 Motivation and Historical Rem<trks 9. t.I COlltact Problem (Signorini Problem) 9.1.2 Variationallnequalitie;; in Social, Fillancial and ~"'an Rgement Sciences 9.2 VariatiOlJUI blequalities and Their Relationship with Other Problems 9.2.1 Classes of VariR.tionallnequalities 9.2.2 Formulation of a Few Problems ill Tenns of VariatiOllal Inequalitie1> 9.3 Elliptic Variational Inequalit.ies 9.3.1 Liolls-Stampacchia Theorem 9.3.2 Variational Inequalities for Monotone Operators 9.4 Finite Element Methods for VariatiOlmllnequalities 9.4. t Convergenclo: and Error Estimation 9.4.2 Error Estimation in Concrete Ca<;es 9.5 EvolutiOlI VariatiOlJUI bwqualities :wd Parallel Algorithms 9.5.1 Solution of Evolution Variational Inequalities 9.5.2 Decomposition tvlethod and Parallel Algorithms 9.6 Obstacle Problem 9.6.1 Obstacle Problem 9.6.2 rVlembrane Problem 9.7 Problems

References 10 Wavelet TheOl'Y 10.1 Introduction 10.2 COIltinuous alld Discrete \Vavclet Transforms 10.2.1 Continuous Wavelet Transforms 10.2.2 Discrete v..:avelet Transfoml and Wavelet Series 10.3 Mult.iresolution Analysis, Wavelets Decomposition and RecOllstruction 10.3.1 tvlultiresolution Analysis (MRA) 10.3.2 Decomposition and RecOIlstruction Algorithms 10.3.3 CUlIllectiull with Signal Processing
, to<l'._..

B.

......

....,..",,"l1>"-I

10.4

to.5

10.6 10.7

10.3.4 rhe Fast Wavelet Transform Algorithm Wavelets and Smoothness of Functions IOA.l Lipschitz Class <l.nd Wavelets 10.4.2 Approximation alld Detail Operators 10.4.3 Scalillg ami Wavelet Filt.ers 10.4.4 Approximation by tvLRA Associated Projections Compactly Supported Wavelets to.5.] Daubechies Wavelet.:> 10.5.2 ApproximatiOil by Family of Daubechies Wavelets Wavelet Packets Problems

References 11 Wavelet Method for Partial Differential Equations and Image Processing 11.1 Introduction ]].2 Wavelet rVlet-hods ill Pmti:tl Differential ami Int.egral Equations ] \.2.] Illtro<luetion 11.2.2 General Procedure 1] .2.3 t-.Hscellaneous Exampies t 1.2.4 Error Estimation Using Wavelet Basis ] 1.3 Intl'Oduct.ioll t.o Sigllal and Image Processlllg 11.4 Represent.ation of Siglwis by Frames 11.4.1 FullctiolWI Analytic Formulation 11.4.2 Iterative Reconstruction ] 1.5 Noise Removal from Signals 11.5.] Introduction ] 1.5.2 Model and Algorithm 11.6 Wavelet Met.hods for Image Processing t] .6. t Resav Spa(:e 11.6.2 Linear alld Nonlinear Image Compression 11.7 Problems References Appendices A Set Theoretic Concepts B Topological Concepts

.n",

'.'_'_""""'''''''10''-''

C Elements of Metric Spaces

Notations and Definitions of Connete Spaces

E Veclor Spaces F Fourier Analysis References Symbols and Abb"eviations

List of Figures
2.3.1 The geomctncal mealllllg of open Example 2.2.2(1) 2.3.2 The geometrical mealllllp; of open Example 2.2.2(2) 2.3.3 The geometrical meaning of open Example 2.2.2(3) 2.3.4 The geometriuJi mf!<'l.niug of open and dosed unit spheres ill and closed UIJit spheres of and closed unit spheres of and clOMd Iwit. spheres ill

CIO,11
2.4.] LCR circuit

3.3.1 OrtilOgOlJaI complClltetlt ill Example 3.3.1 3.4.1 Coometncal meanlllg of the orthogonal decompositloll in R 2 3.5.1 Geomet.rical ilJterpretatioll of the dwracterizatiOiI of projection 5.4.1 Graph of fl(X), h(x), h(x) of Example 5.4.11 5.6.1 Geometrical explanation ill Problem 5.6.5 8.3.1 Basis fUllctiOiI in Example 8.3.1 8.4.1 COilsta.llt boundary elemellts 8.4.2 Linear bOlllldary elements 9.2.1 Rigid punch 9.3.1 Geollletrical interpretation of characterization of projection ill terms of variational inequality 9.6.1 Obstacle problem 9.6.2 EquilihriuIIl of elastic strillg 10.2.1 Haar wavelet 10.2.2 rl/lexican llat wavelet , 10.2.3 Convolution of Haar \vavelet <lIld !p(x) = c-:t 10.3.1 Schematic representation of the decompositiOll algorithm 10.3.2 Schematic represelltation of the reconstruction algorithm

B.

......

, toOl'._..

....,..",,"l1>"-I

La.5.1 Scaling functions (-) and wavelet functions (... ) (Top: N = 2, middle N = 3, bottom N = 4)
11.5.1 Noise suppn.c'SSiUll processing mudel

Q.

.... ,

, toOl'._..

....,..",,"l1>"-I

Chapter 1

Nletric Spaces and Banach Fixed Point Theorem

1.1 Introduction

Tile maill objective of this chapter is to prcscnt the DamICh coutrflctlon fixed point theorem and its applications to the existence of solutions of matrix, integral and differential equatioll&.

1.2

Banach Contraction FixAd Point TheorAm

Sf'1.

Definition 1.2.1 r~t X!w (l non-pmpty on X x X mto the set of real numbers R

and d a function defiuff]

d(.-),XxX-tR
satisf,\'ing tile following (;()llditiollS: (i) d(x,y) = 0 if and only if;r; = y,

(ii) d(x,y) = d(y,.J;) for nIl X,Y E X, and

(iii) d(x,Y)

:s d(:/;, z)

-I-

d(z, y) for all

X,

y, z EX.

The number d(;, y) is called the distance between x and y, d is called tile metric and the pair (X. d) is called a met7'ic !SlJace.

Remark 1.2.1 [t can be verified tilat dC,') is a nou-negative function, that is, d(y, z) ? 0 for any y, z E .r: For arbitrary x, y,;; in .'(, we have

d(r.y)

< d(x,z) + d("y).

",,,. "'_'_""""'''''''10''-''

Taking

,I;

= V in tltis relation, we get

d(x.x) <d(y,z)+d(z,y)=2d(y,z).

Since d(x,x) = 0, we obtain

d(y,::) > 0 for all y,:; E X.

A subset V of a metric space (X,d) is itself a metric space with a metric d l (',') defined bv the relation
Remark 1.2.2

dJ(x,y)=d(x,y)

forx,1IEY

Examples of Metric Spar,p,s

Example 1.2. 1 (a) (I?, d) is a metric space with respect to the metric defilled by d(x,y) = Ix - yl for all x,y E R. (b) (fl, d) is a metric space witll respect to the metric defined by d(x, V) = ma... (lxi, IuD for all x, y E R, . Example 1.2.2

(n 2,d) is a metric space with respect w tbe metncs

.1:

(a) d(x,V) = ((.1l - X2)2 + (YI - Yz)zf/z: R'2.

= (x],yI), Y = (J'z,Yz)

Examplf': 1.2..'/

(R",d) b u metric space with respect tu the lUetrics

a.nd

(bl d(x,y)

~xlx,

- y,l;

(Xl.XZ."

.J;,,)

fl"

and

Y=(YI,YZ. '.y,,) E' R".

Examl)le 1.2.4

Let t p,l < P < ex: denote the set of all sequences of

real numbers {x.} such that.

L Ix. II' <

=,

00

00.

Then f p is a metnc space with

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

respect to tile metric

d(x,y) =

= :?= lx, - yil") 'I, (

1=1

for all x = (:CI, :CZ,' .. , :C",' .. ), and y = (YI> liz,' .. , y,,,'" ) E f1"
Example 1.2.5 Let. bl denote the set of all real-\~dlued continuous functions defined on la, b]. Tilen b] is a metric space witil respect to tile metrics

era,

era,

(a) d(1,g) = 'up If(x) - g(x)l,

a<z<b

f,g E C[a,b].

(b) <1(1, g)

~ ( llf(x) -

g(x)I' dx

) 'I'

f, 9 E Cia, bJ.

Example 1.2.6 Let Lz[a,b] denote the set of all integrable functions 1 2 defined on la, bl sucil tilat 111 is also integrable on la, b]. Then Lz [a, bl is a metric space witil respect to the metric

d(1,g)

llf(x) - g(x)I' dx

),"

f,g E L,la,b).

A sequence of points {x,,} in a metric space X is called a Cauchy lJequence if for any c > 0, there e>"';sts an integer N sucil tilat

Definition 1.2.2

d(:cm,x,,)<t'

foralln,m>N.

It may be obser\'t~d that a sequence in a metric space is a function whose domain is the set of natural numbers and wilose range is a subset of the metric space X. Broadly speaking, tile concept of Cauchy sequence means that tile distance between x" and X m is very small when nand m are very large.

A sequence {X.,} in a metric space is called convergent to tile limit x if, for (' > 0, there exists a natural number N sucil that d(x",x) < {' for all n > N. In this case we write lim x" = x.

Definition 1.2.3

Q . ,>0<1"_""""'''''''IO''-I

......

" ..... 00

Definition 1.2.4 A metric space X is called a complete metric space if its ever.... Caucby sequence is convergent in it..

Examples of complete metric spaces

Example 1.2.7 (a) n, n2 , n" , (p, C [n, bJ with sup metric and .;, In. bj aJ'e examples of cumJllet~ metric SP<lces. (b) (0, I] is 1I0t a completp metric space. (c) C [o,b] with integral metric is not a l:omplete metric space. (d) Tbe set of rational numbers is not a complete metric space. Definition 1.2.5 (a) A subs~t. A of a metric space is called bounded if there is a positive constant AI such that d(x,y) S At for all x,y E A.. (IJ) A subset A of a IIIetric space b called a dosed set if every con\'~l'gent sequence in A bus its limit ill A. (c) A subset. A of a m~tric space is called compoct if every bounded sequence bas a convergent subsequence. (d) A mapping T from one metric space mto ullother met.ric spuce is called continuous if x" -} :r implies that T(:c u ) --+ Tx; that is, lim d(x",:c) = 0 implies tllal lim d(T(x,,).Tx) = O.

,,-10=

.,-4=

Every bounderl and c10sen subset of R" is cumpact.

Remark 1.2.3 It can be easily checked that every dosed subset of a complete met.ric sp:u'P if> eomplet.e. Obsc1'Vution 1.2.1 \Ve obf>erVf' t.bat elements of a metric space are pumts bm., in real life, we normally consider distances between points aud sets and between sets and sets, e.g., the distancf> hetweelJ a point and a linp in the plane I.md tbe distance between a square and a circle in tbe same plalle. This consideration motiwl.tes tbe study of Hausdorff metric

Hausdorff rVletric
Let X bp a met.ric spaCt' with thp metric d and let H(X) delJot.e the set of all subsets of X For any two non-empty elements A. ami B of H(X), we define the distulJce between a point x in A and B by the relation
d(:r,B) = int' {d(:J:,y)/y E B}. (1.2.1)

",,,. "'_'_""""'''''''10''-''

Very often, it is denoted by illL fl(x,?J).

,cD

The Jist.unce bet.ween two elements uf H(X) is defined by tilt> relation

dCA, B) =

:,IUp

inf d(x,y)
(1.2.2)

xEA yEO

= supd(x, B).

'oA
It
CUll

be shown by examples that

d(A, lJ)

oF d(B, A),

(1.2.3)

where

d(8,A) = sup inf d(y,x)

yEn xEA

= sup int' d(x,y).

yEfl XEA

(1.2.4)

Definition 1.2.6 Tile HmL.~dmJJ mctf"ie or the dist.ance bp.twell two elements A and B of If(S), denoted by h(A, B), is defined by the relatiun

h{A, lJ)
Remark 1.2.4

~ max

{d( A. B), d( B, ,1)) .

(U.5)

if H (X) denotes tile set of all closed and bounded subsets of a metric space X, tben h(.4., B) is a metric. In particular, if 11 (n?) is tile set of all dosed and bounded subsets (compuct subsets) of It2, then h(A, B) is a metric.

Contraction Mapping or Transformation

Definition 1.2.7 (ContmcLion Mapping). A mapping T defined on a metric space (X,d) into it.self (T; X -) X) is calle.I a LilJSchit2 continuous mopping if there is a positi\'e real constant 0 such that d(Tx, Ty) < o{/( x, y) for all :r, y EX. If 0 < a < I, then T is called a co7ltmdiorl nWIJpirig. 0' is called the contmctivity factm> of T.
Example 1.2.8 Let T ; R ---J. R ancl Tx = (I + X)I/3. ThClI finding a solution tu the equatiolJ T:r = x is \.-"C]uivalent to solving the equation x 3 - x - I = o. The mapping T is a contraction mapping 011 A = [I, 2J, where the contractivity factor is rx = (3)1/3 -1.

Q.

.... ,

, to<l'._..

....,..",."l1>"-I

Example 1.2.9 (a) Let T:c = ~,T,J~ E [0,1]. Tben T is a contraction mapping on [0, 1] with cont.ractivity function 0: = ~, (b) Let T(x) = J; + lI,:.r E Ii alld a be any fixed element of R. Then T is not. a contraction mapping. Example 1.2.10 Let X = H, [a, b] eX, 1 : [0, b] -4 [(I, b] and let l' exist and sup [1'(x)[ < 1. Then 1 is a contraction mapping on [a, bl into itself.

Definition 1.2.8 (Fixed Point or Fixed Point Property) . .-\ mapping T on a met.ric space (X.d) into itself is said to have a fixed point 11 if tbere exbts 11 E X such tba!
Tu = u.

Theorem 1.2.1 (Banach Contraction Mapping Theorem). Let T X -> X be a contnzct'ion fluzlJpin,q with contmctivity lactm' 0: on a complete metric space (X, d) Into itself Then the mapping T possesses exactly une fIXed puint 1t EX. Morrover, 107 any x EX, the sequence x,T(x), T 2(x), . . , Tk(x) cOlltlnye,<; to th: point. 11.: that 1..<;
lim rk(x) = u.
k-4",,,,

Proof.
and 0

We know tbat T 2(x), = T(T(x))" .. ,TIx)

== T(T{k-l)(:,t)) ,

< 0: < 1.

For any n

0,

d(T m (x) ,T" (xl)

< od(Tm-' (,,) ,T'"-"(x))

-:;: a"'d(x,T"-"'(x))
"'-I/O

-:;: u'"

d (Tl'-l (x), T k (x))

.,-m
< am

1,,=1

a"-lrl(x,T(:1:)) ,

l'=l

by applying contract.ion property k - I times. From tltis relation, it follows that

d(T'''(:r),T''(x))

---J.

0 as m,n-)

00

In other words, {Tk(x)} is a Caucby sequence Since (X,d) is complctp., t.here is a point u E .\" that is the limit of tbis sequence.

...... "'_'_""""'''''''10''--'

Now, we show that T(u) = 'll.lt is sufficient to show that d(T(u), n) = 0, as d(T(u),u) = 0 if and only if T(u) = n. We 11<\\'c
d(T(uj,u)
~

(",T(u)) as k

< d (u,T'(,,)) + d (T'(x), T(u)) < de'll, rk(x)) + ad(u, T k - 1(r) ---J. 0
because
h~

---J. 00,

lim d(ll,T k (x) =Uand

lim d(u, T l -- 1 (x = o.
k-toc

Finally, we show that v E X, T(v) = v. Then

tl

is unique. Suppose there is another fixed puint

~

d(u,v)

d(T(u),T(u) < ml(u,v),

0'

den, v)

nd(tl,1').

This holds only when den, v) = 0 which implies that u = v by Axiom (i) of the md.ric. I

1.3 Application of Banach Contraction Mapping Theorem

1.3.1 Application to Real-valued EqUC1tioH
Let X = R be the metric space of real numbers with IIxll In, h] C Fl; f : [0, b] --+ In, h], a differentiable function such that

l:.rl,

and

Suppose we want to find the solution of the equation :c = f(x). By Lagrange's mealH~dllle theorem, for auy :l',v E [(l,hl, f(:c) - fey) = j'(z)(:cv), where y < z < x. Hence, If(x) - f(y)1 = 11'(2)1 Ix - vi ~ k 1:1: - yl, where

Thus, f is a contr<.tctiolJ rIlapping Oil [a,bl intu itself. By Theorem 1.2.1, t.hcre pxists a unique fixed point x E [a, I)], i.e., f(x) = x _ Therefore, x is the solution of tile equation f(x) = x.

",,,. '.'_'_""""'''''''10''-''

1.3,2

Applica.tion to I'i/la,trix Equation

Suppose we W<lnt to find the solutioll of a system of H line;:Il" algebraic equations with n unknuwns
(111:1':1 +o.12X2 (l21X1

+. +o\"X" = + (/'nJ:2 + .. --r (IZ"X" =

/)1

(1.3.1)

Equivalent matrix formulation Ax = II, wllerc

(111 Ul:l UI"

(I:.n (/Z2

(/Z"

,-" -

(I,,,,
This system can be written as
:CI =(l-Ul1)XI -(/'l2J:Z"'-(II"X,,+b l

X2
X3

= =

-(/21:.1'1

+ (1

-(131 XI -

- (/:l:l)xz - (ttJX3 (132Xt + (1 - 0.;1;1)3

(Jt":r,,

- (t3"X fl

+ hz + V1

( 1.3.2)

By letting

O:'J

-(I,)

+ 0", where
foe i =j
for

iii

Equation (1.3.2) can be wrilten in "be following equi''aleJ11, form:

=

Xj

L
}=1

O!IjX)

+ bf ,

i= 1,2,3,.

,no

(1.3.3)

If J. = (XI,X2,." ,:t ll ) E Rfl, tllen Equation (1.3.1) call be wrnten ill The equivalent form x-A:r+v=x. (1.3.-1)

Lel TJ = X - A:c + b Then t.he problem of finding tllP. solution of system Ax = b is equivalent La finding fixed points of the map T .

Q.

......

, to<l'._..

....,..",,"l1>"-I

Now, Tx - fx' = (I - A)(:r - :1") and we silow that T is a eOlltract.ion under a reasonable condition on the matrix. In order to find a. unique fixed point of T, i.e., a llnique solution of system of Equations (1.3.1), we apply Theorem 1.2.1. In fact. we prove tile following result. Equation (1.3.1) has a unique solution if

" " Lla,jl == LI-a,j +oiJI:s

)=1 }=I

k < 1.

i == 1.2.... . n.

For

J~

== (Xl, :1:2, . . ,x.. ) and x' = (.el , x~, .. . ,X:,), we have

d(T:c; Tx') = (t(y, Vi)

where
V == (YI,V2,'" ,Yn) E R" ( Y1'Y"2,'" 'Yn E ')R" V' = "

Y, ==

" L a,jx) -r b,
"

}=I

V:: Lo:,)x~+h"
)=1

t=L,2, ... ,n.

\Ve have
d(y,y') =
I

sup IY, <i<Jl

v:1
-

sup
1 <,<"

" " L a'jJ:j + h, - L a,}J:~

J=1

b,

j=1

~ n,}(xJ
j=1

"

xj)

< sup

l~'~Jl )=1

" L

lai}llxj - xjl
-

<

sup IXj
1~}<"

j'jl

" L 10';;1 1<;<" )=1

sup

:s
Since

k SllP Ix; 1.'5:}~"

xj I.

" LlalJl:S
j=l

k < 1 fori = 1,2, ... ,nand d(x,x') = sup ):c)-:cjl, we

I~j$"

hflved(TT,Tx') < kd(x,:J.:'),

O:S

k < I, i.e, T is a contraction mapPlllg

."". "'_'_""""'''''''10''-''

on R" into itself. Hence, by Theorem 1.2.1, there exists a uniquf' fixed point x of T in R", i.e., x is a uniflue solution of S.,"StCIfl (1.3.1).

1.3.3
Herf',

ApplicatIOn to Integral EquatiolJ

WI"

proVP the following exist.ence I.heol"f'm for integr:ll f'qwll.ions,

Theorem 1.3.1 Let the JlInctwn J((:r,y) be definell and rneasmnble in the S(/lUlH: A == {(x,y)/o. < x </),0. ~ Y < b}. Further', lel

ant! g(x) E 2((1, b). Then fhe integral eq'llatw'l

I(x) ~ "(x)

+ I' J, 1,(x,y)I(y)dy
J01.

[',

(1.3.5)

l)()ssesses a tt1lujue sol'lltwll f(x) E L 2 (0., b) of the pammete7 11,

every sujJictelltly small val,w

Proof. ping T

For applying Theorem 1.2.1, letS == L 2 , and consider the map

T: L 2 (a, b) -+ L 2 (a, b) Tf == 11
where h(x) == y(x)

llit> [((;r:.y)f(yldy. it> l\(x,y)j(y)dy E L 2 la,b).

b

This definition is valid for each f E L 2 (a,b), hE L 2 (o.,b), t.his call hI" seen as follows. Since 9 E L 2 ((1,b) Md 11, is scalar, it is sufficient to show that
(x) ==

By the r.auchy-Schwarz illPquality

It> /{(x.y)!(yldy ~ I
<

IIx,Y)/(y)ldy

([I[(X,Y)I'dYf '

([ II(YJI'dVf'

.n",

'.'_'_""""'''''''10''-''

Therefore,

1"lxll'

~ ( [ J(lx,yIJ(Yldy ) '
S ([' 1JX,Y)I'd,,) ( [ IJ(1/)I'<1,,) .

[ ' 1,'(')I'd" < ,{ ( [ I/'(X,Y)I'dY) dX,{ ( [ I/(Y)I'd1/) dx

By the hypothesis

and

, { (,{ IJ(Y)I'dY) d, < oc

TIm:;

,pIX)

~ j' [(x,1/)J(y)dy E [,(a, b).

"
1/,!-

\Ve know that L 2 ((1, b) is a complete metric space with metric

d(f,g) = ( [ IJ(.x) - Y(X)I'dX)

Now we show that T is a contraction mapping. We have d(TI, TIl) d(h, ltd, where

h1 (r.) ==
d(",",)

y(x)

+IJ,[b f{(x,y)Idy)dy

~ 11,1 (['

[[ [((.T,y)[J(y) - ],(Y)]dY] , dX )

>I'

S 1,,1 ([' [' 1J(IX,Y)I'dXdY)

( [ ' IJ(y) - J, (Y)I'd,,)

'I'

'I'

B..

..... ,

, to<l'._.. ....,..",,"l1>"-I

by using Cauchy-Schwarz-lluuyakowski inequality. Hence, d(T f, TIl) < ) 01' I/II IJ('(x,Y)ldxdy dUJ Ill By definition of the metric in [2, (

1.

WI' IHlVC

d(f, f,)

~ ( IlflY) -

f,(yll'dy

) 'I'

If

then
dl,'f, ,'f,))
where
~

k(f,f,),

o:s k = I/II ( 111[(T-,VWdXdy

) '/2

<L

Thus T is a contraction and, by Theorem 1.2.1, T has a UlllqUC fixed point, i.e., there exists a unique f* E D.l(fl,b) such that T/* = This fixed is CI uniqu p solution of Equation (1.3.5). I point

r.

1.3..:1

Application to Differential EqucltiolJ

In this section, we apply Theorem 1.2.1 to prove Picard theorem.

Theorem 1.3.2 (Picard TheOlem). Lf:t !(x,y) be a continuous [7mclion 0/ two variables in a n:dangle ::I = {(x, y) / a < x < b, C Y d} ant! satisfy the Li,)Schi/,z conl!itioll of order 1 in the secout! variable y. Furlher; let (xo, YO) be an interior lJOint of A. Then l/H: tlilJt:rentiol equativn

:s :s

-j ~
<X

dy

fix,,,)

( 1.3.6)

has a ?miq'llc solution, say y = g(x) which l)(),Sses thmugh (XO,yo).

Proof. First of all, we show that the problem of determining the solution of Equatlon (1.3.6) is equivalent to the problem of finding the solution of

.n",

"'_'_""""'''''''10''-''

an integraJ equation. If y = 9(X) satisfies Equation (1.3.6) and has the propert.y that !I(3:0) ::::: 110, then intep;ratinp; Equation (1.3.6) from Xo to x, we gel
g(,,) - 0("0)

" y(x) ::::: Yo + 1'" f(l,g(t))dl..

'0

l'

f(t.,g(t.)dt

()'3,7)

Thus <:l unique solution of Equation (1.3.6) is <luh'<l.Jfmt to a unique solution of Equation (1.3.7). For determining the solution of Equntion (1.3.7), we have t.o npply Theorem 1.2.1. Since f(x,y) snt.isnes tile Lipschitz COndition of order 1 in y, there exists a constant fJ > 0 such that If(x, VI) - f(x, Y2) I :::; Q!Yl - V'll. Since f(x, y) is continuous on a compact subset A of R2 , it is bounded and so t.here exists a positive constant m such that. If(x,Y)1 :::; m 'r:I (x,y) E A. Choose a positive constant p such that lXJ < 1 and the rectangle B = {(x,y)/ -p+xo:::; x <p+xo, -pm+yo:::; Y:::; 1Jrn+yo} is contained in
,\,

Let X be the set of allreal-vaJued continuous fllllctions y::::: g(x) defined on [-,) -r ;1;0.1) + 3'0] :such that d(g(x),yo) < ffW X is a closed subset. of the metric space C[J:o - P,Xo +p] wit.h sup metric, hence a complete metric space by Remark 1.2.3. Let T : X --7 X be defined as Tv = It, where

It{;:)::::: yo

1'""

!U.g(tlldl. Since

d(h(x), yo) = sup

1'"
"

f(l,g(tdt

<

m(:r: - xo)

h(x)

E::

x, and so T

is well defined. For g,gl E S

d(Tg,TY1) ::::: tl(h,h l )::::: sup l"'[f(l,Y(l) - fU,gl{t]dl,

<
<

1"
"

"
suplfU,.tJ(tl) - fU,91lf.))ldt
)g(t) -gl(t)ldt

" < qpd(g, yd,

0'

',1'"

."", "'_'_""""'''''''10''-''

where 0:5 k = IJ]J < 1. Hence, T is a contraction mapping of X into itself. By Theorem 1.2.1, T has a unique fixcd point g. EX. This unique fixed point, !/, is the unique solut.ion of Equat.ion (1.a.6). I

1.4

Problems
YII

Problem 1.4.1 Sbow that (R2 , (I), where d(x, y) = IXI for all x = (XI ,X2), Y = (YI, yz) E n 2 , is a mctric space.

+ 1.1":.!

- Y21

P1'Obiem /.4.2 Show that (R Z , d), where d(x,y) = max{lxI Y211 for hil x = (x].:r.z),1J = (YI,y:.!) Ere, is a metric sp&e.

Yd, IX2-

Problem 1.4.3

Showthat.(R2 ,d),wherev(x,y)=

~lx.-1hlz

"

) '/2

for all ;/:, y E R, is a mpl ric span'

Problem /.4.4

Show that. (C[a,bj,d), where (l(f,y)

CI

y(x)1 for all f,y E C[a,bj, is

complete metric

sp<u~.

P1'Obiem 1.4.5

Show tlJ<\t (Lz[a,bj,d), where d(f,y)

g(xW ) '1' , is a complete metnc space.

Prohlp-m 1.4.6

Show 1,lulI,

(I"

1 <p <

00

is" complet.e metric S)lace.

Problem 1.4.7 Let 1n = f'.oo deuote thc set of all hOUllded real se<luences. Then show that m is a metric space. Is it complete? Problem 1.4.8 Show that C[a,bj with integral metric defined on it is not a complete metric space. Problem 1.4.9 Show (.hat h(, '), defilled by equation (1.2.5), is for all closed and bounded set.s A aJld B.
11

metric

.,,,,, "'_'_""""'''''''10''-''

Pmblem 1.4.10 points of T, Pmblem 1.4.11 space (X, d).

Let T

R ---) R be defined by Tx

F'md fixed points of the identity mapping of a metric

Pmblem 1.4.12 Show that tlH~ Danach cont.raction t.heorem does not hold for incomplete metric spaces. Pmblem 1.4.13 Let X = {:r E RI:!: ~ 1} c R and let T: X ---t X be defined by Tx = tX+X-1. Show that T is a contraction rnapiling on (X, d), wl\f're d(x, y) = Ix - yl , into itself. Pmblem 1.4.14 Let T : R+ ~ R+ and Tx = x + C"", where R' denote t.he set of positive real numbers. Show that T is not a c:ontraction lIlappi ng. Pmble71l 1.4.15 Let T: R Z ---t RZ be defined byT(xl,:rz) = (x z ,Xl ). What art> the fixed points of T1 In whal quadrant nf the Xl ,x2-plane is T a contraction? Pmblem 1.4.16 Lct (X,d) bc a completc metnc space and T CI continuous mapping on S into itself such that for some int.cger n, Ttl = ToT 0 1' .. 0 l' is a contraction mapping. Then prove that T has a unique fixed point in X. Pmblem 1.4. 17 Let (X, d) be a complete metric space and T be a contraction mapping on X into itself with contmctivity factor a, 0 < a < 1. Suppose that u is the unique fixed point ofT and XI = Tx, J:Z = 1'xJ, .l:;j = 1'X2, ... ,X" = T(Ttl-l x ) = T"x .. . for any x E X is a sequence. Then prove that
1/3 1/3

(al ([(x"" n) :$
(b) d(:l:""u) :$

am
1
Q

Ii(X,XI) 'r/ Ul.

Q

d(Xm_I,X",) V

m. a

Pmblp-m 1.4,18 Prow t hat. every contract.ioll mapping defincd metric spacc X is continuous, but t.he com'erse may not be true.

Oil

B.

.... ,

, to<l',_"""",,,,,"l1>"-I

References
[I] KC Uoder. Fix!l'1 Point Theorems with Applications to Economic and Game Theory. Cflmbridge. London, ~ew York: Cambridge University Press, 1985.

[2] V1 Istriltescu. Fixed Pomt Theory. Dordrecht, Boston, London: Reidel Publishing Company, 1985.
[3]
]{A

Liusternik, VJ Sobolev. Elements of F\metiulJal Analysb (Third English Edition). New DeJhi: Hindustan Publishing Co., 1974.

[4/ AH Siddiqi. Functional Analysis with Applications. New Delhi: TataI'v1cGraw-Hill Publishing Companv Limited, 1986. [5J DR Smart. Pixed Poiut Theorems. Cambridge: Cambridge University Press, 1974.

.n",

"'_'_""""'''''''10''-''

Chapter 2

Banach Spaces
2.1 Introduction

Around 1920, a young engillceriug student investigated cert<lin mathematical structures combining topological and algebraic properties. He introducf>d the notion of magnitude or length of a vector which led to the sludy of IIwthemi.\tical strUl~tures called uormetl s"aCt~ and a ll-space or COIllplete narmed space. Subsequently, these spaces becamp the foundation of functional analysis. This young man is 1I0W rCognized R.S Stdan Danach, a distinguished Polish mathematician, and tlte complete narmed space is now known as a Banach space. It will be seen that a nonned space is a metric space and CI Banach space is a complete metric space. In this chapter, we present some basic results concerning Banach spaces and mappings defined on them The concept.:> of lIormed space, l3anaclJ space, bounded and lineClr operators and their spaces, collvex fUllctionals, and duals of a normed space Clre discussed. To make the concepts c1p,ar, a number of examples are given. The material presented here is essential for proper understanding of various branches of mathematics, science and technolog.y.

2.2 Definitions and Examples of Normed and Banach Spaces

Definition 2.2.1 A norm (or length function) on a vector space X is a real-valued function, denoted hv 1111. defined for all vectors x E X and which :-;at.isfies the followillg axioms:
(i) (ii)

IIxll > 0; Ilxll = n if and only if . = 0, lIuxll = litlllxll, tX an arbitrary scalar,

...... "'_'_""""'''''''10''-''

(tii) IIx ~ yll < 11.'11 + Ilyll Vx, y E X.

A 1Io1'med slmce or normetl vector space. denoted by (X, a vector space X and a norm Ii-II on X.

IHI). consists of

(a) Axiom ti) rcquir<:>s that the length of it vector is nonnegative and t.hat only zero has the length Zef().

Remark 2.2.1

(b) Axiom (ii) rneallS that the length of tile \'ect,or ox sllould be the length of the vector.

Inl tiUles

(c) Axiom (iii) is the triangle inffjuality. (d) One may observe that these axioms are just an extension of the properties of the absolute vaJuc 10:1 of a real number u.
(e) It can be seen (Problem 2.8.1) that e\'cry narmed space X is a metric space with respect to the metric
d(x,y) =

Ilx - vII,

x and 11 EX.

F\lrtherlllore, d(x,O) = IIx - 0Il = IIxll so that the norm of any element is equaJlo its distance from the origin. Thus, a narmed space is a special type of metric space and all concepts like Cauchy se<luence, conwrgem sequence, closed ness, completeness, compactness introduce<1 in a metric space will be valid for nonned spaces. However, a metric :opacc need nut be a Ilurmed space (8eP Problem 2.8.1). (f) A vpctor spac(' can be a nonupd spaef' with resppct. 1,0 different norms defined on it (see example 2.2.4)
(g) If In Definition 2.2.1 (i) only semI norm.

IIxll :?:

0 holds, then

II . II

is called a

A complete lion lied :"pw.;c X k. utlll'1 a Dll7luch SllfJ.n~, that is, every r.auchy sC<juencp in X (lIx n - xmll -+ 0 as 1I,m --t 00) is a convergent sC<luence (=3:c E X such that IIx n xII -t 0 as n -~ 00).

Definition 2.2.2

(i) A (normed) subspace of a normed space X is an algebraic subspace of X having the same norm functioll as the Ilormed Spfi('P X.

Rem01'k 2.2.2

(ii) A suhspa('f' of a Banach space is itself a Banach space if and only if it is closed .

B.

..... ,

, to<l'._..

....,..",,"l1>"-I

2.2.1

Examples of Normed and B81wclI SPlICes

Example 2.2.1 The vector space R of real numbers is a normed space with respect to the norm IIxli :::: Ixl,x E R (Ixl denotes the absolute value of real number x). Example 2.2.2 The vector spRee R2 (the plane where points have coordinates with respect t.o two ort.hogon<:ll axes) is a nonned space wit.h respect t.o t.he following norms:
l.

11(1111:::: Ixl + IYI ,where

= (x,y)

R Z;

2.

lIall,

max (lxi, Iyl); and

3. 1I(1lb == (x 2 +y2)1/2.
Example 2.2.3 The vector space C of all complex numbers is a normed space with respect to the norm IIzll == Izl, z E C (1,1 denotes the absolute vulue of t.he complex numher). Example 2.2.4 The vm:tor space R" of all n-tuples x == (Xl ,X2,'" , x n ) of real numbers is a normed space with respect to the following norms:
l.

IIxll, ~

" L Ix,l,
1<=1

B.

......

Note: 1. R n equipped with the norm defined by (3) is usually denoted bye,;. 2. R" equipped with the norm defined by (4) is usually denoted by l~ .
, to<l',_"""",,,,,"l1>"-I

Example 2.2.5 The vector space m of all bounded sequences of real numbers is a normed space with the norm Ilxll = sup I.r." I. (Somet.imes exn

or fO() is used in plac p of m.)

Example 2.2.6 TllP vector space .. of all collvergent scquencel' of real numbers is a norm(.-'C! space with t.he norm IIxll = sup/x"l.
n

ExamlJle 2.2.7 The vector space of all convergent sequences of real numbers with limit zero is denoted by Cn which is a normed I'par-e with respect to the norm of (" Example 2.2.8

The vector space f p ,

00,

<

00,

of sequences for

which {; IXtf <

is a IlUnll(.-'d !>p<\t:e with the

nurnl

IIxllp =

[= ]'/'
{ ; l:qlP

Example 2.2.9 The vector space C[a,b], of all real-valued continuous funt:t.ions defined on [a, b] , is a normed space with respect to the following norms:

a.

11111, ~ 11/)1<1" 11/11,:

III"',

b.

(ll/(.')I'dX)''',
a<",<b

and

c.

"'I'

I/(x),.

(11-11 3 is called the uniform convergence norm).

Example 2.2.10 The vector spnce P[O, I], of all polynomials on [0, I} with the norm 11:1'11 = sup Ix(t)I, is n normed space.
0<1<1

Example 2.2.11 Let A be nny non-empt,Y set, and let D(A) the class of all bounded real-vnlned functions defined OIl A. Then 8(.4) is a normed space with respf'r-1. 1.0 the norm II/II = sup 1/(t)l

Q .,_._........

......

'EA
",,"I1>"--I

Note: If A is the set of positive integers, then m is a special case of B(A).

*E/.cam1Jle 2.2.12

Let A be a topological space and let BC(.4) denot' the set of all bounded and continuous real-valued functions defined on .4.. Then B(A) ;2 BC(A) and BC(A) is a nonncd space with the norm Ilfll = sup If(OI, for all f E l1C(.-1).
IE,1

*I\'ote: if A is compact, then ever.y real-valued wntmuous function defined 011 A is bounded. Thus, if A is a compact topological space, then the set of all real-valued continuous functions defined on A, denoted by C(Al BC(A), is a normed space with the same norm a" in Example 2.2.12. If A = [a.b]. we get. the normed sp,\Ce of Ex,Ullple 2.2.9 with 11lb.

'Example 2.2.13

Suppose p > 1 (p is not necL-'Ssarily an inte!,rer). LP denotes the class of all real-valued functions f(t) such that f(t) is defined for <\11 t.. with the poosihle fOxception of a set of meaSllfe 7.Pro (almost everywhere or a.e.), is measurable and If(t)jP is Lebesb'lle imegrnble over (-00,00). Define an e(luivalence relation in 1' by stating that f(t) ""' g(t) if f(t) = 9(1.) a.e. The set of all equivalence classes into which LP is t.hus divided is denoted b.y LP or L p Lp is a vector space and a normed space with respect to the following norm:

'Note: 1. In plnce of (-00,00), one can consider (U,ool or any tinite inten'al (a,b) or any mem;urable set E. 2. f(lJ represents the equivn1cnce class [fl. 3. LP is nut a Ilormed space if the eqllalit.v is considered in the USLl<l.1 sense. However, LP is a semi-normed space, which meaus IIfll = 0, while

UO.
4. The zero elem(>nt of L p is the equivalence class consisting of all such that. f(l) = 0 a.e.

E !"

'Example 2.2.14

Q ., ........

......

Let (a, b] be a finIte or an Infinite interval of the real line Thp-n a measurable function f(t) defined on [a,bJ is called essentially bounded if there exists k 2: 0 such t.bnt the set H! f(t) > k} hm; measure zero, Le., f(t) is bounded a.e. 011 (u,bl Let D'''[a,b] denote the cl<.t:>s of all measurahle and essentially boumled funct.ions f(t) defined on [a,bJ. Loc'is in relntion to C'X just. as we dpfine L p in relation to 1'. L'X) or Lx> is a
",,"IO"--I

I,ormed space with the norm

III1 I = Sl~P I/(t)1 ,

[essential least upper bound

of I/(t)1 or essential supremum of I/(t)1 over the domain of definition of 11.

Example 2.2.15 The vector spaee BV[a,b] of all functions of boull(k'(l variatioll on [a,b] is a normed space with respect to the norm 1II11 = I/(a11+ 1;':- (x), where \ ';.:- (x) denotes the total variation of IU) on [a, bl. Example 2.2.16 The vector space Coo [a, &] of all infinitely differentiahk functions on [a, &1 is a nornwd sphce with rflSppct to tlw following norm:

II/II""

I. ~
b "

ID'/(t)" <It

'I'

l<p<oo,

where D' denotes the i-til derivative. Note: The vector space

eX> [Il. b]

can bf' nonned in illfinitdy manv ways.

Example 2.2.17 Let Ck(Ol denote the spaces of all real functions of n variables defined on f1 (an vpen subset of It") which are continuously liiffc:rentiahlt> up t,o order k. Let n = (al,aZ,'" ,n,,) wherf' (Y,S nJ'P nanncgutive integers, und

101

L a,.
1=1

Then for

f E ell (f1), the followmg

derivatives exist and are continuous:

DO/ =

8 1<>11

Dt"" ... ,af"'n I ,"

lal <_ k.

C k (f1) is a normed space under the norm

Example 2.2.18 Let Cpj (0) be the spa(:e of all infinitely differentiable functions with compact support on 0 (,ill open subset of R"). Cpj (f1) is a normed space with respect tv the following norm:

II/lib" ~

Q,.,_,_........

......

(10 L

IDa /(t)l" (<It

lolS:k

';"

",,"IO"--I

where nand Da f are as in Example 2.2.17. For l.:olllpact :suppurt,:see Definition C.4(7) of Appendix C.

Example 2.2.19 The set of all absolutely continuous functions on [a,b] , which is denoted b:r ..IC {a, b] , is a subspace of BF la, b]. AC[a, bJ is normed space with the norm of Example 2.2.15. Example 2.2.20 the condition
The class LiPa [a. b]. the set or all functions

f satisfviug

II/II. ~

"'p
I>O .r

(I/(X + t~ - /(x)l) < 00

t

is a normed "pace for a suitable choice of (I.

2.3
2.3.1

Basir Propertip.s Clnsure, Densp.np.ss and Separability

Closed, Dense and Separahle Sets

Definition 2 ..1.1 A subset F of a nonned spuce X is called closcd if it contains all its limit points. For an,\' subset }" of a normed space .X, the closure of F, denoted by Y, is the union of Y with the set }., of all limit points of }"; that b, Y = Y u Y', where 1" is t.he set of all limit points of 1"". Sr(a) = {x E Xlllx - (JII < r} is C'alkd opcn sphcr'C with center (J and radius 1" in a normed space L If a = 0 and r = 1, S,.(a) is called the unit open sl,lwn:. If IIx - 1111 ~ 1" holds, t.hen the sphere is called dosed.

Q,.,

......

Remark 2 ..1.1 (i) Closed subsets are important while st.udying the solut.ion of equation, where one looks for approximate sollltiollS by constructing sequences of approximations, all of which belong to a set Y of functions with certain propert.ies. Lf Y is a closed set. and if the seqnence is convergellt, t.he limit ilbo belonb~ to Y, givillg <t cOllvcrgent :sequcllt:e of appruximations in the solution set Y. (ii) It is clear that Y c Y, and V = Y if und only if }' is closed. (iii) It may be noted that C [- J, IJ is IIOt a c1os(xl suhspace in 2 [-1, I] .

........

",,"IO"--I

Definition 2.3.2 Let A and B be two subsets of a nonned space X such that A c B. Then A is dense in B if for each v E B and each c > 0, there is a u E A with IIv - ull x < E.. It is denoted by A. = B.

Example 2.3.1 (i) The set Q of rational numbers is dense in R; tim!; is, q ~ R. (ii) The space C(O) is a dense subspace of L 2 (0). (iii) The spj of all polynomials is nens!' in L"l. (n).
Definition 2.3.3 A Ilormed space X is callPfI countnble subset which is dense in X.
.~epamblp.

if I.h prp pxists a

Example 2.3.2 (i) R is a separaille nortned space Slllce the set ofraLlOnal numbers Q is dense .uld countable. (ii) R" is separable. (iii) L'l. (0) is sepnrable since the set of all pol.ynominl!> with rntional coefficients is dense and countable in L 2 (0).
It may be noted that n normed "pacp ma:y f"ontain morp than om' set

which is dense awl countable.

Definitton 2.3.4 Two normed spaces (X, 11.11 1 ) and (\"IHb) nre said to be isometric and isomorphic to one another if there exists a one-to-one mapping T of X onto }' such that

and T is Iinenr, thnt is,

T(x +y) = T:1' + Ty, 'ix,y c X T(nx) = nTx, 'ix E X,o E R or C.

III other words, T is (n) one-to-one, (b) onto, (c) norm-preserving, and (d) linear. In such a situation, we shaU write X = Y
It may he 7'etlwr'ked that two isometric isomorlJllic spaces can he viewed (IS being essentially the same norflled sl){l.ces in two different guises. The names given to points in two $paces a7't difJen:nt, but all important properlies like disl.anf"e, continuity, r.(ln1Imyenr-p r.losp.dnf'.~s. denseness, .~epambility aTe the same. We shall come acmss such sltuallOns in Seelion 2.5.

Q ., ........

......

",,"IO"--I

Definition 2.3.5 (a) Two normed spaces (X,IHI 1 ) and (X,IHIz) ar(' called topolobrically equivalent or two norms IHI I and 11112 are called equivalent if there exist COilstants k 1 and k 2 such that kl IIxll l ~ IIxl1 2 < k'lllxli l . (b) A normed SP<l!:f' is O'lied finitp.-dimf~tI.~io1tal if thf, underlying vedOl' space is finite-dimensional; otherwise it is mfinitc-dimcnsional
Theorem 2.3.1 equi1Jalen/,. All nonns on a fimte-dmwnslOnal nonned sll(J(;e
an~

Theorem 2.3.2 A normed space is homcomorphic to its open unit ball S, (0) = (x E X / IIxli < I).

2.3.2

Riesz Theorem and Constl'Uction of a Nell' Banad} Space

If V is a proper closed subspace of a normfx1 space X, then a theorem hy Riesz tells us that there are pointl' at a nonzero dist<lllcC from F ~'Iore precisely, we have

Theorem 2.3.3 (Reisz Theorem). of a 1WrtlICd spacc X, and let > o. lI.tll = 1 such that d(x, > 1 -

Lct Y be a IJropc7' closed subspacc Then thcn'. exists an :.r E X with

This theorem also helps to prove that a normed space is finne-dimensionHI if and only if its bounded closed subsets arc compact. The following result provides us tile most useful method of forming a new Banach space from a given Banach ::;pace:
Theorem 2.3.4 I~et X be a closed subslJace of a Banach spacc Y. Then the factor or quotient vector Slmcc FIX is a Banach space with fhe norm

lIy + X I ::: rex {lly + xl]} inf

2.3.3
R, R",

for each y E }'

Dimension of Normoo Spaces

normed spaces. In fact. all real finite-dimensional nonned spaces of dimension n are isomorphic tlJ R" r: [a, b] , 1" L", r [0, IJ, Bll [a, hI, C k (il), (ltc. arc examples of infinitedimensional uormed spaces.

e;:, C are examples of finite-dimensional

Q .,_,_........

......

",,"I1>"'-I

2.3.4 Open and Closed Spheres

I. Consider a nonned space R of real numbers, an open sphere with radius r > 0 and center a which we denote by 5,.(a). This is nothing but M open interval (a - r, (l + r) . The closed sphere Sr( a) is the closed interval [a - r, a + r]. The open unit sphere is (-I, 1) and the do&.-'<i unit sphere is

[-1,11
2. (a) Consider the lIorllled space R'l (plane) with the norm (1) Example 2,2.2). Let
x::= (Xl,."2) E R'l, a:::: (al,a'.!l E R'l.
(S(-~

Then

and

s,

{XE R'/llxd+lxzl} ~ 1}.

g(-~onwt.rkal

Figure 2.3.1 illnstratf's thf' unit spheres.

lllp;-ming of the open ;-Uld dosf'(l

51 :::: Parallelogram with venlces ( -I, 0), (0, I), (1,0), (0, -1). 51 ::= Parallelop'am without sides AB, Be, e D and DA. Surface or boundary of the closed unit sphere are lines AB, Be, CD <llld

DA
(b) If we consider R2 with respect to the norm (2) in Example 2.2.2, Figure 2.3.2 represents the OPl.oU and closed unit spheres. The rpctangle A BCD :::: 5 1 ,51 is the rectangle without the four sides.
5 1 ::= {X::=(Xj,xz)E R2 jmax(lxd.lx zl) 51 ::::

<l}

{x::::

(Xl ,X2) E R'lj max (Ixd, 1.1'21)

< I}

Surface or boundary S is given

h:r

5::= t,,::= (Xl,X2) E RZjmax(lxd,jxzl):::: ::= Sides _-IB,BC,CD and D.-I .

I}

Q ., ........

."."

",,"IO"--I

III

A
-]

>-10-------, ,

-1 D

Figure 2.:i.l

The geometrical meaning of open and dosed uni. sphncs in Example 2.2.2(1)

(c) Figure 2.3.3 illustrate::; the g,oollietric.. .,] meaning uf the open and dosed unit spheres in R2 with respect to norm (3) of Example 2.2.2.

5 is the circumference of the circle with center at the origin and radius 1. 51 is the interior of the circle with radius 1 and center at the origin. Sl is the circle (including the circumference) with center at the origin and radius
L

3. Consider C[U,I] with the norm

11/11

sup
0"':"9

1/(t)l. Let Sr (9) be a

Q."_,_........

.... ,

,,,,"I1>"--I

A ,
"

-. ,
,

Xl

,
C

Figure 2.:1.2

rhe goollJel,rical meaning of open and dosed unit spheres of

Example 2.2.2(2)

closed sphere in C[U, I] with center 9 and radius r. Then

S,(9) = lhEC[O,II/II"-hll<'}

~ {h E r [0, III "'p

I"(x)

h(x)l" ,}

0.<;:"9

This mc:tns that Ig(x) - hex)! ~ r for each x E [0, I] and so hex) e[Hlllot be less than g(x) - r or more than g(x) + r. The region, in which the graph of h(J:) must lie is shown in FigUff' 2.3.4_ The graph of hex) must lie within the region houuded by the twO broken lines. One ofthese is obtained by raising g(x) through a distance r, the other by lowering 9(;;) lrtl the same distance. If h(;r) never goes out of this region, but meets its boundary (the broken lines) at one or morc points, IIh gil = rand 11 lies on the boundary of sphere Br(g). If hex) lies between the broken lines, bill. never reaches either of them, 1111 - gil < r and It E Sr(g) Thus, Sr(g) (open f>phew with center 9 and radins r) represents the region between the broken lines. It is clear that if either II E Sr(g) or It is on the boundary of Sr(g) (It satisfies the condition lilt - gil = r), then It E Sr(g)

Q .,_._........

......

",,"I1>"-"

1 ---='+-----j-:..---,-I'----x,

-1

Figure 2.3.3

The geollwtric1l1 meaning of opcn and dosed unit spheres of Example 2.2.2(3)

~"of-~~~-~-------,I,---_X

Figure 2.3.4

The gt-'Ollletrical meaning of open and dosed unit spheres in

CIa, II

2.4
2.4.1

Bounded and Unbounded Uperators

Definitions and Examples
LeI X and Y be two normed spaces Then

Definition 2.4.1

1. A mapping T from X into },- is raIled all ope.rator or HI.ransfornmtioll. The wlue of Tat:r E X is denoted by T(x) or Tx.
2. T is called a linear operator or linear transformation if the following

conditions are satisfied:

(a) T(x+y) = Tx-t-Ty.for all x,y E X.

(b) T(ox) = aTx. for all x E X amI real

0:.

3. ThP. operator T is called bOllnded if t here exists a real k > 0 I.h" IITxll ~ k Ilxll Vx E X.

SlIell

4. T is called continuous al. a point .ro E X if, given t > 0, there exists a /j > 0, depending on {; and XQ such that IITx - Txoll < (, whenever Jlx - :Loll < 0. T is called c07lti7l1wUS on X if it is continuous at every

point of X.

5. T is called uniformly contmuous If, for ( > 0, t.here exists a J > 0, independent of Xo, such that for ally Xo and x ill X with IIx - ;1'011 < 15, we have IITx - Txoll < t.

6.

IITII =

sup {

1I1~~11I /x i-

o}

is C<l.lled the

1101'711

of the bounded opera-

tor T (For an unbounded operator, the sup may not exist). 7. [f Y = the normed .space of real numbers, thell T !ll.nctir!'llnl and it. is lIsually oenot.pd by F. 8. For the operator T, the set
N
~

n,

i~

called a

n::=

{T(x) E Y/x E
~

Xl

and the set

(x E Xj1'(x)
Sl)(lCCS,

0)

are called the nmyc IUld null

respectively.

Remark 2.4.1 1. When we write IITxll or IITa' - Txoll, it b clear that we ,up considering the norm of Y. [f Y ::= n, then IITxll and IITx - T,r-oll will be replaced by ITxl and ITx - Txol, respectively.

.n",

"'_'_""""'''''''10''-''

2. In recent rears, the study of nonlinear operators has attracted the attention of mathematicians in vicw of their important applications. 3. Two operators T 1 and T z are equal (T. = T z ) if T1x = T'lx for all ,l: E X. 4. The tWO conditions III Dehnitaon 2.4.1 (2) ma,v be replaced by a single condition T(ax + fly) = cJ'x + (lTy, where a ~nd t1 are real numbers and x,y E X, 5. (a) If Tx = 0 for all l' E .\, then T is called the ZC7V 0llemt07', 1l1tll opemto7' or t7'1vial opemtor', (b) If Tx = x for all ;t; EX, Tis G:lHed the ~defltity upaut07' aud is usually denoted by / 6. The first. and second conditions of Defillltioll 2.4.1(2) are known a<; additiw and homogcneous properties, respectively. Example 2.4.1
I. fix) ~x

AllY rcal-valucd functioll

1.R

r R, such as

2. I(x) = sinx or cosx

3. f(x)

e'
J..2

4. I(x) =

or

X,I

arc operators delined on the normed space of real numbers into itself III fact, each olle is a functional. It can be easily seen t.hat all are nonlinear except 1. The following fact can be \'crificd: A real-valued function is linear if a.m.! only if it is of the form I(x) = ox, where a is a fixed real number, x E R.
Example 2.4.2

Let four mappings be defined on HZ as follows:

1. TJ(x,y) = (a.c,ay), where x,y E Rand (\' is a fixed positive real lIumber.
2. T'l(x, y) = (;r cos 0 - y sin 0, x sin 0 + y cos 0); i.e. T is a mapping which rotat.es each poitlt, of the plane about the origin and through an angle

o.

......

3, T 3 (x,y) = x for all (x,y) E R Z

0,.,>0<1"_""""'''''''10''-1

4. T1(X,y) = V for all (x,V) E n 2.

OP = (x 2 + V2 )1/2 OQ 0:(x 2 +y2)1/2,

where 0 is the origill in R2 , P and Q are arbitrary points ill R2 .
If (x,y) is any POlllt P in I.he plalle, then its image (ax,ny) under T is point Q which is colinear with P and the origin and 0: timcs as far from the origin as P. If we take the norm (3) of Example 2.2.2, then

<I((x,Y),(O,O ~ lI(x,y) - \0,0)11 ~ II(x,Y)1I ~ (x'

T1 , T2 , T3 and T1 are linear operators.

+ y') 1/ .

Example 2.4 ..7

LpT. A = (a.)) he an m x

1t

matrix. Then thp equations

'fl'=La;j~J'
)=1

"

i=I,2,3, ... ,m

define an operator T on

T : n" --+ R'"

Tx=y

nn

intu R"'.

wherex=(~1,6,... '~fi) E

R",

and y = ('fl1,1/2 . . . . T/",) E R m .

1 T is linear if

(a) T(J. + x') = T:J: + Tr'. (b) T(ax) ~ aT(x).

LHS of (a)
= T(x U') = T (("

+ W ,(" H;), ..

n H:,)) ,

where

x = 6, ... ",,) x' = (~;,~~, .. ,~:,).

Thus,
n

..

T(x + x') =

La
J=l

"

'J

(~j

+ ~j)

" L (ll)~j -I- L ajj~;

)=1

j=l

= Tx

B.

.... ,

+ Tx' =

lUIS of (fl.) .

, to<l',_"""",,,,,"l1>"-I

LHS of (bl

T(c<x) ~ T(c",("'" ,~,,)) = T(o:6,oz, .. ,at.. )

=

" L
;=1

(lijO(j ::: 0:

" L Q'j~) = oTx = Rl-lS of (b).

J=!

2. T is ullifonnly
Let x' == (ti,t; arhitrary x = (6,6,

COlItilIUMLS

and hence

coTitimwu.~

if .'1

i- O.

~;,) ,~,,)

E nn be fixed and let y' = Tx'. Then. for an E R", ify = T;r., \V~ have

Ily - y'II'

IITz -

Tx'lI'

LQ'J~)
J"'I

..

- LQ,j~;
1=1

"

" - L a,)(; - j)

1='
o

,
n
j=l

L L a'j\<, - ,;l
.==1

'"

Also,

B.... applying the Cauchy-Schwartz-Bunyakowski's mequality, we get

lIy - y'II' '5

~

.=1

(i>,;I')
J

(t 1,,-<;1')
1=1

L L la" I' liz - x'II' ,

.=1 J=l

"

lIy - y'li < 'liz - x'II' ,

when::
("2

L L 10'JI
,=1 )=1

"

.n",

'"'_'_""""'''''''10''-''

Thus, for a given {; > 0, we may choose 15:::: (Ie, provided e t- such that IIx - xiii < {;/c implies IITx - Tx'll :::: lIy - y'll < e(le:::: L This proves that T is uniformly continuous provided A is a non-trivial I1lRuix.

Remark 2.4.2 We 5hall see in Problem 2.8.3 that all linear operators defined on finite-dimensional spaces are continuous. Example 2.4.4
,

LP.l T: P" [0, 1] Le defined as follows:

d dl

TIJ(t) :::: -l)(t)

(first derivative of the polynomial).

P" [0, I] denotes the space of all polynomials on [0,1] of degree less than or equal to n whidl is a finite-dimensional Hormed spRee. T is a linear operator OTi P" [0, I] into itself as the followiug relations hold good by the well-known properties of the derivative: T(PI +"'"1) :::: Tpt + TP2 and T(apl) :::: aT1-'t for Pi and 1-12 E P" [0, I] and for a real number. We
hm'p

Tx" :::: -x" :::: 1Ix.. -

dx

Thlls, UT"II:::: t but IITx"lI:::: n. Tn view of this, T is not bounded and by Theorem 204.1, it is not continuous.

Example 2.4.5 Lei X be a Ilormed S(Jace under a norm IHI. Theil a mapping T , defined on X in the following way:

Tx ~

1"11

is a nonlineClr operawr. If we consider X:::: CiO, 1]' with sup Harm, Lhell

Tf

,up 09'51

If(t)1
~

T(h

+ h)

099

"'p

(I(f,

+ h)(t)1)

09'51

",p

Ih(t)

+ h(t)l,

which may not be equal t.o [ sup

Tft+Th Example 2.4.6

0</<1

IfIU)1 + sup h(t)I]; i.e., T(fJ +!z)

O<t<1

t-

----

Let T be defined on (: as Tx-n , -

...... "'_'_""""'''''''10''-''

where a = fa,,} or a" is the n-th (C, 1) mean or Cesaro mean of order I of the sequence x = {x" 1 E c. Then T is a linear transformation on c into itself.

Example 2.4.7 let. x = {J:,,} E m, then T;r. = y, where y = {:;' } IS a linear trallsfol"lnation of m into Co. Example 2.4.8
Let T bp a mapping on B(A) defined as follows:

Tf

/Uo),

where to is a fixed point of /I, then T is a linear mapping on B(A).

Example 2.4.9

Let T Iw a mapping on C[U,I] which maps every func-

tion of C[O, 1] to its Riemann integral, i.e., Tf =

1
1

f(t)dt. Then T is

linear by the well-known properties of the Riemann integral. We define an operator D on CIa, 1\ by taking the derivati\'f' ()f functioll~ belonging to C[O, I]; i.e., d D / ~ d,!(t).
It is clear that D is not defined for all f(t) E ClO, 1] and even if D f exisr,s, It may not belong to C[a, 1]. If, however, D is defined over a subspace of Cra,IJ which consists of all fUllctions having first continuous derivatives, the range of values of D is also contained in ClO, I]. D is linear but not continuous.

Example 2.4.10 Consider 2(",1' = 2), t.Jw mapping 1 defined below maps 2 into itself. For fx,,} E 2, TXT! = (l'Xn+l + PXn-l, where 0: and f3 are real constants. T is a linear operator on 2. Examl,le 2.4.11 Let M be the subset of Cla,bJ consisting of elements f(t) which haw first and second derivatives continuous on [a, b] such that f(a) = 1'(0) = 0. Let 1J and q belong to At. Let us define T on C[o,b] by Tf = y, where y(t) = I"(t) +lJ(t)f'(t) Tq(t)f(t). T IS <t tiueat' uperat.or on CIa, bJ into AI Example 2.4.12 I(s,t) he a continuous function on the squarp band f E C[a,b]. Then T, defined hy Tf = y, where
[,pt.

B .,,..,.,_......,..,,,,,,,,,"-!

......

0< s:::; b,o:::; t:::;

9(S) = lab /{(s,t)f(t)dt, is a linear operator on C[a,b] into Itself.

71,

flefilled by Til

= 9, where y(s) = 1(8) -

lab /{(s,t)/(t)dt, is also a linear

operator on CIa, b] into itself Some well known initial and bouudary-value prohlems can be expressed in terms of T and T, Consider the LeR circuit as shown ill Figure 2.4.1. The differential equation for the dlarge I Oil the condenser il,
Example 2.4.13 Remmk 2.4.3

La'! + lid! + L = v, d,2 dl C

where v(t) is the applied \'oltage at time t.
If the illitial conditions are assumed
La

(2.4.1)

he 1(0) = 0, dl/(O) = 0 and

R 2 > 4LjC, t.hen 11lf' solution of thi~ differential PQuation is givpn by

I(s) =
c)"s _ e),2S

1
6

"

1((s - t)1I(t)dt,

( ) and ),\,),2 arc the distinct real roots of L)''/. + L'\] -'\2 R>.. + I/C = U. If v E C[O,oo), then so is I and, in this case, the previous equation may be written as I = Tv where T is a line<u operator on C[O, (0). ThllS, Eqllatioll (2.4.1) can be written in the form

where 1\'(s) =

Sf.

where S is a linear operator defined on a subspace of ClO.oo).

Example 2.4.14

Let I(x) E La(-r.,r.) and

I

I(x) ...... 2"ao

+ ~ (a" cosnx + b" sin 1/.J:)

where an =

1:

I(x)oosnxdx.b" = 011

7r

11"

I(x) sinnxdx.

If

T!

-rr

{o,,}, T is a linear operator

LI '

."". "'_'_""""'''''''10''-''

'/I(t)

c
Figure 2.4.1 LCR circuit

Example 2.4.15 Let T; i" --t .e"'~ wherex=(xI,X2, ... ,x", ... )E{p,

~:::: 1 be defined by Tx:::: '!.I,

y= (Yl,1I2, ... ,11,,, .. ,) E p with y,:::: Lo.}xj.

j=l
~ ~

and (a,})
:xl,

is an infinite matrix satisfying the condition

L L lai,l"

<

> I. Then T is a linear operator on C into Eq . p

LetX={x={x,}/I:lx,I<::xJ,x.'s are real}. X is

Example2416

"

a norllled space with respect to the norm

IIxll :::: sup Ix. l Let Tx ::::

I

LX..
i=l

Then T is a linear functional on X but it is not continuous:

~

T(x+y):::: L(x; +y;)::::

,=]
and
~

LX, + LVI:::: Tx +Ty,

1=] .=1

,
U

'flux) :::: LUX, ::::

1=1

LX, :::: oT(x).

.=1

T is lllleaL

.n",

"'_'_""""'''''''10''-''

let x = (l, I,.. ,1,0,0, ... ) be an element of X

having the first n

entries 1 and the rest U. Then I)xlJ = sup Ix,) = I and Tx =

L
=1

T, =

n.

Therefore, T is lIot buundL"(1. rn view of Theorem 2.4.1, it is not continuou:,.

2.-1.2

Properties of Linem' Opera.tors

The following results provide the interesting propertie:, of linear 0plo'rRtors;

Proposition 2.4.1 J. If T is a linea" opemtm', Own TO = O. 2. If ;7;1, X2, ,x" hdr!1lg tn a norm"a space X (J.ud 01,0'2,. "cal numbers, then f07" any linea,' opemtm'T defined on X,

.0" tU'f'

3. A nu flpemt01 T on a n01'1lled space is contmuous 1f and O1dy ,f;r" --jo Xu implies Tx" --jo 1'xo Ix" E X and Xo E X] . 4. The mill space of a nonzcm continuous linea,' opcmtm' is a closed subspacc.

Theorem 2.4.1 ~ct .\' and 1" be 1lOT7Iwd spaces and T a linc(J.1' opemtOf' on X into Y. Thcn the following statemcnts an; equivalent:
1. l' is continuous.

2. T

IS

continuous at the 0'7g17l.

3. T is bounded.

4- 1'5 1 is a bounded subset on".

Theorem 2.4.2 Fm' a bounded linc(J.1' 0l)(;fYltm' T, the followin.fj c07llliti01l.'J me equivalent:
1.

IITxl, } IITII = "'p { Ilxll Ix of' 0

0"

,up {
,,'0

IITxll } IIxll .

2.

IITII = 'ur {kl IITxll < k IIxll} . IITII =

B.

......

3.

"'p IIITxll I 11"11 < I} .

, to<l'._..

....,..",,"l1>"-I

4. IITII ~ ",p {IITxll / IIzll

I)

07>

SUp

IITxU.

11>:11=1

Theorem 2.4.3 The set B IX, Yj of all bounded lincm' 0l)Crat01'S on a nor7ltcd space X mto a llonned space Y is a lloT'1llcd SI)(lCC. If Y is a Banach S1XlGe, then B [X, Yj is also a Banach spQce.

Remark 2.4.4 1. It can be verified that the right-hand side of (1) in Theorem 2.4.2 exist.s, and that IITxl1 IITllllxll holds good for bounded T. 2. In vipw of Theorem 2.4.1, the concepts of continuity and boundcdlles5 for linear operators are equivalent.

s:

Remark 2.4.5 in l'heorem 2.4.3, if we take Y as the Ballne'n space of real /lumbers, then we get the following result. "The set of all bounded linear functional; defined on a lloflTIed space X is a Banach space." This is usually denot.ed by X or Xl and is called the dual SPUCf;~, the cUTlJuyate .~TJace, the adjoint space or fit'st dual space of the normed spaC(> X. X denotes the space of bounded linear functionals of X and is called the second dual of X. For t.he representation of the elements of X for some well known spaces, see Section 2.5.
Proof of Proposition 2.4.1. L T(O + 0) = TO + TO, by the additivity property of T. Also, LHS = T(O). Thus, TO = TO + TO. This implies that TO = O.

2. Since T is linear, the result. is true for n = k; i.e.,

11

= 1,2. Suppose it is true for

T
Then deal'ly
T((frl:l~l

(t a;x;)
=1
..j...

I: a,T(,;) .
=,

fr2X:./

= Q;ITxI

+ fr2Tx2

+ +

+ O'kk) -1- Q;HIXk+J)

+ frkTxj. + O'k+1 Tx k+l;

t.hat is, the result is true for 11 = k + I. By the principle of finite inductioll, the result is true for all finite n. 3. This is a spprial case of Theorem C.2(2) of Appendix C. rn fact, the linearity is not. required.

.n",

"'_'_""""'''''''10''-''

4. Let N be the null space of T =I- 0 such that

x,y E N=> Tx =0 and Ty= 0 => T(;t+y) =0 => x+y EN. x EN=> Tx = 0 => o(Tx) = 0 => T(o:t) = 0 => 0:1: E IV.
Thus, N is a subspace. Let X n E IV and x" -) x. In order to show that N is closed, we need to show that x E N. Since x" E N, Tx.. = 0 for all n. As T is continuous Tx" ---t Tx. so lim TX n = Tx = 0 => x E N.
n-->:I;

I
Proof of Theorem 2.1.1. We shall prove that (1) (3). (3) {::} (4) which means that all arc equivalent.
}

(2), (2) {::}

]. (1) } (2): Suppose T is continuous, winch means that T is continuous at every point of X. Thus, (J) => (2) is obvious. Now suppose that T is continuous at the origin, that i<;, for X n E X with X n ---t 0, Tx" ---t O. This implit'.S that if X n ---t x, then T(x n - x) = Tx" - Tx --t O. That is, for X n --t X in X,Tx" --t Tx ill Y. By Proposition 2.4.]. T is continuous and (2) => (J). 2. (2) :} (3): Suppose that T is continuous at tlw origin. Then X n --t o implies TX n --t 0 by Proposition 2.4.J. Suppose further that T is not bounded. This implies that for each natural number n, we can find a vector X n such t.hat

"'

n II x" II

IITx,,1I

>

1,

By choosing Vn = xn/n IIx,,1I , we see that. Vn ---t 0 but TV" -+> O. This cont.radicts t.he first assumption. ~Iencc, T is bounded wheiJe\'er T is continuous at the origin, i.e., (2) => (3). To pro\'e the converse, assume that T is bounded, i.e., there exists k > 0 such that IITxll -:=; k IIxll. It follows immediat.ely that if XII ---t 0, then TX n ---t O. Thus, (3) => (2).

:1. (3)

Q . ,~._ .._",,"l1>"-'

......

(4): Suppose l' is bounded, that is, there exists k > 0 such that IITxll < k IIxll. From this, it is clear that if lI:tll < 1, then lITxll < k. This means that the image of dosed unit sphere Slunder T is a bounded subset of Y. Thlls, (3) => (4). For the converse, assume that
}

TS I is a bounded subset of Y. This means that TS I is contained in a

closed sphere of radius k centered at the '0' of Y. If x = 0, thcn Tx = x and IITxll ~ k, and the result is proved. In case x =f. 0, IIxll E Sl and so l'

(11:11) ~

k or

IITxll ~

k 11:1:11 This shows that (4) :::} (3).

I
Proof of Theorem 2.4.2. (a) (I) {::} (2): First, we pro\'e that if JlTII is gi\'en by (1), then it is equal to ;nf{kl IITxl1 ~ k IIxll1,

II TII
0'

'up { IITxl1 Ix l' 0 } IIxll

40

IITII > IITxll, x _ IIxll ,m

IITxll < IITllllxll a, TO

Thus,

~0

111'11

is

Ollf'

of th" k's satisfying the r"lat.ion

IITxl1 ::; k Ilxli.

Hence.

1IT11 > mf {klllTxll < k IIxlll

On the other hand, for x f- 0 and for a k satisfying the relation k IIxll , we have III~IIII ::; k. This implies that

(2.4.1)

IITxll

IITxll sup { Wlxt'O }

~k,

or 111'11 < k. Since this relation is satisfied by all k and of :1: and k, we get

111'11

is indepcndent

1IT11 < ;nf (kl IITxll ~ k IIxlll

Dy Equat.ions (2.4.2) and (2.4.3), we h1'l.v"

(2.4.3)

I!TII ~ ;nf{kl IITxll ~ k IIxlll

This proves that (1)
~

(2) .

..... ,

Q . ,>0<1"_""""'''''''IO''-I

(b) (1)

'}

(3), Let

1IT11
Then

,up { IITxll Ix i' 0 } . IIxll

liTxII < IITIIII.'11 < IITII

Thus,

if

IIxll < 1.

,up {IITxll / lI.xll S I} < IITII

Since IITII = sup
such t.hat

(2.4.4)
Xl

{III~~" Ix t- O}' for

an,Y t:

> O. there exists an

t-

0,

IIT"011 Ilxoil > IITII - ,.

By taking y
=: XI /

IIxlli ,(IIYII = I) , we find that

sup (11Txll 11!x1l S I} > IITyll ~

~
1

(II:: II)
Co

IIxollllTxoll > IITII -

Therefore.

sup {IITxll I IIxll < I} 2 IITII

By Equations (2.4.4) <Iud (2.4.5), we find that (1) (e) (1) '} (4), Suppose
<=i'

(2.4.5)

(3).

IITII

sup { IITxll Ix i' 0 } IIxll

Theil, as in the previous case,

IITxll S IITllllxl1 < IITII

if

IIxll -

1,

wI'

Ilxll S 11Th { IITxll / Ilxll ~ 1} '

(2.4.6)

",,,. "'_'_""""'''''''10''-''

Further,

IITx>!1 > IITII _ <, IIx.ll

and

Wlerey= - I

.1:1

IIx.ll

DC

,up { IITxll / IIxll ~ 1 } > IIT"II l!xll

Tim",

<.

,up {

1I~~il / IIxll ~ 1 } > I!TII.

I

(2.4.7)

By Equations (2.4.6) and (2.4.i) we have (I) <* (4).

Proof of Theorem 2.4.3. 1. We introduce in 6[X, VI the operation:. of additiOlI (+) and scalar multiplication (.) in thE" followiug mMner: For T,S E B[X, Y],
(a) (T + S)(x) ~ Tx

+ Sx.
where
0

(b) (oT)x) = oTx,

is a real number.

It call be easily velified that the following relations are satisfied by the above two operations: (;) IfT,SEB[X,Y],thenT+SEB[X,l"].
(i;) (T + S) + U ~ T + (S + U), fo' all T. S. U E BlX. YI.

(iii) There is tm element 0 E 6[X, Yj such that

O+T = l'

for all T E 6[X, Yj

(iv) For e\'ery T E 6[X,Yj, there is a 1'1 E .6[X, VI such that 1'+1'1 = O.
(v)
T+S~S+T
0:1'

(vi)

E S[X,

1'1

for all real

0:

and l' E S[.\'", Y].

",,,. "'_'_""""'''''''10''-''

(vii) o(T + S) == oT"1- oS.

(viii) (0

+ (3)T == oT + pT.
~

(;x) (amT
(x) IT
~

a(jJT).

T.

These relations meau that B[X, YI is a vector space. 2. Now we proWl that

, IITII ~ "'p { IITxll Ixi' 0 } IIxll

is a norm on B[X, Yj and so B[X, YI is a normed space with respect to t.his lIorm or equi\'alent norms p;iven by Theorem 2.4.2. If; is clear that IITII exists. Since {

1I1~~I" Ix t=

o}

is a bounded subset of real Ilumbers, its least

upper bound or sup Inust exist. (a) UTI! ;?: 0 as IITII is the Slip of nonencgative numbers. Let IITII == O. Then IIT:rfl == 0 or Tx :::: 0 for all x E X or T :::: 0, which is the zero element of B[X, Yj. Conversely, assume that T == O. Theil IITxll ==

1I0xll ~ 11011
(b)

0. Th;, mean, that IITII ~ sup {1I1~~11I Ixi' O} ~ 0.

Thus, !!TU == 0 if and only if T == o.

lIaTIl

~ sup {lIal\~~xlI Ixi' o}

{li a (Tx)1I Ix" IIxll ~ "'" {lalllTx ll Ixi' O} 1/11
~
wp

o}

(By applying property (2) of the norm on F)

~ lal "tp {

IITxll } W Ix "

lailiTIl .

Q . ,>0<1"_""""'''''''IO''-I

.... ,

lIT + SII ~ sup {II(T S)(x)1I Ixi' Ilxll IlTx + Sxll } ~ sup { Ilxll Ix " 0

o}
.

By the property (3) of the norm on Y, we have

IITx + Sxll S IITxll + IISxll

This implies that

0'

liT + SII S IITII + IISII

This proves that .o[ X, Y] is a normed space. Now, we pro\'e thM, UrX, YI is a Banadl space providf'd Y is a Banach space. Let {T.. } be a Cauchy sequence 1ll SIX, V]. This means that for" > 0, there exists a natural number N such that liT,., Tmll < (for all n,Jn > N. This implies that, for any fixed \'ector x E X, we have
:l.

IIT"x - Tmxll S II(T" - Tm )lIlkll <, Ilxll,

for n , m > N',

"-->00

that is, {T"x} is a Cauchy Sftluence in }'. 511lce Y is a Banach space, lim T"x = Tx, say. I'\ow, we \'erify that (a) T is a linear operator on X

into Y, (b) T is bounded on X into Y, and (c) I)~, - TJI $"" for 7n > N. (a) Since T is defined for arbitrary x E X. it is all operator on X itJto 1'.

T(x+y) = lim T,,(x+y)

"-->00

= lim [TnT l- Tnyl

.. -->au ,,--> oc " --> 00

as all Tn's are linear

= lim T"x -/- lim T"y

=T;,;+Ty T(ox) = lim T,,(ax) = lim oTnx sinlCe T,,'s are linear ,,--> ex, ,,--> ex, = 0 lim T"x = oTx.

Q.

......

"-->",,,

, to<l'._..

....,..",,"l1>"-I

(L) Since Ttl's are bounded, there exists !If > 0 such that 1I1'"xll ~ AI for all n. This implies that for all n and any x E X, IIT'lxll < I)T"lIl1xJI ~ M Ilxli. Taking the limit. we have
,,-->x

Hm

IITn x11 < AI IIxll .

IITxll < Ai IIxll

lim ["-->00 Tnx
=

Tx. Since norm is a contitmolls function

n-'OO

lim /IT"xll

This proves that T is bounded. (c) Since {T,,} is a Caudly sequcllce, for each t: > 0 t.here exists a posit.ivc integer N such that liT" - T",II < ( for all n,1II > N. Thus, we ha\'e

IITxll.]

IITnx-Tm < IITn -Tmllllxll lIxll xll

0'

fo' n.m> N.

"l~';"

m Il liT - T",II = Slip {1I(T-T )X Ix oF U} ::; (, IIxll

'Ill

for all m

> N;

that is, T", ---t T as

---t

00.

2.4.3

Unbounded Operators
Suppose that T is an opcrator defined as

Example 2.4. 17

Tf
Let

'Yx

ooC'[O.I/.

J"

= sin nx. Then

Ilfnll

sup Ifn(x)1
0<.1'<1

~ 1

foo

all n.

TI,,=ncosnx,

111'1.111=11.

Since 11/,,11 = 1 and T I" increases indefinitely for n ---t 00, there is no finite constant k snch thaI 111'/11 < kllJJI for all f E GI[O, 11 Thus, Tis an unbounded operator.

Q.

..... ,

, t><l'._..

....,..",,"l1>"-I

Definition 2.4.2 An operatorT: X -+ F is called bounlLcd below if alld only if there exists a cOllstant 1lI > 0 such t,hat

IITxll > AI IIxllx' V, E X.

The differential operator considered ill Example 2.4.fJ is IJIlbounded on ColO, II = {f E C [0, 11/ /(0) : /(1) = O} as well. However, it is bounded below on Co [0, 1] as follows: For / E C [0, L]

I(:c) =

'df f -ds < Slip ~ o ds 0S"9 (Ix

Id

or

IITIII

sup
O<x<1

I/(x)1 = 11111,

that is, T is bounded below.

Example f2.4.18 Suppose T : X -+ X is continuous, where X is a Banach space. Let..\ bf' a scalar. Then the operator..\1 - T, where f is the identity operator, is bounded below for sufficiently large..\ Ve17.fication.

II(AI - T)xll

IIAlx - Txll 2 1I('\xlll-IITxll

Since T is bounded, lITxll ::; k IIxli. Hence, II(..\/x - Tx)1I ~ (1..\1 - k) This implies that (..\/ - T) is bounded below for sufficiently large..\.

Ilxll

An important feature of line.u bounded below operal.Ors is that e\'ll though they may not be continuous, t.hey always ha\'e a continuous inverse defined on their range. More precisely, we have following theorem.
Theorem 2.4.1 Let T be a linear boundcd below operator from a rw1Tlied spau X into a nonnclL spar./'. Y. Then T has a continuous invc,'sc T- 1 Imm its mnge R(T) into X. Conve"sely, if there is a wnl.inuous inve,'se T-l : R(T) -+ X, then then, is a ]JU,sltive co'nstu.nt M ,such that

IITxlly

Q.

......

Ilxll x /01'

every x E X.

, toOl'._..

....,..",,"l1>"-I

2.5

Representation of Bounded and Linear Fllnctionals

Represmauon of elements of the dual space X of a lJorrned lill~ar space which are bounded and linear fUllctionals has been studied extensively_ Through this concept we define a new class of Banach spaces called the r<'flexive Banach spaces..4 Banadl space X is called the TcJlexive B07lo(;h. spa(;c if the second dual of X is equal to X; that is, (XT = X" = X (Equality in the sense of isometric and isonlorphic explained earlier). A mapping J : X -+ X" defined by .l(x) = FrC! where F:r:(f) = f(x) 'rf f E X', is called the rw.tuml embcllding. The Hungarian tvlathematician Riesz found a general representation of a bounded alld linear functional on a Hilbert space whicll is discllssed in Chapter 3. Here, we give a representation of the elements of the dual spa<-"CS of R", f p , L2 [0, I] and Lp \Ve also indicate a close relationship between the basis of a normed space X and the basis of its dual X'.

Example 2.5.1 in the form

Any element F of the dual spa<-" of Fr' can be written

F(xl = LCt;X"

(2.5.1)

'00 I
where x =
(XI,X2,'"
n

,x,,) E R",

n =

(CtI,t:X2,'.'

.on) E R". In fact.

F(x) = (x, Y) =

L X;Y;; that is, for every x E R'l, there exists a unique

,=1

y E Fr' such that the value of a bounded and linear functional F can be expressed as the "inner product". ((x, y) is called the inner product of x and y and will be studied in Chapter 3).
Example 2,5,2

Let X = L 2 [0.11 and define functional F on .\ by

(2.5.2)

and .rJ is an arbitmry function. F is linear and bounded and, hCllce an element of (LdO, tJr .

Q.

......

, to<l'._..

....,..",,"l1>"-I

Example 2.5.3

FE (L"f can be defined by

P(J) =

/IX)9Ixl<1x,

-+(J

1 7)

1.

(2.5.31

Example 2.5.4 tional Fi bJ

Let {l{!,} be a basis of X. DefinlO a boundoo linear flllll;-

"-"(:1:) = O'i E fl,

where x = Lo,l{!,.

"

,-,

(2.5.4)

It can be easily checked that F. is linear and bounded. The relationship between {F;} and {l{!i} is characterized by
F,(l{!j) = 0,;

for all i,j.

(2.5.5)

Since 1fJ) are linearly independent, F, are also linearly independent. Thus, the set {F,} forms a ba..,is for the dual space )( *. {F,} is called the d'u.al basis 01' conjugate basis.

Example 2.5.5 Let V be then-dimensional subspaceofL2 [O, I], spanned by the set {1fJ;}:~1 and F be the bounded and linear fUllctional defined by Equation (2.5.2). Then for! E Y, we have
F(J) =

II
;=1

!(x)g(x)dx

10' (t,a,op'(XI) .(xldx

f:a, 10'
0

"(x)op,(x)dx

P(J) = Lo,B"
;=1

"

(2.5.6)

2.6

Algebra of Operators

1. A vector space X, in which multiplication is defiJled having the following properties, is called an algebra

Definition 2.6.1

...". "'_'_""""'''''''10''-''

a. X(yz) = (xy)z

Vx,y,zEX. Vx,y,zE X.

b. x(y+z) =xy+xz

c. (x+y)z=xz+yz Vx,y,zEX.
d. o(XV)
=

(ox)V = x(oV),

for every scalar a and X,V E X.

An algebra is called commutative if

xy = yx

"Ix, V E X

An element e E X is called the identit.y of X if

e:r
Xf'

Vx E X.

2. A Banach algcbm X is a Banach space which is also an algebra such that

IIxyll s I"IIIIYII
Rem01'k 2.6. J

.x, y E X.

If X has an idcntlt.y c, then

In such cases, we often suppose that

lIeli =

1.
IS

Remm'k 2.6.2 In a Banach algebra, the multlphcatlon uous, that is, if Xn ---+ x and Vn ---+ V, thcn x"y" --t ;l11

jomtly contIn-

Ve7ification.
IIXnY" -

\Ve have

xyll

= IIxn(Vn - V)

+ (x n - x) yll < IIx"IIIIY" - yll + 11 x" - xlillyll--> U as n --> 00,

lIx n ll < IIxU n .

which gives the desired result.

Remmn 2.6.3

For cvery element x of a Banach algebra,

Verification. n = m. Then

The result is dcarly true for n = 1. Let it be true for

IIx"'+' II

IIx"'xll S IIx"'lIlIxll S IIxll'" IIxll ~ IIxll"'+' ,

.n". "'_'_""""'''''''10''-''

or IIxnl+111 < Ilxll +' ; i.e., the result is true for m + 1, and thus by the principle of induction, the desired result is true for every n. Thcorcm 2.ti.l
1f.1":
I.~

nl

an element of a Banach algebm.\', Uten the sene$

L x" is ,

r:Ofl1JeT'!}ent

whenever

Ilxll <

1.

For proof of this tllL-'Orem, we need the following lemma. LcnuTia 2.6.1
Let.
XI, X2, . ,

x n , . .. bc a sequence of elements m a Ba-

naell s/Jace X. Then the SeT"1CS

L x"
n=1

is ContlcJ1}cnt in X; that is,

It n

L
k=l

Xk

IS converycnt in X whenever the series of real nllmbcn

L
n=1

IIx,,1I

is

COfltJcrgent.

Proof of Lemma 2.6.1.

\Ve have that

, L
k=p

Xk

is convergent. Then of

L IIXkll-7 0 as 1).(1 -700 and. in turn, the remainder

k=p

:; L

k=p

IIx,ll If L "x""
,.,=1

=,

00

Xn

tends

10

zero. Therefore,

,,=1

00

Xn

is convCJ'gent

I
~

Proof of ThCOl'em 2.6.1. Since IIx"n ~

IIxlI" for all n, therefore, L Ilx"H

..=1

is less than e(jual to the sum of a convergent geometric serie1>. By Lemma

2.6.1,

LX" is convergent ,

Definition 2.6.2 Let B (X) denote the set of all bounded linear operators on a normed space X into itself and S,T E B(X). The multiplication in B (X) is definL--d as follows:

51'(x)

5(1'(x)),

Vx E X.

B.

......

T E B(X) is called mveytible if there exists T-I E B(X) called InlJCrse of

, to<l',_"""",,,,,"l1>"-I

T such that

In view of Theorem 2.4.3, 13 (Xl is a norme<l space amI it is a Bannch space whenever X is a Banach space.
Theorelll 2.6.2

Let X be a Banach SIJat:c. Then 13 (X) is a Banach algebra with T'eS1Ject to thc mllltiIJlication defined in Definition 2.6.2 with identity 1 stich that 11111 = I.
Dy Theorem 2.4.3, 13(X) is a Banach space with respect to the

PT'Oof. norm

IITII

IIxll=1

sup {IIT(x)11J

liT (x)1I } { i1:rU . ~~~

13 (X) is an algebra

<l.'3

the following results can be easily verified:

U(S+T)~US+UT} (S + T)U ~ SU + TU ,VU, S, T E llIX). U(ST) ~ (US)T

a(ST) = (as)T = S(aT) for every scalar a and all S, T E 13(.\'). We have

IISTII ~ sup { IIST(x)1I } #0 IIxl. ~ sup [IISIT(x)lIII1 T(x)lIj. ,." IIT(x) II IIxll
By definition,

liS/I ~ sup {"S(~')"}.

,'to

IIx II

If x' = T(x), then clearly T(x) f:- 0; otherwise USTU = 0 and the dt'-'sired result is obvioU5. In case T(x) f:- 0, we gP-t

S[T(x)]

B.

.... ,

IIT(x)1I <: IISII,

, to<l'._..

....,..",,"l1>"-I

and

118TII < 11811 :~~ { II~~~! II }

Therefore, B(Xl is a Banach algebra.
I(x) = x, '<Ix E X

~ 1181111TII.

11111

,up 1I/(x)11 11"'11=1

,up (lIxlll 11<:11=\

~ L

I
Theorem 2.6.3 If X is a Danach S!Jace and T E B(X) such that 1, then the o!Jcmtor J - T i~ imwrlihle.

IITII <

Proof.

We have

(1- T)(I + T-rT'l -r ... +T") = (I -r T + T'l -r ... + Tn) - (T + T'l =I_r,,+l,
and

+r 3 T

T,,+l)

II Tn-HII

< IITII,,-H -) 0 as n
n~oc

---t 00,

by

Ihp hypOt.JlL-'Sis

IITII < 1.

Hence,

lim (J - T)(Pn ) = I,

where
n

Sinr'e IITII < l. Nul B(X) is a Banach algebra, hy Theorem 2,f1.1, Hm P.,
n-,O;O

exists (say P = lim POl); therefore,

" ..... 00

(I-T)p~

In fact,

(I -T)P

~
~

(I - T)Pn T (I-T)(P- PH)

I +(I-T)(P- PH).

Bill

11(1 -

T)(P - Pn )1I ~

III -

Til liP - Pnll

-7

0 as n

-7 00,

.n",

'"'_'_""""'''''''10''-''

therefore,

(I-T)P
Thu~,

1.

(I - T) il> invertible and (1- T)-I = P, where

P=:1+T+T2 + ... ,T n + ..

I
Example 2.6.1 algebra. FOI
The set of all n x n (n fiwte) matrices is a llanach

..\ -

we can consider the following equh'alem norms (I is the identity matrix): a.

b.

IIAII = SUplaiJI IIAII ~

where 11111 = I
whe"p

L lai,1
i,J

11/11 ~ n

where 11111 =

-Jii-

Example 2.6.2

Let T E B(X) be defille<llIs follows:

I- -

T"
n!

+_

Then

lI e'rllB(x)

< I1III 6Ix ) + II T II 6 ,x) + II T "IIBlx)

1IT'lIu(x)
2!

+... +

,. r

+ ...
for all n .

Since B(.\") is a Banach algebra,

IIT"II < IITlirl

B.

......

, to<l'._..

....,..",,"l1>"-I

where

Therefore ,

,T E SeX).
Example 2.6.3
The operator equation

Tx = y,
has a unique solution x = T-ly if r- I exists and it is hounded.
It ma,}' he observL--d that finding the inverse T- 1 could be quite difficult Lut t.he inverse uf a neighburing operator To (1 0 is dose to T in the sellse that liT - Toll < IIT~ [I/") can be found as follows: Given this condition it

can be shown that

liTo-'ll' liT - Toll I liTo 'II liT Toll' IIx - Toll

- To-'vll < IIT-' T,-' III1YII IITo II' liT - Toll llyll -' < I -liTo' II liT - Toli ,
II T -'y

which gives an estimate for the error Ilx - xoll. Suppose T is a square matrix and the elementl; of the matrix are subject to error which are at most of magnitude f. H To is the matrix actually used for computation, we can assess the possible error in the solution.

2.7 Convex Functionals

III this sf'Ct.ion, we shall discuss the fundamental properties of convex sets,
affine operators and convex fUllctionals .

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

2.7.1

Convex Sets

Definition 2.7. J Let X be a normed space and M its subset. Theil 1. 111 it! called amvex if ux + /3y E A[ whenever J:, y E AI, where a > 0, /3>Oanda+/3= 1 2 Al is calk-'d affine if ox+!3y E AI whenever x,y E AI, where 0+/3 = 1.
Rem7'k 2.7. J

for all
n

Xi

1. A sub~t M of a nurll1\:ld topat:e is mnvex if and only if !I/, i = 1,2, .. ,n, feal 0, > 0, i = 1,2, ... ,n;

La. = l'LO"x, E M .
=1 i=1

2. Every affine set is convex bl1t the converse may not be true. 3. The normed space X, null set 1>, and subsets consisting of a single element of X are convex sets. In RZ , line segments, interiors of triangles and ellipses are its ('onvex subs!'l,s. A unit hail in My nonnecl space is it.. convex subset. 4. The closure of a convex subset of a normoo space X is convex. 5. If T is a linear operator on the normed linear space X, then the nnaf:,'C of a convex subset of X under T is also convex.

Properties of Convex Sets

I. Let {A.} be a 5e(luence of convex subsets of a normed space X. Then,

(a)

n
=
.=1

A, is a convex subset of X

(b) If A, ~ A;+I,i = 1,2, ... , then

U A,
=1

IS

convex.

(c) liminf .-1; =

H=

U nAi is oonvex.
= =
j=1 '=3

2 For any subsets A and B of a nOlllled space X, SUppOSl' that A+8= (x+y/x E A,YE B) ,and O'A= {ox/x E A and a reall. If.-1 and B are convex, then (a) A Band uA, (b) A = aA + (1- a)A for n ::; 1, and (c) (01 + o.,)A = al A + a., A for 0'1 > O,O'z > 0 are convex.

::;

.n". "'_'_""""'''''''10''-''

Definition 2.7.2 Let 111 be a subset of a norme<1 space X. Then, the intersection of all convex subsets of X containing /If is the smallest convex subset of X containing M. This is calk.od the convex Iltill or the convex envelope of M and is usually denoted by CoM. [CoM = n{AdAj's are convex and /If <;::; A,}]. The closure of the convex hull of M; that is, CoM is cal)(ld the dosed convex hull of M.

Properties of Convex Hull

I. The convex hull of the subset M of a normc<l space X consists of all vectors of the form
n

" L
;=1

O,X;,

where the x E M " o ,

> - 0 are real for

i = 1,2, ...

,n and Lo. = ]
;=1

2. let A = n{!l-I/M:> N, AI convex and closed} In other words, A is the intersection of all closed and convex sets containing N. Let Nr; be equal to the convex hull of N. Then A = N r; lMazlll' lemma (for the proof SL"'e Dunford and Schwartz [2])]. 3. Let A. be a compact subset of a Banach space, then GnA is compact.

Definition 2.7,:] let X be a normed space and F a nontrivial fixed functional 011 .Y. Then
J. H = {x E X / F( x) =
C,

r is a real constant} is ('aIled a hyt)crplanc in

x.
2. E .'\/F(x) < c, C is a real constant}, and H 2 = (x E X/F(x) > C, is a real constant} are called half-spaces determined by J-J, wherp 11 is defined in (I).
HI

= lx

Example 2,7.1

lines are hyperplanes in

n2 and planes are hyperplanes

. R' III . Definitwn 2.7.4 I. A hyperplane H IS called the stlpport of U at Xo E l' if Xo E H and U is a subset of one of the half-spaces determined by H. 2 A point Xo of a convex set. Af is called an exf;rf'mc point if there do not exist points XI,X2 E /II and E (0, I) such that Xo = ax) + (1- a)x2'

B.

."

...

, to<l'._..

....,..",,"l1>"-I

Remar* 2.7.2 The extreme points of a closed ball are its boundar)' points. A half-space has no ext.reme point even if it is c1ost.-'(\

2.7.2

Affine Operator

Definihon 2.7.5 An operator T on a nonned space X into a normed space Y is called affine if for every x E X, Tx = Sx + 0, where 5 is a linear operator and 0 is a constant in Y. Remark 2.7.3 U X = 1; = R., then Sx = ax, for x E X and real n. In this case, affine operators (affine functionals) are dfficribed by Tx = nx + b. If 1; = n, T is called an affine fundionill
Theorell1 2.7.1

An operator T is affine if and only if

(2.7.1)

for every choice of Xi E X and real a f

'!I

such that

" L n; =

=,

I.

The proof is a straightforward verification of axioms.

Convex Functionals
Definition 2.7.6 Let J( be a convex subsct of a normed space X. Then a functional F defint.-'(\ on K is called a conve.y, functional if F[ax

+ (1

- n)yJ

< nFx + (1

- n)Fy "In E [O,IJ

and Vx,y E J(

F is called stnctly convex if

F[ax+(I-n)y]<aFx+/l-a)f'yVx.yEf(
F is c1111('(1 function .
CorlOOI!P

and

nE[O,I]

if -F is convex. If X = R'l, F is called a cGrltJex

B . ,Jo<l"_""""'''''''l1>''-I

......

Rem01'k 2.7.4
OI,OZ, ,On

If F is a convex funct.ioll1'l1 on K, Xi> Xz ... :t: n E K, and

are positive real numbers such that.

LOi = I, theu
.=1

(2.7.2) Equation (2.7.2) is cal1ed Jensen's inequality. Sometimes this relation b taken as the dP-finit.ion of a convex functional. In view of Throrem 2.7.1, every affine functional IS convex. The converse of this may not be true. Every linear functional is COli vex.

Example 2.7.2
I. I(x) =

x E (-1,1) f( 1) = f(l) = 2. The function defined in this manner is COIIVCX on [-1,1\.

X

2.

I (x)

= y2 is a convex function on (-00,00).

3. I(x) = sin x is a convex function on [-r., 0].

4. f(x) = Ixi is a eonvex fllnc.Uon on (-00,00) .

5.

I (x) =

eJ" is a convex function on (-00, 00) .

6. f(x) = -logx is a convex function on (0,00).

7. I(x) = xl' for p > 1 and I(x) functionals on [0,00).

-xl', for 0

<

< 1, are convex

8.

I (x)

= x log x is a eonvex funct.ion on (0.00) .

X Z

9. f(x,y) =

+ 3xy + 5112

is a convex fundion on a subset of R 2

LO. l(xI, X2 . .. ,x n ) = Xj is a convex funCl.ion on R.n.

11.

I (XI, X2 ... ,x,,) =

Xn) =

L
=1
c

IXi + b, I, r < n is a convex function on R".

12. l(xI, X2

L 0,
i=1

IXi

+ bill' ,0, > O,p >

I, r < n is a convex

function on F("

B.

.....

, to<l'._..

....,..",,"l1>"-I

13. g(x)

=::

StJP{L:X;y,/y
;=1
j'OIlH'!X

=::

(Yl,Y2, ... ,Yn) belongs to a collvex subset

R"} is a
14. g(x)
=:

fUllction

inf

Of.r. E >.M, whPl"e AI is a

COilveX

slIhsp.t. of R"} is

fI

COli vex fUlIction.

15. g(x) =: inf {llx - yll/y =: (Yl' Y2, ... ,y,,) belongs to a convex subset AI of Rt"]} is a convex function.

Definition 2.7.7 Let F be a functional on a llormed space X into R. In other words, let F be a real-valued fUllctiulI defined 011 X which may t.ake
the values
+00

and

-00.

The set

epi F == Hx.a) E X x RIF(x) < o} is called the ep1!J1Y1]Jh of F. Let !{ be a convex subset of X, then F . all x and y of J(, we have
f{ -~

R is called convex if for

F (AX + (1 - A) y) S AF(x)

+ (J

- A) F(y) 1m

,ul

AE [0, II,

while the expression on thf' right-hand side must be defined. dam F {x E XjF(x) < oo} is called lhe effective domain of P.

It may be observed that the projection of cpi P on X is the effective

rlutnain of P.

Theorem 2.7.2 Let X be a nOf'1lled ..pace (we am take only It vector space). Then, F: X --4 R is ronm::x if and only if pi F is convex.
Proof. Let F be convex and (x,a) and (y,{J) be in cpi F. By tbe definition of cpi F, F(x) ::; 0 < 00 and F(y) < {J < 00. Theil we have

F(AX

+ (1- A)Y) < AF(x) + (I - A) F(y) < ),0 + (1 ),)13.

This implies that

(),x

+ (I

- ),) y, )..0

+ (1

- ,\) IJ) = )..{x, a)

+ (1 -

,\) (y, iJ) E epi F,

and so epi F is the convex subset of X x R.

Q.

......

, to<l'._..

....,..",,"l1>"-I

For t.he converse, suppose epi F is convex. Then dom F is also convex. For x and y E dom F, F(x) < u and F(y) < (J. By hypothesis, "-(x,a) + (1- >.)(y,(J) E epiF for "- E [0,1), which implies that
P (Ax

+ (1

- -') y) S AQ

+ (1 -

-'lB.

If F(x) and F(y) are finite, it is sufficient to take a = F(x) and (J = F(y). Tn casf' F(x} or F(lI) arp -co, it is sufficient t.o take a or 11 tpndillg to -co and we obtain that
F(>'x

+ (1- A)")

S AF(x)

+ (1- A) F(y).

Thus. F is

COil vex.

Properties of Convex Functionals

I. If F and G are convex funetionals, then (a) F max (F, G) are also convex functionals.

+ G, and

(b) F V G =

2. If F is a convex functional, then uF is also a convex functional, where a is a nonncgativc real number.
3. If {Fo } is a sequence of COnvex functionals defined on the sequeJlce of COnvex subsets {Mil} of a nonned space X and /If = nAfo :f:. 1>, then the subset of At on which Fx = sup Fox < co is convex and F
i.<; convex on it.

"

4. The limit of a sequence of convex functionals is convex.

5. Let F : X ---+ Rand G : }. ---+ R be convex functl0nals defined on a Hormed space X and a subspace }/" of R such that the range of F ~ y ~ Rand G is iucrcasin.c;. Then Go F is a convex functional on X.
6. If F is a convex functional, then G(x) = F(x) + H(x) + a (where f1 is a linear functional and a is a constant) is also a convcx functional. 7. If F(x) = U(x) + a, where G is a linear fUllctional and a is constant. then IFI P for p ~ I is also a convex functional.

Continuit:r of Convex Functionals

It may be 110ted that a convex fUllctional nP.ed 110t be contIllUOUS.

",,,. "'_'_""""'''''''10''-''

2.7.3

Lower Semicontilluous(Isc) and Upper Semicontinuous (usc) Functionals

Let. X be a metrizflble space. A reaJ-valued fUllction f defined on X is called lowe,' semico1ltinuous (Isc) at a point Xo E X if f(xo) < lim f(Yn) whp.never Yn --4 Xo ;IS n --+ oc f is called Isc on X if f
n=rn;

Definition 2.7.8

is Isc at overy point of X. f is called upper semicontinuous (usc) at Xo E X if - f is lsc at xo. f is called usc on X if it is usc at each point of X.

Remark 2.7.5

I. Every continuous functIon is lower and upper semi-

continuous, but the converse may not be true. 2. Every convex lsc function. defined on an open subset of a Banach space, is continuous, TheorelTl 2.7.3 Let X be a metrizable spaCt: and F ; X --+ R, l7lCfl F 1S /sr. if find (Jnl,J if the epigraph (Jf F is cl(J!,('fj in X x R.

Proof. Suppose epi F is clC6Cd and let x" --+ x. \V.... want to 'ihow that P is Isc, i.e.

Let
(} = lim F(x,,)
>ROO

1. If () =

00,

t.lwn F(x) <

VJ." E X and so tJw dp.sir....l! result is t.rup

2. Let - ( X l < a < (Xl. Choose:l snbSetjllllCe {x",} such that P(x m ) --+ (Xl as 'In --+ (Xl. Then (x n " P(x",)) E epi F. This implies that F(x) ~ Q = lim F(x,,) and we get the desired result.
..-kG

3 Let Q = - ( X l . Choose a subsequence {x rn } such that F(x m ) converges decrea.<;ingly to - ( X l as m --t (Xl. Then (:I"m, P(x m)) E epi F for m ~ mo, since F(x n ) ~ P(x mo ) for m ~ 1110 and (x mo F(x mo )) --4 (:t, F(x mo )) for 111 --+ (Xl. Hence, (x, P(x mo ) E epi P or F(x) < P(x IllO ) for every mo. But then F(x) ~ Q = -ex, and t.his is a contradiction. Hence, F(x) = Q = - ( X l .

...". "'_'_""""'''''''10''--'

For the converse, suppose F is Isc, and (x n , an) -t (x, 0)

F(x)

(1.<;

n -t 00 with

< lim F(;rn) < lim

0'"

Q.

This implies that (x,o) E epi F, and epi F is closed in X x R Thus, we have provPd t,llP rlesiroo resnlt.. I

2.8
2.8.1

Problems:
Solvoo Problems

P1'Obiem 2.8.1
1. Show that every normed space is a metric space but tim I. the converse

may not be true. 2. Show that

(a) Illxll- lIylll < IIx -"II

(bl

IlIxlI-II"111 ~ Ilx+"1I
for all x, y belonging to a normed space X.

3. Prove t.hat. the mapping!>

J(x,yJ = x+y on X x X illto X,

and
'~,(a,x)

=ox

on R x X into.X,
dll-'

where X is a normed continuous.

Sl:>a(;f'

and R i!>

field of real numbers, arp

4. Prove that a norm is a real-valued continuous function.

S'ohttion. L Let. (X, 11-11) be a normoo space. Let d(x,y) = IIx-yll, x,y E X. d(,) is well defined as x-v E X and so IIx-vll is defined.
(i) d(x, V) ? 0 as
d(x, y) = o{::} IIx - yll = () {::} x - y = 0 (by the first condition on tbe norm). Tberefore, d(x, y) = 0 if and only if .T = y.

IIx - yll > 0, by tbe first property of the norm.

(;i) d(x,Y) 'II' - yll ~ II-(y ~ II. - xii ~ d(y,x).

(iii) d(x, Y)

x)1I

I-lillY - xII

liz - z + Z - yll < IIx - zll + liz - yll

=::
011

(by the third conditioll

the norm).

Thus, d(x,y) < d(x,z) + d(z,y). Therefore, all the conditlOlIS of the metric are satisfied. So, d(,) is a metric. In ortler to show that the COII\'erSe is not true, consider tile following example: Let. X be a spa~ of all complex sequences {x,} and d(,) a metric r1efiued on X as follows:

d(x, y) ~ L., 2; I
.=1

f-.

+ IX,

Ix; - y;1

y,

I'

where x = tXl, X2, .. , X n , ... ) and y = (VI, Y'l , ... ,Yll'" . ). In fact, d(.,) satisfies all t.he Unce conditions of the metric.

(i) d(x,Y) > O,d(x,y) = 0 if and ouly if x = y.

Verification.
Consequently,

lx, l+lx.

Yil

v'l,an >o ~ d 8 01 IXi-YilYil > o Vi. y,l 2'1+lx;

d(x,y) -

>0 - L 21+lxYI i=l .,

-

(Xl

IX; -

Vii

If d(;I;, y) = 0, then

~~

lx, - Yil _ ~ 2' I + lx, -y,1 => => =>

x

IXj-Vil -O~ vl J + IXi v.I IXi-Yil=OVi

~

Xi

=::

Vi Vi

== (:tl, :(.2, ,xn , ... ) == V == (YI ,Y2,. ,Vn,"')" Conversely, if X ==y, tllell lx, -y,1 == OVi and d(J;,V) == O.

...... '"'_'_""""'''''''10''-''

(ii) d(x,Y)

d(y,x).

Verification.
OC '"" 1 Ix, - y, d(x, V) ~ L 2' 1 + Ix _ y

'=J

.,

- :S'
~
~
,''''\

~ 1 1(-I)llx,-Y;1 2' 1 + I lllx; vii

ly;-x,1 L2'I+lv -xl

~ 1
I '

d(y, x)
X=(Xl.Xl, ... X" ... )

(iii) d(x,y) < d(x,z) +d(z,y) wbere

y= (YI,U2, . . lUll"')
Z

(21.22 ... 2 11 " , )

Ver'ijication.
1.

d(x
,V)

~ ~

f +lx,lx---z, ++z,_Y" I < f: I lx, - "I + ~ -'. 2'1+lx, zil

1

~2'1+lx;
2' 1

Ix,-y,1
VII

;=1

zI

z - y ,

.=1

~2il+lzi-y.1 .=1

I>, - v;1

by applying tlw ineCjllaJity given in Theorem D.3 of Appendix D

Thus,
d(,,y) < d(x,z) +d("y).
TheJ"efore, (X,d) is a metric space. This metric space cannot be a normcd space Lecause if there b a 1101'11I IHI such that d(x,Y) IIx ~ vII , t.hen

d(ox,oy) ~ II ox -

oyll ~ lollix - vII ~ 101 d(x, V)

",,,. '"'_'_""""'''''''10''-''

must be satisfied. But for the met.ric under consideration, this relation is not valid as

d(ox, oy) = ::-: 2; 1 + 101

and

f-

101 lx, - y,l lx, y;I'

101 d(x, y)
2.

= I Ix; - v,1 101 L: 2' 1 + Ix--y- I . ~

1=1 "

(a) Ilxll

II" - y + vII S IIx - yll +IIyll, 0>' Ilxll - lIyll SIlx . yll

Also,

lIyll - Ilxll < IIx - yll , -(llxll -llylD SIIx - vII

Tim!>, we get

(2.8.1)
(b) In (a), replace y by -y. Then, we get

IIxll - II - yll S IIx + yll

Since

(2.8.2)

11- yll ~ lIyll, IlIxll-ilylll S IIx + yll,

which is the desired result.

(2.8.3)

3. In order to prove that is continuous, we need to show for f > 0, 3 0 0, such that II (x, y) - (x' ,1/)11 < f, whenever II (x, y) - (x', y')11 0, or equivalently

> <

lI(x - x',y - y'}11

Ilx - x'il + lIy - vII < b.

",,,. "'_'_""""'''''''10''-''

and we have

II (x, y) -

~ (x',

y')11

lI(x + V) - (x' + y')11 ~ lI(x - x') + (y - y')11 < IIx - x'il + II. - .'11 < b ~ .
~

This shows that 1> is continuous. Similarly, 1jJ is continuous as

1I"'(o,x)-.,(~,x')1I ~ 1I0x-~x'II~llox-ox'+ox' ~x'il

lolllx x'il + 10 13111x'll <',

~,x

whenever

lI(o,x) 4.

(~,x')11 ~

11(0 -

x')11

10 - 01 + IIx - x'il < b.

S Ilx - YII. This
I':

By Solved Problem 2.8.1(21. for all x.

Ilixli -llyll]

implk>s that if IIx - yll < 8. then Illxll-HYIlI < 1':. wheJ'e shows that Ilxll is a real-valued continuous functioll.

o. This

A lternativc Solution. If X'l --t X ill; 1t ---t 00; Le., IIx n - xII --t 0 ill; n --t 00, then IIx,,1I --t IIxll in view of the above relation. Thus, by Theorem C.2(2) of Appendix C, IIxll is a real-valued continuous function.

Probie'm 2.8.2 Prove that the set XC of all comugent Sl-'<juem;es in a Ilormer! space X is a /lormed space. Solution.
(XI,:I'2,.

We can easily verify that Xc is a vector space. For x = ,xn , ... ) E Xc, let IIxll = sup IIxn ll. The right-hand side is well
0 a,

" defined as every convergent seCjuellce is bounded.

L

IIxll >

IIx,,1I > O.lIxll

sup Jlx" II = 0 {:::} I)x'lll

0 if aud OVn {:::} x = O.

ooly

if

x ~

O. IIxll

"
3.

Ilx + yll ~ ,up IIx" + Y"II < ,up IIx,,1I Hup IIY" II
II '1 II

IIxll + lIyll .

Problem 2.8.3
the norm

11/11 =

l (

1. Show that C[a,b] is a tlorllled space with re:>pect to

lf (t)1 dt

) -I' but not a Banach space.

" .." "'_'_""""'''''''10''-''

2. Show that R", '-Tn C, C [a, bJ (wit.h sup norm) and BV [a, b] are Banach spaces.

Solo'ion.

1. (a)

IIfll

2 0 ru;

fIt)

[It may be observed that the continuity of f implies the continuity of If!
liftOI Z dt
b

o.

If(t)I' dt

o. Ilfll ~ 0 $> If(t)I' ~ 0 $>

aud so

= 0 implies f(t) = 0 on [a, III] .

(b) 1I0fil
(e)
=

~ ( [ 10J('11' dt) 'I' ~ lal ( [ IN)I' d') >/, ~ loillfli. ~ (t If + g" dt) 'I' < ( [ ,fl' dt) 'I' + ([Igl' dt) 'I'

IIf +gll

by Minkowski's inequality (Th. D.5(b) of Appendix D). Thus, CIa, b] is a uorllled space as it is well known that C la, b] is a \'eCwr span> [Tn fRct, for operat.ions of addition and scalar multiplication, defined
by

11111 + Ilgll

(f + g)(t)

f(t.) + g(t)

(o!!(t) = of(t)
the a.doms of vector space follow from the well-known properties of continuous fUllctions.] In order to show that C [a, bJ is not a Banach space with the illtegral Ilorm, WP I~onsider the following example: Take ft = -I, b == 1

-1<t$O I,,(t) == nt. 10 < t :S ~ { I, Ii < t $ 1.

We can show that {fll( t)} is a Cauchy seCjuence in C [-1, 1J but it can not converge to an element of C [-1, 1]. (See Solved Problem 3.10.3). 2. (a) nil is R Banach space. R" is a vector sp~cP with the opf'rations Vx == (XI,X:.!> .. ,.r",),y == (Yl,Y2,." ,Yn) E:: Rn
x

O'

+ Y ==
OX

(Xl

+ YI ,X:l + Yz, ... ,X" + Yn), nil is a Banach

Rn .

==

(n:XJ,OX2, ... ,OX,,)

We verify a."doms of the norm of 1111.1 of Example 2.2.4. space with respect to all norms defined on it.

......

IIxll == max (Ixd ,IX21 ,... ,1:1:" I) ,x == (xt, xz, ... ,xn) E

Q.o,>o<I"_"""""""IO"-I

II xII 2': 0 is obvious.

lIaxll
:= :=

IIxll

:=

0 {:} Ix. I := 0\1, {:} x

:=

(XI, Xz, ., x"J

:=

O.

min (loxII, !oxzl, , lox.,!) 10Imax(IJ:II,. ,Ixnl)

101 IIxii

IIx + yll = max (lXl + Yll , 1:I:z + Yzl , ... , Ix" ;- Yn!) S ma.x(lxII;- IYII, IX~I + Iml, ,. , Ixnl + IY.d) < ma.,,(IxII, ,lx,,!)+max(IYII, ,IY,,!)
~

IIxll T 11"11
00.

(b) t", =

{x:= {X;}/~I.r.I'" < oo} where I S p<

J, Y =

l", is a vector space with respa:t to operations of additl01l ami scalar multiplication defined as follows:

x = (XI, Xz, ... , x", x+y= (Xl+1Jl,Xz+m,

(V" Yz,

, Y", ... ) E l""

,x"+y,,

),

ax::::: (OXl,OXZ, ... ,ox", ... )

By r."linkowski's inequality and the fact t.hat.

L Ix,l' < L ly,l' <

00,

00

00

00.

;=1

.=1

we have

L
;=1

lx, + y,l' s L Ix,j' + L ly,I'

;=1
00

;=1
00

This shows that x

+y

E C". Since

L lox,l"
.=1

:=

101"

L Ix;!" <
.=1

00.

V scalar

a, we see that ox E C". All the axioms of vector space can be verified easily. Now, we verify the noom axiom' foe the nmm defined by

IIxll ~

(f;

IXil ')

'f' on I,.

.."" . '_'_""""'''''''10''-''

It is clear that.

IIxll ;::: o.
IIxll II rrxll
~ 0

<> Ix,I'

~ (~lrrX;I'
~

r
0\1',

{=}

x, = O\l'i

{=}

== O.

~ (IQI"~IXd'r
Irrlllxll
( [ ; IX, T

IIx + Yll

11,1') 'I' exi", au"

(~lx.I')'I, + (~11Id') 'I,

in view of t.he t-.'1inkowski's inequality and the fact that x = {xd E t p , and y = {yd E t p ' This shows that IIx + yll < IIxll + lIyll In order t.o show that t p is a Banach space, WI' need to show that pvery Cauchy sequence III t p is wnvergent. in it.. let x" = {an E t p be a Cauclw seqUf'.ncc. This implies that forn,m> N Equation (2.8.4) implies that
(2.8.4)

la:' - a;' I < {

for n,m > N and Vi.

(2.8.5)

For fixl..-'l.i 1" Equation (2.8.5) implies that {an is a Cauchy sequence of real numbers and, by t1w Cauchy criterion of real sequences, it must converge, say, to (l.;; Le.
,

nn

n =a; =a,

(2.8.6)

....... 0.::

From Equation (2.8.4), we have

, L la:' - a:" I
>=1

< (I',

for evcry k.

Making m --+

('X)

in t.his

L la:' - /I,I
;=)

n~l;J\ion,

we have
P

< fP for n ;:::

N.

.n",

"'_'_""""'''''''10''-''

Making k --+

00,

we get
Lla~

-a,I P < (.P forn > N.

,=J
TillS implies t.hat

- (:I;" - x) = x - x" Ell"

where hence Furthermure,
forn~N,

x = (x - xn)

+ x.,

tp .

i.e. ,
IIx n -

xII

--+ 0

as n

--+

00.

Therefore. {xn} is convergent to x E t p , which shows that t p is a Banach spacto. (c) c = {{J:,,} I lim '= is a veclor space with operations of addition and scalar multiplication defmed by
n~=

x" x}

+Y

= (Xl +Yi,:I;2

-I-

Y2,. ,x.,

+ lit,,.)

ox:;::: (OXt.nx2,'.' ,0:1;", ... ),

where
x =
(Xi,X2, ...

,x n , ... )

11 = (lJl,m, ,.11",)

belong to c and 0: is a scalar. For x E c, [Ixll = sup Ix"l. This is a norm. We shall now show that. c is a Banach space. Lpt. X tl = {ai'} be a CaudlY sequence in c. This implies that for (. > 0 3 N such that

IIx" - xmll
This implies that

= sup lai'
;

- a:JlI < (. for n. m > N.

laj _a;lll < (. for n,m > N .

(2.8.7)

B . ,~,_ .._",,"l1>"-'

......

This !np.,ans that {an is a Cauchy sequence of real members and, by the Cauchy criterion, for fixed i, I lin a,m
",-t<X>

:=

aJ

(2.8.8)

By Equations (2.8.7) and (2.8.8), la, Since x" E c, we have lim ai'

WE'

Qbtain for n

0.;1 ~ c,
:=

> N.

(2-".9)

a.. , which implies that

la" - ai'l Therefore, lam -akl:=

<f

for i :2: N (f, n).

-a.. +a" -a~ +ak -al"I < lam - a~1 + la;:' - 0 .. 1+ la.. - akl + Jak - Qkl < +(+(-I-( :41':.

10.", -a~,

+a:~

for n :2: Nand m,k ::::: N(,n). Thus, the sequence {a;} satisfies the (:auchy criterion. There p..xists lim OJ := a and hence x := {a,} E c. \Ve find from Equation (2.8.4) that

IIx.., - xII

:=

sup la, -

0:'1 -4 0 as n ---t 00;

Le., x .. -4 x as n -4 00 in c. Therefore, c is a Banadl space. (d) In order to prove that C[a, b] with 1111;:: sup II(t)1 is a Banach
a<t<b

space, we need to show that every Cauchy sequence ill Cia, b] is convergent. Let. (J,,(t)} be a Cauchy sequence, that is, for (> 0,3 N such that

III" -

fmll:= supl/.. (t) - 1",(1.)1 ,

< I':

for n,m

>N

This implies that {I" (t)} converges uniform ly to a continuous function, say, l(l) in C[a, b]. Therefore, III" -/11:= sup II,,(t) ~ l(t)1 ---t 0 as n ---t 00; Le.,

In

---t

in C[a. bJ and so C[a, b] is a Banach space.

:=

(e) Bl/{a,bJ

{I, a real-valued function defined on

.. la,b]/sup L
P k=1

I/(f.k) - I(tk-dl < 00, where P b an arbitrary partitiun of [a,b]; Le., the set of point.s tk slldl t.hat. a := 1-0 < tl < tz ... f." := h} is equal to the class
of functions of bounded \'ariations defined on [a, b] .

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

Let., for

J E BF la,b] ,

11/11 ~ I/(a)1 + V:U),

where I'~"(f) = sup
P

(2.8.10)

k=l

" L If(t~,) -

f(t k -l)1 is called the total variation of the

function fU).

11111 > 0 as 11(0)1 11111

~ 0 ;

1/(0)1

~ 0, and

vl'U) > o. ~1'(f) ~O; 1(0) ~O and II(I,}- I(t, ,)1

~ 0 and
:>

~ 0

for all lk {:} fUd = I(lk-l), and I(a) = 0

I(td = O'rJk :> f = O.

lIaIIi ~ 10/(0)1 + '';l' LlalJ(t,} - l(t,-d]1

k=1

"

~ 101 [II(a)1 + "'P

10111/11 E BV la,bj and h = I + g.
~

t,

I/(I,} - l(tH )1]

Let f,g Fot" any partition of t.he interval la, b] : a = to < /'1 < t'} < ... t" ~ b, Ih(a)1 ~ 1/(0) + g(a)1 <: I/(a)1 + Ig(a)1 and Ih(l,} - h(l,-dl If(td - IUk-1 )1 + If/Uk) - n(fk-1)1 where k = 1,2,3, ... , n. Therdore,

< <

IIhll ~ Ih(a)1

Hup
P

" L Ih(t,) k=1

h(t,-dl
I/(t,) - l(t,-dl

<: 11(0)1 + 1.9(0)1 + ,np L

P
k=l

"

+ sup
P

" L Jg(tk) - g(tk-dl

1:=1

11111 + IIgll

Thus, 11111 defined by Equation (2.8.10) is a norm on BF[a,bj. Now, we shall show that BFla,bj is a Banach space. Let {I,,} be a Cauchy sequence in BI' la, bJ . {/,,{f)} is a Cauchy sequence of real numben;

as
I/mltl -

1,,(t)1 <: I[Iml t ) - 1"lt)] - [/mla) - 1,,10111 + 11m (a) - 1,,(0)1 <: I/m(a) - 1,,(0)1 + V: U", - I") ~ III", - Inll

..... ,

B,.,to<l',_"""",,,,,"l1>"-I

Hence, {j,,(t)} must be collvergent, say to l(t)Vl E [a,bl. l(l) E BV [a,b]

M

sup
p

L IIUI.:) - 1(f.I.:-dl =
k~ I

"

"-HXI

lim sup
P

L 11,,(h) - I,,(f.k-dl <

L"" I

.,

00

" Ilf" - fll ~ 1f"la) - fl.)1 + ,up L If"I'd

P
1,,=1

- f(td - f,,(t,-d + f(t,-,)I

< If,,(a) - f(a)1 + "'I' L If,,(t,) - f(t"l

p
1;=1

"

+,up
p

" L If,,(t,-d k=1

f(t,-,)I

--t 0 as

--t

00.

Hence, j" --t

in BV [a, bl , and consequently B~" [a, bl is a Banach space.

2.8.2

Unsolved Problems

Problem 2.8.1 1. Let C be the space of real-valued Lipsdlitz functions of order 1 defined on [0, II ; that is, the class of functions I SUcil that.

".p
(z,y)E[O,I]>O,Ij, #y

If(x) - f(y)l ~ K(I) < 00.

Ix -

ul

Show that - is a subspiH:e of C [0, II. 2. Let Ilfll, ~ 'up If(t)1 + I<(I) ~
0:9$1

Ilfll + /(1).

Show that 11,11, is a norm on . 3. Show that (C, 11.11 1 ) is a Balladl space.
P7"Obiem 2.8.2 Let C 1 [0,1] denote the vector space of all real-valued functions defined on [0,1] having first-order continuous derivatives. Show that the expressions given below are equivalent norms.

11111,

~ ,up

O<t<l

1If(t1i + II'(t)I}
-

..III, ~

0 . ,>0<1"_""""'''''''10''-1

..... ,

o<t< I - -

"'I' 1f(t)1 + O<t<1 1/("l ,up

Prove that all linear operators, defined on a finite-dimen sional normed space into an arbitrary normel space, are bounded. Prove that the natural embedding J is an Isomct.nc isomorphism of X into X"
P1'Oblem 2.8.4

P1'Oblem 2.8.3

Let 1)(x) =lIfllllx)1, where tional on a nonnOO space X. Show that

P1'Oblem 2.8.5

f is a bounded linear func-

1. p(x + y)

< p(x) + p(y)

"Ix and 11.

."1.:.

2. p(ax) = op(x)Va ;:::

P7'Oblem 2.8.6

and

Let C 1 [0,1] denote the vector space of all real-valued functious definel on [0, lj which have a first-order continuous derivative. 1. Show that p(f) defiued by the expre::.::.ion

p'(J)

~ 1'(0) +

l'

(J'(t))'dt

is a nonn. and that convergence in t.hi~ norm implies uniform cAllwergence. 2. Show that if C l [0, I] is eqUIpped with the norm

11111 ~

'up
09:0:;1

1f(l.)I,

then the functional F define! on C I [0, I] by F(f) = 1'(0) is linear but not COUtiUUOIIS.
P1'Oblem 2.8.7
I. Let X be the vcctor space of all the sequences {x,,}

of complex numbers such that the series that

L n!x" is convergent

Assume

,,> I

lI(x")1I ~

Show that (X, II'ID is a nOrJned space.

L n! Ix"l ""
x

2. Let T: PI -t X b(' defined by T({x,.,}) = :;. Show that T is a bound('d linpar operator and find its norm. Deduce that X is It Banach space.

.n",

'.'_'_""""'''''''10''-''

P1'Oblem 2.8.8 P7'Oblem 2.8.9

En'xy finite-dimensional nonned space is a Banach space. Prove that a nonned space is homeomorphic to its open

unit ball.
P1'Oblem 2.8.10 P1'Oblem 2.8.11

Prove that L p , I ::; 1) <.

00,

is a Banach space.
:.paCf.~:-;

Let X and V be twu rea.l Ballitch

:.uch that

(a) A: D(A) C X --t Y is a bounded linear opf>xat,or. (b) D(A) is dense in X; that is, D(A) = X. Show that rl call be uniquely extended tu a bounded lilw..ar uperator defined on the whole space X.
P1'Oblem 2.8.12
I. Let {Tn} and {y,,} two sequences in a nonne<I space X, and {A,,}

be a sequence of I"f>..a! numbers. Further, suppose that X n --+ x and Un --+ 11, as 11 --+ CXJ, in X and ..\,., --+ ..\ in R as n --t 00. Show that x,., + y" --+ x + y and ..\,.,x" --+ ..\x as n --t 00. 2. If {a,,} is a Cauchy sequence in a normed space, show that {Ax,,} is a Cauchy sequence. 3. Show that a metric d(,) induced by a norm on a normed space X is translation invariant, i.e., d(x + b,y + b) = d(J:,y), where b is a fixed element of X, aIHI d(o::r,oy) = 101 d(x, 11), where 0: is a scalar.
P1'Oblem 2.8.13 P1'Oblem 2.8.14

Give an example of a seminonn thnt is not a norm Show that P 10, Ij with

IIJII =

sup
tElo,1]

IJ( t) I is a normed

space but nal. a Banach space. Let. X = {{x;}/x; = Ofori > n}. Show that X is a nOl"lnOO space with IIxll = sup lx, I but it is not a Banach space.
P7'Oblem 2.8.15

P1'Oblem 2.8.16

If X and Yare Banad} spaces, show that X x V is also

a Banach space.

...", "'_'_""""'''''''10''--'

P7'Oblem 2.8.17 Let X and Y b( two real nonned spaces amI T an operator on X into \,. such that

a. T(x + y) = T(x) + T(y)Vx,y c X.

b. T is bounded 011 the unit ball of X linf'..ar operator. Prove t.hat T
IS

a continuous

P7'Oblem 2. 8.18 Let X and Y be 1l0nl1ed spaces and T an operator of X into l', Show that T is a continuous linear operator if and only if

L u,T(x;),
1=1

for every convf'xgent series

L
;=1

0,:1;.

P1'Oblem 2.8.19

Show that R", C. C[O. 1/, (I" L" are separable normed

spaces,

Problem 2.8.20

Let T,

erO,11

-->

erO,11

T(f)

~
s

l'
:s:

K(s,I)f(t)dl.,
1,0 :::; /,

where Is, f.) is continuous on the unit square 0 Compute IITII.

:s:

:s:

I,

P7'Oblem 2.8,21 Show that if T is a linear operat.or on a nonned space X into a normed space }', then T- 1 is a linear provided it exists. P7'Oblem 2.8.22
Show that the dual of R" is R",

P1'Oblem 2,8.23 Show that the dual of (I" I < P < 00, is f q , where lip + Ilq = 1. Also, show that f" is reflf'.J("ive for 1 < p < 00 but (1 is

non-reflexive. Prove that. the set M""., of all m x n matrices is a normed space with respect t.o the following equivalent norms:
P1'Oblem 2,8.24

B.

......

, to<l',_"""",,,,,"l1>"-I

b. IAII,::=

L L la;JI and .4 = (a.j) E Mm.".

;=1 j 1

on

"

P1'Obiem 2.8.25

Prove that the space of all real sequences converging

to 0 is a normed space with 1I:.r;11 =

L Ix,,-t-t - :.r:" 1< =.

"=1

Problem 2.8.26 Let T be an operator defined on X x },., the product of nonned SP:lCes X 0.1111 Y. SUdl that

a. T(x + x', y + y') :::: T(x, y) + T(x, 1/') V(x,y) E X x Y,(x',y') E X x Y.

+ T(x', y) + T(x', !I')

b. T(ax,{3y) = a{3T(x,y)V(x,y) E X x Y with Q: and /3 being scalars, i.e., T is bilinear. Show that T is continuous if and only if there f'_xists k > 0 '"eh that IIT(x, 11) II < k 11>'11111111 Let

1IT11
Show I.hat

,up
lI"'II:"O I ,lIyll"Sl

IT(x,y)l

IIT(x,1I)11 < IITllllxllllyli.

P7'Oblem 2.8.27 A linear operator T defined on a nOlllled space X UltO a normed space}' is called compact or (complelely contimlOlls) if, for every bounded subset AI of X, its image TUH) is relatively compact, i.e" the closure T(M) is compact.

a. Show that t.he operator in Problem 2.8.20 i" compact.. b. Show that if 5 and T are compact operators, then 5 where a is scalar, are compact.

+T

and aT,

c. Show that every compact operator is bounded and hence continuous.

Problem 2.8.28 Show that the identity opf'xator defined on an infinit<.."dimensional normed space X is not compact.

."", '.'_'_""""'''''''10''--'

P7'Obiem 2.8.29 Let T ; X --t X (X is a nonned space) be defined by the relation T(x) == F(x)z, where z is a fixed element of X and F E X". Show that T is compact. P7"Obiem 2.8.30 Show thai the space of all compact operators on a Banach space X into a Banach space 1" is a Balladl space whidl is a dosed subspace of 6[X, 1"]. P7'Obiem 2.8.31 Let T: lp - f lp, P > 1, be definoo by T(:I;) = y, where y = (xl/I, X2/2, X3/3, . _ . ,x" In, ... ) for x = (Xl ,X2, X3,. . . x,,) E t p . Show that T is compact. P7'Oblem 2.8.32 Let T be a linear operat.or with irs domain D(T) and rangf' R(T) contained in a normoo space X A scalar A such that there exists an x E D(T), J: - 0, satisfying t.he equation T(x) = AX, is called an eigenvalue or characteristic vahle or proper value of T. The corresponding x is called an eigcnvector of T. If ..\ is an eigenvalue of T, the null space of the operator AI -T, N(AJ - T), is called the eigenmanifold (cigenspace) corresponding to the eigenvalUl~ ..\. The dimension of the eigenmanifold is calk><l the multiplicity of the eigenvalue A. The set of A such that (AI - T) has a continuous inverse defined on its range is said to he the 7"e,wltlent set of T and is denoted by p(T). The set of all complex numbers that are not in the re..sohent set is said to be the spectr'UlIl of T and denoted by o(T). R;>..(T) = ()..I _T)-l is called the resolvent of T. Let T:.X -} x, X being a Danach space, and

is called the :;pectml radiul> of T. (a) Let T; X --t X, X bejng a nonne<l space, be a linea!' operator on X and let {A" A2' A3,...} be a set of distinct eigenvalues of T. Further, let {x,,} be an eigenvector assuciated with A",n = 1,2, . . . Show that the spt {Xl, X2, .. . ,:r..,} is linearly independC'Tlt (b) Show that the set of eigenvalues of a C'ompacllinear operator T : X --t X on a nonned space X is coumable and the only possible point. of accumulation is A = 0. (c) Fino o(T), where T is defined as in Problem 2.8.20 on [,2[0, II into itself

...... "'_'_""""'''''''10''-''

(d) Let T ; X ---t X, X being a Banach space, be linear and continuous. If 1>.1 ~ IITII, then A is in the resolvent set ofT. Moreover, (AI - T)-' ;, g;ven by (AI - T)-' ~

I:>-n-' T' and II (AI - Tj-' II

fl=O

S (IAI-IITII)-I . Show that the spectrum of T is contained in the circle P/P.I S limsuPIITfllll/"), which is contained in (A/IAI S

IITID
(e) Show that for a bounded linear operator T, u(T) is closed and p(T) is open.

(f) Let T ; C[a, hI ---t C[a, hI be as in Problem 2.8.20, then (AI - T) -IiI = = v, A -=j: O. has a unique solution ii = A >. -"- T'lv.

n~

Problem 2.8.33 Inverse Operator Theorem; Let T be a bounded linear operator 011 a lIunne<.! SPaL"C X onto a nunned space Y. If there is a positive constant m such that
m

IIxll < IIT(x) II

Vx E X,

then show that ]'-1 ; Y ---t X exists and is bounded. Conversely, show that if there is a continuous inverse T- I ; Y = R(T) ---t X, then there is a positive constant m such that the above inequality holds for all x in X.
Problem 2.8.34

fa' f(x)dx.
bounded,

Let T, C[a.b] -+ C[a.b] be defined by f(t) -+ 9(t) ~

;

Find R(T) and T- 1

R(T) ---t C[a,hl Is T-I linear and

...... "'_'_""""'''''''10''-''

References
[lJ S Banach. Theorie Des Operations Linearites, Chelsea. New York: 1%5. [21 N Dunford, JT Schwartz. Linear Operations, Part 1, General Theory. New York: lnterscience, 1!)58. [31 DH Griffe!. Applied Functional Analysis. New York, Toronto: Ellis Horwood Limited. Publishers - Chichester. John Wiley and Sons. 1981. [4] CW Groctsch. Elements of Apphcable Functional AnalysIs. New York and l3asel: Marcel Dekker, 1980. [5] RD Milne. Applied FUllct,ional Aualysb. Bot;lon, London, Melbourne: Pitman Advanced Publishing Program, 1980. [61 N Nachbin. Introduction to Functional Analysis: Banach Spaces and Differential Calculus. New York and Basel: Marcel Dekker, 1981. [71 AW Naylor, GR Sell. Linear Operator Theory in Engineering and Science. New York, Heidelberg, Berlin: Springer-Verlag, 1982.

[8] IN Reddy. Applied Functional Analysis and Variation Methods. Singapore: \lcGraw-Hill Internntional Edition, Engineering Mechanics Series, 19R6.

[9] AH Siddiqi. Functional Analysis with Applications. New Dl!lIli: Tatai'vlcGraw-Hill Publishing Company Limited, 1986.

[10] JT Oden. Applied Functional Analysis, A First Course for Students of

Mechanics and Engineering Science. Englewood Cliffs: Prentice-Hall, Inc., I!)?!).

B.

.... ,

, to<l'._..

....,..",,"l1>"-I

[If E Zeidler. Nonlinear functional Analysis and its Applicat.ions. I-Ieidclberg: Springer Verl<lg, 1990.

B.

.... ,

, toOl',_"""",,,,,"l1>"-I

Chapter 3

Hilbert Space
3.1 [ntroductjon

As seen in Chapters 1 and 2, functional analysis involves abstraction and formalization of ideas studied in physics and classical mathematics. It seeks to identify the most primitive features of elementary analysis, geometry, and calculus to give them order and structure. and to establish their interrelationship. It simultaneously unifies a bunch of idl!i.u; and exwnds thl!1fi to areM which havp been explored in the framework of classical mathematics or different branches of science and technology. A Hilbert space is a special type of Banach space in which the underlying vector space is equipped with a structure called an inner product or a scalar product which provides the generalization of geometrical concepts such as direction, orthogonality (perpendicularity of two vectors), and angle between two vectors. The inner product is nothing but a generalization of the dot product or scalnr product of vpctor calculus_ Hilbert space with a rich structure of inner produM, is a powerful apparatus to tackle problems of diverse fields of classical mathematics like linear equations, variational methods, approximfltion theory, differential equations. Besides this, Hilbert space methods have played a vital role in resolving problems of llumel'OUS fields of science and technology. A few recent applications will be discussed in Chapters 10 and I L We present in this chapter the results of Hilbert space t1l1..-'Ory whidl are essential for proper understanding of various current scientific and technological developments.

B.

.... ,

, to<l',_"""",,,,,"l1>"-I

3.2 Basic Definition and Properties

3.2.1 Definitions, Examples and Properties of lllller Product Spnce

Definition 3.2.1 Lt X be a vector space uver the field of real or complex numLers. A mapping, denoted by (., .), defined on X x X into the underlying field, is called an inner product of any two clements x and y of X if the following conditions are satisfied:
1. (x+x',y) = {x,Y}

+ (x',y)

2. (o:x,y) = o:{x,y),o: belongs to the underlying field 3. (x,y)

~

(y,x)

4. (x, x) 2 0, 'Vx EX; and (x, x) = 0 if and unl)1 if x = O.

If the inner product ('j') is .-:Iefined for every pair of plements (x, y) ,x and y EX, then the vector space X together with the inner product (.,.) is called an hmer product space or pre-Hilbert space, usually denoted by (X, (.,.}) or simply X.

Remark 3.2.1
1. (x,V) denotes the inner product of two vectors x and y E X. Somf'

authors usc the symbol (.,.) for the inner product.

2. (x,y) denotes conjugate of the number (x,y). Therefore, if it is real, then (x,y) = (x,y) = (y.x). 3. Ln vIew of condition (3) abo\"e, (x, x) must be real (For x ; y in (3), \I"e have (x, x) = (x, x). We know that a complex number z i", rea] if and only if z = z. Tlwrefon'!. the number (x, x) must be rem.
4. Conditions (1) and (2) of Definition 3.2.1 Imply thm the function (.,.) is linear in the first variable x. It is easy to sec that (.,.) is also linear in the second variable y if X is a real vector space. In fael, we have

(a) (x,y

+ y')

= (x,y)

+ (X,yl}'rJX,y,U'

E X.

Verification. (x, y + y') = (y + y', x) by condition (3) of Oefinition 3.2.1. By condition (l), (y + y', x) = (y, x) + (y'! X), and so (x,y +y') = (y + y',x) = (y,x) + (y', x) = (y,a:) + (y', x)

0.

..... ,

,.-.1"_""""'''''''10''-1

by the property of the conjugate of the complex numbers. Again, by condition (3), (y,x)::: (x,y) and (y',x)::: (x,y'). Therefore,
(x,y

+ y')

~ (x, y) -r (x, y')

(h) (x,{3y) ::: {3(x.y)Vx,Y E X and B belonging to the underlying field

Verification. (x,py) = ({3y, x), Ly couditlon (3) but (f3y, x) = (3(y, x) by condition (2). Therefore, (x, (3y) ::: f3{Y, x) = {3 (y, x) = (3{x, Y), using the propert.ies of conjugate numbers and condition (3) of the inner product. On a real vedol' space X, the inner product h) is also lineal' in the second argument y as
(x,~y) ~ ~(x,y).

5. Using the lirst and second condition., of the inner product. 4 (al and (b) of Remark 3.2.1, it CM be seen that

6. (x.O) = 0, Vx E .Y.

(,0) = (O,x) = (O+O,x) ::: (O,x) T (O,x) ::: (x,O)+ (x,O) ::: 2(x,0). If (x,O) 1= 0, dum 1 = 2 (an absurdity). So (x,O) = OVx EX.
Ve,-ification.
7. Lf, for a given element y E X, (x,y) ::: a v x E X, then y ::: 0. (x, y) = is valid for x = y, i.e., (y, y) = 0, which implies that y ~ 0.)

8. The inner product is a continuous function with respect to the norm induced by it (For definition and proof, see Remark 3.2.4 and Solved Problem 3.10.2, respectively).

Example :1.2.1

R2

is an inner product space with the inner product.

(x, y) =

Xl YI

+ X2Y2,

where x::: (XI,X2) E n 2, y::: (Yl,Y2) E R2 .

...... "'_'_""""'''''''10''-''

Example 3.2.2 product

R",n > 1 is an inner product space with thf' inner

(x,y) = LX;JJ"

"

.=J

wherpx=(x\,X"l., . . ,xfl )ER f1 , !/=(YJ,Y2,.

,y,,)ER".

Example 3.2.3 Thp vprtor spa!'!' C [a, b] of the coutinuous function d{'-fined on [a, bl is an inner product SVRce with the inner product
L (f,g) =

lb
[b

I(x)g(x) dx, where I,g E C[a,bl

2. (f,g) =
tu

I(t)g(t) w(t)dt, where f,g E C[a,b] and wet) 2: 0 belongs

C[a,b].

For wet) = 1, we get the inner product of (1). w(t) is called a \reight fUllction.

Example 3.24

2 = {x = {x.d /

n~l Ix.,l
L x,y" .=J
=

< oo} is an inner product

where x = {x,} E 2, Y =

space with the inner vruduct (x,y) =

{y;} E f 2 We caU .2 real if the sequences {x,,) Rre real.

Example .1.2.!; L1.(a. b) = {All Lebesgue int.egrahl~ functions (real- or 2 complex-valued) over (a, b) / 1/1 are also Lehesgue integrable over (a, b)}
is all inner product space with the inner product (f,g) =
f,g E L,(a, b).

lb

Ig dx where

Remm'k .1.2.2
1. An inner product space will be called finite-dimensional if the underlying vector space is finite-dimensional. Somet.imes 1\ finite-dimensional inner product space over the field of culllplex Ilumbers is called Hermitian space 01' unitary space. A finite--dimcnsionnl inner product space over the field of real numbers is called a Euclidean space.

2. Unless explicitly mentioned, all abstract Hilbert spaces are defined over the field of complex numbers.

Theorem 3.2.1 (Cauchy-Schwartz-Bunyakowski Inequality).

all x, y belo1lgi1lg to a1l i1l1ler protluct
SfKICC

Fol'

X, me have

l(x,Y)I' < (x,x)(y,y),

(3.2.l)

Proof. [f y = 0, then {x, y} = 0, by Remark 3.2.1 (6) nnd {V, y} = O. Thercfore, Equation (3.2.1) is satisfied as both sides are zero. Lt y f:- 0,
then (y, y)

f:-

(x y) O. Let>' = ( , ). Then we have

y, '!J

l(x,Y)I'
(Y, y)
~

{:r., y){:z:, y} (y, y)

A(x,y)
~

A(Y, x),

by eunditiun (3) of Defillltiun 3.2.1. But w~ can writc >"{y,x) = X(x,y) by the same condition. Thus,

l(x,Y)I'
(y,Y )

~Ay,x)~Ax,y)~

IAI' (y,y).

(3.2.2)

By using conditions (1) and (2) of Definition 3.2.1 and Remark 3.2.1(4) (a) and (b), we have

o<

(x - >'Y,x - >.y) = (x,x) + (x, ->..y) + {->"y,x} + (->.y, ->..y)

-

~ (x. x) - A(x, y) - A(Y, x)

+ IAI

, (y,y).

(3.2.3)

By Equation.<; (3.2.2) and (3.2.3), we obtam

0< (x x) _ l(x,Y)I' _ l(x,Y)I' -, (y, y) (Y, y)

0'

+ l(x,Y)I'
(Y, y)

0< (x x) _ l(x,y)I' -, (Y,Y) ,

0'

(3.2.4)

B.

.... ,

l(x,Y)I' < (x,x)(y,y) .

, to<l'._..

....,..",,"l1>"-I

This is the desired inequality.

Theorem 3.2.2 Every inner protluct spacr X is a nonned space with resIled to the norm Ilxll = ](x.x)II/2 'r/x E X.

Proof. Since the inner product space X is a veetor space by definit.ion, it suffices to verify axioms of t.he norm (Definition 3.2.1).
1.

0 'r/ x and IIxll = 0 if and only if x = o. Ilxll = [(J:, X)]1/2 , (x,x) ~ 0, and (x,x) = 0 if and only if x = 0 by condition (4) of the definition of t.he inner product. Therefore" (1) is satisfied.

IIxll ~

2.

lIaxll

= lalllxll 'r/ x E X and a real or complex. We have lIaxll = [(ux,ax)]IJ2 = [ao{x,x)]IJ2 by cundition (2) of Definition 3.2.1 and Remark 3.2.] (4) (b). Since an = lal 2 , we havp

lIaxll
3.

~ [ lal (X,X)

'

]1/2

lall(x,x)] 1/2

Inlllxll

IIx T yll " IIxll + lIyll V x, y E X. [Ix+yll' ~

~

We have

+ (x,y) + (y,y) (x,x) + 2 Re{x,y) + (y,y)

(x+y,x+y) (x,x) + (x,y)

by Definition 3.2.] and Appendix D. By the Cauchy-Schwart'lrBunyakowski inequality, Re{x,y) < l{x,Y)1 < (X,x1/2 (y,y)1/2. Therefore, we haw

IIx + YII' <

+ 2 (x, x 'I' (y, y'/' + (y, y) = [[(X.X>]1/2 + [(y,y)]1/2f.

(x, x)

0'

IIx + yll < [(x,x)I'I' + [(y,y)I'1' ~ Ilxll + lIyll

Hence, IIx + 1111 IIxl' + IIvll. This proves that norm on . X and (X, 11)1) is a normed space.

IIxll

= [(X,X)]1/2 is a

0 . ,>0<1"_""""'''''''10''-1

.... ,

RemU1'k :1.2.:1
1.

Ln view of Theorem 3.2.3, the Cauchy-Sdl\v1'lrtz-l3unyakowski equality is written as l{x,Y)1 ~ IIxlillyll.

in-

2. In the Cauchy-Schwartz-Bunyakowski inequality, equality l<x,Y)1 IIxIiIlYII, holds if and only if x and y arc linearly dependent.

Verification. Suppose !{x,Y)1 ::::: IIxll lIyll and x -=F 0, y -=F O. (If either x::::: 0 or y ::::: 0 then. x and yare linearly dependent and the desired result (x, y) follows.) This implies that (x,y) -=F 0 and (y,y) -=F o. If A::::: -(- ) ' then y,Y A -=F 0 and
(x - Ay,x - Ay)

(x,x) -

1~'Y)i
Y,y

[See Equations (3.2.3) and (3.2.4)1

~ II II' - 1I"1I'lIyll' ~ u
x

lIyll'

By Definition 3.2.1 (4), it implies that x - Ay ::::: O. Hence, x and yare linearly dependent. Conversely, if x and "!J are linearly dependent, then we can write x ::::: Ay. This implies that

l(x,Y)1

I(Ay,y)1 ~ IAII(y,Y)1 (Ily Definition 3.2.1(2)) = I-'Illyll' = (1-'1 lIylll lIylI ,Ily Theocem 3.2.2) ~ II-'ylillyll ~ lI"III1YII
~

Ilema1'k ,1.2.4

The norm product.

IIxli :::::

l(x,x)l l/ ".! is said to be the norm induced b:yan inner

2. Since every inner product space is a nonned space, all the definitions given in Chapter 2 can be translated into an innel product space with respect to the norm induced by an inner product For example, a sequence {x n } in all inlier product sp:'U:e X is called a Cauchy sequence if, for f > 0, there exists N such that IIx - x." II ::::: [{x - x,,,. x., - Xon )]l/2 < f for nand m > N; i.e., lim (x n -Xon, X.,.,--too
~

0.

......

X on )

::::: O .

..

,..-1"_""""'''''''10''-1

Hilbert Space
An inner product space X is calk--d a Hilbert space if the normed space induced by the inner product is a Banach space (complete normed space). That is, every Cauch.y "equence {x n } E X with respect to the Horlll induced by the inner product is convergent with respect to this
norlll.

Definition 3.2.2

Example 3.2,.6

L R",

"-2

and L2 la, b) are Hilbert spaces.

2. C[a,b] and P[a,b] are inner product spaces but not Hilbert spaces.
Note: We define an inner product on P [0,11 ns follows:
(J,g) = wherej,gEP[O,l].

l'

f(t)g(t)dt,

Example :Ut. 'l

Let \" =

{f/f

is absolutely continuous on (a, b) with

f and ~ belonging
product:

to 2(/, b) and f(a) = feb) = O}. Then Y is a dense

SUbSlJilCe of Lz(a, b). Y is a Hilbert space with respt--'Ct to tlw following inner

where (.,.) is the inner product of L 2 (a, b).

Example 3.2.8 Let D be an upen 8uL:.et uf R~ and CO' (D), the infinitely differentiable real-valued functions wilh compact support in O. Let

Then (...) is an inner product on CO' (0). This inner product induces the norm

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

However,

C'ir (n) is not a

Hilbcrt space.

Remark 3.2.5
Historically, C is the first example of Hilbert space which was discov2 ered by the c.elebrated Gcrman tn;JI.hcmaticiall David Hilbert around 1910. Although the abstract axiomatization (Definitiun~ 3.2.1, 3.2.2) was given by J. von NeumaJlIl in 1927, tl](> abstract space is itself cal1etl Hilbert space in view of his valuable contributions in this area.
2. In the early stage; of the devdopment of the Hilbert space theory, our present-day Hilbert space (Definition 3.2.2) was assumed to satisfy an additional condition, viz., the inner product space is also separable. That is, up to 1930. a Hilbert space was a separable complete inner product space.
3. We can show that e...ery separable Hilbert space is isometric isomor-

phic to '2.

3.2.2

Parallelogram La.w aIJd Cllaractelization of Hilbert Spa.ce

From the elementary geometry (goometr.y of the plane), we know that the sum of the squares of the diagonals of a parallelogram is equal to thc sum of the sqUa1"es of its sides. The following result gives a. generalization of this result to an inner product space.
Theorem 3.2.3 (Parallelogram Law).
heloflging to an inne,' 1J1"oduct s]Jace. we have Fm' any two elements x and y

IIx + yll' + IIx - gil' ~ 211xll' + 211YII' .

Proof. Since X is an inner product. space, we have IIx + Yll2 = (x + Y, x +y) and Ilx - Yll2 = (x - y, x -V) (Theorem 3.2.2). By Definition 3.2.1 and Remark 3.2.1 (4), we ha.. . e

IIx + yll' =
0'

(x,x)

+ (x,y) + (y,x) + (y,y),

(3.2.5)

IIx + YII'
Similarly,

IIxll' + (x, y) + (y, x) + lIyll' .

(x, y) - (y, x)

IIx . YII' ~ IIxll' -

+ lIyll' .

(3.2.6)

...". "'_'_""""'''''''10''-''

By adding Equations (3.2.5) and (3.2.6), we obtain

IIx + yll' + IIx - Yl"

This is tllf' desired result.

211xll' + 211yll' .

Remar'k 3.2.6 The following exarnplf' shows that the parallelogram law is not valid for an arbitrary norm on a vector space. Let X = C (0, 21T] be the space of all real-valued continuous functions on 10, 21T]. Then C (0, 21T] is a Banach space with the normllfll = sup If(t)l. However, this norm does
0<1<21f

not satisfy the parallelogram law. For example, choose f(i) = max(sint,O) and 9(t) = max( - sin t, 0). Then

11/11 ~ 1,11911 ~ 1,111 + gil ~

Thus,

I,and

III - gil =

I.

211/11' + 211gll' =
Hence,

4, and

III + gil' + III - 911'

2.

III ... gil' ... 111 - 911' t 211/11' + 211911' .

In fact, Theorem 3.2.5 shows that if the norm of a nOrIlled space satisfies the parallelogram identity. the space is an inner product space. Theorem 3.2.4 (Polarization Identity). For any two elements x and y belongirlg to an infie1' product i;!)(I(X, we have

Pmof

By Definition 3.2.1 and Remark 3.2.1, we have

IIx + yll' -lix - yll'

-i IIx

~ (x, x)
=

+ (x, y) + (y, x)+ (y, y) -(x.x) + (x,y) + (y,x) - (y,y)

illx +iyll' ~ i(x,x) + (x,y) - (y,x) +i(y,y)

igll' = -i(x,x) + (x,y) - (y,x) - i(y,y).

By adding these four relations. we obtain

IIx ... yll' - IIx - yll' + i IIx + igll' - i IIx - ,gil' ~ 4(x, y),

.n". "'_'_""""'''''''10''-''

0'

(x, y) =

~ [lix + YII' - Ilx - YII' + i I'" + ;YII' - i IIx - ;YII'] .

I
Theorem 3.2.5 (Jonlan-von-Neumann, 1935). A normed space is an inner product space if and only iJ tile norm oj the normal SIlaCC satisfies the !K.l1'alielogmm law.

Theol'em 3.2.6. A Banach space is a Hilbe,t space iJ and only iJ its norm liatisjicJ. tht: pamlldogwm.
For proof of above theorems, one may see, for example, Siddiqi [13].

3.3
3.3.1

Orthogonal Complements and Projection TheoTenl

Orthogonal Complements and Pmjections

Definition 3.3.1
I. Two vectors

x and y in an inner product. space are cailed orthogonal, denoted by x...Ly if (x, y) = O.

2. A vector x of an inner product space X is called orthogonal to a nonempty subset A of X, denoted by x.lA, if (x, y) = 0 for each
YEA.

3. Let A be a nonempty subset of an inner product space X Then, the set of all vectors orthogonal to A, denoted by A-I-, is called the orthogonal complement of .4; that is, A1- = (x E X/(x,Y) = 0 for each y E: A}.;-\1- = (;-\1-)1- will denote orthogonal complement of A-'4. Two subscts A and B of an mner product space X are called onhogonal denoted by A.1B if (x,y) = 0 \/x E A and \/y E B .

Q . ,>0<1"_""""'''''''IO''-I

.... ,

Remm'k 3.3.1
1. Since (x,y) = (y,x),(x,y) = 0 implies that (y,x) = 0 and vice versa. Hence, xl-V if and only if yl-x.
01'

(y,x) = 0

2. In view of Remark 3.2.1 (6), xl-O for every x helonging to an inner product space. By condition (4) of the definition of the inner producl,

o is

the only vector orthogonal to itself.

3. It is deal' that {O}.1 = X and X.1 = {OJ.

4. It is clear that if ..,11-8, then An B = {O}.

5. Nonzero orthogonal vf'f"tors,
XI, XZ, X3,..

,x n of an inner product

space are linearly independent.

Th~or~J11

3.3.1. Let X be un f1IlIfT !Jwduct space und A its urbitnwy subset. Then the foLlowiflg results hoLd good:
I.

.4. ~

IS

a closed subspace of X.

2. A. n A.1 C {O}. A "1 AJ = (O) if and only if A is a subspace.

3. A C A1..1..

4.

If 8 C

A,

then B1. :::) A1..

Proof.
1. Let x, y E ..11.. Then, (x,z) = 0 'r/ z E A and (y,z) = 0 'V z E A Since for arbitrary scalars a,{3, (ox + j3y,z) = a(x,z) + jJ(y,z), by Definition 3.2.1, we get (f.l:J. + /Jy,z) = OJ i.e., ax + {3y E A1.. So A.1

is a. subspace of X. For showing that A.1 is closed, let {x,,} E A-l and x" -+ y.We need to show that y must belong to A.1. By definition of A1., for every x E X (x, Tn) = 0 'r/ n This implies that lim (x. x,,) =
"---->00

lim (x", x) = 0 (RetlHlrk 3.3.1) Since (.,.) is a continuous function

n~oo

( lim x ... , x) = O. or (y, x) = O. Hcnr:e, 11 E A

n~oo

2. If YEA n A.1 and 11 E A-l; by Remark 3.3.1, y E {O}. 1f A is a subspace. then 0 E .1 and 0 E AnAl.. Hence .4 n:1.1 = {O}. 3 Let YEA, but y If- Al.l.. Then there exist~ an elenwnt z E A- such that (y,z) -I- O. Since z E: A.1,(y,z) = 0 which is a contradiction. Hence, y E ..1.1.1 .

...". "'_'_""""'''''''10''-''

4. Let y E A-L. Then (y,z) = 0 V z EA. Since every z E B is an element of A, we have (y,z) = 0 V z E B. Hence,?I E B-L, and so B-L ::::> A-L.

I
Definition 3.3.2 The angle 0 between two vectors x and Y of an inner product space X is defined by the following relation:

C"'" ~ IIxlillyil'
Remar'k 3.3.2

(x,y)

(3.3.1)

I. By the Cauchy-Schwanz-Bunyakowsb mequality, the nght-hancl side

of Equation (3.3.L) is always less than or equal to L, and so the angle o is well defined, i.e., 0 < 0:::; 1T, for every x ancl y different. from O. 2. If X = R3 , X = (XI,X2,X3), Y = (YI,Y2,Y3), t.hen cosO=.
(x~

+ .r.~ + xnl/2(y~ + Y~ + yj) 1/2

XIYI+X2Y2+ X3Y3

This is a well-known relation in three-dimensional Euclidean space. 3. If xl..y, then cosO = 0; i.e., 0 = Jrj2 In view of tins, orthogonal vectors are also called perpendicular vcct.ors. A well-known result of pl.me geomet.ry is that the sum of the squares of t.he h:t..<;e and the perpendicular ill a right-angled triangle is equal to t.he square of the h.ypotenusc. This is known as the Pythagorean theorem. its infinite-dimensional analogue is as follows. TheOl'em 3.3.2. Let X be an inneJ' l)f'()duct Sl)(I,{"P and x, Y E X Then /0" x.ly, we have Ilx + y!l2 = IIxll 2 + lIyll2.

~ (x + y,x + y) ~ (x,x) + (y,x) + (x,y) + (y,1/1 (by Definition 3.2.1). Since x.ly,(x,y) = 0 and (y,x) = 0 (by Definition 3.3.1 2 and Remmk 3.3. J). Hence" IIx + Yll2 = Ilxll + lIyll2. I
Proof.

IIx + yll'

Example 3.3.1 Let X = fl?, the three-dimensional Euclidean space. and ill = {x= (XI,X2,X3)} be its subspace spanned by a non-zero VC(;tor x The orthogonal complement of M is the plane through the origin perpendicular to the vector x (Figure 3.3.1}.

...... "'_'_""""'''''''10''-''

Figure 3.3.1

Orthogonal complement in Example 3.3.1

Example :1..1.2 Let A be a subspace of Ff generated by the set {( 1,0, 1) , (0, 2, 3)} . A typical e!emellL of A can be expressed as x
(Xl,.:r.2,X3) =

),(1,0, 1) +1!(0,2,3)
X3

- >"i + 2/!j + (>.. + 31!)k

=> Xl =)" X"2 = 2ft,
= ), + 31!
Thus, a t.ypical element of A is of the form (XI,X2. Xl + ~X2). The orthogonal complement of A can be constructed as follows: Let x = (XI ..C2.X3) E A-.l. Then for y = (Yl,Y2,YS) E A. we have

(.r., y) = Xl YI

+ xzyz + XsYs =

Xl YI

+ xzyz + Xs

(YI

+ ~yz)

= (Xl +XS)YI +

(X2+~X1)Y2=0.

Since YI and Y2 are linearly independent, we havf' xl+xa-Oand

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

Therefore,

A-.L = {X = (XI,XZ,X3) IXI = -X3, Xz =

-~X3}

{XEW/X=(-X3,-~X3,X3)}'

3.4

Orthogonal Projections and Projection Theorem

As we know, an algebraic projection P on a vector space X is a linear operator P on X into itself such that p Z = P. A projection P on an inner product space X is called an orthogonal projection if its range and null space are orthogonal, that is, R(P)l.N(P) ((u,v) = 0 VuE R(P),

v E N(P)), where R(P) and N(P) denote, respectively, the range and null space. It can be easily venfied that if P IS an orthogonal projection, then so is f ~ P. Important properties of orthogonal projection are contained in the following theorem:

TheOl'em 3.4.1. SIJace X. Then

Let P be an ortho.qonai proJecton oj an nine" lJroduct

I. Each element z E X can tK: Mitten tl1liQ1.lt:ly as

z = x +y. where x E H(Pl and y E NIP),

2.

11=11' ~ Ilxll' + 11.11' IIPII =

1 for

3. EVefY orthogonal projection is continuous and moreover P oF O.

'!. N(P) and R(P) are closed linem' subspaces oj X.

5. N(P) ~ R(P)' and R(P) ~ N(P)'.

We know that for an algebraic projectioll P on X,

x ~ RIP) + NIP),

13.4.1)

where R(P) n N(P) = {O}. Thus, every element z in X can be expressed in the form z = x + '!I, x E R(P), Y E N(P). To show the uniqueness, let

z = Xl z = Xz

+Yl, Xl E R(P), Yl E N(P)

+ Yz,

Xz E R(P), liz E N(P).

",,,. "'_'_""""'''''''10''-''

Then XI

+ YI

X2

+ Y2;

or Xl -

X2

= Y2 - YI

1. Since XI - X2 E R(P) as R(P) is a8ubspace of ...-Y and Y2 - YI E N(P) as N(P) is a subspace of X, we find that XI - X2 = Y2 - YI = O.

Therefore" the representation is unique.

2. Since z =:r + 11,
X

E R(P) and y E N(P), we have

IIz11 2 = II x + YII 2 = (x + y, x + y) ~ (x,x) + (x,y) + (y,x) + (y,y) = IIxII 2 + lIyII 2 as {x,y} = 0 and
3. By pmts (1) and (2) for all z EX, we have

{JJ, x) = O.

IIP,II' = IIP(x + Y)II' = IIPxll' + IIPYII'

IIPYU = 0 as
2

x E R(P),y E N(P).

Y E IV(P). Thus,

IIPzlI' ~ IIPxll' ~ IIxll'

11,11' .

This means that P is bounded and hence continuolls. Let P 1= 0; then, for X E R(P), we have Px = x. Therefore, IIpxll :::: IIx!l which implies that 11P1i = 1. 4. It is clear that R(P) is t.he null space of f ~P. the result follows.
B~'

Proposit.ion 2.4.1(4),

5. Since N(P)l.R(P), N(P) C R(P)-'-. To prove the assertion, it is sufficient to show that N(P) :::) R(P)'!'. Let Z E R(P).l. Then there exists a uniquc Xo E R(P) and Yo E N(P) such that z = Xo + Yo Since z E R(P).l, we have {z,u} = 0 for all U E R(P). Then 0 = (z,u) = (xo,u) for all tl E R(P). This implies that. Xo = 0 and z = Yo E N(P) which shows that R(P)'!' C N(P). Similarly, we can prove that. R(P) = N(P).l.

I
The existence of ort.hogonal projec:t.ion is guarant.eed Iw t.hc following theorem: Theorem 3.4.2 (Project.ion TheOlem). a 1Iilue,t space X, then
ff Ai
105

a closed .mbspace of

x = AI '" AI".

(3.4.2)

",,,. "'_'_""""'''''''10''-''

--.,- -- ..... -,/

..

.-

.-.,.,-

_~to~~=::'T==:=='A~I-x
Figur.. 3.4.1 Geometrical meaning of the orthogonal decomposition in

, ,

n2

Remark 3.4.1
l. Theorem 3.4.2 implirn that a Hilbert space is always rich in projections. In fact, for every closed subspace M of a. Hilbert space X, there exists an orthogonal projection on X Wh08C range space is 1\1 and whose null SpM'P is M.L

2. Equation (3.4.2) means that every z E X is expressible uniquely in the fOfm z == x+y, where x E A-I and y E lll.L. Since III nM.L == {O} , in 01'del' to prove Theorem 3.4.2, it is sufficient to show that X == At +1\1.L. Equation (3.4.2) is called the orthogonal decomposition of Hilberl space X. 3

(i) Let X == l(l. Then Figure 3.4.1 provides the geomer,ric meaning of the orthogonal decomposition of R2 as

(ii) Theorem 3.4.2 is not valid for inner product spaces (see Problem 3.10.15).

Q.

.... ,

, to<l',_"""",,,,,"l1>"-I

We require the following results in the proof of Theorem 3.4.2.

Lemma 3.4.1 Let At be a closed convex subset of a Hilbert space ).," (md p = inf lIyll. Then there exists a unique x E !If such that II.TII = p.
yEAI

Lemma 3.4.2 Let Ai be a closed subspace of a Hilbe7t space X, x fJ. M and let the distance between x, and AI be p, i.e., p = inf IIx - ull. Thcn

thert'. e:l;isl..<; a unique !lector UI E M such thnt

IIx - wll

"EM

= p.

Lemma 3.4.3 If M is a lJr'O])(:r closed subspace of a Hilbert space X. then there exists a nunzeJV Vect07' u m X IciUch that u LA!. 3.4.4 If At and N an? dosed snbslmces of a flilbert space X, such that M LN, then the .mbspar:e AI + N :::: {x + y E X/x E AI and yEN} 1S also closed.
Remm'k 3.4-.2 Lemma 3.4.2 can h~ rephrased as follows: Let M be a. closed convex subset. of a Hilbert space X and for x E X, let p :::: inf Jlx - ull . Then there exists a unique element wEAl such that p =
ttElII

T~emma

IIx - wll.

w is called the projection of x on M and we write Px :::: w (Definition 3.5.1).

Proof of Lemma 3.4.1. Since AI is <1. convex suL:.et of X, aX-t"(I-u)y E Al, for {'( = 1/2 and every x, y E M. By t.he definition of p, there exisLo:; a sequence of vectors {x,,} in Al such t.hat IIx,,1I ~ p. For .T :::: X" and y :::: X"', we have
"X',,-'-;+"x"'' -" E At 2

and

x"

+ x",
2

Hence,

IIx" + xmll
By the parallelogram law for elemenL<;

~ 2p.
Xtl

(3.4.3)
2

and x m ,

IIx" +xmll Z + 11;1,'" -x mll 2 = 211x,,11 + 211xmll 2 , IIx" Xm

11 2 = 211.T"lI Z -+ 211x m11 2 - IIx" + :em 11 2 ::; 211..r,,11 2 +211xm 11 2 -1p'l (By Equation

(3.4.3)).

""n

"'_'_""""'''''''10''-''

Since 211x lI lI:t --t 2p2, and 211x m l1 2 --t 2p2, we ha.ve IIx lI - .T m1l 2 --t 2p 2 + 2p 2 - 4p 2 = 0 as n, m --t 00. Hence, {xn} is a. Cauchy sequence in At Since At is a closed subspace of a Banach space X, it is complete (See Remark
1.2.3). Thus, {x,,} is convergent ill Af; Le., there exists a vector x in !If such that lim X n = x. Since the norm is a cont.inuous function, we have
,,--toe

lim IIxll ll = lim = 11:1'11. Thlls, :/; is an (~lf>mmlt of AI with the " ..... 00 desired property. Now, we show that x is unique Let x' be another element of AI such x + x' that p = llx'il. Since x, x' E M and M is convex. 2 E AI By the
p =
" ..... 00

II

Tnll

parallelogram la'" for t he element" ~ and '~, we have

+ ;1;' ,
2

-2-+

IIxll'
IIxll'

IIx'II'
2

IIx - x'II'
2

< -2-+

Ilx'II'
2

:c+ x'
2

< p,

which contradicts the definition uf p. Hence, x is unique.

Proof of Lemma 3.4.2. The set. N = x -l AI = {x + v/v E AI} is a c]osf'd convex subset of X, and p == inf 110 - (x + v)11 is thf' distance
of 0 from N. Since -v EO M for all v E AI, p = inf 110 - (x + v)lI. By "eM Lemma. 3.4.1, there exists a. unique vector u E N such that p = f1ull. The vector w ~ x - It = - (u - x) belongs to Af [As v,"e have At = N - x -::::
",+.,eN

{z - x/z E Njx fJ. Al} and AI is a subspace, therefore, - (z - xl E M].

Thus, IIx - wil = lIull = p. w is umque,. For if w is not unique, WI f- W is a vector in !If sudl that IIx - WI II = p. This implies that It, = (x - WI) is a vector in N such that IluIIi = .. r - w,1I = p. This contradicts that u is unique. Hence, w is unique. I
, toOl',_"""",,,,,"l1>"-I

B.

.....

Proof of Lemma 3.4.3.

Let

.T

If-

AI and p = inf

lIEM

11:1' - vII, the distance

from x to Af. By Lemma 3.4.2, there exists a unique element w E /o.l such that IIx -wfl = p. Let n = x-w. u i- 0 as p > O. (lfu = 0, then x -w = 0 and IIx - wll = 0 implies that p: 0.) Now, Wfe' show that ul..M For this, we show that for arbitrary Y E M, (u, y) = O. For any scalar a, we have lIu - O'1JII = IIx - IV - ayll = IIx - (w + ay)lI Since At is a subspace, w + ay E M whenever w, y E M. 2 Thus, w+ay E M implies that lIu - ayll > p = lIuli or lIu - ayll2 -lIull 2': 0 or (u-ay,u-uy) -lIull 2 > O. Sinc.;e (u-wJ,u-ay):::::: (u,n) -a(y,u)2 a(u, y) + aa(y, y) = lIuli - a(u, y) - a(y, u) + lal (y, V), we have,
-a(fI.,y) - a(u,y)

+ lal 2 11Y\l2 2': O.

(3.4.4)

By putting a ber, we get

(3(u,y) in Equation (3.4.4), {3 being an arbitrary real num- 2~ I(u, y) I'

+ ~' 1 y)I' lIyll' ~ o. (u,

in EquatIOn (3.4.5), we obtain

(3.4.5)

If we put a =

l(u,Y)1 2

and b =

IIYII 2

-2{3a + {32 a b ~ 0,

{Ja ({3b -

2) > 0

'rj

real {3.

(3.4.0)

If a > 0, Equation (3AG) is false for all sufficiently small positi\ ~ R. Hence, 2 a must be zero, i.e.. a = l{u,y)1 = 0 01' (u,y) = 0 'rIy E Af. I

Proof of Lemma 3.4.4. It is a well-knO\vn result of vector spaces t.hat AI +N is a subspace of X. We show that it is dosed, i.e., every limit point. of AI + N belongs to it. Let z be an arbitrary limit point of /rl + N. Then there exists a sequence {z,,} of points of M + N such t.hat z.., -t z. (See Theorem C.2 of Appendix C.) !lll..N implies that M n N = {OJ . So, every z" E A:f + N can be written uniquely in the form z" = ;1;" + 1/'" where x" E /l;f and y" E N. By the Pythagorean theorem for elements (x", - x,,) and (Yrn - y,,), we have
(3.4.7)

(It is dear that (x.." - x n ) l-(Ym - Yn) 'rim, n). Since {Z'l} is convergent, it is a Canchy sequence and so liz", - z..,11 2 -t O. Hence, from EquatIon (3.4.7),

...... "'_'_""""'''''''10''-''

wc s(' that Ilx m- x,,11 ~ 0 and !lyon - Y"II ~ 0 as m,n ~ 00. Hence, {x m } and {V,,} are Cauchy se<juences in M and N, respectively. Being closed subspaces of a complete space, [1,.[ and N are also complete. Thus, {x m } and {V,,} w'e convergent in AI and N, respectively, sa;-.' X m -) x E A1 and y" -t YEN. :c + y E At + N as .T E /If and YEN. Then
z =
"->00

lim Zn = lim (x n ,,---too

+ Yn)

= hm Xn
"->00

+ lim

"->00

=x+yEAI+N

This proves that an arbitrary limit point of M is closed. I

+N

belongs to it and so it

Proof of Theorem 3.4.2. By Theorem 3.3.1llf..J.. is also a dosed subspace of X. By choosing N = Ml. in Lemma 3.4.4, we find that At + All. is a closed subspace of X. First, we want to show that X = AI + Ml.. Let X f. AI + All.; Le. AI + All. is a proper closed subspace of X. Then hy Lemma 3.4.3, there exists a nOlw..ero vector u such that u.l..M + Ai l.. This implies that (u, x + Y) = 0, Vx E AI and y E At1.. If we choose y = 0, thcn (u, x) = 0, Vx E M; i.e., U E Ml... On the other hand, if we choose x = 0, then (u,y) = 0 for all Y E M1.; i.e., u E Ml...l.. (Since M and Ml.. are sub-spaces this choice is possible). Thus" U E MoL n M1..l.. By Theorem 3.3.1 for A = M1., we obtain that u = O. This is a contradiction as u =I- O. Hence, our assumption is false and X = 111 + llll... In \'iew of Remru-k 3.4.1 (2), the theorem is proved. I Sometimes, the following 1>tatement is also added in the statement of the projection theorem. "Let Jf be a closed subspacc of a Hilbert space X, t.hen M = AI.L.L (Unsolved Problem 3.10.3)".
Remm'k 3.4.3

1. Let.

X == L 2 (-I, 1). Then X = Mffi 1If.L, where AI =

~

{f

L2 \ -1,1)1 f( -t) = f(t)Vt E (-1,1); i.e., the space of even functions};

{f E L,(-I,I)/f(-t) odd functions}.
M~ ~

-f(tN' E (-1,1), i.e., 'he space of

2. Let X = Ldo,bJ. For t; E [1t,bJ, let M =: {f E L 2 [o,b]/f(t) = 0 almost pverY\"'here in (a,e)); and JIl. = {f E L 2 [o,bJlf(t) = 0 almost everywhere in (e, b)}. Then X = AI ffi All...

...... "'_'_""""'''''''10''-''

Remm'k 3.4.5 The orthogonal decomposition of a. Hilbert space, i.e., Theorem 3.4.2, has proved (Illite llseful in potential theory (Weyl, [17]). The applications of the results concerning orthogonal decomposition of Hilbert spaces can be found in spectral decomposition theorems which deal with the representation of operators 011 Hilbert spaces. For example, for a bounded self-adjoint operator, T, (Tot"y) is represented by an ordinary lliemanu-Stieltjes integral. For details, see Kreyszig [9), Naylor and Sell [II J .

LetX=Lz [O,7I"1 andY= {fEL2 [O,7I"lffisconstant}. It can he checked t.hat. Y is a closed subspace of X and hence itself a Hilbert. space. Every h E ~ [0,71"] can be decomposed as

Example 3.4.1

h = f -I- g,

f E V and 9 E VJ. (By Projection Theorem).

For pxamplp, 11 = sin x can bp decomposPd as sin x = fl -I- (sin x - a), a E 1'. The constant n is determined from the orthogonality of 9 = sin x - n to every element e in Y
(e,,q) =

I' 1 e{sinx 0

a)dx = U

= e(2 - (m).

2 Since c is arbitrary, we have n = -. Thus. til(> projection of h = sinx on Y

is the functIOn f(x) = -. Let us define an operator P on Lz [0,71"1 by

n
Ph~ P(f+g)~f,

fEY

Pg =
SlllX -

(l

9 E 1;.!.

,
-

I' 2 (10 (sinx-7I")'!tlx )'1' I" 4 1 [( 1 (sin'! x ..... j'T2 - 11" sin,r:}dx )]'1'
0

( 2"
IIsinx - ell -

-I- :; - 71"

8)1/2

::::: 0,545

[1~ (sin z x ...... ,] _ 2CSinX)tlot-] II'!

[; -I-c27f _ 4C]I/Z

...... "'_'_""""'''''''10''-''

3.5

Projection on Convex Sets

We discuss here the concept" of projection and projection operat.or on U)[lvex ~t.s which are of vital importane p in such diverse fields as opt.imization, optimal control and variational inequalities.

Definition 3.5.1 (a) Let X be a Hilbert space and /( c X a non-empty dosed conveJ\ set. For x E X, by projection of x on [(, we mean the element z E K denoted by PIr) such that

IIx - PK (x)lI x <: IIx - yllx Vy E 1< or IIx - zll = ~MJ( IIx - yll
or(x-z.x-z) < (x-U.x-y}YyE
1(.

(3.5.1)

(b) All uperat.or 011 X into K, denoted by PJ(, b calk--d the llroJEctum opemtor if PK(X) = z, where z is th(> projection of x on 1<.

Theorem 3.5.1 (Existence of Projection on Convex Sets). Let f{ be a closed wnvex subset of a Hilbert space X. 'fIlIOn for any x in X, there is a unique element in [{ closest to x; that is, there is a unique element z in [( such that (3.5.1) is satisfied. Theorem 3.5.2 (Variational Charaderization of Projecdon). Let I( IJe a close(l convex set in a Hilbe,t Slmce X. For any x E X, z E is the projection 0/ x if and only if
(.T-z,y-z):$"O'tlyEK.

f{

(3.5.2)

Proof of Theorem 3.5.1. The proof follows from r,mmfl 3.4.2 find Remark 3.4.2 if we observe that the set x - l< consistinf;;i. of all elemems of the form :r - y for all y E J( is a dosed convcJ\ set. Proof of Theorem 3.5.2. Let z be the projection of x E X. Then for anya,O:$" a <1, since [\' is convJ\, O'y+(l-a)z E [( for all y E Ie Now,

IIx - ("" + (1 -

a)z) II' ~ 9(a)

(3.5.3)

is a twice continuously differentiable function of a" Moreover, g'(a) = 2(x - 0''1/ - (1 - a)z, z - y)

g"(a)

*-y,z-y).

(3.5.4) (3.5.5)

.n",

'"'_'_""""'''''''10''-''

0> -2
Q<

Figure 3.r.l

Gf'Olllf'l.rical illl.('xprp/.atioll of the characl.erizal.ion of projection

Now, for z to be the projection of x, it is clear that 9'(0) 2': 0, which is (3.5.2) [n order to prO\'e the converse, let (3.5.2) be satisfied for some element z in K. This implies that. 9'(0) is 1l0lHlcgative, and b) (3.5.5) 9"(0') is non negative. Hence, 9(0) S 9(1) for all y E [( such that (3.5.1) is I satisfied.

Remm'k .'/.5.1 A geometrical interpretation of the characteri7,at.ioll (3.5.2) is given in Figure 3.5.1. TIle left hand side is jlL"t the cosine of the angle 8 between lines connecting the point PIx) == Z with t.he point x and with the arbitrary point y E K, respectively, and we have t.hat cosO S 0 as 0 > ; necessalily holds. Conversely, for every other point
z' E l{, z' t- PK(x), there exists a point y' E [( so that the angle between rr the lines connecting the point z' with point.., x and yl is less than 2

Theorem 3.5.3 The ])mjection opemtoT Pj( defined on a /-lilbert sJKlce X into its non-em])ty closed convex subset [( 1JOssesses the following pmpe,-ties:

la) IIPK(U) - PK(v)1I 5 lIu - vII ~ U,v E X (b) (P,du) - Pj(v), u - v) > 0 V U,v E X (c) PJ( does not satisfy (PJ(u)-PJ(v), u-vOVU,vE.\'

(3.5.0) (3.5.7) (3.5.8)

.n", "'_'_""""'''''''10''--'

(tl) PK is nonlinear

(e) PI{ is continuous.

Proof of Theorem 3.5.3.

1.

(a.) In view of (3.5.2).

(PJ{(u) - U, PK(U) - v) :'5, 0 'r:/ v E J<.

(3.5.9)

Put

Ul

in (3.5.9) to get

(PK(ud - Uj,PK(ud - v)

< U 'r:/ v

E J<.

(3.5.10)

Put u = U2 in (3.5.9) to get

(PK(U2) - U2,Pr,(U2) - v) :'5, 0 'r:/ v E }{.

(3.5.11)

Since PK(U2) and PIud E J<, choose v = PK(U2) and v = PJud, respectively, in (3.5.10) and (3.5.11); then"fore we get.
(PK(ud - til, PK(uJ) - PIU2) :'5, 0 (PK(U2) - UZ,PK(UZ) - PK(ud} $ 0

(3.5.[2)

(3.'>.13)

By (3.5.12) and (3.&.13), we get

(PK(ud Ut -

PK(u..d + U:l, PruJ) - PK(UZ)

< 0, (3.5.14)

(PK(ud - PK(UZ),PK(Ul) - PKtuz)

:'5, (Ul - Uz, PK(Ul) - P,duz),

IIPK(ud - h(u,)II' < lIu, - u,IIIIPK(ud -

PK (u,)II,

(3.5.161

by the Cauchy-Schwaltz-Bunyakowski int:quality, ur

(3.5.17)

...", "'_'_""""'''''''10''-''

(b) It follows from (3.5.15) X

as IIPK(ud -

2 PJu2)1I ?: 0 'r:/ UI,U2 E

(cl Lf J< = X then PI< = 1 (Identity Operator), then

{U -

v, u - v} > 0 if U

i- v.

However, if [( i- X; t.hat. is Uo E X and tlo Uo but PK(PK(tl{)) = PK(110). (d) It is clear that P1\ is non-linear.

K, then PK(UO)

i-

(e) Continuit:v follows from (3.5.17) I. cause Un --t u implies that PK(U,,) --t P1\(u) as n --t 00. I

3.6

OrtllOnormal Systems and Fourier Expansion

Fourier Expansion in Hilbert Space

Definition 3.6.1 A set of vectors {u,,} in an inner product space is called orthonormal if
(3.G.l)
where Ii.j is the Kronecker delta. Theorem 3.6.1 dependent.

An orthonormal set of non-zero vectOl's is linearly in-

Proof of Theorem 3.6.1. the linear combination

Let {ud be an orthonormal set. Consider

+O'"u,,=O.

0'Iu\+a2U2+"

0= {O,UI} = {0'1111 +a2U2+

+anu,,,ut}

= 0'1 (Ul' Ul) + 0:2(112, Ul} +

Since (u;, Uj) = Ii,j, we have

+ a" (u,,, UI)'

O=O'I(UI,ltl} oral =OasUl

i-O.

.n .. ,

"'_'_""""'''''''10''-''

Similarly, we find that 0:2 = 0:3 = ... = set. of linearly independent. vectors.
Example 3.6.1

0 ...

= O. This shows that {u,} is a

(i) In the first place, it may he noted that any non-zero set of ort.hogonal vectors of an inner product space can be converted into an orthonormal set by replacing each \'ector u, with u, j lIu; II . (ii) Let us consider the set {co..<;nx}:"o ill L2 [-Jr, Jr] which is orthogonal.

c: '-os x, c: cos 2x, is a set of orthonormal vectors in 2 [-Jr, Jrl . v2Jr vJr vIr (iii) Let X = ~ [-1, 11. u = 1, v = bx E 2 [-1, 1], where b is a constant n and v are orthogonal as
Ii'>'

(u, v) =

" j-, bxdx = ["X2'] '_, = 0

[J

=2 ifu=v=l. The functIOns

u = nj lIull =

I/V2 and v = vi IIvll = /fx are orthonormaL

Definition 3.6.2
1. All orthogonal set of vectors {1}i.} in an inner product space' X is

called an orthogonal basis if for any u EX. there exist scalars such that
00

0;

u = L:Oit/!;.
;=1

If clements VIi are orthonormal. then it is called an orthononnal basis. 2. All orthonormal basis {'1ft;} in a Hilbert space X is called maximal or complete if there is no unit vector Wo in X such that {t/Jo, WI, 2,' .. } is an orthonormal set In other words, the sequence {W.} in X is complete if and only if t.he onl~' vectol' ort.hogonal to each of Wi'S is the null vector. 3. Let {wd be an orthonormal basis in a Hilbert space X, then the numbers 0; = (u, t/J,) are called the Fourier coefficient.s of the element u with rcsjwct. to thf' system {,pol and 'L,o;t/!, is called the Fourier series of the element u .

B.

.....

, to<l',_"""",,,,,"l1>"-I

Theorem 3.6.2 (Bessel's Inequality). basis in a lfilber't space X. F01' eaeh U E X,

Let

U'd

be an O1,thon01Tnal

L I(u,1",)1' < lIull'. ,-,

=

(3.6.2)

Theorem 3.6.3 Lci {l;\} be a eountably inJinite orlllOnomwl sci m a Hilbe,'t space X. Then the following statements holt!.
f. The mfinite senes

L
=

a"tP", when! On 's an~ seala7'S, eonverge,'1I! tmrl converyes,

n=1

only i/ the SC7"ies 2. If

L 10,,12
,,=1

L Q"tPll
..=J

convCfges ant/

tl.=

= La"'IjJ,, == LfJ"'IjJ,,,
,,=1 ,,=1
2
C>O

then

0"

= {3"

/01' aun Ilfld

Ilull = L la,,1 .
"=1

Proof of Theorem 3.6.2.

\Ve have
N

,
> O.

u- L{U,\C'i)tP;

=,

L.H.S.
N

Ilull' - 2 L
i-I

(u, ",,)(u, ,p,)

+ LL(u,>!,.)(u,,pj)(v,,,,pj),
=1 j=1

by using properties or inner pluduct,

lIull' -

L l(u.,p;)I'
1=/

...", "'_'_""""'''''''10''-''

by applying th fact that

('I/J., tP)}
This gives us

= 0,) = 0 if i
= I iri=j.

i= j

L l(u,.,,)I' < lIull'

.=]

Taking limit N --+

00.

we get

L 1(. ",)1' <: 11.11'.

=,
Proof of Theorem 3.6.3.
1. fRt

00

L a"'I/J,,
n=1

hp. convergent. and a.,,"ump thai

00

L an'~""
...=1

or equivalently

N-too

lim

u "" a" 'I/J" L..J

"=1

= 0
00

(u,'Ij;",) = {La'ltP",'l/Jm )
...=1
00

= La"(tp,,,tpm)
11=1

m= 1,2,'

= Of/!

(by Uetillitioll 3.6.]).

By the Bessel inequalit},. we get

m=1
00 ,

which shows that

L laml
,,=1

cOllverges.

.,.... "'_'_""""'''''''10''-''

To prove the converse, assume that finite

SUIlI

L: lanl
0=1

COli verges.

Consider the

s" ==

'=J

" L a,tP;.

We have

II'" - '",II' ~ (

" L

a,-p"

" L
'="'+1

a,-P,)

" L
==",+1

10:.1 2 ---t 0 <lli n,m ---t 00

This mealls that {s,,} is a Cauchy sequence. Since X is complete, the SC<jUellCe of partial sums {s,,} is convergent in X and therefore the series

L a.,tP" converges.
;=J

2. We first prove that 1It111 ==

We hayf'

,,=1

IOn 1 .

IIull

L
n=1

10:" 1 = (tI, u) -

L L (a"t/J,,, u",W",)
"=1 m=1

- (tt, U

L
n=1

Ct'l

tP,,)

{L a" 1/'."
,,=J

tv

tI -

L
n=1

tv

0." V'n}

Sincc result.

L o:"t/J"
n=1

converges to

tI,

the AI COllverges to zero, proving the

[f U =

L a.,1/"1 = L lJ""I/J,,, then

n=1 ,,=1

by part 1,

implying that an = {3". for all n.

I
Let Y be the dosed a flilbc,"t space X .

Theorem 3.6.4 (Fourier Series Representation). subspace spanned by a countable o,-tJlOnormaJ set {'l};,}

HI

B.

.... ,

, toOl',_"""",,,,,"l1>"-I

'l1len every element u E Yean be written uniquely as v ~ L(v.,p,)V';.

,=1

(3.6.3)

Theorem 3.6.5 (Fourier Series Theorem). Fm' any O1'l1wnormal !'let {1p,,} m a flilheft spaee X, the followf.ng statements arc eqU1.'vu.1ent:

(n) elW'Y

in X r.fln he

7'Cp'1'~~r.ntnJ

by the Fmn'ir.,>

sn'ir..~

in X; Owt

i.~!

=
u= L(U,l/J,)"I,IJ,.
,=1

(3.6.4)

(b) For any pair of vectors

tl,

v EX, we have

=
(u,v) = L(u,W,)(v,,)
i"" I

(3.6.5)

=
wheH~ 0,

L 0.,(3"
.=1

00

= (U,l/!;) =

coefficients of 1).

Fom"ie,' coefficients ofu.(3 = (v,W.) = Fourier {Relation (3.6.5) is called the ParSelJa1 formula}.

(e) Fm> any"U EX, one has

lIull' ~ L l(u,V',>I'.
.=1

00

(3.6.6)

(rl) Any subspace V of X that contams {1p;}

IS

(lense m X.

Proof of Theorem 3.6.4. Uniqueness of (3.6.3) is a consequence of Theorem 3.6.3(2). For any U E Y, we can write

u = Jim ""' o.,t!,;,

/'.'-->00

;=1

L.J

"

M >N

as Y is closed. From Theorems 3.4.2 and 3.6.3, it follows that

II
M

Ai
V -

B.

......

II
, to<l'._..

- L (v, 'P,)V', <

=,

L(Q"V',)
;=1

....,..",,"l1>"-I

and as N --+

00,

we get the desired result.

Proof or Theorem 3.6.5. (a) => (b). This follows from (3.6.3) and the fact t.hat {'I/J;} is orthononnal. (b) => (c). Put tt = v in (3.6.5) t.v get (3.6.6). (a) => (d). The statement (<1) is el:jui\'alent t.o the st;Jtement that the orthogonal projection onto 5, the closure of S, is the identity. In view of Theorem 3.6.4, statement (d) is equivalent to statement (a). I

Example .'/.6.2 (Projections and Orthonormal 11ass).

1 \Ve now show that the Fourier analysis described above can be used w construct. urthogonal projections onto closed subspaccs of Hilbert spaces. Let {tP,,} be an orthonormal basis of a Hilhert. sparp. X. Then each U E X can be written as
=

11

L O'.ntP" = L O'.n'l/;n + L
.=1 .,=1 '1=1\' .... 1

00

00

0'."'1/1,,,

where on's are Fourier coefficients.

Let the set {'I/;I, th,'" ,'Ij!N} forms a basis for all lV-dimensional subspace F of X. The operator P defined by

Pu =

L (u, tp")'l/J,,
n=l

is an orthogonal projection of X ont.o Y. rn fact, P is linear and p 2 = P; that is

N

P(U+1!) = Pu+ Pi!: P(u+v) = L{U+1!,'I/;n)'I/;"

,,=1

L(U.'I/J'I)tP'l + L{v,'I/J,,)'I/;.l
n=l
N

(by Definition 3.2.1(1))

,,=1 = PU+ P1)

P(>.u) = >'P(u) , P(>.U) = L(>,u,lbn)<!'"

,,=1
N

= ,\ L(u,"!")W,,, (by Definit.ion 3.2.1(2)).

11=1

...... "'_'_""""'''''''10''-''

p2 U

=:

P(PU) = L(PU,lp"}W,,
..=1

L (L (u, "m)"m, V',,)""

.,=oJ m==l

= L(u,1jI,..)
n=J

L ('1/1",tJi",)'l};",
m=J

=:

L {u, 1/,,, )1,('

0=1

(by u<;jng orthonorrnaJity of 1/ln )

= Pu

Vu.E.'\..

Thus, p2 = P.
2. R(P) = V To show that R(P)J...N(P), let, v E N(P), and such that (v, u) = (ii, Pu)
N
=:

it

E R(P)

=:

{v,

L (tt, 'ljl.,)'l/;.,)
..=1

L(1!,W,,)(U,vJn)
=1
N

= (L(V''''')'''''u)
=\

= (Pv, v).

Since v E N(Pl. we have Pv

=:

0 and hence N(P)J...R(Pl.

Now, we show that th.. projection error, that is, u - Pu. is orthogonal to

Y:
FbI' V E V, (u - Pu,v)
=: (

L
n=N+1

00

o:"1/;,,,1!)::::

L
.,=N+l

00

a.,('1/'".1!) --t U as IV --t 00.

Furthermure,

II Pull

:::::
=:

L I(u, tPn}12 , and if

,,+'1

1)

E Y,
1""

lIu _1)11 2

, , Ilv - Pull + IIv - Pull

-1)

(U

-r PU- Pu,tt - v

Ptt- rU}

v E

lC

.n",

"'_'_""""'''''''10''-''

I-Ience, t.o mnke lIu ~ vll 2 as small as possible u Y, we must take v = Pu; that is,
vEl'

inf

IIti - vII

Uu - Pull

Remark 3.6.1 Results of Theorem 3.6.3 can be stat.ed in alternat.ive = tcrms as follows: For an element u EX, L (u, W;)t/J. converges to uQ = Pu
in the closed subspace Y spanned by the 1/'. 'so The vector u is orthogonal to Y.
;= I
-1l()

U -

Pu

3.7 Duality 8Jld Reflexivity

3.7.1
[II

lliesz Representation Theorem

this section, we prove a theorem which gives the representat.ion of a bounded linear functional defined on a Hilbert space, and wit.h t.he help of t.his theorem, the relationship between a Hilbert spacp and it.s dual is studied.
Theorem 3.7.1 (Riesz Representation Theorelll). If f is a bounded linear ftmctional on a Hilbert space X, then tliere edsts a unique vecto, y f' X such that f(x) = (x,y) Vx E X. and II/II = lIyll.

Proof.
1. Ln the first place, we prove t.h"t there f'xists an element y sneh that [(x) ~ (x,y) .x E X.

(a) If f = 0, then f(x) = 0 \Ix E X. Therefore, y = 0 is t.he vcctor for which (x,O) = 0 Vx EX. We have seen that (x,O) = O. Thus, the cxistence of vector y is proved when f = O. (b) Let / =I- O. By Proposition 2.4.1, the llull space N of f is a proper clasen subspace of X and by Lemma 3.4.3, therc exists a non:lero \'ector u E X such t.hat ul-N We show that if 0: is a suitably chosen scalar, y = o:u satisfies the condition of the theorem. (c) [fx ENe X, then whatever be o:,f(x) =O=o:(x,tI) as tll-N. Thus, f(x) = (x,o:tI). Hence, the existence of y = o:u is proved for all x E N .

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

(d) Since u.LN, u 2 0' lIull , then

r;. N; let

;r

= u E X - N. If f(u) = (u,au) =

Therefore, for a =

Hon of the theorem in this case, t hat is, /(11) = (u, au). (e) Since u

11,,11

f(u~, the vector y =

0 And so

au satisfies the condi-

N, f(u}

-#

~~:~

is defined for any x E X Then f(x - Bu)

Consider x - Bu. where B

= ~~:~.

= f(x)

(3/(u) = O. This implies that:l: - (3u E N. Every x E X can be written as x = x - f3u + Bu. Therefore. for each x EX, I(x) f(x - ~u + ~u) ~ I(x - ~,,) + I(~u) ~ I(x - ~u) + ~/(u) (f i, Uncar) = (x - (3u, au) + !3{u, au}
~
0'

by (c) and (d), wh~re

f(x -/1u)

= --:r

I(u)

lIuli

(. Smce x - (3u E N, hy (c),

I(u) = --, ) .

= (x -

Bu,au) for every a and so for a

This gives us /(x) = (x - {3u,au} + ({3u,au) = (x, au). Thus, for all arbitrary x EX, there exists a vector y 'au such thai I(x) ~ (x.y). 2 We lIOW sbuw that y is unique. Let y\ and Y2 L~ twu vectors such that f(.1:) = (X,1/') Vx E X ('Ind J(x) = (X,Y2) Vx E X. Then (x,y\) =:: (X,Y2) Vx E X or (x,y, - Y2) = 0 '<Ix E X. By Remark 3.2.1(7), Yl - Y2 = 0 or YI = Y2. This proves the uniqucness of the vector y. 3. For each x EX, there cxisls a unique y E X such that

lIuli

I(x) = (x,y) 0'

I/(x)1 ~ l(x,y)l

By the Cauchy-Schwarz-Bunyakowski inequality,

Hence, By the definition of the norm of a functional, we have

l(x.y)1 <ilxll lIyll I/(x)1 " lIyllllxll 11111 " Ilyll

(3.7.1)

...... '.'_'_""""'''''''10''-''

Lf lIyll = 0 or y = 0, UJen I/(x)1 = l{x,O)1 = 0, for all x and so

11/11 ~ ,up {'ll~lll/x" o}

by Theorem (2.4.2),

0; i.e.

IIIII ~ II_II. Suppose _ ,,0. Then

(3.7.2)

or

11/11 > lIyll By Equations (3.7.1) and

(3.7.21. we have

11/11 = lIyll .

I
Theorem 3.7.2 Let y bc afixal clemcnt of a Hilbert space X. "fhenthe junctional f" (lcfined below, belongs to X
/,,{x) = (x,Y)

1m'

x E X.

The mapping 1jJ ; y

I.

--i

f ll of X into X satisJies the following pmpe'ties:

11,,(_)11 ~ II_II
+ yz)
~

:J. ?,b(y.

= ?,b(1Il )

+ ?,b(yz).

4. ",(a_)
5.

a"(_).

V,

is one-onc.

Remark 3.7.1 If X is a real I-lilbert space, then in property (4) of '1);, we have V!(o:y) = o:'I);(y). Then Theorem (3.7.2) means that every 1'&,\1 Hilbert space X can be identified with its dual, i,e., X = X. (See Definition 2.3.4 and Remark thereafter.)

Proof.

Ln order to show that

/11

EX, we need t.o verify that

...... "'_'_""""'''''''10''-''

Vefifimlion.

By the definition of the inner product,

(b) I,(ox) ~ ol,(x).

Verifimlion. //I(cxx) = (ox,y) = a(x,y), by t,hp definition of the inner product, or /lI(OX) = o/y(:I:).
(c)

/y is bounded.
Verification. lfll(x)1 = l(x,y)1 ::; IIxJl lIylJ, by the CauchySchwarz Bunyakowski inequality. Thus, there exists a k = /lYIl > 0 such that l/y(x)1 ::; k IIxli. That is, /11 is bounded.

Now, we verify the properties of1)J.llw(y)1I = II/yll. By the second part of Theorem 3.7.1, II/yll = Ilyll Helice, lI'1/I(y)1l = !lYIi. Since Theorem (3.7.1) states that for every / E X', there exists a unique 1J such that /(x) = (x, y) 'r/:r. E X, the llIapping 'l/; : Y --t /y is ont.o. 'l/;(YI

+ yz)(x) = /1J1+Y2(X) = (X,YI + yz) = (X,YI) + (x,yz) (by Remark 3.2. 1(4a)) ~ I,,, (x) + I,,(x) ~ ,pry, )(x) + ,p(y,)(x)
~

(,p(y,)

+ ,p(y,))(x) Vx
+ .'(y,)

EX.

Hellcl'..
,pry,

+ ,,)

,'(y,)

w(ay)(x) = /Oy(x) = (x,ay)

= o(x,y) (by Remark 3.2. I (4b)),

<I'(oy)(x) ~ "<I'(y)(x)V x E X.

.n". "'_'_""""'''''''10''-''

Therefore, ljJ(oy) = 01P(y). Let 1/!(yt) = fj!(Y2)' Then fVl(X) = fmV.r:)Vx E X or (x,y.) = (x,Y2) or (X,YI - Y2) -- o. By Remark 3.2.1(7), YI - ifL = O. Hence, Yl = 112 Conversely, if YI = Y'l, then (x, Yl -Y'l) = 0 by Remark 3.2.'I(4a) or (x, Vt) = (x,Y'l) or 1/J(yd = 1,b(Y2). Hence, 1p is one-one I

Remark 3.7.2
1. Theorem 3.7.1 was proved by the Hungarian mathematician lUesz around 1910. It is one of the most important results of Hilbert space. It has sevcral applications of vital importance. Applying this theorem \\"e prove in the next subsection that every Hilbert space is reflexive. 2. Since ROl, 2, L2(a,b) are Hilbert spaces, by Theorem 3.7.1, bounded linear functionals defined on R". 2, and L2 (a.b) are. relspectively. of the following forms: (a) For F E (m'r, thcre exists a unique a = (aI, a2, a3,' .. ,all) E R" such that

..

F(x) = LX,lI"

where

X=(XI,X2,X3,' ,x.,)ER"

.=1
= (x, a) - (a, x).

(b) For F E (2)- , there exists a unique a 2 such that

~

..c;:

(aI, a:.:,a3, . .. ,a" ... ) E

F(x) = LXiiT;,
1=1

where

x::: (XI,X2,'

,x",'" ,) E 2

(x, a).

(c) For F E (2)-, there exists a ulliQue lJ E 2 such that

F(I)

~
~

f(t)y(t)dt,f E L,

(1,9)

3.7.2 RefJexivit.r of Hilbert Spaces

Let us recall that a Banach space X is called a reflexive Ban<lch space if it can he identified with its HeCond dual (X'r = X". This means that a Banach sp<lce X is reflexive if there exists <I mapping J on X into X-'

Q .,_,_......,..",,"IO"-I

......

which is linear, norm preserving, 1-1 and onto. In view of Problem 2.8.4, to verify whether a Banach space is reflexive or not, it is sufficient to show that the natural mapping is an onto mapping. Theorem 3.7.3
Eve7'Y Hilbert space X is 7'eflexive.

Proof. To show t.h<lt the ll<ltur<ll embedding J : x --+ Fx of X into X" is an onto m<lpping, we h<lve to show th<lt for <In arbitrary clemcnt FE X', there exists z E X sud1 tlmt J" = F. Let 9 be <I functional on X defined as follows:

Properties of 1j; are given by Theorem 3.7.2. Using the properties of 1j; and the fact that FE X" , \\e see th<lt

g(XI

+ X2)

= F(1};(xl

+ X2 ~ P(1'(x,) + rp"'(,ul,(-'-x,'))
of(1,(x)),

9(ax) ~ P(,,(ax)) ~ aF(>!>(x))

~

and

19(x)1

IF(>!I = !P(,,(x))1 < I!PII 1I>!>(x)l1 I!PII IIxll

Hence, 9 is a. bounded linear functional on X. By Theorem :1.7.1, there exists a unique z E X sud1 thaI

P(1'(x)) = (x,z) PC,,(x)) ~ P(>!>(x)) ~ (x,z) ~ (z,x).

(3.7.3)

Prom the definition of the natural imbedding J and Thcorem 3.7.2, (3.7.4) From Equauons (3.7.3) and (3.7.4), we have It = F which is the desired result. I

Q .,_,_......,..",,"IO"-I

.... ,

3.8
3.8.1

Operators in Hilbert Space

Ac{ioint of Bounded Linear Uperators on a Hilbert Space

Let. X be a Hilbert space ,:IIld T : X --t X be a boundt--'d linear operator 011 X int.o itself. Thp-I! the adjoint oJ.!~mtor T* is defined by
(Tx, y) = (x, T*y) V :/', Y E X.

Definition 3.8.1

Remark 3.8.1
L T always exists.

Verification.
/y(XI

Let (Tx,y):::: fy(x).

'&2) = (T(xl + X2),Y) = (TXI ,V) -\- (TX2,Y) = fY(XI) + /y(X2) J,(ox) ~ (T(ox),y) ~ a(Tx,y) ~ oJ,(x).
-l-

Now, 11,,(x)1 = I{Tx,Y)1 < IITxU 1I11J1 (by Cauchy-Schw<:lltz-Bunyakowski inequality) < IITllllxllllyfi < k IIxll for a fixed y E X. Thus, Jy E X and by the Riesz theorem, there exists a y* E X such that

(Tx,y)

J,(x)
---t

(x,y') Vx EX.

Thus, T induces a IiHear map y defined on X into itself.

y. and we write y. = Ty, where T* is

2. T* is bounded, linear and unique.

Verification. IIT'xll' ~ (T'x,T'x) ~ (T(T'x),x) < IIT(T'x)1I IIxll b~' Caudly-Schwartz-Bunvakowski inequality. Since T is bounded, there exist.s k > 0 such that IIT(T'x)1I
Hence,

<: kllT'xll

II(Tx)II' < k IlxllllT'xll,

or

lI(rx)1I '5: k IIxli.

Hence, T" is bounded. For aHz E X,

(z,T(x+y))

(Tz, (x-<-y)) = (Tz, x) -<- (Tz, y) = (2, (TOx) + (2, TOy) = (z. T"x + T"y),

(z, [T'(x

+ 1/)1- [T"x + T"lIJ)

= 0 V z E X.

ThIS implies (See Remark 3.2.1 (7)) that r(x+y) = T'x+T"y V z E X. (z,T(ox))
~

(Tz,ox)

= Q(Tz,;r.) = o(z,TO(x))
~

(z,oT(x)),

(z,T(ox) - or(x))
Hence,

0 V 2 E X.

r(ox)

or(x).
operators Tt and Ti. such

Thus. T" is linear. Suppose that for a givcn T, there exist that

tWO

(Tx, y) = (x, T,y),

and

(Tx,y) = (x. Tiy).

Then wc have
(x, Till) - (x, T;y) =

u,

(x,Tjy

T;y) = 0 V x E X .

Q .,-,_......,..",,"I>"-I

..... ,

This implies that Tty = T;y 'r/ y EX; i.e., Tt = T;. Hence, the adjoint of T is unique.
3. The adjoint operator of T" is denoted by
T~

Example 3.8.1 Let X = R" be a real Hilbert space of dimension nand foraJl x : (XI,XZ, ,x,,) E R", let T be defined as follows:

Tx =y

where y= (UI,Y'l,'" ,Yn); ~h =

.. a"x L

and (ai;) is an n x n

j=J

matrix; and T is bounded linear operator on R" into Itself.

2i

.. =L
j=1

aj;Yj

Thus, if a bounded linear operator T on R" is represented by an n x n matrix (a,;), thf' adjoint T of T is reprf'Sented by (aJ .), transpose matrix of (a;j).

Example 3.8.2
Tf(t) =

lb

Let X = Lz(a,b) and let T : X --+ X, (defined by

]{(8,t)f(t)dt, where ]{(s,t), a

<

s ::::: b, a

<

<

b, is a

Q .,_,_......,..",,"IO"-I

..... ,

continuous function) be a. bounded linear operator. Then,

(Tf,g)

~ [(Tf(t))9(') ds

~ [ ( [ f{("t)f(')d') 9(,)d,
= [f(S)

(t

J{(S")9ft)d') ds

~
~

l'

f(') ( [ [{is, t)9(')dt}I'

(J, T" g),

where

T-g =

by TW g =

1"

f\(t,s)g(t)dt Thus, the adjoint operat,orTw ofT is given

It,s)g(t)lIt.

Example 3.8.3

Let X = (2 and T: z -t 2 be defined by T{(OI,02,

03""'O", ... )} = (O,OI,O't,

,a",). For

(Tx, y) ~
~

" I: a,fi,+,
k=1

(x,T"y),

where Tw {(PI, th, (33, ... )} = (fh, ~3, ... ) . Thus, the adjoint of T b T. T is known as the right shift operator and T the left shift operator. Theorem 3.8.1 Let T be a bounded linear operator 011 a llilbe,t space .X into itself. Then its adjoint olJemtm' T has Uie following properties:

1.

I~

= I

wlten! I is the identity oJK:mtor.

2.

(T+S)'~T"+S.

3. (aT)'

~
~

ill" .

,. (TS)'

ST".

",,,. "'_''''''''''''''''''''''10''-''

5. T"" = T.
6.

IIT"II ~ IITII IITII'

r- and (Tw)-l
=

7. IIT"TII =

8. 1fT is invertible, then so is

(T-1f

Proof
I. Sincc Ix::=:c 'Th; E X, we have (/x,y) ::= (:c,y), and by the definition of /., (lx,y) ::= (x,ry). Hence, (lx,Y) ::= (x, J*y), or (x,y - ly) = OVxE X. Thus, y - /*y =0 or ly =y; i.e. 1::= I".

2. (T + S)" = T" + S" > (T + 5)" (x) = T"(x) + 5" (x)Vx E X. By the definition of (T -I- 5)" 'rIx EX, we h,we T -I- 5)2, x) = (,2, (T -I- S)' x}. Also, T + Skx) = (T,

+ Sx,x)
+ (S,.x)

= (Tz,x)

= (z, T"x)
TllU~,

+ ("S"x).
im~lies

(z,(T+S)*x) = (z,T*x+Sx) Vx and z E .Y, which that (T -I- S)x::= Tx -I- Sx

3. aT) "x) = (', (aT)" x)

aT)"x) ~ a(Tz,x) = a(x,T(x))

= (x,err"(x).

Th(!se two relations show that (2, (oTt x - orx) = 0 'rIz and x. Hence, (aT)'::= ill' .
4. (TS)(x) = T(S(x))

TS) (x),y) = (T(S(x)),y)

= (Sx, T'y) = (x,S"(T"(y)) = (x,(S"T")(y))

On the other hand,

TS) (x),y) = (x, (TS)"y) (x,(S"T")y) = (x, (TS) " y),

0' (TS)" = SOT".

5.
(Tx, y)
~

(x, T'y)

T')' x, y),

or T-r")x,y) :::: 0 'fly E X, where T'~ :::: (T)*. I-Ience, T=T"*.

6.

IITII ~ IIT'II $> IITII < IIT'II and IIT'II < IITII. We h"e 1IT'(x)II' ~ (T'x.T'x) ~ (T(T'(x,x) < IIT(T'(x))1I IIxli < 1IT'(x)1I IITII Ilxli.
IIT'(x)1I <;

IITII IIxll

This Implies that IIT'II < IITII (See Theorem 2.4.2). Applying this relation to T", we ha,ve IIT"II S IIT"II. However, T'~ :::: T and, the,elme, IITII <; IIT'II
7. IIT'TII < IITII' as IIT'T(x)1I <; IIT'II IITllllxll ~ IITII' IIxli (in view Theorem 2.4.2 and the previous relation). On the other hand. IITxll' ~ (Tx, Tx) ~ (T'Tx, x)

of

< IIT'Txl1 Ixll

m IITxll IITII' .

IIxll' , < (IIT'TID'I' IIxll m' IITII' < IIT'TII.

<; II T 'TII

Hence,

liT' Til

8. Since ,. = T and IT-I:::: T-lT :::: I, tIT-I)" :::: T' :::: I; and and by (4), (TT-l)* = (T-1)*To Therefore, (T-I)*T" = T and consequently (T)-l = (T- 1 )*.

I
Remark 3.8.2
1. The definition of the <Idjoint opennor may be extended to bounded linear operators defined on a Hilbert sp<:Ice X into another Hilbert sp,we Y in the following manner:
(T3:,y}V :::: (x,T"y)x 'r/x E X, Y E Y

.\'

T" : I'-+X.

Q.o,

.... ,

......,..",,"IO"-I

2. Let T : X --t V be a bounded linear open:ltor on a Hilbert space X into another Hilbert space Y. Then the null and range spaces of T ,md its adjoint To are related by the following rf'Jations:
(aJ (R(T))" ~ N(T')

(b) R(T) ~ (N(T'))"

(e) (R(T'" ~ N(T)

(d) R(T') ~ (N(T))".

3.8.2 Self-Adjolllt, Positive, Normal and Unital)' Operatul:';

Definition 3.8.2 X into itself. Then
Lt T be a bounded linear operator on a Hilbert space

1. T is C<:l.lled self-adjolllt or HermItIan if T

== T*.

2. A self-,Hljoint operator T is called a positive operator if (Tx, x) > O'Vx E X. It is called strictly positi1Je if (Tx,x) == 0 only for x := O. Let Sand T be two self-adjoint operaLors on X. We S<ly that 5 > T if ((S-T)x,x) 2:: 0 'V xEX.

3. T b called normaL if TTo == ToT.

4. Tis c<:l11ed unitary ifTTo

== ToT == I, where 1 is the identity operator.

Example 3.8.4
1. T given in Example 3.8.1 is self-adjoint if the matrix (aij) is symmetric, i.e.. a'J == aJ, 'V i <:Iud j.

2. T given in Example 3.8.2 is self-adjoint if K(s,t) 3. The identity operator 1 is self-adjoint. 4. The null operator 0 is self-adjoint.

== Tt,s) 'V sand t.

Example 3.8.5

tet T: 2 (O, 1) -t 2(0,1) be defined as follows:

Tf(t)

'f(l).

T is a. self-adjoint operator. T is bounded, linear,

(Tf,g)

Q .,_,_......,..",,"IO"-I

.... ,

l' Tfgdt~ l'

tf(t),q(l)dt,

and

(f,Tg)

l'

fTgd'

l'

f(t)'9(')d'

~ ['f(t)9(')dt ~ [tf(')9(')d'", 'E (0,1).

Hence,

(Tf,g)

(f,Tg).
Lct X = 2, and T; 2 --+ 2 be defined as T({ad) =

De} {T . T

Example :.1.8.6
IS

bounded, linear and

~
~

(Tx, y)

'\' eL., Tfi,

k=l

00

(x, Ty).

Thus, T is self-adjoint.
Example 3.8.7 T given in EX<lInpl p 3.8.1 is ]JOsitiv p if Ihe matrix (at;)

is symmetric and positive. Let T ; X --+ X be given by T = 2iJ, where / is the identity operator. Then T is normal.
Example 3.8.8

TT

~ (2i')(2il)"

= -4/,
and
T'T
Example 3.8.9
~

(2U)"(2il)

-4J.

T given itl Example :1.8.1 is unitary if the matrix atj

coincides with the invPffip mRtrix of ai}.

Remark 3.8.3

1. Every self-adjoint operator is normal but t.he converse b not true, in

general. 2. Every unitary operator

IS

normal but the converse need not be true.

Q .,_._......,..",,"IO"-I

.... ,

Verification.
1. Let T be self-adjoint. Then T = T :::} 1'T' = 1'2 and 1"1' = 1'2. Hence, T*T = 1'1'. and l' is normat For the converse, consider l' as given in Example 3.8.8. 1" = (2i)/' = -2il :f:- T. Hence, T is not sell:'adjoint. 2. Since T i~ unitary, TT = T = 1"1' and su l' is nonnal. For the converse, we consider the abo"f' 0perator where 1"1' = T'T = -4[ :f:/. Hence, l' is normal but not unitary. The theorems given below provide interesting properties of self-adjoint. po~itive, normal and unitary opemtor~.
Theorem 3.8.2 Lct X bc a n,al flilbert Slmee. Thc set A(X) of all selfadjoint operators of a /-filbert SlKlce X into itself is a closed subspacc of the Banach slmce 6(X) of all bounded linear operatm"s of .Y into itself and, therefm"e, it is a real Banach space containing the identity olJemtor. Theorem 3.8.3 Let 1'1 and 1'2 he two self-adjoint opemtors on a flilbcrl space X. Tlum tJleir Ilmdll.ct 1'11''2 is Sdf-(uljoint if and only if 1'1 1''2 = 1'21'1 Theorem 3.8.4 A n operator l' on a liilliert space X is self-adjoint if and only if (Tx, x) is '"Cal V x E X. Theorem 3.8.5

1. T/ l' and S Gf"C two lKJsil1ve opemtors defined on a flilbert space X Sllch tJwt TS = ST, tJlen ST is posrtitle.
2. Tf l' is a bounded linem" operator. then 1'*1' and 1'1'*
an~

positive.

Theorem 3.8.0 The set of all normaL operators on a /-filbert space X is fI closed snbset of 6(.\'"), the Banach Slmee of bouuded Imw1' uperators 0/ X into itself, which contains the set A(X) of all self-adjoint olJerators and is closed under scalar multiplication.

...... "'_'_""""'''''''10''-''

Theorem 3.8.7 {fTi andT2 are lwrrnal opemtors on a Hilbert Sllace X with the llrollerly that either commutes with the adjoint of the othef>, then T 1 + T2 and T 1 T z are lwnnal. ThL>orem 3.8.8 A bounded Linear opcmtor T on a Hilberl space X ih nonnal1f Mld only if liT xII:::: IITxll fOf> every;r: E X. Theorem 3.8.9 Theorem 3.8.10
[fT is a llOf11lai ollCm/.or, fllen

IIT:l1I

IITU:l.

Let T be a bounded linP.ar opemtm> on a Hilbert Sllace T+T X into itself and T* be its adjoint. Further, suppuse that A = 2 and

T- T'
2i

(T = A

+ iB and T" :::: A - iB. A and B are called noal and

imaginlJ.fll parl.s). T 1S normal if and only if AB = BA.

Theorem a.M.ll A bounded lmoor operatorT on a Hilbert sllace X 1nto itself is unitary if and onty if it is an isometry of X onto itself.

Proof of Theorem 3.8.2.

LRt A Rml B Iw self-adjoint operators,

(wi

+ BB)'

~ (nA)'

+ (,38)' (by Thromn 3.8.1(2))

(by Theorem 3.8.1(3).

= cul*

+ {3D*

Thus, for real scalars er and {J, (nA + (JB)* = erA + {JB; i.e., erA + (JB is self-adjoint and the set of all self-adjoint operators A(X) is a subspace of B(X). To show that this subspace is closed, it is sufficient to show that if (Tn} is a sequence ()f self-adjoint operators 011 X, lim T., = T is also self-adjoint. We have ,
,,~=

liT - T'II

liT -T" +T" - T; + T; - T'II < liT - T"II + liT" - T,;II TilT; - T'II ~ liT - T"II + liT" - T"II + liT; - Til
~

as Tn's are self-adjoint. Thus, the limit of right-hand side tends to 0 as n --+ 00 and consequently T :::: T". [ E A(X) as 1 :::: [. by Theorem 3.8.1.(1) I

.n". '.'_'_""""'''''''10''--'

Proof of Theorem 3.8.3. Let T I T2 be self -adjoint. Then (T IT2)' = T IT 2. By Theorem 3.8.1(4), (Tt T2)" = TiTiand as T t and T2 are selfadjoint, (TIT'll' = T2T I . Thus, T IT2 = T 2T I . Conversely, let T IT2 = T 2T I . Then,

(x, (T,T,)' (yl)

~ ~
~

(x, TiT,(yl) (x,T,T,(yl) (x,T,T,(yl),

or (x, TIT2)" -T1T 2)y) =0. Hence, T I T 2 is self-adjoint. For proving Theorem 3.8.4 we require the following lemma: Lemma 3.8.1 If T is a b01lnded linea?' then T = () if (Tx,x) = O'r/x E X.
opt~rator

on a

Hilb(~rt

sIma; X,

Proof

We have

(T (ax + py) ,ax + f3y} - 101 (Tx,X) - 1f31 (Ty, y) = a{3(Tx,y}+aB(Ty,x) 11S = aQ(Tx, x} + (fO(Ty, x) + a{3(Tx, y}
~

+fJfJ(Ty,y) -101' (Tx,x) -lfJl' (TlI,Y) afJ(Tx,y) + ofJ(Ty,x) = RIIS.

By putting

0:

= i aUll

f3 = 1 in the above equatioll, we gel

i(Tx, y} - i(Ty, x} = 0,
(3.8.1)

and by putting

0:

= 1 and {3 = 1 in the above equation, we get

(Tx,y) + (Ty.x) = 0

13.8.2)

From Equattons (3.Kl) and (3.K2), we find that (Tx,y) = U 'r/ x and y. Hence, Tx = 0 'r/ x,Le. T = o. I Proof' of'Theorem 3.8.4. Let T be self-adjoint. Then (Tx, x) = (x, Tx) = (Tx,x) = (Tx,x). Hence, {Tx, x} is real 'r/ x E X Let (Tx,x) be real 'r/ x E X. Then,

0 . ,>0<1"_""""'''''''10''-1

......

(Tx, x) = (Tx, x) = (x, Tx) = (T"x, x),

((T- T")(x) ,x}

OVx E X.

By ~mma 3.8.1, T - T* :::: 0 or T:::: T"; i.e., T is ::.clf-adjoint.

Proof of Theorem 3.8.5. 1. Sim:e T h; self-adjoint, (T 2 x,x) (Tx, Tx) 2': 0; i.e., T2 > O. LeI T ::j:. O. \Vp defin<> a sequence of operators {T.,} in the following manner;

T1

::::

II T II

T z :::: T) - T?

T3 =Tz-Ti T4 =T3 -T; T..+1 :::: TO] -T;.

It can be verified that T., 's arc self-adjoint. where 0 S. Tn 5 I and

" L
i=l

T/x --+

Tix (For vedfication, one may see (I, pp. 415-416J). Since 5 commutes with T, it must commute with every Tn. This implies that. (TSx,y)
~

~ IITII (S lim "T,'x, x) " L

liT II (ST,x,x) "

~ IITIlIi"n

" I: (ST,'x, x)
i=l

" :::: IITllli~ L (STi'L, T,x).

,=J

Since 5 2': 0 implies that (ST;x. T,x) 2: 0 for every i, we have (TSx. x) > 0: Le., TS is positive. 2. (TT"x,x):::: (T"x,T*x}:::: liT xII'] > 0:::> is positive. Similarly, 2 (TOTx , x) :::: (Tx, T~ J as YO' =: T. Hence, (T*Tx, x) :::: IITxll 2: 0; i.e., TOT is positivc. I

Tr

Proof of Theorem 3.8.6. To prove the c1osedne&l, we 8how that the limit of the SffjUence of normal operators is a normal operator. Lct lim T" = T. Then lim T; = T*. 11-+= '1-->00
IITT* - T*TII = IITT* - T'1T~

+ T'1T.: - T.:T'1 + T~T'1 - TTII < IITT' - T,.T~II + IIT,.T; - T;T"II + IIT~Tn - TTII -+ 0
a8n-too

B.

.... ,

, to<l',_"""",,,,,"l1>"-I

which implies that TTo = T'T; Le., T is normal. By Remark 3.8.3(1), every self-adjoint operator is normal, and so the set of normal operators contains the set of self-adjoint operators. (aT) (aT)O = nCt (TTO) = oo(T"T) as T is normal. Thus" (aT) (aT)" = (aT)" (aT), and hence aT is normal. I Proof of Theorem 3.8.7. We have TIT'; = T';TI and T'}]t = T I T2. Using this fact and the fact that Tl and T2 are normal, we get
(Tl

+ T2) (T, + T2)" =

(Tl + 12)(Tj + T;) = TITi +T2Ti +T,Ti +T2Ti,

and
(T1

+ T2l" (Tl + T2) =

= =

+ Tn(T! + T2) TiTl + TiTl + T 1 oT2 + Ti T 2 TiT! + TlTi +T"}.T; +T;T2,

(Ti

which shows that T l + T2 is normal. Similarly, it can be seen that T 1T 2 is normal under the given condition. I Proof of Theorem 3.8.8. liTO xII = IITxll :} IIToxll 2 = IJTxU 2 :} (T'x, T'x) ~ (Tx, T.x) <} (TT'x, x) ~ (T'Tx, x) <} ((TT' - T'T) x, x) ~ o V x E X. In view of this relation, Lemma 3.8.1 li\les us TT = ToT Thus, T is normal if and only if IIToxll = IITxll. Proof of Theorem 3.8.9. By Theorem 3.8.8, IIT'lxl1 = IITTxll = IIToTxl1 for every x, which implies that IIT'lII = IIToTII. By Theorem
3.8.1(7), IIT'TII

= IITII' and w IITII' = IIT'II.

TT'
~

Proof of Theorem 3.8.10.

Let AB = BA, then we have

(A + W)(A - iB) = A'l + B 2 + i(BA _ AB).

and
T'T
~

(A - iB)(A + W)

= A'l

+ B2 + i(AB - BA)

Since AB = BA, TT' = TOT; i.e., T is lIorlllal. Conven;ely, let TT" = TOT; i.e.,

.4 2

+ B 2 + i(BA - AB)

= 04 2

+ B 2 + i(AB - BA),

...... '.'_'_""""'''''''10''-''

0'

AB - BA::::: BA - AB,
0'

2AB::::: 28A.

Hence, AB::::: BA.

We need the following Lemma for the proof of Theorem 3.8.11:

LemUla 3.8.2

if T IS a bounded lillear operator nn a Hjlbert space X, then the jollowm9 conditions arc equivalent:
I.TT~l.

2. (Tx, Ty) ::::: (x,y) V x awl y.

3.

IITxll

IIxll

V x.

Proof.

(I):::} (2): Let ToT =

r. Then (T*Tx,y}::::: (x,y) or (Tx,Ty):::::

(x,y). (2) :::} (3) : Let (Tx, Ty) ::::: (x, y) V x and y. Then, for y = x, we have

IITxll' = IIxll' 0' IITxl1 ~ IIxll (3) =} (1) , Let IITxll = IIxii V x. IITxll'
0'

Theu

IIxll' =}

(Tx, Tx) ~ (x, x)

J

:::} (TTx,x}::::: (x,x)

TT-l)x,x) =OVx.
Then, by Lemma 3.8.1, we get T*T-l =0 orT*T =1

Proof of ThcoreUl 3.8.11. Let T be unitary. Then by Lemma 3.8.2, T is an isometric isomolllhism of X onto itself. Conversely, jf T is all isometric isomorphism of .X onto itself, then T- 1 exists, and by Lemma 3.8.2, we have

T'T=l

:::} (TT)T- 1

:::::

IT-lor T(TT- 1 )

=::

TT- 1 ,

.n",

'"'_'_""""'''''''10''-''

TO - T- J , _
0'

TT

=::

TT- 1

=::

r,

which <;hows that T is unitary.

Let T be an operator on a Hilbert space X into itself, then a nonzero vector x sum that T(x) =:: ),x, ), being a scalar, is called an eigeuvectoror chaructcristic vector or proper vectorofT, The corresponding ), is called an eigenvalue or characteristic value. Theorem 3.8.12 The proper values of a self-adJomt 0lJCmtOf> are real numbers Two proper vectOf'S corresponding to two different Ilf"Olle,> values of a self-adjoint operator are orthogonal. Theorem 3.l:S.13 The pTOIJer values of a umt.ary opcmtor are complex numbers), such that 1),1 = 1.

Definition 3.8..1

Proof of Theorem 3.8.12. Let T be self-adjoint and T(x) want to show that), is real, i,e. ), =:: "X.

),X,

We

(Tx, x) ~ (x, T" x) ~ (x, Tx),

{),x, x} = {x, AX},

0'
),(x, x) _ X(x,x),

Let A. ::j:. A2, T:r As T =:: To,

=::

AIX and Ty = A2Y. Then (Tx,y) = (),\x,y) = ),\{x,y).

(Tx, y) = (7, TOy) = (x, Ty) = (x, ),2Y)

=::

A2(X, y).

...", "'_'_""""'''''''10''--'

Thus, AI(X,y} = A2(X,y). Since AI

1:- A2,(X,y) must be zero. Hellce, 1:- O. Then

xl-V

I
Proal' of Theorem 3.8.13. Let Tx = .\x, x

(Tx,T.'C) = (),x,),x) =-- ),'X(x,x)

1>'1' 11'11'

On tile other hand, (Tx,Tx) = (x,T"Tx) = '~1I2 as T is unitary, i.e., T'T ~ 1. Hence, 1>'1' IIxll ~ 11'11 => 1>'1 ~ l. I

3.8.3

Adjoillt of all UIlboUIlded

LiIle~lr

Opemtvr

In Section 3.8.2, WI" have studif'd the cOllcept of the adjoint of a bounded linear operator on a Hilbert space. This concept may he extended to unbounded linear opewtors on a BaJlach space, in general and on a Hilbert space, in particular. From the point of view of applications, the concept of adjoint of unbounded linear operators on Hilbert spaces is more useful and, therefore, we discuss it here in brief.

Definition 3.8.4 Let T be an unbounded linear operator on a Hilbert spac~ X and assume that the domain uf T, V(T), is dense in X. The adjoint
operator of T, T*, is defined by

(Tx,y) = (x,T*y) for all x E V(T),

lJ E V(TO),

where V(T*) = {y E X/(Tx,y} = (x,z) for some z E X and all x E V(T)}. For eac.h such y E V(T), the adjoint 0lwral.or To of T is defillf~d in tR.rms of that z by z = TOlJ.

Remark 3.8.4 If T is also closed, then V(TO) is dense in X and To is dosed. Morcuver, T*' = T; that is, V(T**) = V(T) and the operators agree on these domains. For dosed operators, see Section 4.7.

Exarnple 3.8.10 and

Let X = L 2 (a,b) and T

b~

defined by (Tf)(t) =

V(T) = {J E X / f is absolutely continuous with

E X and J(a) =

o} .

0 . ,>0<1"_""""'''''''10''-1

..... ,

Then

(T[,g) ~

'd[ -1g(')d'

" d

~ [[(')y(')J: ~

" f
"
t>

d [(,)_/9 d'
<t
~

[(b)g(b) -

dg [(O-d'
dt

(f,T'g),

where

(T*g):::: - <1y and V(T):::: { g E X/g is absolutely continuous dt

dg with - E X,g(b):::: 0 ) . <1,

Definition 3.8.5 If the linear operator T on a Hilbert spoct< X is 1-1 on V(T), then t.he linear operat.or T-I on X defined 011 V(T-I) :::: R(T) hy
T-' (T(x}) ~ x,
is called the inverse of T. Theorem 3.8.14 Let T be a 1-1 linear olJerator on a ffilbert slJace X such that T- I exists and V(T) :::: veT I):::: X. Then T* is also /- /, and
[0'

all x

VeT)

Proof. The theorem will be proved if we show that (T-I)T*::::: 1 and I T*(T- )* :::: r. For verification of the first relation, take any y E Vcr). Then for every x E V(T-I), we haveT-lx E V(T) and so (T-'x,T'g) ~ (TT"x,g) ~ ({x,g) ~ (x,g).
This implies that roy E V((T-l).) and (T-l)" Ty:::: (TT-I).y::::: rOy =:: y. In order to verify the second relation, take an arbitrary y E V((T-- 1 )*). Then, for every x E VeT), we have Tx E V(T- 1 ) and, therefore,

(Tx, (T-')' g) ~ (T"Tx,y) ~ (x,g).

This shows that (T-I)* y E VeTO) aJld T*(T-1).y:::: y.

Definition 3.8.6 An unhounded linear operator T on a Hilbert. space X is called symmct7'lc if T* extends T in the sense that T :::: T on VeT)

.... ,
0,0,>0<1"_""""'''''''10''-1

and V(T) 2 VeT). In this case, (Tx,Y) = (x,Ty)'v' x,y E VeT). A symmetric operator is self-adjoint if V(T') = VeT). It can be checked that every self-adjoint operator

is symmetric.

Example 3.8.11
and VeT) =

Let .\'

= L 2 (a, b), and T

be defined by (T f) (t)

= <I'! d(l

, df . <I'! . { f E .\ / f and dt are absolutely continuous with dt 2 E .\

and f(a) = 0 = f(b)} Then,

~g ~ so that (Tg)(t) = -/" where V(T) = { . and -/ are absolutely g E X/g ( ,~ (t

continuous with self-adjoint.

~;

E X and g(a) = g(b) =

o}

= VeT). Hence,

is

and T be defined by T (o.~.) = {~k T is self-adjoint and one-one. The subspace r2 = V(T- I ) is everywhere dense.

Example 3.8. 12

Let X =

}.

V(r- I ) is the set of all sequences {Ilk} E 2 such that

00

k 2 1BI 2

< 00. Then

I is defined on V(T- 1 ) by T-I ((/h}) = {"fA} . T-I is linear. the inverse Let {cd bf' the ha<;is of 2, where PI. is the vector of all t.he components which are zero except the k-th one which is 1. T-I is unbounded as

r-

/;-=1

and

By Theorem 3.8.14, (T- 1 ). = (T )-1 = T- 1 Therefore,

r- 1 is self-adjoint.

B.

......

, to<l',_"""",,,,,"l1>"-I

3.9
3.9.1

Bilinear Forms and Lax-Milgram Lemma

Basic Properties

Definition 3.9.1 Let.X be a Hilbert space. A In<lpping a h,) : X x X --+ C on X x X into C is caned a sesquilinem' functional if the following conditions are satisfied:

2. a(ox,y) = oa(x,y).

3. a(x,y) -1-yz) = a(x,Yd +a(x,yz).

4. a(x. By) = [3a(x, y).

Remar'k :J. 9. 1
1 The sesqUilinear functional IS linear m the first variable but not so in the second variable. A sesquiJinear functional which is also linear in the second variable is called a bilinear 101m or a bilinear functional. Thus" a bilinear form n(,) is a mapping defined on X X X into C which satisfies conditions (1) through (3) of Definition 3.9.1 and (4).
a(x,{3J/) = (3a(x,y).

2. If X is a real Hilbert space, then the concepts of sesquilinear functional and bilinear form coincide.

3. An inner product is an example of a sesquilinear functional. The real inner product is an example of a bilinear form. 4. If a( .. ) is a sesquilmear function, then y(x, y) = a(y,x) is a sesquilinear functional.

Definition 3.9.2

Let a() be a bilinear form. Then

1. a (.,.) is called syrnmel1ic if a(x, y) = a(l', x) V (x, y) E X x X.

2. a (".) is called l)Osltive if a(x, x) 2': 0 Vx E X. 3. a (".) is called ]Jositive defi7litc if a(x,x) implies that x = O.

> 0 Vx

E X and a(x,x) = 0

Q.

......

4. F(x) = a(x,x) is called quadmttc form .

, to<l',_"""",,,,,"l1>"-I

5. a (".) is called bounded or continuous if there exists a constant Jo.J such that

>0

G. a (., .) is said to be coercIve (X -coercwe) or X-elliptIc if there eXIsts 2 a constant a > 0 such that a(x,x) ~ it IJxll for all x E X. 7. A quadratic form F is called real if F(x) is real for all x E X.

Remar'k 3.9.2
1. If a(-,) : X x X --t R, then the bilinear form a(,) is symmetric if a(x,y) =:a(y.x).

2.

lIall

=:

sup

#0,,0

IIxll1l1l11 #0,,0 sup la(x,1I)1.

~

latx,y)1 =

sup

(-=- ...JL) IIxll' 111111

II x ll=lIl1l1=!

It
3.

is c1ea, that la(x,y)1

11"11:=1

11011 IIxll 111111

IIFII ~ sup IF(xli.

4. If ah') is any fixed sesquilinear form and F(x) is an associated quadratic form on a Hilbert space X. then a(x,l/) =: HF(x + y)F(x - y) + iF(x + iy) - iF(x - iy)].

Verijicahofl.
F(x

By using linearity of the bilinear form a, we have

=:

+ y)

a(x + ll,X + y)

=:

a(x,x)

+ a(y,x) + a(x,y) + a(y,y)

and
F(x 1/) =:

a(x - y,x - y)
~econd

=:

a(x,x) - a(y,x) - a(x,!})

+ a(u,y).
(3.9.1)

By subtracting the

uf the above l...-'l.juation from the first, we get

=:

F(x + y) - F(x - y)

2a(x,y)

+ 2a(y,x).
+ 2a(ty, x),

Replacing y by ill in Equation (3.9.1), we obtain

F(x

+ iy) - F(x - ty)

=:

2a(x, iU)

",,,. "'_'_""""'''''''10''--'

F(x

+ iy) -

F(x - iy) = 2ta(x,y)

+ 2ia(J/,.r.).

(3.9.2)

tl'1ultiplying Equation (3.9.2) by i and adding it to Equation (3.9.1), we get the re~ult. Theorem 3.9.1 1f a bilinear fo,m n(',') is bounded aud symmetnc, theu lIali = IIFI, , who'e F is the associated (Juudmtic functional. Theorem 3.9.2 Let T be a bounded liuoor 01Jet'otor on a Hithe,t space X. 'Then the cortlplex-valwd function a(, .) on X x X defined b1l
a(x, y) = (x, T1I)
(3.9.3)

is a bounded bi/moo,' f01m on X, and Jlall :::: IITII. Conve"sely, let a(,) be a bounded bilinear fOfm O'i a HilIICT't s]Jace X. Tlte'i there exisf<s a unique bounded linear opemtor T on X such that

a(x,y) = (x,T1I) V(X,lI) E X x X.

(3.9.4)

Corollary 3.0.1 Let. T be a bounded liuear opemtor on a Hilbe,t space X. 'l11en the COrtllJlex-valued function b(,) on X x X defined by b(x, y) :::: (Tx,y) is a bounded biliflear f01711 Ofi X aud IIbll = IITII. Conversely, let b(, .) be a bounded bdiuear fo,m on X 'Then there is a unique bounded linear o])t'ratnr'T on X .~Ut'h thath(x,y) = (Tx,y}\f(x,y) E X x X CorollHl'y 3.9.2 1fT is a bounded linear o]Jerator on X, l.hen

IITII=

,up
Ilzll=lIi1l1=1

l(x.TlI)I=

'up
IIzll=III1I1=!

I(Tx,y}l

Theorem 3.9.3 Let T IIC a bouflded liuoor o]Jerator on a Hilbert space X. Then the following statemeuts ar'e equivalent:
1. T is sdf-lldJuml.

2. 'The bilinear fo,m a(,) on X defiued bya(x,y) = (Tx, y) is symmetno. 3. 'The quadratJc form F(x) on X defined by F(x) = (Tx,X) '8 real.

",,,. "'_'_""""'''''''10''-''

Corollary 3.9.3 If T is a bounded sdf-adjoillt operator on X, then IITII 0 ,up I(Tx,x)l.

II x ll=1

The following lemmas are needed for the proof of Theorem 3.9.1.
Lemmu a.H.l A bduwar form a(x,y) 1$ symmctnc if awl 07tly if tlle associated quadratic functiollal F(x) is real.
P7'Oof

Jf a(x,y) is symmetric, then we have

F(x) = a(x,x) = a(x, x)
~

F(x).

This implies that F(x) is real. Conversely, let F(x) be real, then by Remark 3.9.2(4) and in view of the relation
F(x) = F(-x) = F(ix) (FIX) =a(x,x),F(-x) =alx,x} =a(-x,-x),

and
F(ix) = a(ix,ix) = ita(x,x) = a(x,x),

we obtain,
a(y,x) =
=
;:0

~ [F(x + y) -

F(y - x) F(x - l/) F(x - l/)

+ iF(y + ix) + iF(x

iF(y - ix]

~ [F(x + y) ~ [F(x + y) -

- iy) - iF(x + iy)]

ill)]

+ iF(x -t- iy) - iFtx -

= a(x,y).

Hence, a(,) is symmetnc.

Lemma 3.9.2 a(, -) 18 IlOuuded if aud ouly 1f tlte assoctated quadrauc fOfm F is bouuded. If a(, .) is bounded, then II FII ~ lIall ~ 2 II FII .

Proof

Suppose a('j') is bounded. ThcJl wc haw" sup

11;,.11=1

!F(x)1

= sup )a{x,x)1 ~ sup 11"11=1 1I;,.1I=IIYII=1

la(x,y)1

lIall,

...". "'_'_""""'''''''10''-''

and, therefore, F is bounded and IIFII :-:; lIali. On the other hand, suppose F is bounded. From Remark 3.9.2 (4) and the parallelop;ram law, we: get

lo(x, y)1 < ~ IIFII lIlx + "II' + IIx - "II' + IIx + iyll' ~ IIx - i"ID

~ ~ II FII2 (IIxII' + 11,,11' + IIxll' + 11"11')

=
0'
,up
II x ll=lIvll=1

IIFII (1Ixll' + 11"11') , lo(x,")1

~

211F1I.

Thus, a(,) is bounded and

Iiall :-:; 211FII.

Proof of Theorem 3.9.1. By Lemma 3.9.1, F is real. In view of Lemma 3.9.2, we need to show that lIall :-:; IIFII. Let a(x, y) == l'e"", where l' E n, l' ~ 0 and 0' E n. Then by using Hem<lrk :1.9.2(4) ;md hearing in mind that the purely imaginary terms are zero, we get

lo(x,")1 ~ 0 ~ a(e-wo,") ~ ~ IF(x' +y) + F(x' where x' -", l'e- w . ThL., implies that

y)l

IO(X,1111

~ IIFII (lIx' + "II' + IIx' - "II')

~ IIFII (lIx'1I2 + IIYII 2)
,up Ilx' 11=111111= I

==
0'

(b.v the parallelogram law).

lo(x',,,)1

IIFII,

lIall ~ IIFII.

Proof of Theorem 3.9.2. 1. Let T be a bounded linear operator on X. Then a(x,y) == (x,Ty) sat,isfil<S the following condition:
(a) a(x + X'd,)

== (x + x',Ty) == {x,Ty} + (31 ,Ty) == a(x,y) + a(xl ,y)

..... , Q,.,..-I"_""""'''''''IO''-I

(b) a(ilx, y) = (ax, Ty) = o(x, TV) = il(! (x, y),

(c) la(x, v)1 = inequality.

I(x, Ty)1

IITII IlxlJ lIyll, by the Cauchy-Schwanz-Bunyakows~

la(x,

This implies that

sup
11.1" 11= 111111= I

y)1 < IITIII or IInll

IITII In fact, Ilnll =

liT II ("e Equation (3.9.7)). 2. For the converse, let a(,) be a bounded bilinear form on X. For any y E X, we: define III on X as follows:
f,(x) ~ a(x,,,)

(3.9.5)

We haw

III(x\
and

+ X2)

= a(xi

+ X2, y) =

a(xl, V)

+ a(x2' y)

11I(ax)

= a(ax,y) = oa(x,V),
la(x,,,)1 < lIallllxll Ilyll,

If,(x)1 ~

Thus, 111 is a bounded linear functional 011 X. By the Riesz representation theorem, there exists a unique \'cctor Ty E X such that

III(x) = (x, Ty)V x E X,

and
IITyll ~

(3.9.6)

11/,11

11011 11,,11

13.9.7)

The operator T : V --t Ty defined by T(y) = (x, Ty) is linear in view of the following relations:
(X,T(iIY)) = foy(x) = (x,al') = o(x,y) = Hly(x) = a(x,Ty) (X,T(ill') = (x,oTl'),

(x, T(oy) - o(T(l,))) = 0 V x E .\ (x, Tll,

'* T(oy) ~ aT(y)

EX,

~'-''''''''_._''-I

... ,.,

+ ~/)) = 11I + y'(x) = (x,y + yl) - (x"y) + (x,y') = [1I(x) + [~(xJ = (x,Ty) + (x,Ty') = (x,T(y) + T(y'))V x

0'
(x, IT("

+ "')1- [T(,,! + T(y')J)

T(y

~ 0 V x E X,

which gives

+ y') == T(ll) + T(y').

~

&juation (3.9.7) implies that IITII

we have

lIali. By

Equations (3.9.5) and (3.9.G),

a(x,y):::: fll(x)

0::

(x,TJ/)'v'(x,y) E X x X.

Then, fol' every fixed JI E X, we get (x,Ty):::: (x,Sl/) or (x, (T - S)l,) == O. This implies that (T - S) y == 0 V Y EX, i.e. T:::: S. This proves that there exists a unique bounded linear operator T such that a (x, Y) :::: (x, Ty).

I
Proof of Corollary 3.9.1.
I. Define the function a(x, y) on X x X by
a(x,,,)
~

b(y,x)

(x,Ty).

fiy Theorem 3.9.2 , a(x,y) is a bounded bilinear form on X and Iial == IITII. Since we ha\' b(x, y) :::: a(y, x); b is also bounded bilinear

on X and

II/,ll ~

n II=IlIlIl=1 ...

suv

[b(x,y)[ ~

IIxll=lIyll=1

sup

la(",x)1

lIall

IITII

2. If b is given, we define a bounded bilinear forlll a on X by

a(x,y):::: fl(y,xt

Again, by Theorem 3.9.2. there is a boum.lcd linear operator T that

a(X.l/) = (X. Ty)V (x,y) E X x X.

011

X such

Therefore, we have b(X,l/)

= a{y,x) = (y,Tx) = (Tx,y)V (x,y)

E X x

x.

I
Proof of Corollary 3.9.2. By Theorem 3.9.2, for every bounded linear operator on X, there is a bounded bilinear form a such that a(x,y) (x, Ty) and lIall ~ IITII . Then,

lIall

sup
Ilxll=III1II=1

la{x,y)) =

sup
lI""II=llyll=1

(x,Ty).

".". "'_'_""""'''''''10''-''

From this, we conclude that

IITII ==

sup l(x,Ty}j. IIzll=II11II-1

Proof of Theorem 3.9.3. (1) :::;. (2) : P(x) == (Tx,x) == (x,Tx) == (Tx,x) = P(x). 111 view of Lemma 3.9.1, we obtain the result. (3) :} (2) : By Lemma 3.9.1, F(x) == (Tx,x) is real if and only if the bilillear form a(x,y) == (Tx,y) is symmetric. (2) => (1) : (Tx,lI) == a(x, y) == a(y. x) == "{T"'lIcc == (x. Ty). This shows . x"} that T* == T that T is self-adjoint, I Proof of Corollary 3.9.3. Since T is self-adjoint, we can define a bounded bilinear form a(x,],) == (Tx,y) == (x, Ty}. By Corollary 3.9.2 and Theorem 3.9.1,

IITII

lIall

IIFII

== sup IF(x)1

IITII

11"'11=1 == sup I(Tx.x)1 11"'11=1 ~ ,up I(Tx,x}l. 11"'11=1

The following theorem, known as the Lax-tvlilgram lemma proved by PD Lax and AN Milgram in 1954 [10], has important applications in different fields. Theorem 3.9.4 (Lax-Milgmm Lemma). Let X be a HilIlCrt s]Jace aud a(,) ; X x X --t R a bouuded bilinear form which is coercive Of' X -elliptic iu the se71se that then, exists a > 0 such that a(x, x) > 0' IIxlI~ V x EX. Also, let f : X --t R be a bounded linear fUflctional. Then then? exists a unique element x E X such that
a(;c,y) == f(JJ) "IV E X.
(3.9.b)

Proof of Theorem 3.9.4. M > 0 sl1c;h t.hat.

Since a(,) is bounded, there exists a constant

~ M

I" (v, v)1

IIvll IIvll

(3.9.9)

Q . ,>0<1"_""""'''''''IO''-I

......

By Corollary 3.9.1 aud Theorem 3.7.1. there exi.<;ts a bounded lilwar operator T 011 X and III E X 5ut:h that <Juation a(u,v) == f(v) can be rewritten

as
(ATu, v) ~ (Af" v),

(3.9.10)

(>.Tu - )"Jv.v) = 0 '<Iv E X.

This implies that (3.9.11) We will show that (3.6.11) has a unique solution by showing that for appropriate values of parameter p > 0, t.he affine mapping for v E X, V --t V - p()"'Tv - >'/y) E X is a contraction mappinp;. For this, we observe that

IIv -

p..\Tvll:.! = (v - pATv, v - pATv)

= dvll
2 -

2p{)"Tv, v)
2fX' IIvll'

+ p2 11ATvll 2

(by applying inlier product axioms)

,; Ilvll' -

+ p' Ai' IIvll' ,

a(v,v) = (>.Tv,v) ~ allvll

and

(by the coercivity),

(3.9.12)

II>.Tull ~ AI IIvll

(bv boundedness of T).

Therefore, (3.9.13)

IIv-pATvll

< (1-2fX,+P'Ai')

1/'1

IIvll

(3.9.14)

Let Sv = v - P (ATv - A/y ). Then

IISv - Swll
~

II (v -

P (ATv - Af,, - (w - P (ATw - Af,,)11

~ lI(v - w) - p(AT(v -

w)11
(3.9.15)

~ (1-2pa+p'M,),/2l1v-wll (by 3.9.14).

",,,. "'_'_""""'''''''10''-''

This implies that S which is equivalent Banach contraction fixpd point. which is

is a contraction mapping if 0 < 1 - 2pa + p2 M 2 < 1 to the condition t.hat. p E (0,20:/ !lI 2 ) . Hence, by the fixed point theorem (Theorem 1.2.1), S has a unique t.he uniqup solut.ion. I

The following problem is known as the abstract variational problem:

Problem 3.9.1

Find an plement x such that

a(x,y) = I(ll)

for aU y E X.

where a(x,y) and

I are as in Theorem 3.9.4.

In view of the Lax-tl'liIgram lemma, the abstract wl.riational problem has a unique solution. Note: A detailed and comprehensive description of the Hilbert space theory and its applicat.ions is p,iven by HelmherK [8], Schechter [12] and Weidmann [16].

3.10
3.10.1

Problems
Solved Problems
If X ::j;. fO} is a Hilbert space, show that

Problem 3.10.1

IIxll =
Solution.
x::j;. O. Then

,up Ilyll=1

l(x,Y)I

If x = 0, the result is dearly true as both sides will be O. Let

IIxll

IIxll'

(x,x)
=

Ilxll

= (x,

TIXfi)

sup (X,l/) lIyll=1 ~ sup IIxll lIyll, (by the Cauchy-Schwartz-Bunyakowski inequality) 111111=1
=

<

IIxll . IIxll =
sup (xd/)' 111111=1

This Implies tlmt

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

Let X n ---t x and y" ---t y in the Hilbert space X and an ---t a, where a,,'s and a are scalars. Then show that
Pf'Oblem 3.10.2
L x,,+ 11" ---t x

+ y.

3. lim (x",y,,) = (x,y),

n--><Xo

Solution.
function.
l.

rhe last relation ::;how::; that the inner product is a continuous

II (x" +y,,) - (x + Y)II

, ~ ((x" -x) +(Y" -y) ,(x" 2

x) +(Y" -11)) = ((x" -x) ,(x" -x+ ((y" -y) ,(y" - y + ((x" - x) , (y" - y)) + ((y" - y) , (x" - x.

Since x" ---t x and y" ----lo y,lIx" - xll = (x" - X, X n - x) ---t 0 as n ---t 00 and lIy" - Yll2 = (y" - y,y" - y) --t 0 as 11 --t 00. By the Cauchy-Schwartz-Bunyakowski inequality, we ha.ve

I((x" and

x), (y" -

YI < IIx" - xlllly" - 1111-+ 0 as n -+ 00,

I((Y" -

y), (x n

x1 < IIY" - yllllx" - yll -+ 0 as n -+ 00.

In view of these relations, we htl\'e lI(x" ---t 00; i.e., X n + y" ---t X + y,
2

+ V,,)

- (x

+ Y)II

---t lJ

as

lI(a"x" - ax)1I = (o'"x" - ax.a"x" - ax) = (a"x" - ax" + ax" - ax, a"x" - ax" + ax" - ax) = (a"x" - ax,>, anx" - ax,,) + (a"x" - ax", ax" - ax) + (ax" - ax, ax" - ax) + (ax" - ax, a"x" - ax n )
~

Ja" - al IIx" II + 10" - 0'1 IIx,,1I IIx" - xilial + lal' IIx" - xii' + lal Ilx" - xII IIx,,1I la" - al -+ 0
00

as n ---t Hence, a"x n -> ax

under the given conditions.

11

as

--t 00.

.n",

"'_'_""""'''''''10''-''

3.

I(X.,Yn} - (x,Y}1 ~ I(xn,y,,) - (X,Yn)

:=

+ (x,y,,)

- (x,Y)1

l(x,,-x,II,,)+(x,Y,,-Y)1
X,

< I(x" -

+ J{x, y" - y}1 <: IIx" - xII IIYnll + IIxlllly" - yll

Y,,)I
0 such that IIx,,1I < 00. Therefore, l{xn,Yn)

Since :t n --t X and y" --t y, there exists /ll M, lI.rr! - xII --t 0 and lIy" -"II --t 0 as n --t -(x,Y}I-t 0 as n --t 00; or

>

{XIllY,,} --t (x,y)

as n --t 00.

In view of Theorem C.2 (2) of Appendix C, the inner product (-,.) is continuous.
Show that R", f 2 and L 2 (a, b) are Hilbert spaces while C[a, bJ is not a Hilbert space.
Pf'Oblem 3.10.3
Soh~tion.

L. R" is a Hilbert space. For

x, 11

nn,

lJ.'

define the inner product by

(x, 11)

:=

.=1

XilI.

(a) (x+ x' ,Y)

" == E(x; +a;)y,

=1

:=

" +L L xilli" X~Yi == (x, 11) + (x',y) .

.=1 i=l

(b) (o'x,y):::
n

L
=1

"

(o:x,)", =
n

0:

L X,Yi:= o(x,y) .
;=1
" "
:=

(c) (x, Y)

:=

L
0=1

XiY;

:=

L ,Y. :::: L Xill; := L Xill; .=J .=1 ;=1

.=]

(x, 11)

(dl (X,X):= LXiX; ==

.=1

"

" L xf
(x,x):= 0

==>
} :}

{x, x}
;=1

> 0 and

LX;
,O}

x;=o
x~(O,O, ..

>
:}

x=o.

.n",

"'_'_""""'''''''10''-''

Thus, (., -) is an inner product on Rtl and it induces the norm given by

/lx/l ~ V(x.x) ~ ( ~xf

)'/2

Rn is complete with respect to any norm and so R" is a Hilbert space.

2_ 2 is a Hilbert space: For x = (XI,Xt, ... ,r", ... ),y = (Y1,Y2, ... , y,." ... ) E 1.2 , we define the inner product (-,-) on 2 by (x,y) =

L x'Yi .=1
(a) (x

"

+ x', y)

L (Xi + a;)(fJ;)
,=1

where
,x~,_-)

x , = (' , , x;,x t
(x+x',y) =

LX;!!. + LX;!!;
.=1 '=1

= (x,y)...L.(x',lI)

(bl (ax,y)
(e)

=
~

L(ax;)Yi
=1

=a
~

~ x/il,
1=1
~

= a(x,y) .

(x, y)

=
EXiY;
.=1

= E X/til = E YiXi = (y,x)

i=1 1=1
2

(d)

(x,x)

L
~

00_00

x, Xi =

L Ix.1

.=1

;==1

(X,X)
(x,x)

0
lJ

as all the terms of the series

:}

,=1

L Ix.1

are positive

L
;=1

Ixl

= u :} Ix.1 2 = 0 } Xi = U"i/ i

x= (0,0,-- ,0, ,0) = 0 E 2In view of the above relatIOns, 2 is an inner product space with
:}

the mner product

(x, y)

iE x,y..
2

Let

x"

{ai"')} ~=I'

k =

L, 2, 3, ___ and {x,,} be a Cauchy sequence in 2. From the relation

a(m) - a(") - <" k Ii; k Lk==1

every fixed k

la(m) - (,t") I ~ Ilx - x 11 we obtain that for 1 the SL"quence {ai")}OC is a Cauchy SL"quence of ,,==
k
2 ",n,
1

00

.n",

'.'_'_""""'''''''10''-''

complex numbers and, therefore, it converges to a complex number, say ak. (\-Ve know that C is complete.) Let

For every t" > 0, we have

IIx", - x,,11
\f m

;=1

L la~")

00

- a~")1

<

t: 2 for

all m 2': n(t:) and n 2: n(t") and, therefore, for every fixed index
1)

2': 1, k'fl la~m) - a~n)1 < t"2

> n(t") and n > n(t"). Letting

\f

1fI.

tend to 00, we obtain

k~1 lak - a~")12 < t"2

,

n 2': n(,,) and

Now, we let p tend to 00 and we get

Llak-ain)1
k=l

00

~~'v'n>n(f).

(3.10.1)

The SL"quence {x - x,,} =

{Uk - a~") }:l ' therefore, belongs to 2,

and since 2 is a vector space, the sequence x = (x - x n ) + x" E '1' Prom Equation (3.10.1), we conclude that Ilx - Xn II < t: V n ~ n (l) : i.e. {x n } ilS convergent in 2. Thus, 2 is a Hilbert space.
3. L 2 (a, b) is a Hilbert space: 2 ta, b) is a V(~tor space, (See Appendix b E). We define the mner product by (f,y) = J'l J(t.)y(t) dt V J,y E L,(a, b).

(a) U+h,g)

~ {U+h)(t)9(t)dt ~ {f(t)9(l.)dt+ {h(t)9(t)dt ~

dt

U, g) + (h, g).
(b) (af,g)

~ {(a/l(l.)g(t)
f(t)g(t)dt f(t)f(t)dt

~ aJ:U)(t.)g(t)dt ~ aU,g).
g(t)f(t)dt

(e) U,g)

~[ ~[

l'

~ (g,/).
as If(t)I'

(d) U,/)

~ [If(t)I' dt => U,/) ?: 0

o. U,/)

?:

o ;mply;ng

{1/(t)I' dt ?:

If(t)1

0 a.e.

B.

.... ,

J=

0 a.e.

, to<l',_"""",,,,,"l1>"-I

Thus, C) is an inner product on 2 (a, b) which is complete with respL"Ct to the norm

IIfll ~

(t If(I)I' dl)
11111

1/2

mduCL"CI. by the above inner product. For

1 E 2 (a, b),

we have

If(I)1 dt

~ (If I, I) S IIfll
~ (v'b

(by the

CSB inequabty)
(3.10.2)

a) IIfli.

Suppose {In} is a Cauch.\, sequence in 2(a,b). By induction, "'e construct an increasing sequence of natural munbers HI < Hz < ... nA' < 1 ... such that III", - 1,,11 < 2k V 111 ~ Hk and n > Hk Consider the sequence D.7(3), we have

{/"HI -ln~}:1

E: L 2 (a,b). By Theorem

Jo
"

["'

L: IfnH' (I) k=l

f..(I) Idl ~

'[" L: Jo Ifnw (I) A=I "

00

f .. (I)1 dl

< (v'b-a)Dlf",+,(I)-f.. (t)11

k=1

<

(v'b=G) L: 2' ~ v'b k=1

a < 00.

By Beppo-levi's theorem (see Appendix D) the series

f,,~(t)1

L
k=l

I/"k+1 (t)-

converges for almost every t E [a,b} and so does tlw series

= I",(t) + L[f"~+l(t) - I,,~(t)1 = lim I"k(t). ,~=

k=1

k-too

As a consequence, the functlOlI f defined a.e. on [a,b} by I(t) lim I,,~ (t) is finite a.e. and Lebesgue measurable on [a, b}. Further-

.n",

'.'_'_""""'''''''10''-''

more, Theorem 0.7(3), we have

If(tl - fn, (til' dt

~ [ [~Ifn,,,(t) - f",(t)r dt
~
m-+=

lim

j" [f: If",+,(t) - f",(tl

(l

l] ,

dt

k=l

~ ,~~':o
<
<
This implies that (f fure, we have I = (f -

[t.Vn.., - fn,

I] ',U

~"."oc (t,llfn,+, - fn,lI) , dt

,~~':o (
I...

t; 1)' ~ (1)'
m

2'

2'->

I ...,) E

In

2 (a,b) and T E 2 b) . For n p

(a,

III - fnpll
L I

< 2P~I' There> n", >J.' ohLain

1 I

Ilf - f,,11 < Ilf - f",11 + Ilf" - f",11 < 2P-' + 2P + 2P < 2"-" This shows that {/... 1 converges to 1 E .z (a, b) and hence 2 (a, b)
a. Hilbert space.
4. C [a, b] is an Inner product space but not a l-lilbert space: For

is

1,9 E

In, b],

we define t.he inner product b.\, (I, g) =

Ia

f(t.)g(t.) dt.

It can be seen, as in the case of 2 (a,b) , that (.,.) salisfiL"S the axioms of the inner product. vVe nOW show that it is not complet.e with respect to the norm induced by this inner product, i.e., there exists a Cauchy sequence in C [a, b], whidl is Ilot convergent in CIa, b] . Leta=-I,b=l,and
for for for

-1<t.<O
0 < t ~ lin 1/<t<1

be a sequence of real-valued continuous function on [-1,1]. We prove the desired result by showing that this is a Cauchy sequence with respect to the norm indua"CI. by the inner product

Ilfll ~

, L ,/(t')I' dt) 'f'

B.

......

, to<l',_"""",,,,,"l1>"-I

but it is not convergent t.o a function \-Ve ha\'e (t.alw m > n),

lEe [-L, L].

/.,,(t) -/,,(t) ~ 0 fo' - I "t ,,0 o < Im(t) - I.,(f.) < L - I,,(t.) for 0 <t < L
and so

11/'0 -/,,11'

= ( / , /",(t) = /'
-I

-/,,(t)I' dt)

I/",(t) -/n(t)I' dt +

r' Jo I/.,,(t) -/,,(t)I' <It

II -/,,(t)I' <II

" [ I I -/n(t)I' dt
=

r'I" II -/n(t)I' <II + /.'

1/"

t J,
I

"

11 -/n(t.)1 2 dt,

as 1-1,,(t) =0 for-<t<lor -

"

11/", -/nll' "

'ln

11 - nIl' dt"

11/"
0

dt = -,
11

"f", - Inll <

-

y"

form> 11.

This show:> t.hat Vt > 0, m denotes the int.egral part of

> 11 2': IV(l) [N(l)

l~ + L],

where

L::~]
(" for

sequence ill C [- 1, 1] . Suppose continuous funct.ion on [-1, I] and f > 0; 3N (f:) such that n > N(l). Let -L < 0: < O. Since U" - 1)2 is positive,

,~,

11m - 1,,11 < t:. Hence, {I,,} is a Cauchy In --t lEe [-1, 1]; i.e. I is a real-valued
III" - III <

!..~ (fn(t) -/(1))' dt <

t,

(fn(t) -/(1))' dt
for 11

'fi"twl<.. . ~........._"-'

... ,,,

==

III" - 111 2 < ("2

2':

N (l),

t' J_, j2(t)dt ~ ~<}

Hence,

as IIl(t) == 0 for - 1 < t

0:,

Vf;

> O.

j~ f'(t)dl ~ O.
C [- L, 1]) Then, by Equation (3.10.3), 12 (t.)

(3.10.3)

The mapping t --t fl(t) is positive and continuous on [-1,0:1 (as

:=

f E
:=

011 [-

L, 0:] . Thus, I(t)

N]:=

0,
n

L <tS;a,where-l<o:<O.lctO<{3< land

[h]

+1. For

>

v:= sup(N(t:),Nd,

lJ.'

ha.ve /,,(t)

:=

1 for t E [iJ,

11,111" - III < {.

Then,

(1- f(t.)'dl
<

(1"(1) - f(t.)'dl f(I) dl

r' J_, (I"(t.) -

$"

V"

and, consequently,

l
f(t)::::o
I(t)
:=

(1 - f(I))'dt

~ O.

(3.10.4)

The function t --t (1 - f(tl)2 is positive on [.e,I] as f is continuous On [-1,1)]. Then Equation (3.LOA) implies that [1 - f(t)J2 := 0 Vt E [.e,l] or 1(1) := 1 for t E [13, 1], where 0 < f3 < L In short, we have shown that

for -l~t.<O} 1 for U < t ~ 0

(3.10.5)

This is a contradiction to the fact that f is continuous (J given by Equation (3.10.5) is discontinuous). Hence, our assumption that in ---t f E 0[-1, 11 is false, and so 0[-1, 1] is not a Hilbert space.
Pf'Oblem 3.10.4 Let V be a bounded linear operator on a Hilbert spar:e /-h into a Hilbert space H2. Then III - V'FII < 1 if and only if V x E

...... "'_'_""""'''''''10''-''

II x ll=1 operator on HI'

HI> 0

< inf IWxII ~ sup IIV xii = 11\.'"11 < .,J2, where 1 is the identity
11"'11=1

Solulion.

Since 1- V'I' is self-adjoint by Corollary 3.9.3,

III

-V'VII ~

,up
11"'11=1 1I:r1l=1

l(x,(1

V'V)x>!

,up 1I-IIVxll'l.

Therefore, the condition that III - VVII conditions

<

1 is L"quivalent to the two

11"'11=1
and sup
IIxll=1

,up (I -IIVxll')

~ 1-

11"'11=1

'nf Ijllxll' < 1

(111'xll 2 -

I) = sup 11\1 xll 2

11"'11=1

L < l' ,

which is equivalent to 0< inf 11"'11=1

.11-"xIl 2 < sup IlVxIl 2 <2.

Ilxll=\

3.10.2

Ullsolved Problems

Pmblem 3.10.1

1. Cive an example of a nonseparable Hilbert. space_

2. If A is any orthonormal set. of a separable Hilbert space X, show t.hat A is counta ble. Prove that pvery separablf' Hilbert spao' is isometrically isomorphic to 2.
Problem 3.10.2 Problem 3.10.3 At = MLl Problem 3.10.4

If M is a closed subset of a Hilbert space X, prove that

If A is an arbitrary subset of a Hilbert space X, then

prove that

0 . ,>0<1"_""""'''''''10''-1

......

I. A..l. = (A..l...l.)..l..

2. AU ~

[AJ.

Problem 3.10.5 Show that for a sequence {x n } in an inner product 2 space, the colltlitiom; 1I:r"II-t IIxll and (x,,, x} -t IIxll imply IIx n - xll--+ 0 as n ---j. 00. PJY,1Jlem 3.10.6 Let en = (0,0"" .1,0",,), where I place. Find P{c,,} for F E (f2
j::; III

r.

the n-t.h

Problem 3.10.7

Let

J(

{x =

(XI,X2,

xn .) E f

/lx..1<

l/n

for all n}. Show that K is a compact subset of 2. In an inner product space, show that 111111 if and only if IIx . yll ~ 111"11 - lIylll
P1'Obiem :J.10.l$

IIx + yll = IIxll +

Problem 3.10.9
I. (a) Show that the vectors XI (1.2.2).x2 3 X3 = (2, -2, 1) of R are orthogonal.

(2.1.-2),

(b) Show t.hat the vectors (1/3,2/3,2/3), (2/3,1/3, -2/3), (2/3, -2/3,1/3) of [{3 are orthonormal. (c) Show that f p , P i- 2 is not an inner product space and hence not a Hilbert space.

Problem 3.10.10

Consider R 2 with norm

IIxlll = Ixil + IX21.

(a) Show that every dosed convex set in mum norm.

(R2 , 11.11 1 ) has a point of mini-

(bl Show. by an example. that this may not be unique. (c) What happens with

IIxlioo =

lIJax

flxll, IX21F

P1'Obiem 3.10.11 Let L 2 ( - ( I , a) -t L 2 (-a, a), where 0 < a < 00, blo definL"CI. by y = Px, where y(t) = 1/2 [x(t) + x( -t)J. Find the range and

r;

B.

......

domain of P. Show that P is all orthogonal projection.

, to<l',_"""",,,,,"l1>"-I

Problem 3.10.12

Let T : L z (-00,00)

---oj.

L z (-00,00) be defined by

f(t) foc t>O T f(I.) ~ { _ f(t) fOl t>O

Show that T is a unil,ary operator

Problem 3.10.13

For any elements x, y,z of an inner product space,

z
1

show that

liz - xii + liz - "Ii = 211x - yll + 2

Problem 3.10.14

if we have

IIx + nyll

Show that in an mner product space, Xl..l1 if and only = IIx - ayll for aU scalars a.

Problem 3.10.15 Let E[-I, lJ ~ {J E C[-I, II/fH) ~ f(t), fOl all I E (- I, I]) and 0 [-I, 11 ~ {J E C [-I, 11 / f( -t) ~ - f( t) foc all t E [-I, I]} . Show that C[-l,lj -= 8[-1, IJ810[-1,IJ.

(a) Let X = R Z Find /11-'- if /11 = {xl. x = (xJ,xz)

f-

0EX

(b) Show thaI Theorems 3.4.2 and 3.7.1 are not valid for inner product spaces. (cl Let H = {I E L z [a. bl /1 is absolutely continuous with l' E Lzl(l, oJ}. Show that fOlIE L z [a,bJ there is agE H such that F(f) = Ig dt is a solution of the differential equation .q(t) - g"(t) = l(t) with the boundary condition g'en) = g'(b) = o. Let H be an inner product space and y a fixed \'ector of H. Prove that the functional l(x) = (x, y)Vx E H is a continuous linear functional
Problem 3.10.16

J:

Show that every bounded linear functional F on 2 can be represented in the form
Problem 3.10.17 F(x) =

L x;fl;,
.=1

00

Q.

......

, to<l',_"""",,,,,"l1>"-I

Problem 3.10.18

Let T C 2
=:

---oj.

C'l. be defined by

T(x)

(XI

+ iX2, XI - iX2), where x ::: (XI, X2).

Find T". Show t.hat. rT ::: TT" = 21 Find !(T + 1'") and

i. (1' -

T).

Problem 3.10.19 If Ilxnll ~ 1, n ::: 1,2, .. , in a separable Hilbert space H, then lliscuss the properties of FXn~, where FE H" and X'lk is a SUbsL~

quenc p of {x,,} Show that T4 is a self-adjoint operator provided T is a self-<U..ljoint uperatur.

Problem 3.10.20 Problem 3.10.21 Let l' ~ an arbitrary uperator on a Hilbert space H into itself, and if a and f3 are scalars such that lal ::: 1131 Show t.hat aT + (J1' is normal. Problem 3.10.22 Show that the unitary uperaturs H into itself form a group. Problem 9.10.23
UII

a Hilbert space

Let T he a compact. normnl operator on a l-filbert space X, and cunsider the eigenvalue problem (M - T) (x) ::: yon X. If), is not an eigenvalue of 1', then show that this equation has a unique solution x ~ (AJ - T)-' y.
Problem 9.10.24

Prove the results (a) through (d) of Remark 3.8.2(2).

Pf'Oblcm 3.10.25 If S, T: X -) X, where X is an inner product space sudl that {S;l:,x)::: (Tx,x)Vx E X, show that S::: T. Pmblem 9.10.26

Gi\'e an example of a self-adjoinl operator without

eigenV'rllues. If H is a finite-dimenslonalumer product space, show that every isometric isomorphism H into itself is unitary.
Problem 9.10.27

Q.

......

Let X be a l-lilbert space. If Tis a self-adjoint operator (not necessarily bounded) ml'\pping a subset of X into X such that
Problem 9.10.28
, to<l'._..

....,..",,"l1>"-I

D'r = X, prove that the operator U = (T - i)(T - i)-I is defined on all of X and is unitary.
Problem 9.10.29 Let T be a compact normal operator on a Hilbert space H. Show that there exists an orthonormal basis of eigenvectors {en} and corresponding eigenvalues {Pn} such that if x L{x,en)e n is the
n

Fourier expansion for x, then T(x) = LII,,{x,en)en'

n

Q.

......

, to<l'._..

....,..",,"l1>"-I

References
[1 ] G Bachman, L Narki. F'unctiOlln\ Analysis. New York and London:
Academic Press, 1966.
[21 AV Balakrishnan. Introduction 1.0 Uptimizat.ion Theory in a Hilbert Space, Lecture Notes in Operation Research and Mathematical System. New York, Heidelberg, Berlin: Springer Verlag. 1971. [3J SK Berberian. Introduction to Hilbert Space. Oxford: Oxford University Press, 1961. [4J Ii Dautray, JL Lions. Mathematical Annlysis and Numerical Methods for Science and Technology. Vol. 2. Functional and Variational l\lethods. Heidelberg, New York: Springer-Verlag, 19R8. [51 L Debnath, P Mikusinski. Introduction to Hilbert Spaces with Applications. 2nd ed. San Diego, London: Academic Press, 1999. [61 N Dunford, JT Schwartz. Linear Operators Part l. General Th('{)ry, Interscience, New York: Interscience, 1958.

[7] P Halmos. Int.roduction to Hilbert Space. New York: Chelsea Publishing Company, 1957.

[8] G Helmberg. Introduction

to Spectnu Theory in Hilbert Space. Am-

sterdam, London: North Holland Publishing Company, 1969.

[9] E I<reyszig. Introductory Functional Analysis wit.h Applications. New

York: .John \-Viley and Sons, 197,1\. [101 PD Lax, AN Milgram. Parabolic equations, Contributions to the theory of partial differential equat.ions, Ann. Math. Studies, 33: 167-190, 1954.

",,,. "'_'_""""'''''''10''-''

[11] AW Naylor, GR Sell. Linear Operator Theory in Engineering and Science. New York, Heidelberg, Berlin; Sprin~er-Verla~, 1982. [121 M Schechter. Operat.or Methods in Quantum Mechanics. New York: Elsevier North Holland, Inc., 1981. [131 AH Siddiqi, Functional Analysis with Applications. New-Deihl, New York; Fourth Reprint, 1993. Tata tvlcGraw Hill.

[14J GF Simmons. Introduction to Topology and Modern Analysis. New

York: McGraw Hill, 1963. [15] AE Taylor. Introduction to Functional Analysis, John Wiley, New York: John \-Viley and Sons. Inc.. 1958.

[16J J Weidmann. Linear Operator in Hilbert Spaces.

berg: Springer-Verlag, 1980.

~ew

York, Heidel-

[17J J-T Wey!. The method of orthogonal projection in potential theory.

Duke Math. J., 7: 411-444,1940.

...... "'_'_""""'''''''10''-''

Chapter 4

Fundamental Theorem::;
4.1 Introduction

There are five basic results which, along with their coll~quences, have played an important role in such diverse areas of mathematical sciences as approximation theory, Fourier analysis, numerical analysis, control theory, optimization, mechanics, mathematical economics, and ordinary and partial differential equations. These results are known as Hahn-Banach Theorem, Banach-Alaoglu Theorem, Uniform Doundedness Principle, Open ~'Iapping and Closed Graph Theorems. Topological forms of the HahnBanach Theorem b'llarantcc the preservatiolJ of propertieb when a functional defined on a subspace is extended over the entire space. 1t has been established beyond doubt that the existence and uniqueness of solution., of problems in different areas of science and technology are closely related with the topology and related COIl\'crgence of the underlying space. We discuss here strong, weak and weak' topologies and related cOIl\'ergence. More recent concepts of convergence like convergence of COIl\'ex sets [15], r convergence [1,4] and 2-scale convergence [81 have found interesting applications. The Uniform Bounded Principle hroadly shows thaI, uniform boundedness of a sequence of bounded linear operators is guaranteed by the boundedness of the set of images. A converse of this result, often called the Banach-Steinhaus Theorem, is given as a solved example. It states tha.t uniform boundedness of a sequence of bounded linear operators, along with pointwise converf:,ccnce on a dense subset, is suflicient to ensure the pointwise convergence of the sequence on the entire space. The Open Mapping Theorem or thc Banach Theorem states that. a one-to-one continuous opcrl'\tor from one Banach space onto another has a continuous inverse, that is, the operator maps open sets into open sets. 'The Closed Graph Theorem demonstrates the continuity of a linear operator with a closed graph. In this chapter, we discuss the above mentioned results and their

Q.

......

, to<l',_"""",,,,,"l1>"-I

consequences. References for further reading are gi\'Cll at the end of the chapter.

4.2
.:1.2.1

Hahn-Banach Theorem
ExtensiOJI

ron/)

of Hahn-Ballad} Tlworem

Theorem 4.2.1 (Hahn-Banach Theorem). Let X be a ret.ll vectur SP(lCt~, 111 a subs!mce of X, (lnd 1) a ret.ll function defined on X satisfyin!} the fulluwing conditions:
1. 1)(X + y) :$ p(x)

+ p(y),

2. p(ax) = 0:7)(X),

v x, Y E X

(l1Id positive real

0'

Further, suppose that. f IS a lmem' functional on M such that. f(x) :$ p(x) \:j x EM. Then the,,~ e:ri,~ls a linear functional F defined on X for which F(x) = f(x) V;l; E !If and F(x) :$ p(x) \:j x EX. In othel' words. thne e:t'ists an extens1Qn F of f having the PI'O]JCfty of f
Theorem 4.2.2 (Topological Hahn-Banach Theorem). Let X be a normed space, Al a subspace of X . and f a bounded linear functional 071 111. Then then: extsts (I bounded linear functimwl F f)fI X such that
J. F(x) ~/(x) ~xE Ai,

IIFII

11/11

In other words, there exists an extension t' of f which is also bounded linear and preserves the norm. The proof of Theorem 4.2.1 depends on the following lemma:

Lemma 4.2.1 Let X be a vector' space and M its pl'O]JCr subspaa. For Xo E X - M. let N = [M U {xo} J. Furthermore. suppose that f is a linea?' functional on Ai and p a junctional on X satisfying the conditions in Theorem 4.2.1 such that. f(x) :$ p(x) \:j x E M. 'Then there exists a linear functional F defined on N such that F(x) = f(x) \:j x E AI and F(x) <p(x) ~xEN .

Q.

......

, to<l',_"""",,,,,"l1>"-I

In short, this lemma tells us that Theorem 4.2.1 is valid for the subspace generated or spanned by AI U f Xo } .

Proof. Since l(x) <; ptx), for x E Ai and f is linear, we have, for arbitrary YI, YI E M

1(YI)-1(Y2):$ P(YI +XO-Y2- XO)

$ P(VI

+ xo) + P(-Y2

- xo),

by condition (I) of Theorem 4.2.l. Thus, by regrouping the terms of Y2 on olle side and those of YI on the other side, we havp

Suppose YI is kept fixed and Y2 is allowed to vary over AI, then Equation (4.2.1) implif'!l t.hat. the !leI. of rf'al numhers { -p( -Y2 -xo) - 1(Y2 )/V'l EM} has upper bounds and hence the least upper bound (See Remark A.l( (1)). Let ex = su p { -1)(-112 - xo) - 1(Y2)/Y2 E At} If we keep Y2 fixed and VI is allowed to vary over M. equation (4.2.1) implies that. the set of real numbers {P(VI + xo) - f(yd/vI EM} has lower bounds and hence the greatest lower bound (See Remark A.l (I)). Let {3 = inf{p(YI + xo) - f(yd/YI EM}. From Equation (4.2.1), it is clear that. ex <fJ. As it is well known that. between any two real numbers there is a always a third real number, let l' be a real number such that (4.2.2) It may be observed that if ex = {3, then l' ::::: ex = p. Therefore, for all yE M,wehave

-p( -y - xo) - fry)

~ 1

<p(y + IO) - fry).

(4.2.3)

From the definition of N, it is clear that every elemem. x in N can be written as

x=y+),xo,

(4.2.4)

Q . ,>I"_""""'''''''IO''-I

......

where 7.0 ~ M or Xo E X - Ill, ), is a uniquely determined real number and y a uniquely determined vector in M. We now define a real-valued function on N as follows:

F(x)

F(y

+ ),x,) =

f(y)

+ ),~,

(4.2.5)

where l' is given by Equation (4.2.2) and x is as in equation (4.2.4). We shaH now verify that the well-detinet..! function F(x) satisfies the desired conditions, i.e.,
1. F is linear.
2. F(x) = f(x) V x E M. 3. F(a:)

< p(x) 'tt x

E N.

1. F is linear: For

ZI ,Z2 E N (ZI ::: VI +),1 XO, Z2 ::: V2 + ),2XO)

F(zi + Z2) ::: F(VI

=

+),1 Xo

+ V2

),2XO)

F( (VI + Y2) T (),I + ),2)XO)

+ V2) + (),I + ),2)1' f(VI) + f(V2) + ),11' + ),2/,

::: f(vi
=

a"

f is linear; ur
F(zi + Z2) ::: [f(Vd + ),11'1 + [f(Y2) + ),21']
= F(zd + F(Z2)

Similarly, we can show that F(ltZ) = pF(z) 'r/ zEN and for real/t. 2. If x EM, then), must be zero in Equation (4.2.4) and then Equation (4.2.5) g;vffi F(x) = f(x). 3. Here, we consider tllf~ cases. (See cquatioll (4.2.4)). CMe 1 ), - O. We haw' seen that F(x) ::: f(x) and as f(x) S 1J(X), we get that F(x) < p(x). Ca<;e 2 ), > O. From Equation (4.2.3), we have
l' :$ p(V

+ xo) - f(v)

(4.2.6)

Since N is a SUbspace, vi), EN. Heplacing V by vi), in Equation (4.2.6), we have l' :$1J(vl), + xo) - f(yl),) or l' :$ p (t(y + ),xo)) - f(yl),) By condition (2) of Theorem 4.2.1, p (x(V + ),xo)) ::: xp(v + ),xo), for ), > 0, and f(v I),) ::: f(v) as f is linear. Therefore, ),1' :$ 1J(Y + ),xo) - f(y)

",,,. '.'_'_""""'''''''10''-''

or f(y) +),1' < p(y + ),xo) Tlms, from Equations (4.2.4) and (4.2.5), we have F(x) < p(x) V x EN. Case 3 >. < O. From Equation (4.2.3), we have -p(-y - xo) - fry) '" Replaclll,l!; 11 by yl>'
In

>.

(4.2.7)

E<luation (4.2.7), we ha\'e

-v -p ( T
0'

- Xo )

f(y/),)

< >,

-y -p ( T-xo ) <,+f(yf>')

= >+ xf(Y),
as

f is linear, i.e.,
-1) (

-y T

-;1:0

1 < 1'+ >..f(Y).

(4.2.8)

rVlultiplying (4.1.8) by>., we have

(the inequality in Equation (4.2.8) is reversed as >. is negative), 0'

Since

-t > 0, by condition (2) of Theorem 4.2.1, we ha\'e

and so

(-),) (-D
Q.
.....
, to<l',_"""",,,,,"l1>"-I

pry + Axo)

:> F(x),

F(x) ~ 1)(X) V x E N .

'Proof of Theorem 4.2.1. Let S be the set of aU linear functionals F such that F(x) = f(x) V x E AI and F(x) ::; p(x) V x E X That is to say, S is the set of all functionals F extending f and F(x)::; p(x) over X. Sis nonempty as not only does f belong to it but there are other functionals also which belong to it by virtue of Lemma 4.2.1. We introduce a relation in S as follows. For F I , F2 E S, we say that PI is in relation to F2 and we write F. < F2 if DF. c DF2 and Fd DFI = FI (Let DFI and DF2 denote, respectively, the domain of PI and F 2. F2/DFI denotes the restriction of F2 011 the domain of F I ). S is a paltially ordered 5PI.. The relation < is reflexiw as F. < Fl. < is t.ransil,ive, because for F 1 < F 2 ,F2 < F3, we have DFl C DF2, DF2 C DF:!. F2/DF[ = F I and F3 /DF2 = F2, \vhich implies that DFl c DF:! and F:!/DFI = Fl. < is antisymmetric. For F I < F2;
DFI C DF2

F2 /DFl =FI

DF2 C DFI FJ/DF2 = F 2

Therefore, we have

We now show that ewry totally ordered subset of S has an upper bound in S. Let T -== {Fo } be a totally ordered subset of S, Let us consider a functional, sa.v F defIned oyC!" UDF". If x E U DF", there must be some
o
0

o2 ol o2 or DFo2 C DFo Let DFo ] C DF,,2. Then x E DFo2 and so x + Y E DFo2 , or x + y E UDFo ' Let x E UDFo , then.1: E DFol which implies that
tlX

U DF" is a subspace; Let x,y E U DFo . This implies that and y E DF ' Since T is totally ordered, either DF C DF

a such that x E DF", and we define F(x) = Fo(x). F is well defined and its domain U DFo is a subspace of X .
x E DFoI,

E UDFo

I:;f

F is well defined: Suppose x E DFo and x E DFv . Then by the definition of F, we have F(x) = Fo(x) and F(x) = Fv(:r.). By the total ordering of

real Ii. This shows that UDFo is a subspace.

...... "'_'_""""'''''''10''-''

T, either F" extends Fv or vice versa, and so F,,(x) :::: Fv(x) which shows that F is well defined. It is dear from the defimtion that F is line<U', F(x) :::: I(x) for x E DJ :::: M and F(x) :::::: p(x) 'i x E D F . Thus, for each F" E T, F" < F; i.e., F is an upper bound of T. By Zorn's lemma (Theorem A.2 of the Appendix A), there exists a maximal element fr in S; Le., fr is a linear extension of I, F(x) :::::: p(x) and F < fr for every F E S. The theorem \"m be proved if we shO\" that D F :::: X. We know that D F C X Suppose there is an element x E X 'Ouch that Xo If. D F' By Lemma 4.2.1, there exists fr such that fr is

linear, F(x):::: f\x) 'ix EDt, and i'(x):::::: p(x) for x E [DFU {xo}]. (F is also an extension of J). This implies that F is not a maximal clement of S \"hich is a contradiction. Hence, D F :::: X. I 'Proof of TheOJ'em 4.2.2. Since I is bounded and linear, we have If(x)1 < IIJllllxll. V x. (,;ee Remack 2.4.4). if we define p(x) :::: 1I/1111xll, then p(x) satisfies the conditions of Theorem 4.2.1 (see Unsol,,(..>(1 Problem 2.8.5). By Theorem 4.2.1, there exists F extending I which is linear fmd F(x) < p(x) 'V x E .\". We have -F(x) == F(-x) as F is linear, and so by the abo\'e relatioll, -F(x) < p( -x) ~ IIJIlIl - xII ~ IIJllllxll ~ PIX). Tim" IF(x)1 < PIx) ~ 1I/1111xll which implies that F is bounded and

IIFII

II x ll=1

,up IFlx)1 < IIJII

(4.2.9)

On the other hand, for x EM,

I/(x)1 :::: IF(x)1 < I)FlllJxll, and so

llxll=!

IIJII ~ ,up

IJ(x)1 ~

IIFII

(4.2.10)

Hem:p. by Equations (4.2.9) and (1.2.10),

11/11:::: IIFII.

Remm'k 4.2.1 Tlw Hahn-Banach Theorem is also valid for 110rmed spaces defin(..>(1 OYer the complex field.

1.2.2

COllsequences of tlJ Extension Form of tll Hll/m-BaIUldl Theorem

The proof.__ of t he following important results mainly depend on Theorem 1.2.2:

...", "'_'_""""'''''''10''-''

Theorem 4.2.3 Let w be a 1l0mero vector in a normed space X. 'Then fhere exists a contifltlOus linear functional F, defilled on the entire slJace X, such that IIFI! = I and F(w) = IJwll. Theorem 4.2.4 fhen w = O.
If X is a normed space such fhat F(w) = 0 V F EX',

Theorem 4.2.5 Let X be a nonned space and M its closed subslmcc. FtIT'I.her, assume tJwt wE X - M(w E X but w /Il). Then fhen~ exists FE X' such Ihat F(m) = OfomUm E At. and F(w) = l. Theorem 4.2.6 Let X be a nonned SlJace. M its subspace and w E X stIch thai. d = inf IIw - mil > O. "'EM (It may be observed that this condition is satisfie<l if Ai is closed and w E X - M). Then there exists F E X' \vith IIFII = I, F(w) '" 0 and F(m) = 0 for all m E M. Theorem 4.2.7

If X' is sepamble, tJlen X is itself separable.

Lct M =

Proof of Theorem 4.2.3.

f: M ---+ R such that f(m)

[{w}]

= {mjm= AW,>. E R} and

= >'llwlll is linear

[f(ml +m2)

= (>'] +),2)lIwll,

>.\ wand m2 = >'2W or f(m, + 1H2) = (>'] + >'2)lIwll = ),dlwll + >'211wll = f(mJl + Itm2)]. Similarly, I(/lm) :::: IJ/(m) 'V IJ E R. 1 is bounded (If(mll ~ lI'\wll ~ IImll and", If(m)1 < kllmll whe,-e 0 <k< 1)
where m1 :::: and few) :::: IIwll (If m = w. then), = I). By Theorem 2.4.2,

Ilfll ~

n"'I~1

,up If(m)1 ~ ,up 1'\llIwll ~ ,up "'EM 11"'11==1 IImll::::]

IImll

Since f, defined on Al, is linear and bounde<l (and henc(> continuous) and satisfies the conditions few) = IIwll alld 11/11 = 1; by Theorem 4.2.2, there exists a continuous linear functional F over X extcnding f such dlat IIFII = I and F(w) ~ IIwllI Proof of Theorem 4.2.4 Suppose w '" 0 but F(w) :::: 0 for all F E X'. Since w =/:. 0, by Theorem 4.2.3, there exists a functional F E X' such

...... "'_'_""""'''''''10''-''

that IIFII :::: 1 and F(w) :::: IIwll. This shows that F(w) i- 0 which is a contradiction. I~ence, if F(w) :::: 0 'V F E X*. then w must be zero. I ProofofTheore1l14.2.5. LetwEX-Mamlr1:::: inf
mEM

IIw-mll.

Since

AI is a dosed subspace and w M, d > O. Suppose IV is 1.llI" subspace spanned by tV and M; Le., n E N if and only if

n :::: AW + m,
Define a functional on N as follows:

), E R, mEAl.

(4.2.11)

lIn) I

A.

is linear and bounded: I(nl + n2) :::: AI + ),2, where 101 :::: ),1 W + m and 112:::: ),2W + m. So 1(11) l- 1l.2) :::: I(nt} + l(n2)' Similarly, /(J-l1O) :::: J-l1(n) for real J-l. Thus, I is linear. In order to show that I is bounded, we net-'<I to show that there exists k > 0 such that I/(n)1 < kllnll 'V n EN. We have IInll ~ IIrn + Awll ~ II-A (-'; -w)1I ~ IAIII-'; -wll Since -rnA E AI and d:::: inf IIw - mll, we see that d. Hence, 111011 : :-: 1),ld,
0'

By definition, 1/(n)1 ~ IAI < IInll/d, 0<' l/(n)1 < kllnll, where k > ~ > O. Thus, I is bounded. n:::: W implies that), :::: I, and therefore 1(10) :::: I. 11 :::: m E M implies that), :::: 0 (see Equation (4.2.11)). Therefore, from the definition of I, I(m) =... O. Thus, I is bounded lineal' and satisfies the conditions I(w) :::: 1 and I(m) :::: O. Hence, by Theorem 4.2.2, there exists F defined over X such that F is an extension of I, and F is bounded linear. i.e., FE X. F(w):::: 1 and F(m) :::: 0 for allm EM.

IAI

mEM

II--J:!- - wll : :-:

IInll/d.

I
Pl'Oof of Theorell1 4.2.ti. Let N be the subspace spanned by M and w (see Equation (4.2.11)). Define I on N as I(n) ::::),d ProCet.>(ling exactly as in the proof of Theorem 4.2.5. we can show that I is lineal' and bounded on N[I/(n)/:::: /),Id Since II(n)1 <

~ IInll],

I(w):::: d

i- 0 and I(m):::: ufoI' aJi m E M.

I.

IInl!'

we have

II/II <
For arbitrary
.

(4.2.12)

> 0, by lhf' definition of d, there must exist an m E AI such that IIw - mil < d -I- t. Let z :::: II~-:~II' Then liz II II 1I~-~~1111 I and I(z) :::: IIw I mil I(w - m) :::: d/llw - mil [l3y definition, I(n) :::: )'d; if
::0::

::

n :::: W

fIl,

then), :::: I, and so

B.

..... ,

, toOl',_"""",,,,,"l1>"-I

0'

(4.2.13) By Th(.-'Ol'em 2.4,2. implies that

11/11 ==
d~.

11/11 >

sup 1/(m)l. Since IIzll == I. Equation (4.2.13) IImll=l Siuce to > 0 is 3I'bltrar)', we have

11/11 2

I.

(4.2.14\

From Equations (4.2.12) and (4.2.14), we hmre IJ/IJ == I. Thus, f is bounded and linear, I(m) == 0 'r;I mE IH,f(w) #- 0 and 11/11 == I. By Theorem 4.2.2, there exists F E X such that F(w) -10 O. F(7Il) == 0 for all m E Al nnd IIFII ~ 1. I

Proof of Theorem 4.2.7. Let {F,,} be a sequence ill the surface of the unit sphere S of X*[5 == {F E X"/IIFII = such that {F],Fz , ... ,F,,} is a Jense subset of S. By Theorem 2.4.2, IIFII == sup IF(v)1 and so for

1}]

to

> O.

IIvll=l

there ('xisl.s

11

E X sudl that

IIdl
and
(I -

~ I.

dllFll

IFt")I.

(4."-15)

Putting

==

t in E<luation (4.2.15), there exists v E X such that

and

ZIIFIl
Let {v,,} be a sequence such that

IF(o)l

and

.n",

"'_'_""""'''''''10''-''

and AI be a subspace spann<.-'(] by (vn}' Then Mis separablp by its construction. in order to prove that X is separable, we show that M = X. Suppose X #- 111; then there exists wE X, W 1. M. By Theorem 4.2.6, there exists F E X * such that

IIFII

~ I

F(w)
and

i' 0,

(4.2.16)

F(m) = 0 "1m E JIJ.

In particular, F( v n ) = 0
I

'r/

n where
~
-

"IIF"II < 1P,,(v,,11

-:; 1F,,(1In )

IF,,(v,,1 - P(v,,) T F(v,,11 F(t},,)1 + IF(1I.,)I.

Since IlvnU = 1 and F(v,,) = 0 'V n, we ha:';c

(4.2.17)
We can rhooflf! {Fn } SHch t.hat

" ..... '>0

lim IIIFn - F)II ~

(4.2.18)

because {F,,} is a dense subset of S. This implies from Equation (4.2.17) that

Thus, using Equations (4.2.16) (4.2.18), we have

1~

IIFII ~ IIF -

Fn

+ Fnll <: IIF - Fnll + I/Fnll < IIF Fnll + 21/F Fnll,

0'

1~

I/FII

0,
IS

\vhich is a comradiction. Hence, our assumption

false and X = M.

B..

..... ,

, to<l',_"""",,,,,"l1>"-I

4.3
4.3.1

Topologies on Normed Spaces

Strong and
~Veak

Topologies

A IJUfmed :,pace can be <I topological space in many ways 8b one can define different t.opologies on it. This leads tn alt,f'rnati"f' notions of continuity, compactnt'SS, and convergence for a given Ilormed space. As shown ill Chapter 2, every IlDrmed space X is a metric space and hence it is a topological space. More IJrecisely, a basis for the strong lopology is the class of open balls centered at the origin
S.(O)

{x E Xlllxll < ,..

I(F,x)1 <

,. > a}.

(4.3.1)

A weak tOl)ofo!J1J is gcneratt-'(] hy the base comprising open sets determined by hounded lincar funclionals Oil X; namely, 8,,(0) =

{x E X/IF(x)1 =
r

r,

for all finite family (4.3.2)

of elements FE X',

>

o}

UTI is t.he topology gCIlf>nll.oo hy the norm (Equation (4.3.1)) and T z is the topology induced by linear functionals (Equation l4.3.2)), then 1"2 c T\; that is, T2 is weaker than T\. Let X be the dual of a Banach space Y; that is, X :::: Y. Then the weak"-topology (weak-star topology) is generate<l by the base at the origin consisting of the open set.s

{x EX:::: }" /1(Fv,x)1 = IFv(x)1 <- r, for all finite family of elements v E Y, r > O}, (4.3.3)
where

(Fv,u) = (u,v)

UEY',

'/JEY.

(4.3.4)

{Fv/v E Y} defin(.-'c! by (4.3.4) is a subspace of X = Y".

It may 1)(> nbserv('d that th(' notions of weak topology and 1/}"-topology coincide in a reflexiw space, in general and Hilbert space, in particular.
Rema1'k 4.3.1 (i) We will prcn'e 11\ Section 4.5 that the dosed unit ball of 1 he dual space X' of a Banach space (in fact a Hilberl space) is not compact in the strong topology while it is compact in the weak'-topology. (ii) The weak topology of a normed space X is equivalent to its norm topology if and only if it is finite-dimensional (see, for example, Wehausen

[21]).

...... "'_'_""""'''''''10''-''

4.4
4.4.1

Weak Convergence
Break Convergence in Bamlch SP;lCes

Let X he a Ilol'med spac:e, X' and X the first and second dual space of X, I"Pspectively. Then I. A sequence {xn} in X is calk-'d weakly convcrycnt in X, 111 symbols Tn ~ X or X n ...... x, if there exists all element x E X such that lim If(x n )
Jl-Hx>

Definition 4.4.1

f(x)1 = 0 for all f E So; i.e., fol' . > 0, there exists a natural number N sudl that I/(x,,) - f(x)1 :S" for n > N and V f E X'. 2. A seqUllCto: {I,,} in X' is called weakly' cOllvergent t(J f in X' if lim If.. (x) - f(x)1 = 0 for all x E X.
n~=

1. If X is a Banach space \vhidl is the dual of the nOffil(.-'d space Y, and if we bear in mind that Y is a subspace of its second dual, then we define weak' cOllveQ;enr-c <J.,!; follows: A .sequence {x n } ill X is called weakly' cOllvergent to x E X if lim Ix,,(y) -x(Y)1 = lim Iy(x.,) - y(x)l =

Rema1'k 4.4. 1

o V Y E }'.

.,~oo

It is dear t.hal. t,hp p)pnwnls of Y define t.he linetlf fund.ionals

.,_=

on}'*=X, 2. [I. Illay be wrified that cvery convergent sequence in X is weakly convergent but the converse may not be true. Howe'V'Cr, these I.wo notiolls are cquivalent if X is finite-dimem;ional. 3. Weak convergence implies weak convergence but lhe converse is not true in generaL However, these two notions are equivalent if X is a reflexive space. 4. A sequence {X.,} in X converges to x in the weak tOpology if and only if it conwrges weakly.
Definition 4.4.2 I. A subset A of a llormcd space X is called comlmcf, in the weak fOl)(JIO!J'!J or weakly compact if every S<."'quence {X.,} in A contains a subsequence which converges weakly in A. 2. A subset A of a normed space X is called compact ill the weak topology or weak compact if e,rery S<."'quence in A contains a subsequence which is weakly conwrgent in A.

The st.atements of the following theorem can bc easily vcrified: Theorem 4.4.1 1. The limit of a weakly cOllveryerll SelfUCrlCC is ullique. 2. Every conv1yenl scqtlence in a normed slX1ce is weakly co7weryenf,.

...... "'_'_""""'''''''10''-''

3, If {X.,} and {y.,} converge weakly to x and y, respecti'vely, tlien {X.,} + {y.,} converges wcakly to x + y. 4. If {x,,} converges weakly 1.0 x and a sequence of scalars {a,,} converges 1.0 (1;, then {(I;" x,,} converyes weakly to (l;X.

Definition 4.4 .."1

I. A sequence lx,,} in a nonued space X is called a

weak CatJchy 8eqtJence if {f(x,,)} is a Cauch:y sequence for all f E X. 2. A nonned space X is called weakly complete if every weak Cauchy sequence of elements of X converges weakly to some other member of X,

Theor'em 4.4.2 In the normed space X, every weak Cauchy sCf}tJence is bounded and hence evenJ weakly cOflVeryent seqUf.-'1lce with limit x is boumled, i.e., thesel. {lIxl.dl/k= 1,2,3, ...} ts boundedinX and

IIxll <

liminfllxkll
k---.oo

Also, x belongs to Ihe S1!bsrlGce of X genemted by the seqtJence {x,,}.

For proof, see Bachman and Narici [2]. Now, we mention some theorems giving the dlaracterizations of weak convergence in spaces ,>' L ,>, and CIa, bl. The proofs can be found, for example, in Kantorovich and Akilov [10] and Liusternik and Sobolev [12/.

TI leorenI4.4.3
(p,t

A seqtJeflce {} ,xu = x"

(u) (u) (u)) E ( 0) .oz , .. "ok ,'"

,(I;k, . . . )

<

<

00

convemes weaklu to x =

((1;).(1;2, ...

e"

if and

omy if 1. Ilx" II ::;: M (AI is a posi/.ive constant) for all n. 2. For every i, (I;~") --t 0. as n --t 00.

Remm'k 4.4.2 In view of the above theorem fol' bounded sequences, the concept of the weak converge.nce in ,> is equivalent to the coordinatewise convf'l'gence.
Theorem 4.4.4 A sequence {f,,(I,)} in ,,(0,1), I < P < convergent to f(t) E L,,(O,I) if and only if 1. IIf,,1I '5 At V 11. (M bemg a positive constant)
is weakly

00,

2.

, tJ

!.)" f,,(t)dt 1t' f(t)dt for arbitmry >. E [0,11.

---lo

...". "'_'_""""'''''''10''--'

Theorem 4.4.5

A sequence {fn(t)} in C[a,b] is weakly convergent to f(t) in CIa, b] if and only if 1. Un(t)1 :$ M, where AI 1S al)Ostltve com/,ant, 10 = 1,2,3, ... awl

1E

[a,bJ.
a~oo

2. lim !,,(tj = flO for every t E [a,b]

After slight modification of the proof of Theorem 4.4.5, we get the following th(.-'Orem (See A.H. Siddiqi [17]).

A sequence of continuou.~ fUllclions {f,~(t)} converges weakly (uniformly in k) f,o a conlintJOtLs fUllclion f(t) if and only if /. If~(t)I::::: AI where Af is a rlOsilive constant, tJUiformly in k,n 1,2,3, ... , andt E [a,b]. 2. lim f~(t) = fU) uni/onllly in k for every t E [a.b].
Theorem 4.4.6
"-+00

J.A. Siddiqi [18, 19], Mazhar and A.H. Siddiqi [13, 14] and A.H. Siddiqi [17J ha:';e applied Theorems 4.4.5 and 4.4.6 to prove several inten':Sting result.s concerning Fourier coefficients, Walsh-Fourier coefficients and summability of trigonometric sequences.
Example

Rema7'k 4.4.3

4.4.1

Show that weak convergcncedocs 110t imply convergence

in norm.

Solution. By the applicatioll of the Riesz representation theorem to the Hilbert space L 2 (0. 211"), we find that

f(x) ~ (x,g) ~ Jo
Consider the sequence

r'" X(t)9(1)<lt.

(4.4.1)

{x" (t) } defined below

~,i.::n.::".::t . =-

X n () t

rr

for n=I,2,3, ...

2

We no\.... show that

{x n is weakly convergent in L (0.21T) but is not (/.)}

norm-convergent ill L 2 (O, 21T). "'rom Equation (4.4.1), we have

f(x,.,) = (x,."y) = rr

'1'"
0

sinnt y(t)dt. rr

(4.4.2)

B.

......

, to<l',_"""",,,,,"l1>"-I

The right-hand side of Equation (4.4.2) is the trigonometric Fourier coefficient of g(t) E 2(0, 1l"). By the Riemann-Lebesp;ue theorem concerninp;- the behavior of trigonometric Fourier coefficients, -

IT

11"
0

sinntg(t)dt-+Oas n -+ 00,

or f(x.,) -+ U as n -+ 00; i.e, {X.,} COI1\'crp;es weakly to O. \\le have

IIx. - 0Il ~ II x.1I ~

1,. (

Ix.(tll'dt
1/2

)'1'
_,_

(1o
1 ( Since 1l"

2)'r )

sin ntl 2 d ) 2 t. rr

v1l"

r.;.

,.

sin 2 nt rlt

,n

:f:. 0 'V n, IIx" - 011 cannot tend

to ... .ero, .

and

therefore {x,,} cannot converge in the nOlTll. Thus. a weakly cOI1\'ergent sequence {x,,} Ileed nut be convergent iu the 1I0rlll.

Example 4.4.2 vergence

Show that weak convergenw does not imply weak con-

Solution. Let X = f l (we know that f l is the dual of ('() and Y = eo. Then Y c:< fl. Thus, the dual of Y is X. Let {x lc } be a sequence in X defined by the relation
x~ = 0
~

lfkij ifk=j.

For Y = (YI,Y2,Yk, ... ) E Co, let xk(y) = Yk (x k belongs to the dual of C{); Le., it is a bounded linear functional on eo). Since Y E C{), lim Yk = 0 and so lim xk(y) = lim Yk == O. Therefore.
k--4OO

k---+oo

k---+oo

the sequence of the dual space of Y = Co converges ,..'eakly to zero. Now, if z E X = foo with z == (Zl, Z2, ... ), then xk(z) = Zk. Since Z E foo, {Zk} is bounded with respect to k but need not convcrge to zero as k -+ 00. In fact, if z = (1, I, ... ), then x k ( z) -+ 1 as k -+ 00. Therefore, {x k } does not col1\'erge weakly. This example shows that ""cak convergen("(> does not imply weak convergence and, by Example 4.4.1, it does not imply norm convergence.

{x k }

Q.

.... ,

, to<l',_"""",,,,,"l1>"-I

Prove that the notions of ",~a1; and strong convergence are equivalent in finite-dimensional normoo spaces.
Example 4.4.3

Solution.

See BachmaJ1 and Narici [21 and I<reyszig [111.

jn

4A.2

"leak COlJvcrgelH.;e

Hilbert Spm;es

By virtue of t.he Hipsz representation theorem, a sequence {x,,} in a Hilbert space X is weakly convergent. to the limit x E X if and only if lim (x,,, z) ::::
'1---'00

(x, z) for all z E X

Theorem 4.4.7 (The Weak Compactness Property). BVe7'!1 bo7trufed scqucnce of elements in u IMber'! SIK1ce X mnLaim a wmkly mmwrgent subscquence.

Proof. Let {:l:d dellote the boundLu sequence with bound !vi, i1xkll :5 Ai. Let Y be the closed subspace spaJmed by the elements {xk} Suppose that Y -.L denotes the orthogonal complement of Y Consider the sequence (Xl, x,,). Being the bounded sequence of real numbers, we can e},."tract. from it. a cOll\'ergent. subsequence by the Bolzano-\Veierstrass theorem. Denot.e this subsequence by (Xl'X:,). Similarly, (X2,X:,) contains a convergent subsequence o:~ == (X2'X~), Proceeding in this manner, consider the diagonal sequence x~. For each m, (xm,.r.~) converges, since for n > m, it is the subsequence of tllf' convergent sequence o:~'. Define F(x):::: lim(x,x~) n whenew~r t.his limit, exists. This limit df'arly exist.s for finite sums of the form

a:, : :

..

x:::: LOkXk.
k=l

which are dense in Y. Hence, for any y E Y, we can find a sequence y" such that IIY" - yll -+ 0 and
F(y,,):::: Iim(y".x~).

'"

Now

where

I(y -

yp,x~~)1

< Ailly - y,,)1-+ 0 as

p -+

00

unifOlTIlly in m.

Therefore, {(y,x::~)} converges. Since for any in y-L, (Z,Xk) = 0, it fo11o\','8 that F() is defined for e,'ery element of X in view of Theorem 3.4.2. It can be seen that F is linear. F is continuous as IF(Ym - y)1 lim HYm - Y, x~)l < AIlIy", - yll --+ 0 for y." --+ y . By the Riesz representation theorem
F(x) == {x.h} for some 11 in X. in fact in F.

It is also clear that 111111 S Theorem 4.4.8 stmngly to x.

"'1 as IF(xJl = l{x,h}1 < Mllhll.

--+ IJxll.

S1t1JIJOSe x n -" x amlllx,,11

Then Xn COllVe7yes

Proof.

We know that

IIx'

l -

xII

,=
~

(x" - x, X n - :1:)

IIx.II' + IIxll' -

(X.,x) - (x,x.),

n-4oo

lim

IIx n -x!l2

=
~
~

limllx,,1I 2 +lIxIl 2 _ n-4CO (xn,x)- n-fCO (x, x,,) lim lim

211xll' o.
(x, x) - (x, x) ~

211xll' - 211xll

I
Theorem 4.4.9 Let {x n } C071verye weakly to x. 11len we can subsequence {x".} such that its arithmetic mean
e~:t7'flct

converyes stmngly to x. Proof. Without loss of generality, we CaJ1 assume x to be Jlero. Let the first term of x,,~ be x'" = Xl' By ",~a1; convergence. we can choose X"2 such that I(xnl ,x"2)1 < 1. Having ch~1I x""'" ,xn.' we can dearly choose x"k+1 so that
[(x""J:nH,)1

< k'

i = 1,2,3, ... ,k.

..,,,. "'_'_""""'''''''10''-''

By the uniform boundedness principle (Theorem 4.6.1), Therefore,

1 '

IJxn~1I ~

M < 00.

k LX",
1=1

I
Corollary 4.4.1 space X. Then

Let F( ) be a contmuous convex junctw1wl on a Iii/bert

~

F(x)

lim inf F(x n )

/07' all X n -"

X.

P,oof. Let us consider a subsequence and, if necessary, renumber it so that lim inf F(x,,) = lim F{x m ) and fnrther renumber the sequence so that
by Theorem 4.4.9, n , convexity of P,
I " L
Xm

converges strongly to x. Then we ha\'e by

Hence,

F{x,,) = lim n ( ; F(XA;)

I"

> lim F n (;XA;

(I" )

= F(x).

4.5
..... ,

Banach-AJaogJu Theorem

In til(' first place, v"e observp that the dosed nnit ball in :=t Banach spacp need not be compact in its nonn topology. To prove the statement, consider

Q.

, to<l',_"""",,,,,"l1>"-I

the normed space i z. The dosed unit ball 51 of i 2 , namely

is both bounded and dosed. That is, 51 is a bounded closed subset of lz. We now show that 51 is not compact. For this, it is sufficient to show that there exists a sequence in 51, every subsequence of which is divergent. Define a sequence {x n } in 51 in the following manner. X n = (0,0, ... 1.0... ) (All coordinates are zero and n-th coordinate is I). Thus,
Xl

= (1,0,0,
~

"

(0,1,0,

) )
)

X3~(0,0,[,0,

= (0,0, x'l = (0.0

Xp

,1,0,0 ) 1.0 )

1 at p-th coordinate 1 at q-th coordinate.

For]J::j;. q, IIx p - xqlJ =

... )1/2 =

~ Ix p - xql Z

) '/2
= (0

+ ... ;.

1z -t-

12

.,J2. Therefore, the sequence {X.,} and all its subsequence:, are

divergent. As seen above, the dosed unit sphere of i z = l2 is not compact with respect to its norm topology. However, the following theorem proved by Alooglu in the early forties, often called Banach-Alaoglu Theorem, shows that the closed unit sphere in the dual of every normed space is compact with respect to the weak topology. X is a 1I0rme(1 s1wce a7uI X 1.5 its dual. Then the closed umt sphere Sj = { / /11/11 :::; 1S compact wall resper:t to Vie weak' tOlJOJOfJIJ
SU1JIJOSe

Theorem 4.5.1

EX

I}

Pmof.

Let C.,

= [-IIX!l,II:dl]

for

E X amI C

II c
rEX

cross-

product of all C.,. By the Tychollofftheorem (Th. B.2(1) (C is a compact topological suLspace of R''). If! E Sj, then If(x) I :::; Ilfllllxll < IIxll and so /(x) E C.,. We can consider Sj ~ C where / E Sj is associated with (J(x.),/(xz), .. ,!{xn) ... )ECforxl,x2, ... ,Xn, ... EX.

...... '.'_'_"''''''''''''''10''-''

then 9 E r [as S; ~ C = C in view ofTh. B.2{2)], <lndlg(x)1 ~ IJxll Now, we show that 9 is linear. Let c > 0 be given and x,y E X. Since evr:ry fundamental neighborhood of 9 intersf'('ts Sj, thp,re pxist.s an 1 E Sj such that

Since C is compact, in view of Theorem B.2(5), it is sufficient to prove that Sj is a closed subspace of C. For this. we show that if 9 E S; then 9 E 5j; that. is to say, S~: Sj, whidl implies that 5j is closed. let 9 E S; ;

Ig(x) - f(x)1 Ig(g) - f(x)1

19(X+9)-f(x+9)1 <
Siu("p 1 is linear, I(:/:

, <3 , <3 ,

+ Y) - I(x) - I(Y)
- fix

= 0 and we htnp

19(x +

g) - 9(X) - g(g)1

~ 119(x + y)

+ y)]- [9(X) - f(xll - 19(9) - f(g)]1

< Ig(X + gl - fix + g)1 + Ig(X) - f(xil + 19(9) - f(911

< -+-+-=. 333
As this relat.ion is true for arbitrary c > 0, we have g{x + y) = g(;r) + g(y). In the same way, we caJ1 show that g{ox) = og() for all real o. Thus, 9 is linear and bounded. Moreover.:[:f:. 0,

, , ,

1I~1119(x)1 < 1 m 9(11:11) S I,

as 9 is linear (for x 9 E Sj.
== O,g(x)

= U and so Ig(x)1

= lJ

< I). Therefore,

4.6
4.6.1

Principle of Uniform BOllndedness and Its Applications

Principle of Uniform Boundedness

Theorem 4.6.J (The Principle of Uniform Boundedness or BanachSteinhaus Theorem). Let X be a Banach space, Y a normerl SIJUce

.."" "'_'_""""'''''''10''-''

Gn(l fT;} a sequence of ooun(le(llinea7> opcmtors over X 1.nto Y such that,

{T;(x)} 1S a bou7l(le(1 subset ofY for all x E X. Then fllTill} is a boun(JC(J subset 0/7>eal numbers, i.e., {T,} is a b0'U1ulC(1 sequence in the normed slKlcC IlIX, YJ.

Remark 4.6.1 L This prinCIple was first discovered by the French mathematician Hemi Leon Lebesgue in 1908 during his itwestigalion concerning the Fourier series but the principle in its general form wa,.<; given hy Stefan Banach and another famous mathematician H. Steinhaus. 2. Some RussiaJ1 mat.hematiciaJ1s call Solved Problem 4.8.2 as the BanadlSteinhaus theorem (See Kantorovidl and Akilov [10]). 3. Completeness of the space X is essential for the validitv of this principle (~L- SoIvL-'C1 Pl"Ublel1l 4.8.3).
Proof of Theorem 4.6.1. III the first. step, we shall show that the hypothesis of the theorem implies the following condition. For some w E X and some positive A > 0, there exists a constant k > 0 such that [ITfi (x)[1

< k,

(4.6.1)

for all nand 3' E S),(w). In the second step, we shall show that this condition implies the desired result. Suppose Equation (4.6.1) is false, i.e., for any sphere S with an arbitrary point as the center and an arbitrary r > 0 as the radins. there exists an integer 710 aJ1d w E S such that

(4.6.2)
Since the function f(x) = !IT"'o(x)1I is a real-valued function of x that is greater than r at xo, the continuity implies that there rtlust be a neighborhood of w in which f(x) > r; furthermore, \\'f' can assume that this neighborhood of w is wholly contained in S. Symbolically, we are asserting the existence of a neighborhood of w, ST such that ST C 5 and [IT".(x)[1 > r V x E 5,. If we choose l' = 1, then by the above fact, there exists some closed set 51 such that S\ c S and an integer n SUdl that lIT'l1(x)1I > 1 V xES. F\trther, we can assume thatdiaJlleterufS 1 , Ii(SI) < 1. We repeat theabm'e procedurt> for S\ instead of S, and 7> = 2. Thus, we get a closed set S2 C S. SUdl thaI. Ii{S2) < 1/2 SUdl that for some 712, IIT"2(x)1I > 2 V x E S2. Continuing this process, v"e get a sequence of closed sets {Sj} satisfying tht> conditions of Theorem C.2(7) of Appendix C, and hence by this theorem, there exists some point y E nSf which implies that !IT,,.,(Y)II > i for every
i.

Q . ,>0<1"_""""'''''''IO''-I

.... ,

Thi will imply that the hypothesi of the theorem that {IITi(x)lI} is a bOli ded subset of Y \;j x EX, cannot be true, and this is a contradiction. Hence, our assn ption i.s false and quation (4.6.1) holds good. et quatio (4.6.1) be true and x be such that Jlxll < A. Then

IITn(x)11

IITn (x w) - Tnwll
w)1I

< IITn (x

+ JlTflwll

(4.6.3)

Since II (x + 1U) - wll == lIxll < A, x + w E S).. (w). This Implie , by Equation (4.6.1), that IITn (x + 1u)1I < k for every n. Since IITn wlI < k, Equation (4.6.3) implies that V n IlTn x11 < 2k for all x such that !lxll < A. et x be any nonzero vf-'!'tor and ('onsider

Ilxll"

AX

Then

The above result also holds good for ;,r = O. Fronl the definition of the norm of an operator, we have forall n; Le., theorem.

IITnl1 <

2kjA

{HTnll}

is a bounded sequence in B[X, Y]. This proves the

Several applications of hem"em 4.6.1 in the domain of Fourier analysis and summability theory can b fOUlld in Kantorovich and Akilov [10] I Goffman and Pedrick [6L Dunford and Schwartz [5], and Mazhar and Siddiqi [13, 14].

4. 7 Open lVlappillg and Closed Graph Theorenls

4. 7.1 Graph of a Linear Operator and 010 'ednesti Property

Let }{ and Y be two nurrned spaces and D a Stl bspace of ~ .~. Then t e linear operator T, d ed on D into Y, is called closed if for e 'ery convergent sequence {x n } of points of D with the limit x E ..' t SUdl that th sequence {Tx n } is a convergent sequenc with the limit y, xED and y = Tx. TIlis DleallS that if lim X n =;:: x E )(, and lim TXn = y, then

Definition 4-. 7.1

xED and y

= Tx.

n---+oo

00

Definihon 4-.7.2

Let .." t and Y be t

normed spaces, D a subspace of

X, and T a linear operator defined on D into Y. Then the set of points

G T = {(x, Tx) E X )( Yjx ED} is called the {fI"U1J!I of T

Remark 4.7.1 If X and Yare two normed spaces, then X x Y is also a normed space with respect to the nOlTll lI(x,Y)1I = !lxl!. + IIYllz, II Iii and Ii . liz are nonns on X and Y, respectively. We see in the next theorem that Gr is a subspaCf' of X x F and hence, a normed subspace of X x Y with respect to the norm

II (x, Tx)1I

IIxll + IITxll

foe

xED.

The following theorem gives a relationship bptween the graph of a linear operator and its c10sedness property.
Theorem 4.7.1 A lilleor olJemtor T is a closed subspace.
IS

dosed if and only if its gml)!I G r

PnJOJ. Suppose that the linear operator T is closed, then Vie need to show that the graph G T of T is a closed subspace of X )( F. (D is a subspace of the normed space X, T : D -+ Y). In other words, we have to verify the following relations:
1. If (x,Tx),(x',Tx') E GT;x,x' E D, then (x,Tx)

+ (x', Tx')

E Gr -

2, If (x, Tx) E G T , XED, then A{x, Tx) E GT V real A,

3. For x., E D. if (X.,. Tx.,) E GT converges to (x,y). then (x,y) E Gr.

I
as T is lin~;u'. SiJl(~f' D is a subspace, x+x' E D. Th~refore, (x+x', T(x+x')) E GT 2. A(x,Tx) = (AX,ATx) = (Ax,T(Ax as T is linear. AX E D as D is a subspace. Therefore, (AX, T(.\x) E G T . 3. (x."Tx.,) converges to (x,y) is equivalent to
lI(x",Tx") - (x,Y)1I ~ lI(x" -x,Tx" -y)1I
~
0'

Verification.

1. We have (x, Tx)

+ (x', Tx')

= (x + x', T(x

+ x')

IIx" - xII + IITx" - "II

IITx,., -

~ 0,

IIx" - xII -+ 0 and

vII -+ 0

n -+

00,

...". "'_'_"''''''''''''''10''-''

that is,
,,---+oo

lim x' l = x and

,,---+0:.

lim TX n = y.

Since T is dosed, xED and y = Tx. Thus, (x,y) = (x, Tx), where xED. and so (x,y) = (x, Tx) E G T . To prove the converse, assume that G T is closed and that x" --+ x, X n E D for all n, aJ1d Tx" -+ y. The dl-'Sin.>d rl-'Snlt is provl->d if we show that xED and y = Tx. . The gh'en condition implies that (xn,Tx n ) -+ (x,y) E GT. Since G T is dosed, G T = GT; i.e., (x,1}) E G T . By the definition OfGT,y = Tx,x E D.
1. In some situations. the above characterization of the dosed linear operator is quite useful. 2. If D is a closed subspace of a normed space X and l' is a continuous linear operator defined on D into another normed space V", then T is closed (Since x" -+ x and l' is continuous, Tx" -+ Tx which implies that if TX n -+ '!.J. then 1'x = y and morelwer. xED as D is a closed subspace of

Remark 4.7.2

X.)

3. A closed linear operator need not be bounded. (The condition under which a closed linear operator is continuous is given by the Closed-Graph Theorem.) fbI' this, we consider X = Y = e[O, 11 with sup norm (see Example 2.2.9). and P[O.I] the normed space of polynomials (see Example 2.2.10) We know that prO, I] C C[O, 1]. .Let T . D = pro, 1J -+ [0,11 be defined df as 1'1 = dt (a) T is a linear operator; see Example 2.4.\l (b) Tis dosed; let In E P[O,I].
,,---too

lim

In = I TI
=

and lim

'1---+=

dl"
dl

= g. To prove that

is dosed, we need to show

that

dl
dt

= g. (It may be observed that the convergence is uniform.)

We ha,'e

Q.

......

, to<l',_"""",,,,,"l1>"-I

Differentiating this relation, we obtain

g=-=Tf.

df

d'

(c) T is not bounded (continuous); see Example 2.4.9,

Definition 4.7.3 An operator T on a normed space X into a normed space Y is called open if l'A is an open subset of F whenever .4. is an open subset uf X; i.e., T !lIap:,; OpCIl sets into open sets,

4.7.2

Open AJapping Theorem

Theorem J.7.2 (Open Mapping Theorem). If X and Y are two Banach spaces and T is a continuous linenl' operator on X onto Y. then T is an owm operator (open mapping). The proof of this theorem requires the following lemma whose proof call be found in Simmons [20, 235--236]. Lemma 4.7.1 If X aM}1" an: Banach spaces, and ifT is a continuous linear olH->f'ator of X onto F, then the image of eadl open sphere centered on the origin in X contains an open slJhere centered on the origin in F, Proof of Theorem 4.7.2. In order to prove that T is an open mapping, we need to show that if 0 L<; an open set in X, then T(O) is also an open set in Y. This is e(Juiw:tleIJt tu proving that if y is a point ill 1'(0), then there exists an open sphere centered on y and contained in T(O). Let x be a point in 0 such that T(x) == y. Since 0 is open, x is the center of an open sphere, which can be written in the form x + Sr, contained in O. By Lemma 4.7.1, T(Sr) contains an open sphere, say S~, ' It is clear that y + S~, is an open sphere centered on y, and the fact that it is contained in T(O) follows from the following relation:
y + 5;, ;

Y + T(S,)

= T(x) + T(S,) = T(x + 5,) ; T(O).

The following result is a special case of Theorem 4.7.2 which will be used in the proof of the Closed-Craph Theorem.

...... "'_'_""""'''''''10''-''

Theorem 4.7.3 A one-to-one contimwus linear operato,' of one Banach space onto another is a homeomorphism. [n particular, if a one-to-one linear operator T of a Banadl sTXIce onto itself is continttotU>, it.5 inver.~e 1'-1 is atdomaticaUy contimwus.

4.7.3

The Closed-Graph Theorem

Theorem 4.7.4 (Closed-Graph Theorem). If X and Yare Banadl spaCl:lj, and if l' is a li1tl:ar 01Jf.->f'ator of X into F, then T is continuous if and only its graph G T i~ closed.

Remark 4.7.3 In view of the following result of topology, the theorem will be proved if we show that l' is continuous whenever GT is closed
'If

f is a continuous mapping of a topological space X. into

n Hausdorlf space F, then the graph of the product X x y'

is a closed subset of

Proof of Theorem 4.7.4. Let XI be the space X renormed by IIxlll = IIxll+ IIT(x)lI S, ; " nocmed space. Since IIT(xlll S IIxll +IIT(xlll ~ IIxlh, T is continuous as a mapping of XI into Y. We shall obtain the desired result if we show that X and Xl have the same topology. Since IIxll :5: IIxII + 1I1'(x)1I = IIxlh, the identity mapping of Xl onto Y is continuous. If we can show that XI is complete then, by Theorem 4.7.3, thi~ mapping is a homeomorphism and, in turn,.-\ and Xl have the sanlf> topology. To show that XI is complete, let {x,,} be a Cauchy sequence lllit. This implies that {x n } and {1'(x n ,} are also Cauchy sequences in X and Y', respectively. Since X and )' are Banach spaces, there exist vectors x and y in X and F, respectively, such that IIxn - xII ~ 0 and 1I1'(x.,) - yll --t 0 as n --t 00. The assumption that G T is closed in X x Y implies that (x,Y) lies on G'r and so 1'(x) = y. FUrthermore,

IIx" - xiI. = IIx" - xII + IIT(x. - x)1I ~ IIx. - xII + IIT(x.l - T(x)1I ~ IIx. - xII + IIT(x") - yll -+ 0 as n
Therefore, XI IS complete.

-+

00.

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

4.8
4.8.1

Problems
Solved Problem6

Let {T.d be a sequence of continuous linear operators of a Banach space X into <t Bannc.h spncp Y such that lim 1'..,(3:) = 1'(x)
n~oo

Pf'Oblem 4.8.1

exists for every x EX. Then prove that T is a continuous linear operator

and
IITII ~ lim inC IITnll.
n~oo

Solution.

1. T is linear: (a) Wf> haw> 1'(x + y) = lim 1'.., (x

--->00

+ y).

Since P&h Tn is linear,

1;,(x + y) ~ Tn(x)
n ---+00

lim 1'.. (x --r y) =

=
~

+ Tn(y) lim [1'n(x) + Tn(Y)] n---too

lim 1',,(x)

n---+=

+ lim

"'-+00

T,,(y)

T(x)

+ T(y),

0,
T(x

+ y)

T(x)

+ T(y).

(b)

T(ax) = lim 1',,(03:)

n~oo

= lim ul~,(x)
n~oo

as each Tn is linear

T{ax) =

n~oo

lim 1'n (3:) = uT(x).

Thufl, T is lineRI'. 2. Since lim l~(x) = 1'(x)

n~oo

II 10m Tn(xHI = IIT(x)lI,

n~oo

as a norm is a continuous function and T is continuous if and only if x" --t x implies TXt} --t 1'3: (see Proposition 2.4.1(3).

.n",

'"'_'_""""'''''''10''-''

Thus, llTn (x)l1 is a bounded sequenCf' in Y By Theorem 4.6.1 (the principle of unifonn boundedness), 111',,11 is a bounded sequence in the space B[X,Y]. This implies that
IIT(x)1I ~ lim IITn (x)1I < lim in! 1I1~lIl1xll.
n ---> =
,,---> ""

(4.8.1)

By the definition of

111'11, we get 111'11 :S: lim inf 111',,11

n~=

(see Appendix 0.4 for

lim inf and Definit.ion (2.4.1(6)) for 111'11.) In view of ~luation (4.8.1), T is bounded and hence contmuous as l' is linear.

Problem 4.8.2 Let X and Y be two Banach spaces and {T,,} a sequence of continuous linear operators. Then the limit Tx = lim Tn(x) exists for ,,-->= every x E X if and only if
1.

IITnll:S:

M for n = 1,2,3,
n~=

2. The limit Tx = lim Tn(x) exists for every element x belonging to a dense subset of X.

Sol1ttiou. Suppose that the limit Tx = lim T,,(x) exists for all x E X. n~= Then clearly Tx ==- lim T,,(x) exists for x belonging to a dense subset of

X.

"-->00

IlTn l1 < p..,

for n = 1,2,3, ... , follows from Problem 4.8.1.

n~=

Suppose (1) and (2) hold; then we want to prove that T(x) ==- lim

l~(x)

exists, Le., for > 0, there exists an N such that 1I1~(x) - T(x)11 < ( for n> N. Let A be a dense subset of X, then for arbitrary x E X, we can find x' E A such that

IIx - x'il < 0,0 > 0

We haye
IITn(x) - T(x)1I " IIT,,(x) - Tn (x')11

and arbitrary.

(4.8.2)

+ IITn(x')II- T(x')I1
> N,

(4.8.3)

By condition (2),
IITn (x') - T(x') II <
(I

for n

(4.8.4)

.n",

'.'_'_""""'''''''10''-''

as x' E A, a dense subset of X. Since Tn's are linear,

1I1~(x)

T,,(x')11 ~

liT" (x - x')II

By condition (1) and Equation (4.8.2), we have

IIT,,(x - x')11 '" IIT"lIl1x - x'il

< Alb for all n.
From Equations (4.8.3) - (4.8.5), we have
1I1~,(x)

(4.8.5)

- T(x)1I

< {l,1O + l] < l

for all n

> N.

This pro\"es the desired result. Show that the principle of uniform boundedness is not valid if X is only a normed space.
Pf'Oblem

4.8.3

Sohttion. To show this, consider the following example. Let Y = Rand X be the normed space of all polynomials
x = x(t) =
with the norm

L u"t'l
n=O

where an = 0 V n > N,

IIxli = Stlp lanl.

X is not a Banach space. Define T., . X ---t Y

as follows:

"
Tn(x) ~

,,-,

La,

IIT..II
4.8.2 Unsolved Problems
4.8.4

k=O

is not. hounded.

Pf'Oblem

Prove the Hahn-Banach Theorem for a Hilbert space

Show that p(x) = limsupx n , where x = (x n ) E: too,x n real and satisfies conditions (1) and (2) of Theorem 4.2.1.
Problem 4.8.5

[f p is a functional satisfying conditions (1) and (2) of Theorem 4.2.1, show that 1)(0) = 0 and 1)(-x) ~ -p(x).

Problem 4.8.6

.n". "'_'_""""'''''''10''--'

Problem 4.8.7 [f F(x) = F(y) for every bounded linear functional F on a normed space X, sho"" that x = y. P7'Oblem 4.8.8 A linear function F defined on ing conditions, is called a Banach limit:
Tn,

satisfying the follow-

1. F(x)~Uifx=(Xl,X2,X:j,.. ,xn, ... )andxn >O'r/n.

3. P(x) = [;fx=(I,I,I, ... ).

Show that lnninfx n S; F(x) S; limsupx" V x = (x n ) E Banach limit

Tn,

where F is a

Problem 4.8.9 Let Tn = An, where the operator A: .2 --t .2 is defined by A(XI,X2,X3,X4, ... ) = (X3,X4,"')' Find
n ---+ =

Hm IITn (x)lI, IITn ll and

,,---+ 00

lim

11~1I.

Problem 4.8.10 Prove that the uormed space prO. 1\ of all polynomials with noml defined by Ilxli = suplO'II, where 01,0'2, . . are the coefficients

of x, is not complete

P7'Oblem 4.8.11 Show that in a Hilbert space X, a sequence {xn} is weakly cOIl\'ergcnt to x if and only (x n , z) converges to (x, z) for all z E X P7'Obiem 4.8.12 vergence in norm. Problem 4.8.13 flexive.
Show that weak convergence in 1 is equivalent to con-

Show that all finite-dimensional normed spaces are re-

B.

......

, to<l'._..

....,..",,"l1>"-I

References
[1] U Attaouch Variational Convergence for Funct.ions and Uperators. Applicable Mathematics Series, Boston, Mass, London: Pitman (Advanced Publishing Program), 19R4. [2]

Bachm:l.Il, L Narici. functional Analysis. New York and London: Academic Press, 1966.

[3] AV Balakrishnan. Applied F\mctional Analysis. New York: SpringerVerlag, 1976.

[4' G Dal Maso. An Introduction to r-collvergence. Boston: Birkhauser, 1993.

(51 N Dunford, JT Schwartz. Linear Operatoft!. Vol. 1, New York: Interscience, 1958.

[6] C Coffman, G Pedrick. First Course in Functional Analysis. Englewood Cliffs, New York: Prent.ice-Hall, Inc. 1965.

[7] CW Croetsch. Elements of Applicable Functional Analysis. New York

and Basel: Tvfarce\ Dekker, Inc., 1980.

[8) U Hornung. Homogeni:t..ation and Porous rvledia, Interdisciplinary AppJ. Math. Series. New York, Berlin: Springer-Verlag, 1997. [9] T Husain. Open tI.'Iapping and Closed Graph Theorems. Oxford: Oxford Press, 1964.

[101 LV Kant.orovich, GP Akita\!. Functional Analysis in Normed Spaces.

London: Pergamon Press, 1964.

[111 E Kreyszig. Introductory Functional Analysis with Applications. New

...... "'_'_""""'''''''10''-''

York: John \Viley and Sons, 1978. [12] LA Liustemik, V.J Sobolev. Elements of Functional Analysis (Third English Edition). Nc\v Dplhi: Hindustan Publishing Company, 1974. [13] SM Mazhar, AI-l Siddiqi. On FA and Fe summability of trigonometric sequences. Indian J Math 5:4G1-466, 1967. [14] SM Mazhar, AH Siddiqi. A note on almost A-summability oftrigonometric sequences. Acta Math 20;21-24, 1969. [15] U tvlosco. Convergence of convex sets, Advances 1969.
In

rv1ath. 3:510-585,

[16] AH Siddiqi. Functional Analysis with AVvlic.:ations. New Delhi, New York: Tata McGraw-Hill, 4th print, 1993. [17] AI-l Siddiqi. On the summabilit.y of sequence of Walsh functions. J Austral fI.'lath Soc 10:385-394, 1969. [18] JA Siddiqi. The Fourier coefficients of continuous functions of bounded variation. Math Ann 143:103-108, 1961. [19] JA Siddiqi. CoefHdent properties of certain Fourier series. Math Ann 18:242-254, ]9G9. [20] CF Simmons. Imroduciton to Topology and Modern Analysis. New York: McCraw-Hili, 19G3. [2l] J" Wehausen. Transformations Math J, 4: 157-1G9, 1938.
III

linear topological spaces. Dtllie

...... "'_'_""""'''''''10''-''

Chapter 5

Differential and Integral Calculus in Banach Spaces

5.1 Intrnduction

It is needless to emphasize that the classical calculus is the backbone of

science and technology and, in fact, a systematic study of vast areas of human knowledge cannot be conceived without notions of finite-dimensional calculus. However, in mally emerging areas of science anei technology, the inevitability of the knowledge of calculus in spaces of functions is yet to be fully recogni:t..ed_ In this chapter, \liP introduce the concepts of differentiation and integration in function spaces and discuss basic results which have been successfully applied in diverse fields. More precisely, we discuss Gateau aud Frechet derivatives and their analogues of classical results such as mean "alue theorems, chain rule, implicit function theorem, Taylor's formulae, fundamental results of distribution theory and Sobolev spaces. Spaces of \'ector valued fUllclions are also presented. Ceneralized gradients (subdifferentiRIs) are introdur:ro, Applications of these resnlts me elnhorated in subsequent chapters,

5.2
5.2.1

The Gateaux and Frechet Derivatives

TIle Gateaux Derivative

Throughout this section, X and Y denote Banach spaces oyer R, and T denotes an operator on X into Y (T ' X --t Y),

Definition 5.2.1 (GiitOOltX DeJivatiIJe).

Let x and t be given ele-

...... "'_'_""""'''''''10''-''

ments of X and
. ]1m
'1--->0

T(x

+ l}t) 7]

T(x)

D1')1 =, 0 lx

for every t E X, where 1] ---t 0 in H. DT(.-r:)t E V is called the value uf the Gateaux dcn:vativc of T at x in the direction t, and T is said to be Giiteau.?: differentiable at x in the direction t. Thus, the Gateaux derivative of an operator T is itself an operator often denoted by DT(x),

Remm'k 5.2.1

(a) If T is a linear operator, then

that is, DT(x) = T for all x E X.

DT(x)t = T(t),

(b) If l' = F is a real-valued functional on X; that is, T: X ---t R, and F is Gateaux differentiable at some x EX. then
DT(x)1

~ [~ F(x + "t)] .,=() , (7)

(5.2.2)

and, for each fixed x EX, DF(x)t is a !mear functIOnal of t EX. (c) It may be noted that the Giiteaux derivative is a generalization of the idea of the directional derivative ",-ell known in finite dimensions. Theorem 5.2.1 vided it CXlStS.
The Gateaux derivative of an oPl-"1tltor l' is unique pro-

P7'fJOf. Let two operators Tt(t) and T 2 (t) satisfy (5.2.1). Then, for every t E X and every 7} > 0, we havE'
IIT,(t) _ 1',(1)11 =

II (T(X HI; -T(x)

1',(11)
-1;(<)) II
T(x

_C(X + ":: <

7}

T(x)

1'(x + 7]1) - 1'1 (t)

+ l}t)
7]

- 1'2 (t)

---t 0

as 7} --+ U.
Therefore. 111'1 (t) - T 2 (t)J1

= 0 for all t E X which implies that 1'1 (t) = T 2 (t).

...... ,">0<1"_""""""""10"-'

Definition 5.2.2 (Gradient oj a Ji\mctional). Let F be a functional on X. The mapping x --t DF(x) is called the .Q1lUlient of F and is usually denoted by yo F
It may be observed that the
~".adient

yo is a

mhppin~

from X into the

dual space X' of X.

Example 5.2.1 e'1 = (0,1, fl,

_ .__ ..),

Let X = R n , Y = fl, ,en = (0,0, .. ,0,1). Then

~

t::l

(1,0,0, ... ),

DF(x)e,

-8 '

of
x,

i=1,2, ...

,n,

where F R" --t R and aOF is thp partial derivative of F.

x.

Example 5.2.2

Let F: R2 --t R be defined by

Example 5.2.3
F(x) =

ut F: n2 --t R be defmed m;
XIX2

Q.

..... ,

x, +x'1' 2
2

(Xl'

X'1) -=F 0 and F(O) = 0

, to<l',_"""",,,,,"l1>"-I

Then

where t = (t l ,t2)' DF(OH exists if and only if

It is clear from this example that the exIstence of the partial derivatives is not a sufficient condition for the existence of the Gateaux derivative. Let X = ROl, F: ROl -4 R, x = (XI, . ,x,,) E fl" and t = (t t2, ... ,t,,) E R". If F has continuous partial derivatives of order 1, " then the Gateaux derivative of F is
Example 5.2.4

DF(x)' ~

k=ol

" L: 8F(x) t,. iJ x,

(5.2.3)

For a fixed a EX, the Gateaux derivative at a,

DF(a)'

[t
k=l

8F(x) ,,]

ax,

(5.2.4)
:I'=a

is a bounded linear operator on fl" into R" (we know that (R")' = DF(a)t can also be written as the inner product

R").

DF(a)' = (y, t),

wheIT'
Y

(5.2.5)

~ (8F(a) 8F(a) 8F(a)) " ' B " " ' uX" ' !! UXI X2

Example 5.2.5 Let X = R", Y = R"', and F = (FI , F2, F3 , , F,.,,) : R!' --t fl'" be Gateaux differentiable at some x E flOl. The Gateaux derivative can be identified with an m x n matrix (atj). If t is the j-th coordinate vector, t = j = (0,0, ... ,1,0, .. ,0), then

lim

F(x + 1)t) - F(x) _ DF(x)t = 0,

1J

Q.

..... ,

'}-'O

, to<l',_"""",,,,,"l1>"-I

implies

F;(r. -t-1/e,) - F,(x) I1m

'1-->0

TJ

_ 0 -a,,-,

for every i :::: 1,2,3, . . ,m and J :::: 1,2,3, ... ,no This shows t.hat F,'s aE(x) have partial derivatives at x and 0 ' == aij, for every i :::: 1, 2, 3, ... , m and j:::: 1,2,3, ... ,no Th(' Gateaux derivative of F at x has thl" matrix representation

x,

of, (x)
ax, oFm(x) ax,

of, (x)
8x"
:::: (a,j).
8Fm (x) ax.

(5.2.6)

This is called the Jacobian matf'ix of F at x. It is clear that if m :::: 1, then the matrix reduces to a row vector which is discussed in Example 5.2.4.
Example 5.2.6 Let X:::: C[a,b],Jtl, 11) he a wntinuous real function on [a,b] x [a,b] and g(v,x) be a continuous real function on [a,b] x R with

continuous partial derivative

~~ on [a. b] x R.

Suppose that F is an operator

defined on CIa. bl into itself by

F(x)(s) "

f{(s, v)g(v, x(v))>lv.

(5.2.7)

Then
DF(x)h =

l
+ t)

f{(s,v) [:x9(v,X(V))] h(v)>lv.

Theorem 5.2.2 (Mean Value Theorem). Suppose tllat the/unehonal F has a Giiteaux tle,'ivative DF(x)t at every point x EX. Then /0'- any points x, x + t EX, there exists a ~ E (0,1) such that
F(x

- F(x)

DF(x

+ 'I)t.

(5.2.8)

",,,. "'_'_""""'''''''10''-''

Proof.

Put

~(o)

= F(x

+ ot).

Then

""(0) -J~ [",(o+~t"'(O)]

= lim F(x
tl..... o

+ ot + pt) {3

F(x

+ ot)
~,

= DF(x

+ at)t.

By the Mean Value Theorem for real function of one variable hpplied to we get
~(ll -<;{Ul

= (,)'(0 for somef. E (U,]).

This implies that

I
5.2.2 The FhklJet Derivative
Defimiton 5.2.3 (Pre-chet De7-ivative). Let x be a Iixed point in a Banach space X and Y be another Banach space. A continuous linear operator S : X --t Y is called the Frcchet derivativc of the operator T : X --t Y at x if
T(x

+ t) -

T(x)

S(t!

+ ",(x. t!.
- 0. -

(5.2.9)

(5.2.10)

or, equivalently

Iltll~o

,.

IIT(x + t) - T(x) - 5(1)11

Utll

(52

= .

. .11

The Fh~het derivative of T at x is usually denoted by dT(x) or T'(x). T is called Frcchet diffc,-entiablc on its domain if T'(x) exists at every point of the domain.

Remark 5.2.2
of real function

(a) Lf X = R, }' = R, then the c1<lssical derivative I'(x) : R --t R at x defined by

I'(x) = lim I(x

f40

+ t) t

I(x)

(5.2.12)

...". "'_'_""""'''''''10''-''

is a number representing the slope of the graph of the function f at x. The Fnkhet derivative of f is not a number, but a Iillear operator on R into R. The existl"nce of the classical derivative j'(x) implies the existence of the Frechet deriwttive at x, and by comparison of Equations (5.2.9) and (5.2.]2) writteu in the form
I(x

+ t) -

I(x) ~ !,(x)t

+ t.Q(t),

(5.2.[3)

we find that S is the operator which multiplies evcry t by the numbcr j'(x). (b) [n elementary calculus, the derimtive at a point x is a local linear approximation of the giVl"Jl function in the neighborhood of x. Similarly, the Frechet derivative can be interpreted as the best local linear approximation. 'We consider the change in T when its argument changes from x to x + t, and then approximate this change by a linear operator S so that T(x -'- t)
~

T(x)

+ 5(t) + E,

(5.2.14)

where E is the error in the linear approximation. Thus, E haR the same order of magnitude as t except when S is equal to the Fnkhet derivative of T. E = O(t), so that E is much smruler than t as t --+ O. [n this way, the Frechet derivative gives thp best linear approximation of T near x. (c) It is clear from the definition (Equation (5.2.1I)) that if T is linear, thl"n ar(x)
~

T(x),

that is, if T is a linear operator, then the F'rernet derivative (linear approximation) of T is T itself.

Theorem 5.2.3 if an opemt01" has the Frechet tlerivative at a point, then it has the Gateaux derivative at tilat point and both tle,"il!utives /talJe equal
vahu,.~.

Proof

Let T; X --+ Y. and let T have the Frechet derivative at x. then

Hm

liT (x + t) - T(x) - 5(t)1I ~ 0

11'1140

IItll

for some bounded linear operator S ; X --+ Y. In particular for any fixed nonzero t EX, we have
Hm
'1-.0

T(x + ryt) - T(x) _ Sit)

1/

_ I' IIT(x + 'It) - T(x) ,m II 1Jt II f)--?O

5('1')11 IIII ' .

",,,. "'_'_""""'''''''10''--'

This implies that S is t.he Gateaux derh<ative of T at x.

In view of Theorem 5.2.1, Gateaux derivative is unique and hence the Fnkhet deri'<ative is also unique. Example 5.2.7 shows that the converse of Thl.-"Orem 5.2.3 is not true, in general.

Theorem 5.2.4 Let n bt: an open .mhset of X and T : n --t Y have the Frechet derivative at an arbitmry point a of n. '.l1,en T is continuotls (it n. This means that evC1Y Frechet diffenmtiable operator defined on an open subset of a Banach space is continuous.

ProofThen

For a E 0, let E > 0 be such that a + tEn whenever

IIT(a + ,) - T(a) II ~

IItli < f.

IIS(t) + <pta, ')11

-> 0 '"

11'11 -> o.

This me-ans that T is contlllUOUS at a.

Most of the results of classical calculus can be easily generalized to FrecheL derivatives. For example, the usual rules uf sum and product in case of functions of two or more variables apply t.o Fnkhet deri'<atives. \Ve now present extensions of the Chain Theorem, tile Mean Value Theorem, the hnplicit Function Theorem and Taylor's Formula to Frechet differentiable operators.

Theorem 5.2.5 (Chain Rule). Let X, Y, Z be ,wi Banach spaces. If S. X -~}' and T: Y --t Z a,'C Frechet diffe,'Cntiable at x and y = S(x) E Y, respectivelu. then U = T 0 S is Frechet differentiable at x and
V'lx) ~ T'(S(x)S'lx).

Proof.

For x, t EX, we have

VlX

+ ,) -

V(x) ~ T(S(x
~ T(S(x
~

+ t

- T(S(x))
- Tty) T(y),

+ t)- S(x) + S(x))

T(,

+ yl -

where z = S(x

+ t)

- S(x). Thus,

Q.

.....

IIV(x + ,) - V(x) - T'(Y)'II ~ o(lIzll)

, to<l',_"""",,,,,"l1>"-I

Since liz -

5'(x)tll ~ o(lItll), we get IIU(x + Ii - U(x) - T'(y)5'(x)tll

+ t) - U(x) - T'(y)z + T'(y)z - T'(y)5'(x)tll ~ o(lItlll + o(lIzlll
~ IIU(x

In view of the fact that S is continuous at x, by Theorem 5.2.4, we obtain

IIzll ~ o(lItlll

and w

U'(x)t = T'(5(x))5'(x)t.

I
We requin> the following notation in the Mean Value Theorem and Taylor's formula: If a and b are two points of a vector space, the notation

ja,b) ~ {x~ na+ (l-a)bE X/a E [0, I])

)a,bj~

{x=aa+ (I-a)bE X/a E (0, III

are used to denote. respectively, the closed and open segments with endpoints a and b. Theorem 5.2.6 (Mean Value Theorem). Let T: J( --+ Y, when; J{ is an open convex set containing a anti h, Y is a normal space anti T/(x)

exists for well x E

la,bl -

aml T(x) is

conti7l1WUS

on -[a,b]. Then all. (5.2.15)

IIT(b) - T(a)1I

<

,up
l'E]a,b[

IIT'(y)lIl1b

P1'Oof.

Let FEY' and the real function 11, be defined by

h(a)=F((T(I-n)a+n(b)))

[maE [0.11.

Applying the Mean Value Theorem of the classical calculus to h, we have, for some a E [0, IJ and x = (1 - ala + ob,

F(T(b) - Tla)) ~ F(T(b)) - F(T(a))

~

h(l) - h(O) ','(a) ~ F(T'(x)(b - a)),

where we have used the Chain RuIn and t1lf' fact that a boundecl linear functional is its own derivative. Therefore, for eacb F E F*,

IIF(T(b) - T(a))11

<: IIFII IIT'(x)1I

lib -

all

(5.2.16)

Q.

.... ,

, to<l',_"""",,,,,"l1>"-I

Now, if we define the function Po on the subspace [T(b) - T(a)] of Y as

FO(A(F(b)) - F(a) ~ A,

then UFoll = [[T(b) - T(a)I[-I. If F is a Hahn-Banach extension of F to o the entire space Y (Theorem 4.2.2), we find by substituting in (5.2.16) that

~ IIF(T(b) - T(a))11

<:

IIT(b) - T(a)II-'

IIT'(x)lIl1b - all,

which gives (5.2.15).

Theorem 5.2.7 (Implicit Fuuction Theorem). Suppose that X, },., Z aTe Banach 8paces, C is an open subset of X x Y and T ; C --+ Z is continuous. Suppo.~P further that for some (XI, YI) E C, (i) T(xJ> yd = 0, (ii) The Frichet derivative ofTh') when x isflXed is denoted by Ty(x,y), called the pmtial Frichet derivative with respect to y, exists at each 1Joint in a neighbof'lwod of(xl,Yd and is continuous at (x,y), (i;;) [T,(x"y,)t' E !lIZ, Yj. Then then! is an open subset D of X contammg Xl and a umque continuous mapping y : D --+ Y S1J,ch that T(x,y(x)) = 0 and Y(XI) =:: Yt .

Proof. For the sake of convenif>.nce, we may take XI =:: 0, Yl =:: 0. Let A = [Ty(O,O)]-1 E 6[Z, YI, Since C is an open set containing (0,0), we find that

oE C, = {y E Y/(x,y)
for all x sufficiently small, say we define a function

E CJ

Ilxll < Ii.

For each x [laving this property,

by

S(x,y) = y - AT(x,y).
In order to prove the lhrorf>JIl, we must prove (i) the eXIstence of a fixed point for S(x,) under the condition that Jlxll is sufficiently small, and (ii) continuity of the mapping x --+ y(x) and y(xd =:: Yl. Now

SlI(x,yHu) = u - ATy(x,y)(u),

...... ,"_'_""""'''''''l1>''-''

and AA- 1 = .4Tl/(0, 0); therefore, assumptions on T guarantee the existeJ1Ce of 5 y (x,y) for sufficiently smallllxll and lIull, and
S,(x,y)(") ~ AIT,(O, O) - T,(x,y)](u).

Hence,
IIS,(X,Y)II

< IIAII

IITv(O, 0) -Tv (x,Y)II

Since Tv is continuous at (0,0). there exists a constant L

IIS,(x,Y)1I

> 0 such that

(5.2.17) Since T (5.2.18)

< L,

for sufficiently slUHlIllx!l and Ilyll, say, Ilxll < f'J ~ 8 and is continuous at (0,0), there exists an t" < f] such that
IIS(x,O)1I ~ IIAT(x,O)1I < ,,(1- L)

lIyl! ~ fZ.

for all x with IIxli f. We now show that 5(x,') maps the closed ball 8,(0) = {y E }'"j lIyll < fd into itself. For this, let IIxll t" and lIyll t"z. Then by (5.2.15), (5.2.17) and (5.2.18), we have

s:

s:

s:

IIS(xy)1I S IIS(x.y) - S(x,O)1I

O~a<1

+ IIS(x.OIIi

S sup IIS;(x,Y)IIIIYII + IIS(x,O)1I

< Lfz+fz(I-LJ=f'l.
Therefore, for Ilxli < f, 5(x,): S'2(0) --t 5'2(0). Also, fOl" YI,Y2 E 5'2(0), we obtain by (5.2.15) and (5.2.17)
IIS(x,y,) - S(x,Y,)1I

S sup IIS,(x,Y)III1Y' - Y,II

111111.:0;'2

< Lily, - y,lI

The Banach Contraction Mapping Theorem (Theorem 1.2.1) guarantees that for each x \vith IIxl! < f, there exists a unique y(x) E 8'2 (0) slIch that
y(x) ~ S(x,y(x)) ~ y(x) - AT(x,y(x)),

that is, T(x, y(x) = U. By uni(juencss of y, we have y(O) = Usince T(U, U) = O. Finally, we show that x -t y(x) is continuous. For if IIxdl < t" and ,,1:211 < t", then selecting Yo = y(X'l) and y] = 8(x"yo), we have by the error buund fOl" fixed point iteration UII the mapping 5(x],') (Theorem 1.2.1, Problem 1.4.17)
I IIY(x,) - y(x,)11 S I _ L lIyo

y, II

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

'Ve can write

Yo - y, = y(x,) - S(x" y(x,)) = S(x" y(x,)) - S(x" y(x,))

= -A[T(X"y(x,)) -T(X"y(x,))].

Therefore, by the continuity of T, Ily(xz) - y(xdll small for IIx2 - xIII sufficiently small. I

CIlJ1

be made arbitrarily

Corollary 5.2.1 Jf, in addition to conditions of Theo,y;m 5.2.7, T,,(x,y) alsu CXfsts on the O]Jen set, and ts contmuuus at (XI, YI), then F ; X --+ y(r) has a Frechet tle''ttJative at Xl given by
(5.2.19)

Proof.

We set x =

XI

+ 11 and G(h)

= F(x)

-1/1.

Then G(O) = 0, and

IIG(h) + IT,(x"y,Jr' loll < II IT,( x" y,) I 'II II T ,(x, , y,)G(h) + T, (x" y, )1,11,
and
Ty(Xj, ydG(h) - T,,(Xl, ydh = - T(XI + h, YI + G(h)) + T(x],

yd

+T1I (Xl, yJ}G(h)

+ TAxI, yJ}h.

If 01,0 2 are numbers in (0, 1), then

IIT1I(XI ,ydG(h)

+ T,,(Xl, ydhll <

+ sup
9, ,92

sup liT", (XI

9, ,92

0 1 11., Yl

+ 02G(h))

- T, (x, ,y')l1 111,11

liT, (x,

+ 0,1" y, + O,G(h))

-T,(x"y,)III1G(hlll
Thus, applying continuity of T"" T1I for that on fix - xIII < 0, we have

> 0, we find that

0 = 0() such

IIG(h) + IT,(x" y, Jr' 7~(x" y,)hll

< IT,(x"y,Jr'<ll + IIIT,(x"y,Jr'T,(x"y')lIIhll

I

IIIT,(x" y,)] 'II

The coefiicient of IIhll can be made as small as needed as 1111,11 --+ 0. Thus,

Q.

......

IIF(x) - IF( x,) - IT, (x, ,y, Jr' T, (x, ,y, )(x - x,)11I ~ o(lIx - xdll
, to<l'._..

....,..",,"l1>"-I

I
Definition 5.2.4 If T: X Y is Fnkhet differentiable on an open set n C ~Y and th first Freche derivative T' a x E n is echet differentiable at X, then th Fr'ch t derivativ of T ' at x is called the second aerivative of T at 3; and is denoted hy Tit (x).
It Inay be observed that if T . ~,r -4 Y is Frechct differ n iable on an op n s t n eX, th n T' is a Inapping on )( into B[X, Y]. Consequently, if Til (x) exists, it is a bounded linear Inapping from X into B[X, Y]. If Tn exists at every point of [2 I then Til : X -+ B [X, B[X, Y]) .
TheoreIU 5.2.8 (Taylor's ForIUula for Differen iable 'nne ions). Let T ; n c X -t Y and let [a, a + h] be any closed segment in n. If T i.s Prechet differentiable at a. then
T{a

+ h) =

T(a)

+ T'(a)h

lIhlk(h), lim f(h)

h-i'O

= O.

Theorem 5.2.9 (Taylor '8 Fonnula for w' ce Freche 'iffereutia Ie Functions). Let T : 0 C )( Y and let [a, a+ h] be any closed segment lying in O. 1fT is differentiable in 0 and twice differentiable at a, then

T(a + h) = T(a) + T'(a)h + -(T"(a)h)h + IJhJj2 (h).

2
h-i'O

lim (h) = O.
Wf'

For pl'OO(<;, of the5P two theorems and other related results, Cal an [2J, Dieudonne [7], ashed [111.
Example 5.2. 7

rf'fer to

Let T : R 2 -+ R be defined by
if x ~ 0 and y ~ 0 if x

= y = o.

It can be vedfied hat T has the Git, eau d rivativ at (0,0) and its valu is O. However) i is not Frechet differentiable at (0, 0) as

Example 5.2.8 Suppose T . fl --t R is a functional on a T-filbert space If. Lf Tis Frechet differentiable at some x E fl, then its F'rechet derivative T' must be a bounded linear functional on If, that is, T' E fl'. By the Riesz Representation Theorem (Theorem 3.7.1), there exists an element y E 11 such that T' (x)( z) = (z, y) for every z E 11. The Frechet derivative T'(x) can thus be identified as y which is called the gradient of T with x denoted by V'T(x), that, is, T'(x)v = (V'T(x),v) \;J v E 11. Example 5.2.9 Let T : fl -~ fl be a rounded linear operator on a Hilbert space If into itself and F : If --t R be a functionnl defined by F(x) = (x, Tx). Then the Frechet derivative of F is given by

F'(x),
where T' is adjoint of T.

(x, (T + T')x),

Example 5.2.If) Let T be a self-adjoint bounded operator on a Hilbert space IJ and (() the quadratic functional defined by
",(x) ~ 2(Tx,x).
1

Then ~ is Frechet differentiable and gradient of r.p, \lr.p = T. Furthermore, r.p is convex if T is strictly positive.

Example 5.2.11

Let T: ~ --t R be defined by

T(x) =
Then

X\X2

+ x?

Example 5.2./2

Let T: C[Q, 1] --t C[Q, 11 be defined by

T(x(t)) ~ x'(t).
Then

T'(x)z = 2xz or dT(x) = T'(x) = 2x and T"(x) = 2J.

B.

......

, to<l',_"""",,,,,"l1>"-I

Example 5.2.13

Let F. 2(0,1) --+ R be defined by

F(J)

l'

!'(tjdt.

Then
,11"(1) ~ 1'" (I) ~ 2 f d'P(I) ~ 1""(1) ~ 2[.

Example 5.2.14 defined by

Let the Hammerstein operator T ; CIa, b] --+ C[a, b] be

(Tu(t)(x)

~ [ ' [((x, t)f(t, "(tdt,

J ; [a, hI x R --+ R are ~ven functIOns.

where f( (".) ; la, hI x la, h] --+ Rand Lf f is infinitely differentiable, then

d'T(u(t))(,) ~ " {((x,l) of (t, u{t))z{l)dt.

" f

au

In other words. t.he Fnkhe1. derivative of the Hammerstein operat.or is anot.her integral operator with the kernel

J((x,t)~~ (t,u(t)).

Example 5.2.15

Consider the map T: R --+ X, defined by

T(t) =
CAl;!;

where x EX, J-l E 6(X) and c A is defined as in Example 2.6.2 Then

dT(t) = AeAtx = AT(t).

5.3

Generalized Gradient (SubdiHerential)

Definition 5.3.1 (Lipschitz ContmU'tty). Let n eX, T an operator from n into}'. We say that. T is Lipschitz (with modulus 0: >0) on n, if

T b called Llf'~chitz ncar x (with lIIudl11u~ a) if, for ~ome r; > 0, T i:; Lipschitz with modulus 0: on S,(x). Lf T is Lipschitz near x E n, we say that T is loc.aily Lipschitz on n. 0: is called the Lipschitz exponent.

.n". "'_'_""""'''''''10''-''

Definition 5.3.2 (Monotone Opemtors). is called monotone if (Ttt-Tv,1J -v)

Let T: X --t X', then T

> 0 for all u,v E X

Note: (".) denotes the dualit,y betwl--'CII X and X"; that is, ah:io the value of Tu - Tv at u - v. in Hilbert space seWng, (".) becomes the inner product. In this chapter, we have used the notation (".) also for the duality. T is called strictly monotone if

(Tu -TV.1J - v}

> 0 for a1l1J,v E X,

> 0 such that

T is called strongly monotone if there b a constant k (Tu - Tv, u - v) ~ kll1J -

vW for all 1J, v E X

Definition 5.3.3 Let T : Ii --t 21f' be a multivalued operator on H into 11'. fhe operator T is said to ue monotone if
(~-7J,u-vOfora1l1J.vE11andfora1I~ET(u)

and 7/ET(v).

A monotone operator T is called maximal monotone if it is monotone and there does not exist T : Ii --t 21f' such that T is mOllotOllf' and Cr(T) ~ Gr(T); that is, the graph of T does not have any proper extension which is the graph of a monotone operator.
Definition 5.3.4 (Genemlized Gmdient). The generalizal gratlient (subdifferential) of F: X --t R at x E X, denoted by 8F(x), is t.he set
aF'(x) ~ {F" E X'/}'(h)

< F(x + h) -

F(x).

hE

Xl.

An element F ' of 8F(x) is called a subgradient or support functional at x. Theorem 5.3.1 [15J Let F: X --t R be a convexfunctionai on a nonned space X, and F have a Gateaux derivative at x E X I then F(x) hw> a unique generu/ized gmtlient at x anti DF(x) =:: DF(x). C07l1!ersdy, if F(x) < 00 anti the genemltzed gmdient 8F(x) retll/ces to a umque subgratlient, then F(x) has a Gateaux derivative at x anti 8F(x) =:: DF(x). Theorem 5.3.2 1/ F: ll-t Ru {oo}, where H is a 11ilbert space, then the genemlizal gradient 8F(x) is a monotone operutor.

...... "'_'_""""'''''''10''-''

Proof.

If 8F(x) or 8F(y) is empl,), then clearly

(oF(,,) - of(y),x - y)" 0

is :;atisfied. [f this is not the case, choose. Ft E 8F(x) and F2 E of(y). ThcII
(F.,x - y) > F(x) - F(y) for all y E H} (F"y - x) > F(y) - F("j

(5.3.1)

By changing sign in (5.3.1), we get

(-F" x - y)

> F(y)

- Fix).

(5.3.2)

By adding (5.3.1) and (5,3.2), we get

(FI

F2 ,x-y):::-:

o.

Hence, of(x) is a monotone opcrator

Example 5.3.1
f(x)

Let
~

Ixl,J . R -+ R

of (x)
8f(x)

= {sgux} if x
~

1= 0,
~

x sgnx=!3i'

[-1, I) if x

O.

For more details of generalized gradient, we refer to Outrata, I{0ltvara and Zowe [12] and Rockafellar and Wets [13].

5.4

Some Basic Results from Distribution Theory and Sobolev Spaces

0 . ,>0<1"_""""'''''''10''-1

.. , ...

Disl,riLmtion theory was developed by the French rVlathematician Laurent Schwartz around 1950 to resolve the discrepancy created by the Dirac delta function whose value is zero at all point:; except one point while its Lebftigue integral is 1. This function was introduced by thl" famous physicist P.M. Dirac around 1930 which contradicted the celebrated Lebesgue t.heory of integral. A class of Lebesgue integrable functions was introduced by the Russian scientist S,L. Sobolev around 1936 which has been found very useful in many areas of current interest, and now known as Sobolev space. The results related to this space have provided a sound foundation of modern theory of ordinary and partial differential equations analytical as well as numerical methods Some important results of these two topics are men~ tioned in this section.

SA.l

Distributions

Let 11 be a positive integer. A vector of n-tuples 0 = (01)0'2,.,. ,0',,), where aI, i = 1,2, ... ,n, are non-negative integers, is called multi-index of dimension n. The number 101 = of the multi-index. For given o. fJ, 0

L 0:" is caJled the magnitude or length

1= I

+ fJ

means

(0\

+ fJ 1 , 02 + fJ2 , On + fJ,,)

o! = 0:\102!0,,!

C$ Xl>

(~) - (3!(00~{3)!

= X~' X~2 ... x~" where x = (x\. X2, . . x n ) E flu.

We say that multi-index 0, (3 are related by 0 < f3 if o. :::; f3. for all i = 1,2,3, ... ,no We are familiar with the concept of the classical partial derivative for a function of n variables f = f(x), x = (Xl, X2, . .. , XII)' Laurent Schwartz introduced the concept of multi-index and a new notation of derivatives given 01 (>1 below. In the new terminology, Do will denote the expression .<Wl ",0 ..
UXI Ux"

DO f will be called derivative of f of order 101. For n = 1,

01

= 1, DO f =

ux,

~l f

which is the classical derivative of a function of single variable denoted

by -/ (XI = x). For n = 2,0:1 '" I, 0:2 ~ 1, 1 I = DO f This is nothl x XI 0 x 2 ing but the partial derivative of the function of two variables f(Xl' X2) which is denoted by
OX10X2
2

a~
=

~j . \Ve have also

OXIOX2

D(l,l}f

OX20Xl

~j

= OX2

(~) ox,

which is equal to

= ,:>0 UXI

(~j)
UX2

for

E C=(O). In distributional

derivatives to Le defined later, v"e shall !lot distinguish between

o'j
O:l:10X2

..... ,
B,.,to<l',_"""",,,,,"l1>"-I

we have

All functions are defined on a bounded subset 0 of R" into R. The boundary of 0 is denoted by r or ao and 0 = 0 + r. We say that 1 E 2 if 1/1 2 is Lebcsgup integrablp on O. As we have seen in Chapter 3, L 2 (!1) is a '/2 Hilbert space with respect to 11/11 = (~1/12dx ) and (1,g) = Igdx (Where k1dx stands for

Throughout our discussion n is a Lounded subset of R" with Lipschitz boundary r. A technical definition is slightly complicated; however, broadly it means that the boundary will not contain cuspidal points and edges. for example, in two and three dimensions, domains having the boundary satisfying Lipschitz condition are circles, squares, triangles, spheres, cubes, annuli, etc. In one dimension. 0 = (a, b). Broadly speaking, the function representing such a bouIII..lary will be smooth or piecewbe smooth and will have no singularities. A function 1 defined on 0 Into R is said to satisfy the HOlder condiOon with exponent "\, 0 < ,,\ ::::: 1, if there exists a constant M > 0 such that
If (x) - f(y)1

/ll

I(X1.X2 .... ,x,,)dxJ, .... dx.,).

< Mllx - ylI'

fo' nIl x.y E ll.

where 1111 is the Euclidean norm on R". K = Sllpp 1 = {x E OJ I(x) i- O} is called t11f' support 01 1 If K is compact, 1 is said to have compact support. It can be easily seen that Co(O), the space of functions with compact support and having continuous derivatives of all order, is a vector space with respect to usual operation:,. A sequence {1>,,} in Co is said to converge to an element 1> in Co(O); namely, 1>" -4 1>, if 1. there is a fixed compact set i{ c 0 such that supp 2.

1>"

C K for all n.

1>"

and all its derivat.ives converge uniformly to (x) and its deriva,. tives, that is, DO" ----+ DO for all 0: uniformly

.n", "'_'_"_"""10"-"

Definition 5.4.1 (Test Functions). CO'(u) quipped with the topology induced through the convergence is called the space of test functions and often denoted by 'O(!1). In other words, a test function is an infinitely differentiable funct,ion on R" identically zero outside of some compact set. Definition 5.4.2 A bounded linear functional defined on 'O(!1) is called a (listribution ur Schwartz distribution.
It may be noted that the space of distributions is nothing but the dual space of 'O(!1) and is often denoted uy 'O*(!1) or 'O'(!1). If!1 = n", we simply write '0*. Thus. a functional F defined on 'O(m is a distribution if it :.atisfies: (i) P(", + ",) = P(",) + Pi,,) (ii) F(a",) = al"(",) (iii) If lP.. --t lP, then F(lP... ) --t F(lP).

Definihon 5.4.3 (Distributional Derivative). Distributional derivative of distribution or generalized derivative is a continuOI1~ linear functional denoted by DO F defined as follows: For F E '0(0), (DO F,lP) = (_I)'O'(P,DO",) fo, nil '" E D(fl). Definition 5.4.4
A function

I:

--t

R is called lornlly integrable if for is Lebesgue integrable

every compact set [( C!1, (

over any compact set J(" It mar be observed that all Lebesgue integrable functions and consequently 11.1.1 c.ontinuous functions are locally integrablf' over la, bJ. If 0 = Sr is equal toa ball of radius T

lK III dx < 00; that is, I

> Oandcenter (0,0) in R 2 , then I(r)

Ixl

,.

-,1'

is locally integrable on Sr. Locally integrable funct,ions I can be identi-

fied with distributions F, defined as follows: (FI' <p) = then (Ff,lP) =

~ IlPdx.

If 10 = 1,

l(x)lP(x)dx. F,

=I

or simply F

= f.

Example 5.4.1
exp [ { 0,

(a) Let
-I ] 1 <x<3

(x)~

(x-l)(x-3) ,

outside the open interval (1,3),

.n",

'"'_'_""""'''''''10''-''

(x) is a test function with support [1,3]. (b) Let (x) = exp(-x 2 ), x> 0
= 0,
J:

<0

(x)

is a test function.

Example 5.4.2 F(</J) = 0, F is linear and wnl,inutJus on V(fl), fl = (a, b) and so F is a distribution.

Example 5.4.3
UOII.

Let F(</J) =

i"

j(x)(x)dx, j is a locally integrable

function F is linear and continuous on V(fl) and therefore it is a distribu-

Example 5.4.4

Let F() =

1"

1(x)1 2 dx, F is continuous on V(fl), but

not linear; therefore F is not a distribution.

Example 5.4.5

The Dirac delta distribution is defined as

(6, ) ~ (O) fo, all E V(ll).

ois linear and continuous on V(fl) and hence a distribution.

Two distributions F and G are equal if (F,) = (G,</J) for all </J E V(fl) such that supp() C (a, b). The Heaviside function 11 is defined by

O,

X<O

Il(x)=

~,
1,

x=O
x

> O.

Let Ii I be defined by

x<o x> O.
II and II] genera!.!'! t.he same distribution and such functions are identified in distribution theory.
, to<l',_"""",,,,,"l1>"-I

B.

......

Definition 5.4.5 (Regulm' and Singular Distribution). A distriLmtion F E V' is called a regular distribution if there exists a locally integraLle function f such that (F,({.') =

In f({.'dx,
,

(5.1.1)

for every lI' E V. Utherwise P is called a singular disttibutwn. In case n=--l,

(F,,,)

f(x),,(x)dx.

(5.1.2)

Rema1'k 5.4.1 (a) It can be verified as follows that (5.4.1) and, in pnrt.icular, (5.4.2) defines a distribution. Since any lI' in V(R) has bounded
support contained in [a, b], the integral

of the product of an integrable and a continuous function). Thus, F is a \\~ll-defined functional on V(R). Linearity is clear as
(F,lI'l

(I f(x)({.'(x) exists (being integral

',

+ lI'2)

(F,o,,)
Let <p" -t <p, then

1
b

1 t
b

f(X)(lI'l

-l-

({.'2)(lx

r' = }(I f(X)lI't(x)dx

f(x)",(x)dx

~ (F,,,,) + (p.",)
~ o(F,,,).

f(x)(o"lxdx

I(F,,,,,) - (F,,,)I

I
"

"f(x)(",,(x) - ,,(x))dx

',

::; supl<p,,(x) - ({.'(x)1

I"

f(x)ldx --+ 0 as n --+

00

This means that F is continuous. (b) One can consider (F,lI') as an average value of F and distributions as objects that have average values in the neighborhoods of every point. In general, distribution~ do not have values at points which resemble physical interpretation' when quantity is measured, the result is not the exact value at a single point.

."'" "'_'_""""'''''''10''-''

Example 5.4.6 Let n be all open or just a measurable set in R". The function F defined by

is

fI

disl.rilmtion. It is a regular disl.ribution since

(F,<p)

1R"

xn<P<!X,

where Xn is a characteristic function ofn. [n particular, if n=(O,oo) x .. (0. (0). we obtain a distribution
(l-I,lp) =

11 1
00
00

00
lp(x)dx1dx2 ... dxn

which is also called the Heaviside function.

Example 5.4.7
by

Let a be a multi-index. The functional F on D defined

(F,<p)
~

D"<p(O)

is a distribution. In particular, for n = I, F defined by

(F,Y') = Y"(O) = the value of the first derivative of Y' at '0'

is a distribution. In case n = 2, F defined b.y
D<p (F.<p) ~ -8 (0).

x,.

k = 1. 2.

is a distribution.

Example 5.4.8 Ixl is a locally integrable function which is differentiable in the classical sense at all x except X = O. However, the generalized or distrihutiomtl derivative of Ixl is calculated as follows:

(Ixl',<p) ~ (-I)(lxl,<p')

~.
=

I:
L oo

Ixl<p'(x)dX

Loo

[0

Xlp'(x)dx _

Jo

[00 xlp'(x)dx

_ r"

<p(x)dx +

10

r= <p(x)dx

...... "'_'_"_"""10"-"

integrating by part and using the fad that lj) vanishes at infinity. Let a function sgn (read as signum of x) be defined b.y
-I

sgn(x) = Then

forx<O forx=O forx>O.

<Ixl

,lj)} =

i:
fI _ -

sgn(x}lj)(x)dx

for all cp E D.

This means that I;cl l =

Example 5.4.9

Sb'11(X).

Let us consider the Heaviside function H defined by

{O 1
00

for x :S 0 forx>O

H is locally integrable and can be identified with a distribution. The gCD.eralizpd derivativf' of this is the Dirac distribution

(11',<p)

-1 ~ -1
~

H(x)<p'(x)dx
00

<p'(x)dx

~ <prO).

Hence, lJl(X} = o(x). The generalii'..cd (distrihllt.ional) dp.rivati\'f' of I.hf' Oirar delta distribution is defined by
Example 5.4. 10

The n-th derivative 0(") of 0 is given by

Definition 5.4.6 (Operutions on DlstributioTLs). Let F and G be distributions and 0: amI /3 be scalars, then F + G is defined by
(aj (Addition) (F.,. G,<p) ~ (F,<p)

+ (G,<p) 10< all <p E 'D.

(b) (Scalar multiplication) of is defined by (oF,l;?) = a{F, l;?), for all l;? E D where a is a scalar.

...... "'_'_""""'''''''10''-''

(c)

[f

P is a distribution and h is an infinitely differentiable (smooth) function, then hF is defined by

(hP, lp) = (F, hlp) for aU
lp

in D.

It may be remarked that smoothness of h is essem.ial for defilllng the operation product.

Theorem 5.4.1 Let F a1ul G be distributio1ls and h a smooth jundion. Then their tlistributio1l derivatives satisfy tlle following relation:

(F+CY=F'+G'

(oP), "- aF' (hFY = h'F+hF'.

Proof

F + OJ', ,,)
This implies (F

-(F + G, ,,')

-(F, ,,') - (G, ,,')

(F',,,) + (G',,,) ~ (F' +G'.,,).

+ C)' =
~

F'

+ C'. Similarly, we get (aPY

= aF'.

hFj',,,)

=
~

=
= =

-(hF,,,') -(F, hlp') b.y Definition 5.4.6(c) -(F, (h,,)' - h',,) -(F, (hlp)') + (F, h'lp) by Definition 5.4.3 (hF',jJ) + (h'P,lp) by Definition 5.4.6(c) {hp i + h' F, lp}.
h'F

Thus, (hFY = hP'

+ h'F =

+ hP' .

It ma.y be observed that the distribution generated by the derivative of a function f: R ---t R is the same as the derivative of the distribution fi these two possible ways of interpreting the symbol F ' for a differentiable function f are identical. The advantage of distribution theory over ordinary calculus is that e\'ery distribution has a derivati\'e.
Theorem 5.4.2 IfF is any distribution, then DaF (Definition rq.3) is a llist,ib1ftioll fm' any multi-index a. Tn particular, F ' llefined by {F' , lP} = - (F, lp') is a llistribution.

..... , 0,0,>0<1"_""""'''''''10''-1

P1'OOf.

(i) (O"P,),<p) ~ (_I),ol(F, O"(),<p

~ ),(_I)'OI(F, O<p),

oc (DO F. ),<p) (ii) (OO(F+G),<p) ~ (_I)I"'(p+G,D<p)

~ (_I)I"'[(F,D"<p) + (G,D<p)]
~ (_I)lol(p,D<p) + (_I)I"I(G,D"<p)
~ ~

),(0 F. <pl.

(OF,<p)

+ (DoG,<p).

Thus, DC> is linear. \Ve now show that DC> is continUOUS. Let V'.. --+ V'; that is, IIV'.. - V'II --+ 00 as n --+ 00, then IIV'~ - V"II --+ a a<; n --+ 00 and, in general,IIDlp" - DUlpll ~ 0 as n -) 00 by a \'tell-known result on uniform convergence I(DOP,<Po) - (DOF,<p)1 ~ 1(_l)I"I(p,Do.po) - (_I)O(P,O"<p)1
~ I( -1)1 0 1 DO 'Po - DO'!')! (}~

S IWIIIID<p" - D<pll ..., 0 as n --oj. 00 by CSB inequality,

and thf' fac!. that IIFII is finitf' and IIDorp" - DV'II--+ 0 as n --+
00.

Convergence of Distributions

Definition 5.4.7 (Cotwergence of Distrtbution). A sequence of distributions {F.. } is called convergent to a distribution F if
(F.. ,V') --+ (F,rp) for every V' E D.

It may be noticed that this definition does not involve the existence of a limiting distribution towards which the sequence tends. It is given strictly in terms of the sequence itself unlike the usual definitions of convergence in elementary analysis, where one is recjuired 1.0 introduce the concept of limit of a seCjuence before the definition of convergence.

B.

......

, to<l'._..

....,..",,"l1>"-I

Theorem 5.4.3 (Distributional Convergence). Let F,Fi>F2 , .. , F" ... be locally mtegmble functums such that Fn --+ F uniformly in each bmmded set, then D Pll --+ D F (in the sense of distf"ibution Definition 5.4. 7).

(DO Fn, <p) ~ (_1)1 0 1 (F", DO <p) _, (-I )Inl (F, DO <p) ~ (DOF,<p)
for every test function lj). This prov(-'S the desired result. For a I, we obtain that
(F~,lj)} -~ (F',f/)}

wheue\'er (F",lj)}

---t

(F,f/)} unifoI"Tnl.y.

I
Example 5.-4.11
Let /,,(x) = JT(1

+ n 2 :c 2 )

We show that the sequence /,,(x) converges in the distributional sense (Definition 5.4.7) to the Dirac delta function o(x). Let lj) be a test function. To pro\'e our assertion, we must show til,lt

Or eqUlvalently,

i:

/ll(x)f/)(x)dx - f/)(O) --+ 0 as n --+

00.

Since

J
we
ha\'(~

OO /,,(x)dx = JOO

-00

( 2 2) = I for alln E N, _ooJTI+nx

n&

I:

In(x)<p(x)& - <p(0)
=

0 . ,>0<1"_""""'''''''10''-1

.... ,

I:
00

I:

In (x)(<p(x) - <p(0&

I" (x)(<p(x) - <p(Odx

I" (x) (<p(x) - <p(0&

In (x)(<p(x) - <p(Odx,

Figure 5.4.1

Graph of h(x), h(x), /3(x) of Example 5.4.11

where [-a,aJ is all interval containing the support of lj). Consequently,

I:

fn(X)'I'(X)dx - '1'(0)1

< '1'(0)

I:

~
+

I:
t

fn(x) ('I'(X) - 'I'(O)dx fn(x) ('I'(X) -'I'(O))dx

fn(x)dx

'1'(0) l~ fn(x)dx .

By direct integration, we obtain that

lim lj)(O)
tl---+<Xl

J-<Xl

r-

a j,,(x)dx

Now we show that

lim

B.

.... ,

'1---+00

(_nnfn(X)('I'(X) -'I'(O)dx ~ O. J_

, to<l',_"""",,,,,"l1>"-I

We have

I:

f.(x)(I'(X) - I'(O)dx

<

I:

If.(x) 1I'(x) - <p(O)ldx

max 1,,'(x)1

Eo

Ixf.(x)ldx.

This holds because IIp(x) - lp(O)1 -:::: maxllp'(xll Ixl. by the mean value theorem It l:an be S(-,ll that

Therefore, we have the desired result.

Example 5.4.12

Let fn(x) be a<; in Example 5.4.11. Then

'( ) 2 n'x ") f "x=--( l+nx ")2--+u(X Tr

in the sense of distribution. Hovlever, f~(x) ---+ con\'ergence (elementary analysis con\'ergence).

a in

the sense of classical

Theorem 5.4.4 (Termwise Different.iation). If F, F1 , F2,' ., F" ... Uf-e distributio1lS such that. F" --+ F as n --+ 00, then F:, --+ Fl. In fact, this ,-esulf is true for any 1nulfi-i1lltex OJ that. is,

Proof

(F~,lp) =

-(Fn,lp') --+ _(F,lpl) = (P,lp') Also,

(DaF",,,) = (-I)'ol(Fo , DO,,) --> (_l}l o l(F.DO,,) = (D"F.,,),

for f'''f'ry test funct.ion

It may be obsen'ed that for tennwise differentiation in the classical case the sequence {Fn } must converge unifonnly and the differentiated sequence also must com'erge uniformly. It may be noted that Theorem 5.1.4 allows us to differentiate selies of distributions term-by-term .

B.

......

, to<l',_"""",,,,,"l1>"-I

The Int.egral of a Distribution

Definition 5.4.8 (Inteb'l'al or Antiderivat.ive of a Distribution). Let F be a distribution. Then a distribution G is called an integral or antiderivative of F if G 1 = F; that is, first distributional derivati\'e of Gis equal to F.
Theorem 5.4.5 (Existence of Integral of a Distribution). distribution F E V' (R) has an integral.

Every

Proof.

Let lPo E V(R) be a fixed test function such that

I:

I'o(x)dx

~ 1.

Then, for every test function lP E D(R), there eXists a test function lP\ E V( R) such that

where K =

i:

lP(x)dx and

i:

lP=Kcpo+lPl, lPd:c)dx = O. Let FE VO(R). Define a

functional G on V(R) by

where Co is a constant alld

,p is the test function

defined by

Then G is a distribution and G'

0.-

F.

Theorem 5.4.6 Let F be a distribution such that it"" dist,'ibutionaillerivatitle vanishes. Then F must be a constant junction. In particular, any two integrals of a distribution differ by a constant junction.

Proof. Vie get

Let lP E D(R). Using the notation of the proof of Theorem 5.4.5, (F,<p)
~

(F,J<<po +<p,) = (F,J<<po) <p(x)dx,

~ (F,I'o)

B.

.... ,

I:

+ (F,<p,)

, toOl',_"""",,,,,"l1>"-I

since (F,lPt) ::: -(F',1jJ) == O. Thus, F is a regular distribution generated b.y the constant function C == (F,rpo). Let C 1 and C z be two integrals of a distribution F. Then C~ == F, C; =. F. This implies that C~ - C; = 0 01' (C t - C 2 )' = O. Hence. by the first part. G 1 - G z is e{jual to a constant function. This proves lhe theorem. I We conclude this section by certain remarks which are <lUlte uscful in the next subsection.
"Remuy'k 5.4.2

(i) If f E C m (11), then all the classical partial derivatives of f to order m are also generalized derivatives (distrihutional derivatives). (ii) Let 1 be locall.v intebrrable such that 1 E Lt(K) for any compact subset K of 11. Then

F(I')

f(xl'l'(x)dx,

'I' E D(O)

defines a distribution on 11 which Vie denot.e by Fl' Two distributions FI and FI} are equal if and only if 1 = 9 a.e. (iii)S(R") =1= {/ E C oo ;'r;/a,{3E Am, xODPI -+ Oaslxl-+ oo} is the space of functions of Coo of rapid decay at infinity. This is not a normed space but a complete metric space with respect to the metric
dU,g) =

o,{3ENn

ao"

+ d oB 1

dfiPU - g) (

9)

1,9ES,
Q

where dap(f) = sup

",ERn

Ixo DB l(x)1 and Q o f3 are chosen such that L

It can be proved that S is dense in L p for all p with I $" p < 00 but S is not dense in Leo. S is also a \'ector space. For F E S. the Fourier transform of F is defined as

0,"

a {3

:::

1.

F(y) = Ff(y) =

r J

,-"'f(x)dx,

y E RH

flo>

The vector space of all continuous linear functionals on S, denoted by $I, is called the space of tempered distributions. Each element of 5' is a distribution and is called a tempered distribution An interesting characterization of tempered distribution call be found in [5, p. 508, see H"(R")J. The elements of L p , I $ p $ 00 can be identified with tempered rlistributions (and in particular S c 5'). Lt may be noted that an arbitrary distribution F will not have a Fourier transform, at least as a distribution. However, we can define a Fourier

.n",

"'_'_""""'''''''10''-''

transfonn denoted by F for each tempered distribution T E S' as (t,cp) :::: (T, rp) for all cp E S. For more information about Fourier transform of tempered distribution, see [5J.

5.<1.2 Souvlev Space

11"'(0) :::: {f E Lz (0)ID"1 E :1(0),10:1 < m}, 111 being allY positive integer, is called the Sobolev space of order m. 11 m (O) is a Hilbert space with respect to the inner product

(I,g)II-,O) ~

L:
lol-:=::",

(DO f, Dag)L"O)

Form = 1,0:::: (a,b),DOj:::: df <Ix

(I,g)",(o,,)

~
~

(I,glL"a,,)

+/

' l
a

1 \(X (

df ' d/xg) ,

f(x)g(x)dx +

l'

Ldn,b)

df d - -dg x " dxdx

It can be easily \'crified that 1-/1 (a, b) is a HilLert space. For m :::: 2, n = Sr = {(x, y)/x z + yZ 5 r} :::: cirde with origin as the center; or n:::: {(:c,y)/a < x < h. c < Y < d} :::: a rf'rt;mgle with sides of lenbrth b - a and d - c
(I,g)""O) ~

L: (D a f, Dag)L"O)'
0 1 19

where

(0,0),(0,1),(1,0),(1,1),(0,2),(2,0)

10:1 :::: 0:1

+ 0z

5 2,

L
1 19 0

(D" j, DOg) ::::

<

D(O,O)

f,

0(0,0)9)
Ldfl)

.n",

"'_'_""""'''''''10''-''

where

= Df! denotes the boundary of f! For [.2(r), see Appendix D.

H"'(r) is a Hilbert space with respect to

The restriction of a function

Q.

.... ,

f E Hm(f!) to the boundary r is the tmce of

, to<l'._..

....,..",,"l1>"-I

and is denoted by f Ii that is, f I = f(r) = mlue of f at f.

Ho"(fl) = tf E H"'(H)/f l = OJ = closure of Dm) in H"'m) H"'(R n ) = Ho"(Rn),m 2': O.

The dual space of Ho"(H), that is, the space of all bounded linear functionals on HOfl(O) is denoted by fI-m(o).

f f

E H-"'(O) if and ollly if

=

L
IQI~'"

D'~g for some 9 E L 2 (fl).

Sobolev Space with a Real Index: HS(R") A distribution is a tempered distribution if and only if it is the derivative of a continuous function with slow growth in the usual sense, that is, a function which is the product of P(x) = (1 + IxI 2)k/2, kEN by a bounded continuous function on Rn. For s E R, HS(Rn) is the space of tempered distributions F, such that

where F is the Fourier transform of F. HS(R n ) eqUipped with the inner product

(F,G),~

JR"
==--

{ (l+lyl')'F(y)G(y)d<

with associated norm

IlFlis

(!Rn (I + lyI2)SIF(y) 12dy) 'I') is a Hilbert

space (for proof see Solved Problem 5.6.7). If 8\ > 82, then ffSl(R n ) C H S 2(R n ) and the injectiun is continuous (Solved Problem 5.6.7). For 8 = mEN, the space HS(R n ) coincides with H"'(R") introduced earlier. It is interesting to note that every distribution with compact support in Rn is in HS(Rn) for a certain 8 E R. For more details, see Dautray and Lions [5]. Theorem 5.4.7 (Green's Formula for Integration by Parts).

(0

in

v6udx +

in

grad u. gmd vdx = t v : df

(ii) k(U6V-V6U)dX=

Q.

......

[(U: -V::)

dr.

, to<l'._..

....,..",,"l1>"-I

If. is clcw' that. (i) is a genemlization of the integration by pa1'tS f01'7lmla

stalL'll below (for n

== l,n =

[a,b])

I""

ul/(x)v(x)dx = u'(b)v(b) - u'(a)v(a)

_I"
"

u'(x)v'(x)dx.

Theorem 5.4.M (The Friedrichs Inequality). Let n be a bounded domain of R" with a Lipschitz boundaly. Then there exists a constant k, > O. depending on the given domain. sudt that [01' every f E HI(O)

1I/11)",n, <5 k,

{t.l (:!,)' dx l I'dr}

+

(5.4.3)

Theorem 5.4.9 (The Poincare Inequality). Let n be a bounded domain of R" with a Lipschitz boundmy. Then fheJ'C exists a comtant. say k 2 , depending on n such that 1m' eve'y f E H 1 (!1)

The above two inequalities hold for elemcnl.'l of z(I1).

F07" f E HI (,11) such that

In

j(x)dx = 0

" 1 81 ' II/II , 2(0) < k. L a ' k] > 0 constant depending on fL

J=I

x,

['2(0)

Definition tL{.9 (Sobolev Space 0/ Dishibutions). Let H'"(fl) denote the space of all distributions F such that D F E L z(l1) for all 0,101 $ m equipped with the norm
(5.4.4) Here, the denvatlves are taken in the sense of distributions and the precise meaning of the statement D F E L.1 (11) is that there is a distribution FI constructed in Remark 5.4.2 with f E Lz (l1) such that D F = Fl' Hence.

liD FIIL"n, - 'up 'i'

0,

I(F,Dorph2{ojl

II rp II L2(O}

(5.4.5)

",,,. "'_'_""""'''''''10''-''

Theorem 5.4.10
product.

Htn(fI) is a Hilbert space with 7'spect to the inner

(F,G) =

L
101:5m

(DOF,DoG)L2{0)'

(5.4.6)

t-,'IQI'e generally, if H"""(fI) or H"',P(fI) denotes the space of all functions f E Lp(fI) such that D f E Lp I P < 00, lal < m, then this space is a Banach space. Wm,P(fI) is usually called Sobolev space with index p or generalized Sobolev space

s:

pfYJOJ. It is easy to check conditions of inner product \Ve prove completeness of H"l(fI). Let {FA-} be a Cauchy ~uellno in Hnl(fl). Then for any la, < tn, {DO h} is a Cauchy sequence in L2(fI) and since L2(fl) is complete, there exists FO E L,dfl) such that

Now, since DO Fk is locally integrable, we may compute

}DOF,(,,)

In

DOF,(x),,(x)dx

~ (_1)1

In

F,(x)DO,,(x)dx

~ (_1)l o l(},(x), 0"),,",,

Also

"E 0(0).

Thus,

and

We also have
IIFDoF, -

Fpoll ~ 'up I(Ditll<i'

FO

If'

L~(fl)

,,,)1

-+Oask-+oo,

so t.hat DO F p<J = FF'Q; Q < m and the distributional derivative of the distribution associated with Fa. Hence, H"'(fI) is complete.

FO is

...". "'_'_""""'''''''10''-''

Let for 1 < P <

00,

11111, ~ (kl/(x)I'dx)
and

'J,

(5.4.7)

11/11",., ~

'J,
L (oSlo I::;", IIDO/(X)I'dX)
Q

(5.4.8) (5.4.9) (5.4.10)

DI(x)~

'Ix,
=

DO/~I

(/, uh.,\Qj
(/,91",.,
Note that

~ / gdx
Q

~ ( O:::lolSm IDogdX) LIDo

(5.4.11)

(5.4.12)

(/,g)m,2 =

2::::
:5l o l:5 m

{DO /, DOg)L.,(Qj'

(5.4.13)

For m = 0, we obtain that II/II",,, = Jl/II" and (/, g)0,2 = (/, gh.,(Qj' Conditions of norm for 11/11." can be easily checked using Minkowski's inequality. Completeness of H"""{f2) can be proved on the lines of the proof of Theorem fiA.l O.

Definition 5.4.10

H~"'''(f2)

denotes the closure of CO'(f2) in H"',"(m.

More precisely, this definition mealls that ('vcry / E H"""(f2) belongs to /-I~"'''(f2) if and only if there exists a sequence {I,,} in CO'(f2) with 11/0 - III --> 0 as n --> 00.

Theorem 5.4.11 H~"'''(f2) is a Banach space, I < P < 00, with nonn 11,11""" defined by Equat.ion {5.4.12} and H~",2(f2) is a Hilbert space with the scalar product. (-,.) given by Equat.ion (5.4.13).

Q . ,~,_ .._",,"l1>"-'

......

5.4.3

Tbe Sobolev Embedding Tbeorems

Definition 5.4.11 (Embedding OrJerot07). Let X and Y be Banach spaces with X ~ Y. The embedding operator j : X --t Y is defined by j(x) =x for all x E X. The embedding X c;: Y is called continuous if and only if j is continuous. that is

IIxJly < kllxll x

for all x E X and k is a constant.

The embedding X ~ }'" is called compact if and only if j is compact, that is, it is continuous and each bounded sequence {x,,} in X contains a subsequence {X"k} which is convergent in Y More generally, we may define an embedding as an lIlJecuve linear operator j : X --t Y. Since j is injective, we can identify'll. with j(u), in this sense, we write X ~ Y. Let

(In l' + t. un') 'J' 1If1!>"o ~ (lnt. Uf,)' dX) 'J'

11111", =

(5.4,14,

(5.4.15)

Theorem 5.4.12 Let 0 be a bounded region in R" with n > I. Then (a) The norms 11111t,z and 1I11kz,0 aI'C equitJalerlt. on Hri'z(O). (b) The embedding H1.2(n) (usually derwted by H(O) ~ Lz(O)) iB compact. (c) The embeddings

an, compact.
Theorelll 5.4.13 Let n be a bounded region in R", n 2 1 and have Bufficiently smooth boundmy, that. is, an E CO,1 (11 n = 1, then 0 is a bounded open intenJal). Then (a) (Density). COO(O) is dense in H1,'l(O). (b) (Compact Embedding). The embedding H1,Z(n) <;;;; Lz(n) M comlJact.

.."" "'_'_""""'''''''IO''-o!

(cj (Equivalent N01T1iSj. Eaclt of the following fI01T1iS

(5.4.16)

(5.4.17)

(In case n

1, we take /00 f 2daO = (f(a2 + (f(b2)

11/11..,

1$

an equuJalent

1lOfTTl on H I ,2(n), namely

~ (ll/(X)I'dX + l ~~ ,dX) 'I'

(11111' +

~~

11'1

')

(dJ (Regulalify). P01' m - j

> n/2,

the embedding

is continuous, that is, each continuous /unction f E Hm,2(n) belongs to Gi(n) ajtf.-'1 changUlg tlte vt.Uue of f on a set of n-dimensional Lebesgue mea..~ure zef'O, if necessalY. (ej (Gencmlized boundalY function). Thf.-'1"C ensts only one lineal" continuo'U..~ opemt01" A : /-/1.2(n) --+ 2(an) with the pl"Ope,ty that, f01' each f E CI(!t), the /unction Af: an --+ R is the cl(l.,~sical boundary function to Uj that is, Af IS the restriction of f: 0 --+ R to the boundary an. Tn the case when! / E /-/1,2(0), the functi011 Af E 2(an) is called the generolized boundary /unct.ion of f. The values of Af an~ uniquely detefTTlined on up t.o changing tlte values on a set of surface measUf"C zef'O. If I E Ht,z(n), t.hen Af = 0 in z(O): that is.

ao

Af(x) = 0 f01" almost nil x E an.

(5.4.18)

Corollary 5.4.1 CO,I. Then

Let n be a bounded ,"Cgion in R" with n

> 1 a1ld an E

Q.

......

(aj COO(O) is df.-"1ISe Ul Hm,Z(n) 101" m = 0, 1,2, ....

, to<l',_"""",,,,,"l1>"-I

(b) The embeddings

L,(O) " H"'(O) ;, H"'(OP H"'(O) ;, .. ,

G7'C

compact.
H~Il2(fI) 1Jftth m

(c)

II I E 111 -I,

that is,

DO/=

I, then A(Df ) = 0 m L 2 (l)fI) lor 101 0 onl)fI 101'0: lol < m-l, I = Oonl)fI.

>

<

B100d/y:.peaJ..'ing, embeddiny theorem.~ givl; condit.iofts under which Soboh'tJ spaCf'.~ a1'C embedded in spar.f:s 01 con!.inUO'M lunct.ions, C k (.0) /n one dimensIOn (n = l)fI is a sl1bset 01 the lwi/ine and the functions in H'(.o) 2 a1'C contin1101U!. For .0 C R (11 = 2, subsets 01 the plane), we n;quil'C that 2 (fI) in order to gil,amntee its continuity a function be in H

Remark 5.4.3 (i) Equivalent norms !Ilb,2 and II lh,2,o given, respectively, by (5.4.14) and (5.4.15) on H~,2(fI) are applied for solving the first boundary value problem for linear second order elliptic equationI'>. This corresponds to the classical inequality of Poincare- Friedrichs. (ii) For solving the second-order (respcctivel)' t.hird-order) boundary value problems, we require the equivalence of t.he norms I! lb,2 and 1I'lIi,2 (respectively II 111.2 and IIIW) on H 1 ,2(fI). This also corresponds to classical inequalities obtained by Poincare and Friedrichs. It may be observed that norms II . Ih, II ,112, II IIJ and II . 114 defined below are equivalent,

IIl1h 11111,

~ ([(/')'dX+ [Idx ')'P ,1I/11,~ ([(/')'dX+I'(a)+I'(b) ,p

=

(1' (I' +

(/')')dx

)'/'

and

11/11. =

(" 1

(/')')dx

)'/'

where f' denotes the derivative uf I in t.h~ classical sense. (iii) Compactness of certain embeddings is usPr! to prove till-' equivalence of norms on Sobolev spaces. (iv) For solving (ljgell\'alue problems, we rCQuirc the compactness of the embedding HJ2(fI) C L2(fI). (v) Regularity statement of Theorem 5.4.13 (d) is used to prove regularity of generalized solutions. The main idea is to show that I E H"l2(fI) for sufficiently large mj that b, the generalized solution I has generalized derivatives up to order m. Theil, for j < m - n/2, we obtain I E cJ(.o); that is, I has classical derivatives up to order j. Tn particular, H m,2(fI) 0;;;

Q.

......

, to<l'._..

....,..",,"l1>"-I

C",-l (f!),m = 1,2,3, ... , if 11 c R. In R 2 and R3, we have that

H'",2(11) ~ C"'-2(O), m = 2,3 ....

(vi) The bounda!)' operator A is vital for t.he formulation of boundary conditiollS in the generalized sense. For example, let f E H1.2(11) and 9 E H 1,2(f2). Then the boundary condition

f=gonon
is to be undet'stood in the sense

Af

Ag m L,(O);

that. is. Af(x) = Ag(x) for almost all x E fJl1. For proofs of the above results, we refet to Zeidler [161.

5.5

Integration in Banach Spaces

In this section, we present somt> important definitions and properties of spaces comprising functions on a real intenal [0, T] into a Banach space X. Such spaces and their properties are of vital importance for studying parabolic differential equations, modeling problf>.1TlS of plasticit)', sandpile growth, superconductivity, and option pricing.

Definition 5.5.1 Let (f2,A,/~) be a finite measure space and X a Banach space. u: f2 --t X is called stmngly measuHible if there exists a sequence {un} of simple functions such that lIu.,(w) - u(w)llx --t 0 for almost all w as n --4 00. Definition 5.5.2 (Bochner Integral). Let (n,A,I.t) be a finite measure space, and .t a Banach space. 'Then we define the BocJmrr mtt:.gn.u of a simple function u : n --t X by

for any E E A, where Ci'S are fixed scalars. The Bochner integral of a strongly measurable function u ; n t X is the strong limit (if it exists) of the Bochner integral of an approximating sequence {u,,} of simple functions. That

i~, f Ud,.t =

Q.

......

lE

lim {U., d/~. 'HoolE

, to<l'._..

....,..",,"l1>"-I

Remm'k 5.5.1 (i) The Bochner integral is independent of the approximating sequence. (ii) If u is strongly measurable, u is Bochner integrable if and only if lIuOlix is integrable.

Definition 5.5 ....J L,,(O,T;X) t < P < surable functions f : [0, T] -4 X for which
T

00

consists of all strongly mea-

1 1I/(t)II~dt < 00.

Theorem 5.5.1 em. ([0, T], X) ('onsisting of all cordinuom functions f : [0, T] -4 X. that have continuous derivatives up to onh"f' JIl Ofl [0, T] ts a Banach space with the norm
(5.5.1)

Theorem 5.5.2

L,,(O,T;X) is a Banach space with the nOlm

II/II ~ ( 111/(t)lI~dt
T

)'1'
1Ilitl~

(5.5.2)

Let X be a Hilber't space. thf.-"n L,dO. T; X) is a IJilbert space the inne, pl'Oduct
{f,g)r",\O,T;Xj

rr;spcet to

1
T

(f,g)x df..

(5.5.3)

Remur'k 5.5.2 (a) In L,,(O,T;X), two functions art> identically PQual if they are equal except on a set of measure zero. (b) Loo(O,T; X) denotes the space of all measurable functions which are essentially bounded. It is a Banach space with the norm

II/II ~ es, 'up 1I/(t)11x.

O<t<T

(c) If the embedding X <;;;, Y is continuous, then t.he embedding

Q.

......

L,(O, T; X)
, to<l',_"""",,,,,"l1>"-I

L.(O, T;

n,

is also continuous. (d) Let X" be the dual space of a BanaCh space X, then (L"t 0, Tj .\) the dual of Lp(O, T; X) can be identified with Lp(O, T; X")j that is, we can write (Lp(O, T; Xl)" = L,,(O, T; XO). (e) Proofs of Theorems 5.5.1 and 5.5.2 are on the lines of Solved Problems 2.8.3 and 3.10.3, <'Ind Problem 2.ftl!}.

r'

Definition 5.5.4 (Generalized Derivative). Let I E LdO, T: X) alld 9 E L 1 (0, T; Y) where X and Yare Banach spaces. The functioll 9 is called the n-th gcncmlizcd dCr1vativp of the function I on (0, T) if

<p''''(t)f(t)dt =

(~I)n

<p(t)g(t)dt

fm all <p E C!!,,(O,T). (5.5.4)

We writp 9 = I(ll). (a) (Uniqueness of generalized derivative). The n-th generalized derivative is unique, that b, if his o,lIother n-th generalized derivative, then h = 9 almost everywhere on (0, T); that is, h = 9 in L 1 (0, T; Y). (b) (Relationship between generalized detivative and distributions). Let I E L 1 (0, T; X), then a distribution F is associated with I by the relation

Remark 5.5.3

F(<p)
for each
n,

<p(t)f(t)dt

fo' all <p E C!!"(O, T).

F(n}

this distribution has an n-th derivative

defined by
(5.5.5)

{F("} ,rp) = (- 1)" (F, rp("})

for all rp E G;'" (0, T).

If (5.5.4) holds, then

F(>lj

can be represented by
(5.5.6)

As we know, the advantage of the distribution concept is that each function I E L 1 (0, T; X) possesses derivatives of every order in the distributional sense. The geIlf'ralized derivative (Definition 5.5.4) singles out the cases in which bv (5.5.6), the n-th distributional deriv:ttive of I can he represented by a function 9 E L 1 (0, T; F). In this case, we set I("} = 9 and write bnefly

L,(O,T;X),f'''' E L,(O,T;l').

Q.

......

Theorem 5.5.3 (Generalized Derivative and Weak Convergence). Let X and }.' be Banach spaces and lct thc cmbedding X C Y be continuous.
, toOl'._..

....,..",,"l1>"-I

Then 'it lollows Imm

lin}

= Ok on (0. T)

loy' all k and fixed n

>

I.
~ 00,

and Ik ----' I in L,,(O, T; X) as k ~ 00, Ok ----' 0 in Lq(O, T; F) as k 1 <p,q < 00 Ulat 1('1) =0 ()n (O,T)
(See Zeidler [16, 419-420], for proof).

Theorem 5.5.4 Foy' a Bmwch spaa' X, let 11"1"'(0, T; X) denote the space 01 all lunclwlls I E L,,(O, T; X) such that 1('1) E ,,(0, T; X), when! n :$ m and 1('1) de110te the n-lh genemlized derivative 011. Then HJIl"(O, T; X) is a Banach space with thc norm

1I/11lf'".~(o.T,xi =

~ II/(i)IIL.,(oT'X)

./,

(5.5.7)

II X zs a l-Jilbc,t spacc mId p with thc imlc,' product

2, UIC11 J-fm,2(0,TjX) is a Hilbert space

U,0),,n.2(o,T;X) =

1
o

T m

L:.U(l},O(I))xdt.
;=0

(5.5.8)

Remm* 5.5.4 Theorem 5.4.10. (b) For x < y

(a) Tl1e Proof of Tl1eorem 5.5.4 is similar to that of

Ilf(y) - f(x11lx S [111'('lllx

dt

(5.5.9)

holds (c) Tl1e embedding H 1 ,2(O,T;X) c C(lO,T], H), where H is a Hilbert space, is continuous, that is, there exists a constant k > 0 sucl1 that

Example 5.5.1 Let X = l', IE cn([O,T],F), n 2: 1. Tl1en tl1e continuuus n-th derivative 1('1) ; [0, T] ~ F is also tl1e generalized nth derivative of Ion (O,T). For I E C1([0,T];Y) and tp E CO'(O,T), (tpf)' = tp'l + tpl'. We obtain tl1e classical int.egration by parts formula

Q.

......

, t><l',_"""",,,,,"l1>"-I

Repeated applications of this fonnula give

5.6
5.6.1

Probl"ms
SnhrP.(} Pmhlems

Problem 5.6.1 If the gradient 'VF(x) of a function F: X --+ R exists and II'VF(x)1I < M for all x E 1(, where I( is a convex subset of.t, then show that
!F(u) - F(v)1

< Mllv - vII

Solution.

By Theorem 5.2.2, we have

!F(u) - F(v)1 ~

IF'ru + .I(v - u))(v - v)1 ~ 1(" F(v + .I(v - v)), (v - v))1 < II" Frv + .I(v - u)lIl1v - vII " Mllv - ull
I(

as II 'V F(tl) II :0:; M for all A)'U + All E 1(1.

'U

[Since

I(

is convex; for all

1[,

vEl(, (I -

Problem 5.6.2 Let f . j{J > R possess continuous second partial derivatives with respect to all three variables, and let F : Gl[a,b] --+ R be defined by

F(x)

f(x(t),x'(t.),t)dt.

Show that the F'rechet derivative of F, dF(x)h, is given hy

["[8f d ax' dF(x)1i =}" ax - lit (8f)] lull

+ [8f ]' ax,li CI'

...". "'_'_"_"""10"-"

Sol1ttion,

F(x

+ h) - F(x)

l ~ l (~~
~
-4

(f(x(t)

+ h(t),x'(t) + h(t),t)

- !(xlt),x'(t),t))dt (x(t),x'(t),t))h(t)

+ :'~, (x( t), x' (f), t) h(t)) dt + ,'( h, h)

where r(h,h)

==

O(lIhIlC[a,bl)' i.f'"

11111 C'la,b] I

Ir(h, h)1

0 '"

IIhllolo,'l -4

dF(x)h

~
~

l [~~(X(t),x'(t),t)h(t)
}a

.. ;;,(X(t),X'(t),t)h(t)] dt

r' [a! _!!- (8!)] hdt .. [Of h]' OX dt OX' OX' "
[.(of.

after integration by part.

Problem 5.6.3

X --+ R be a bounded symmetric bilinear form on a. Hilben. space X a.nd J a functional 011 X, often c..alled "energy
a(,) : X
X

functional", defined by

J(u) == '2a(u.1l) - F(u),

Find tl1e Frechet derivative of .J

where FE X*

Solution.
J("

Fbr

811

arbitrary ,

A.. ~

X,

+ )

"a(" + ,,, + ) - F(" + )

I

~ "a(",,,)

+ "a(,,,) + "a(",) + "a(,)

- F(,,) - F()

...", "'_'_""""'''''''10''-''

by tl1e bilinearityof a(, .). Using the symmetry of a(,), [a(11,) = a(,u)], we get
J(u+)

ga(u,,,) - F(u)} + (a(",) - F()} +

I

~,,(,)

= J(u) + (alu,,,) - F()} + 2a(,1,

IIJ(u + ) - J(u) - {a(",) - F())II

1IlIx
< ~ MIIlIxlllIx
- 2
Tl1is implies that

lla(,)1 2 1IlIx

1Illx

as a(, .) is bounded.

. IJI,,+) - J(u) - (a(u,) - F()}I 0 = , \,m 1Illx-+o 1IlIx

~

dJ(u)= a(",) - F().

Since J defined in t.his problem is Frcchet differentiable, it is also Gateaux different.iable and DJ(u) = dJ(u). The derivative of this functional is oft.en used in optimal control problems and variational inp.qualities.

Problem 5.6.4 Prove that a linear operator T from a Banach space X into a Banach space 1- is Frechet differentiabl~ if aud only if T is bounded.
Solution. Let T be a linear operator and F'n~chet differentiable at a point Then T is continuous (and hen('f' hounded) by Theorem 5.2.4. Conversely, if T is a bounded linear operar.or, then IIT(x + t) - Tx - Ttll = 0, proving t.hat T is Frechet differentiable and T' = T.

Problem 5.6.5 [{ depending on

Prove that for n such that

E cO'(O),n c R2, there is a constant

",,,. "'_'_""""'''''''10''-''

Solution. Let f E Cgo(O). Consider a. rect.angle Q = la, bl x [c,d] as in Figure 5.6.1 with IT c Int Q. Note that f vanishes outside O. Then
f(x,y) ~ , IJyf(x,t)dt foc aU (x,y) E Q.
By Hiilder's inequality. we get

, 1

Integrating over Q, we get

Problem 5.6.6 show that

[ f'dx <

Let Q be a closed square in R 1 , with side length I, then

JQ

JQ i=1

[t (~")'
VXI

dx

for all fECI (K).

Solution.

\Ve consider the square

Q ~ (("'J) E

R', -1/2" E,"" 2).

Let fEet (Q), and let x = (~, 1}) and y = (a, (3). Then

f(x) - fry) ~
By the inequality (a + bf

[' [" J" !t,mdt+ J f,,(,t)dt.

o

-:; 2(a2 + b2 ) and by the Holder inequality

+ fry)'
- 2f(x)f(y)

(J(x) - fry))' ~ f(x)'

< 2 ['I' [(I,(t,m)' .. (I,,(,t))'ldt.

J-t/2

Integration over Q with respect t.o (x,y) yields

Q.

......

, to<l'._..

....,..",,"l1>"-I

d ---- ------T------,
___ ______ c ~ ~__~

()

Figure 5.6.1

GoomeLl'ical explanation ill Problem 5.6.5

Tl1is is the desired result. Here,

IE.

~~;

CO(O) is equal to the space

of continuous functions on n into R; Ck(O) is equal to the space of all functions defined on n into R with compact support whose derivatives up to k-tl1 order ex.ist and are continuous; and Ctf(O) = space of all functlOllS defined on

n
00

Ck(O) is equal to

n int.o R

10=0

having detivatives of all orders.

Problem 5.6.7 For each real numbers s, the space HS(R") has the following properties:
(i) H"(R") is a Hilbert space.

(ii) If s. 2': 82 then HS'(R") C

Q."to<l'._......,..",,"l1>"-I

.... ,

HS~(R")

a11d the injection IS continuous.

Solution. (i) \Ve verify the completeness of H"(R") as checking of inner product conditions are straightforward. Let {Fj} be a Cauchy sequence in /-I"(R"), then {I + IYI 2 }1/28F,} is a Cauchy sequence in L 2 (R") which is complet.e and so {(I + IYI 2)1/'l8 Fj } -+ G in L 2 (FC'). Hence, G is a. tempered distribution and since (1 + lyI 2)-1/28G is of slow growth (I + /YI 2)-1/28 G is a tempered distribution. Thus, there exists a tempered distribution F such that (I + IYI 2)-1/28G = F. Then F(I + lyI2)-1/2" fr E L2 (FC'): that is. FE H"(R") and Fj -+ F in H"(FC'). (ii) This follows from the fact that (I + lyI2)"~ (I + IYI 2)", if 81 > 82.

s:

Problem 5.7.1

Let F ; G[G, F(f)

11 --t

R defined by

(tf(t)

+ (f(,))'l<ll.

Then find the Gateaux derivative of F.

Problem 5.7.2

Let F ; ~ --t R be defined by

F(x)

= x~ +:tlX~,

where x

= (XI,X2)

Find the Frechct derivative of F.

Pmblem 5.7.3

Let. T; R --t

too

be defined by

"1 t '" T(t)= ( l,t'2 . " " ' -n. " ) "
Find the Fn'ichet derivative of T.

Problem 5.7.4 Let HI and H 2 be two Hilbert spaces and T E H(H I , H 2 ). For y E H2, let F: HI --t R be defined by F(x) ~ IITx -

yll' /2.

Prove rhat

Q.

......

\7F(x) = T*Tx , to<l'._..

ry.

....,..",,"l1>"-I

Problem 5.7.5 (a) Let II, h : R n -t R be Lipschitz near x, then sl10w that 8U, + j,)(x) c 81>(x) + 8j,(x). (b) If II is Lipschitz near x and h is continuously differentiable on a neighborhood of x, then show that

8U, .,. j,)(x)

8J, (x)

+ v h(x).

Problem 5.7.6 Let I; R tl --. R be Lipschitz on atl open set containing tl1e line segment [x, yl_ Then show that there is a point U E (x,y) surh that Jly) - J(,) E (8JI")," - x). Pl'Oblem 5.7.7 Let F : X - t R be a functional. If F hl~ a suhgradient at x, then show that F is weakly lowe! semicontinuous at x. [f F has a subgradient at all .1:" in some open convex set g ex, then show that F is convex and weakly lower semicont.inuous in I<. Problem 5.7.8 Let F X -+ R be a convex functional on an open convex set [{ of X. Then prove that at every x E I<, there exists at least one subgradient.
(X~_1}-1

Problem 5.7.9
a test funclion.

(a) Justif:,' that rp(x) = Is the function '(/I(X) =

e 0,

if Ixl <. I, is ot herwlse,

{(xi + ... ~ x~ -

1)-1 a test

functIon on R"Y (b) Show that rp(o-x + {3x), I(x)rp(x), and rp(kl(x); where a and (3 are constants, rp is as in (a), I(x) is an arbitrary smooth function and k is a positive integer, are test functions. (c) Let if! I ,rpz . ... rp" be test functions on R. Then

is a test function on R". Show that )[/(t) - l(s)II.'( <

Problem 5.7.10
is a Banach space.

it

lI/'(lI)Uxdu. where X

Problem 5.7.11 give exalIJpl~ .

Define the concept of convolution for distributions and

Q . ,~._ .._",,"l1>"-'

.... ,

Problem 5.7.12

Show that D"o(),t-It)(rp) = 1),1\0 Dao

(t - ~), w!Jere

0(') denotes the Dirac delta (unction.

Problem 5.7.13 Show that the sequence of regular distribution {sin ntl converges to the zero distribution. Problem 5.7.14
Let H!(O),O

such t!Jat first partial derivative

ux. that H!(O) is a Hilbert spN:e with respect

c; R Z denote the set ~1 E Lz(O), i =

to

of all

E Lz(O)

1,2, T!Jen show

the inner product (j,g)!

10 (1g + vI vgldx where

Show further that

Problem 5.7.15 (llelliell'8 Lemma}- If 0 is a bounded region in R'Il, 71 > I, then prove that embedding H~' (0) C Lz(O) is compact. Problem 5.7.16
Prove Green's formula, namely, the equation

Problem 5.7.17 Show that the embedding Loo(O, T; X) : Lp(O, T; X) is continuous for all I S p :S 00, Problem 5.7.18
Let

1 : R --+ R

be defined by
i( u

-f:. 0

ifu=O'

Show l!Jat (a.) if P > I, then

1 is SU'lCtly monotone,

...". "'_'_""""'''''''10''--'

(b) if P = 2, then

J is st.rongly monotone.
ShOl'''' that for u E L:dO, T: H) and
tho'" L

Pmblem .'i.7.19

E L 2 (0, Tj H),

'fi._.. . ~. . . . ~ __,

.'"''

References
[J. Il Adams. Sobolev Spaces. New York: Academic Press, 1975
[2) H Cartan. Differential Calculus. Paris: Herman/Kershaw, 1971. [3] PG Ciarlet. Introduction to Numerical Linear Algebra and Optimization. Cambridge: Cambridge University Press. 1989. [4] RF Curtain, AJ Prit.chard. Functional Analysis in Modern Applied Mathematics. London, New York, San F"':aneisco: Academic Press, 1977 [51 R Dautray, .JL Lions. Mathematical Analysis and Numerical tvlethods for Sciencf' and Tffhnology. Vol 2, Berlin-Heidelberg-New York: Springer-Verlag, 1990. [61 L Debnath, P Mikusinksi. Introduction to Hilbert Spaces with Applications. 2nd ed. Sandiego-London-Boston: Academic Press, 1999. [7] J Dieudonne. Foundation of Modern Analysis. New York: Academic Press, 1000. [8j DH Griffe!' Applied Functional Analysis. Chichester: Ellis Horwood Limited Publishers, 1!J81 [9J C\V Groetsch. Elements of Applicable Functional Analysis. New York and Basel: Marcel Dekker, 1980. [1OJ RD Milne. Applied Functional Analysis. Boston-London-Melboume: Pitman Adwmced Publishing Program, 1980. [11] MZ Na<;hed. Differentiability and related properties of nonlinear operators: Some aspects of the role of differentials in nonlinear functional

11"'_'_""""'''''''10''-'' ......

anal,vsis. tn: LB RaIl, ed. Nonlinear Functional Analysis and Applications. New York, London: Academic Press, 1971, pp 103-309. [12] J Outrata, M Kocvara, J Zowe. Nonsmooth Approach to Optimization Pl'Oblellls with Equilibrium Constraint~. BOStOIl, Dordrecht, London: Kluwer Academic Publishers, 1998. [13] RT RockafeIler, RJ-B Wets. Variat.ional Analysis. Berlin, Heidelberg, New York: Sprin;er. 1998.

[141 RA Tapia. The differentiation and integration of nonlinear operators.

In: LB Rail, ed. Nonlinear Functional Analysis and Applications. New York, London: AUldemic Press, 1971, pp 45-108. [15] A \Vouk. A Cours of Applied F'unctional Analysis. New YorkChichester: A \Vile~' lnterscience Publication. John Wiley and Sons,

1979.

[l61 E Zeidler. Nonlinear F'unctlona1 Analysis and its Applications II A.

New York-Berlin-HeidI berg: Springer-Verlag, 1990.

......
B,.,to<l',_"""",,,,,"l1>"-I

Chapter 6

Optimization Problems
6.1 Introduction

The ha<;ic elements of optimization can be found in the Cak1.llus courses where maximum and minimum (extremum) problems are concmed with those values of the independent variables for whidl a given function attains its maximum or minimum valu~. A result of the ~lehrated nl<tthematician Pierre de Fermat 8tates that if a differentiable real-valued function of one variable has a maximum or minimum at a point, then its derivat.ive is zero at that point. In this dlapter, we introduce the concepts of maximum and minimum, often referred to as extremum problem in the case of functionaJ, and discuss their relevance to other problems of applied areas, especially solutions of boundary value problems. Here, we restrict ourselves to optimization of smooth functions; however, readers may find detailed accounts of !Jon-smooth optimization problem:s in [2, 3, 8,11,14,19]. Current developments concerning algorithmic optimization and related softwarf> may be seen in [4, 7, 9,12, 15].

6.2

General Results on Optimization

Let X be a normed space and J a real-valued function defined on a non~empty closed convex subset [{ of X. The general optimization problem denoted hy (P) is to find an element u E [( such that Jtt~) < J(v) for all v E IC If such all element u exists, we write

Definition 6.2.1

f(u)

tJEK

inr f(v).

0 . ,>0<1"_""""'''''''10''-1

.... ,

In this situation, we say that f has a mlllJnlUm at u. If J( -::j:. X, this problem is referred to as the constrained optimization problem, while the caseJ( = X is called the unconstmined optimization problem.

Definition 6.2.2 Let.4 be a subset of a nonned SI)8CP X and fa realvalued function on A. f is said to have a local or 'Y3lative minimum (maximum) at a point Xo E A if there is an open sphere S,.(xo) of X. such that
f(xo) ::; f(x) (f(x) ::; f(xo)) holds for all x E S,.(xo) n A. If f has either a relative minimum or relative maximum at Xo, then f is said to have a relative extremum. The set A on whidl an extremum problem is defined is often called the admissible set.

Theorem 6.2.1 Let f ; X -t R be a Giiteaux diffenmtiable junctional at Xo E X and f have a local extremum at Xo, tllen Df(xo)t = 0 for all t E X.

Pmof. For every t E X, the function f(xo + ot} (of the real variable 0) has a local extremum at 0 = O. Since it is differentiable at 0, it follows from ordinary calculus that

~ [ do /(xo + fitll
I

a=O

~ o.
1

This means that D f(xo)t = 0 for all t E X which proves the theorem.

Remark 6.2.1 (i) It follows immediately from Theorem 6.2.1 that if a functional f ; X -tR is Frcdlt differentiable at Xo E X and has a relative extremulll at xo, then dT(xo) = O. (ii) Let f be a real-valued functional on a nonned space X and Xo a sohltion of (P) on a convex set [(. If f is a Gateaux differentiable at xo, then
D f(xoHx - xo) ~ 0

for all x E 1<.

Since [( is a convex set, o E (0. 1) and x E 1<. Hence.

Df(xo)(x - xo) =

Verification.

Xu

+ o(x xo))

xo) E

J(

for all

......
0,0,>0<1"_""""'''''''10''-1

[(~f(xo + o(x -

L=o ;: -: O.

Theorem 6.2.2 [Jet [( be a convex subset of a normed space X. {l} 1f J : [( ---) R is a convexfunclion, then (P) has a solution u whenever J has a local minimum at u. {2} 11 J ; 0 C X. -tR is a convex function defined over an open subset of X containing K and J is Frcchet diffenmtiable at u point u E [(, then J has a minimum at is a solution of (P) on if and only if

u(u

K)

J'(u)(v - u) ?: 0

for el!ery v E K.

(6.2.1)

1f K is open, then {6.2.1}

JS

equwalent to (often called Elder's equation)

(6.2.2)

Pn)()f.

(I) Lt v = u
J(u + ow)

+ w be an.y element of K. By the convexity of J,

0::;0::;1,

< (l - o)J(u) + oJ(v),

whidl can also hI' written as

J(u+ow) - J(u) < o(J(v) - J(U)) ,

Siuce .I has a local minimum at n, there exists 00 such that 00 > 0 and 0::; J(u + oow) - J(u), which implies that J(v) > Ju). (2) By Remark 6.2.1(ii), the necessit.y of (6.2.1) holds even without COJlvexity assumption on J. For the sufficienc.y part, we observe that
J(v) - J(u)
::::>:

J'(u)(v - u)

for every v E K.

Since .I is convex,

.1((1- 0)11.

+ ov) < (1

- 0).1(11.)

+ oJ (v)

for all

E [0,1],

J(u+o(v-u))-J(u) J(v)-J(u) >

0 '

J(v) - J(u)

> lim

J(u+o(v

-u) -J(u)
=:.

0-.0

J'(u)(v - u)

> O.

This proves that if J'(u)(v - u)

> 0, then J has a minimum at u.

.n",

"'_'_""""'''''''10''-''

A functional J defined on a normed space is called coercive if lim

00.

II x ll_' oo

J(x) =

Theorem 6.2.3 (Existence of Solution in R"). Let K bc a nonempty, closed convex subset of Fr' and J : R" --t R a continuous Junction which is coen;ive iJ [( is unboundcd. Then thcre exists at lcast one solution
of(P).

Pwof. Lt {ud be a mimmizing sequence of J; t.hat is, a sequence satisfying conditions u~. E J( for every integer k and lim J(U.k) = inf, J(v). k~oo "en This sequence is necessarily bounded, since the functional J is coercive, so that it is possible to find a subsequence {u~.,} whidl converges to an element u E J( (J( being closed). Since J is continuous, J(u) = lim J(Uk')
k' ~ -.)

inf J(vl whid] proves the existence of a solution of (P). "oK

Theorem 6.2.4 (Existence of Solution in Infinite-dimensional Hilbert Space). Lct [( bc (L non-empty, convcx, closed subsd oj a scpamble Hilbcrt space Hand J : H --t R a convex, continuous functional which is cocrcivc iJ [( is unbounded. TIlen (P) has at least one solution.

Proof. As in thl' previous theorem, K must bP bounded under the hypotheses of the theorem. Let {Uk} be a minimizing sequence in [(. Then by Theorem 4.4.7, Iud has a weakly convergent subsequence Uk' ----" u. By Corollary 4.4.1 J(u) < liminf J(Uk')' Uk' ----" u which, in turn, shows that U is a solution of (P). it only remains to show that the weak limit u of the sequence {Uk'} belongs to the set J<. For thib, let P denote the projection operator associated. with the clo:-Pd, convex set J(; by Theorem 3.5.2,
w(

E K implies (Pu - u,w( - Pu)

>0

for every integer f

The weak convergence of the sequence {w,} to the element u implies that

o < h =(Pu lim

u. w( - Pul = (Pu - u.u - Pu) = -jfu - PuII 2

::;

O.

Thus,Pu=uanduEK.

B.

......

Remark 6.2.2

(i) Theorem 6.2.4 remallls valid for reflexive Banach

, to<l',_"""",,,,,"l1>"-I

space and continuity replaced by weaker condition, namely, weak lower semi-continuity. For proof, see Ekeland and Temam [6) or Siddiqi [20]. (ii) The set 5 of all solutions of (P) is closed and convex.
Verification. Lt u.,lIz be two solutions of (P); that is, Ul. Uz E S. OU] + (1 - 0 )uz E Ie 0 E <0, 1) as J( is convex. Since J is convex.

J( OUI
vEl>

+ (1

- O)U2) :5 oJhJ.d
vEl>

+ (1

- O)J(U2)

Lt ). = inf. Jh!) = J(u.) and ). = inf, J(v) = J(U2), then

). <

J(OUI -l-

(J - o)uz) :5

0).

+ (l -

0).

= A,

that is, ). = J(OUI + (1 - O)U2) nnplying QU] + (1 - o)uz E S. Therefore, S is convex. Lt {u,,} be a sequence in S sudl that u" ---7 u. For proving dosedness, we need to show that u E S. Since J is continuous

J(U) = lim inf J(u n ) :5 ). < J(u).

n~oo

This gites

J(u)

=).

and so U E S.

(iii) The solution of Theorem 6.2.4 is unique if J is strictly convex.

ft . VeN cahon.

--ILtUl,U2E 5 anduJ rU2. Then

UI+UZ 2

E 5 as S' IS

convex. T hereJore, J '

("'+"')
2

= ).. Smce J IS stnctly convex,

...

This is a contradiction. Hence,1/.. ;;j. uz is false and UI = U2.

6.3
6.3.1

Special Classes of Optimization Problems

Convex, Quadmtic and Linear Programming

For ]{ = {v E X/'P,(v) < 0, 1:5 i < m', 'Pi(l!) = 0, m' + 1 < i < m}, (P) is called a nonlinear' pmgmmming problem. If 'Pi and J are convex functionals, then (P) is called a convex pmgramming pmblem.

...... '.'_'_""""'''''''10''-''

For X = R", I,' = (v E Rfl/<p,(v)

S di ,

J SiS m}, J(v) =

~(AV,v)n

(b, v), .1 = (u,,), all

11

x n positivc definite matrix, and <p;(l!) = Laijl!j;

j=1

(P) is called a quadmtic progmmmmg problem. If J(v) =

L aiVi,
,=,

X =

R",J(={VERtl/ta;;l!j<d;,
)=1

l<i<m},A=(OiJ ), lIxnposi-

tivc definite matrix, then (P) is called a linear programming pmblcm.

6.3.2

Calculus of VariatiOllS alld Eu/e,.-LagI'8JJge Equatioll

The c1a:,sical calculus of variation is a special case of (P) where we look for the extremum of functionals of the typ'"

J(1/.) =
tl

it> F(x, u, l/)dx

("'(X)

~ :) ,

(6.3.1)

is a twice continuously differentiable function on [a, bl, F is continuous in x, u and u ' , and has continuous partial derivatives with respect to u and

u' .
Theorem 6.:~.1 A neccssm7/ condition lor the functional J(u) to ltal!C an extremum at u IS that u must sati..~fy the Eulc,>-Lagrangc cquation

iJF d (iJF) =0 au-dx au'

in [a, b] wtth tJ,e boundary condition u(ll) = Pmof
J(u
Let u(a) = 0 and u(b) = 0, {hen
=
Q

(6.3.2)

llnd u(b) = (3.

+ av) ~ J(u)

it> [F(x, u.J.... av, u ' + OV') " ')

F(x, U, U') d.r.

(6.3.3)

Usmg the Taylor series expansIOn

F(.x,u+ttV,U +(11) = F(x,u,u +fi vau +V au'

(iJF ,OF)

+ 2f v an

0'

(aF aF)' + ... ,

+V'{}II.'

B.

.... ,

, to<l'._..

....,..",,"l1>"-I

it follows from (6.3.3) that

J(u + OV) = J(u)

+ odJ(u)(v) + 2Tdz J(u)(v) + .

Q'

(6.3.4)

where the first and the second Frechet differentials are given b.y
dJ(u)v =

d J(U)l!=
Th~ nec~sary

' (OF + l '( of l

a

V {Ju

v' {Jul

I'

l!au+ V

OF) dx OF)' dx. '{)u' OF) dx.

(6.3.5) (6.3.6)

condition for th~ fUIlctlOual J to have an extr~mum at u is that dJ(u)t, = 0 for all v E ezra, bJ such that l!(a) = l!(b) = 0; that is

0= dJ(U)l! =

'( of + l un
a
V

Vi au'

(6.3.7)

Integrating the second term in the integrand in (6.3.7) by parts, we get

l [~~ -,~ (:)] vdx

[[~ -:. (:)]VdX-O
of _ <Ix (iJF) au ~ au'

[v ~~J: ~ O.

(0.3.8)

Since v(a) = v(b) = 0, the boundary terms vanish and the n('''Cessary condition becomes for all v E CZ[a,b],
(6.3.9)

for all functions v E ezra, bj vanishing at a and b. This il'i possible only if

__ 0
I

(see Problem 6.5.1)

Thus. we have the desired result.

6.3.3

AljnjlJJjzation of Energy FunctIOnal (Quadratic Functjonal)

1 J(v) ~ :;a(v,v) - F(vj,

A functional of the type

(6.3.10)

where a(, .) is a hilinear and continuom; form on a Hilbert space X and F is an element of thp dual space X of X, is called an elle,gy jUllctiollal or a quadratic functional.

.n",

'.'_'_""""'''''''10''-''

Theorem 6.:1.2 Let a(,) be coercive and symmebic, (ltld K a nonempty closed convex subset of X. Then (P) for J in (6.3.10) has a '1miq'Ue solution on I<.

Pmof. The bilinear form induces an inner product over the Hilbert space X equivalent to the norm induced by the inner product of X. In fact. the assumptions imply that

v'Qllvll < (a(v,v) )

-1' S JiiGIT IIvll,

(6.3.11 )

where lIal) is given III Remark 3.9.2. Since F is a linear contllluous form under this new norm, the Riesz representation theorem (Theorem 3.7.1), there exists a unique element v E X such that

F(v) = a(v,v)
as

for e\'ery v E X.

(6.3.12)

In view of the remark made above and (6.3.12), (6.3.10) can be rewritten

I I 1 J(v) = '2a(v,v) - a(v,v) = '2a(v - v,v - v) - '2a(v,v)

'2(v-v,v-v)- '2(v,v)
for all v E I< and unique v. Therefore, inf J(v) is eljwvalent to inf
vEK vEK

Ilv - vII.

Thus, in the present sit-

uation (P) amounts to looking for the projection x of the element v on to the set I<. By Theorem 3.5.1, (P) has a unique solution. I

Example 6.3.1
in R"'. Since

Let X = I< = R", J: v E R"

J(v), where J(v) =

the norm

~IIBv - cll~" - ~lIcll~n,

B is all m)(

11

matrix, and

1IlIm denotes

J(v) =

~(BtBV,v)" -

(8 f u,v)",

the problem is one of quadratic programming if and only if the symmetric matrix is positive definite. In this case, the existence of this problem follows from Theorem 6.3.2. We :shall :show ill Chapt~r 7 that fillllillg the solutions of boundary value problems representing physical phenomena can be formulated in terms of finding a minimum of the energy functional.

...... "'_'_""""'''''''10''--'

6.4
6.4.1

Algorithmic Optimization
Newton Algorithm
~UJd

Its Generalization

The Newton method deals with the search of zeros of the eCjuation F(z) = 0, F : U C X -+ Y, X and Yare nOrlned spaces, in particular, for X=Y=R, F:R-tRorX=R"andY=R!', p:Rn-tRnandUan open subset of X (open interval of R or open ball of R")' Once \\"e have this method, the functional F can he replaced by p' or V' F to obtain the algorithm for finding the extrema of F; that is, zeros of F' or V' F which are extremum points of F. One can easily check that if F : [a, b] --t Rand IF'(x)1 < 1, then F(x) = 0 has a unique solution, that is, F has a unique zero. for the function F : U c R --t U the open subset of R, the Newton method is defined by the sequence

n,

F(u,)
'Uk+!

= tlk - P(Uk)'

k > O.

(6.4.1)

tJo is an arbitrary point of open f;f't U. The geometric meaning of (6.4.1) is that each point Uk+1 is the intersection of the axis with the tangent at the point Uk. This particular case suggests the following generalization for the functional F : U c X ---t V: For all arbitrary point Vo E U, the seCjuence {ud is defined by
Uk+l

Uk -

{F'(ud} -] F{ud,
Uk

(6.4.2)

under the assumption that all the points R n , F(u) = 0 is equivalent tv

lie in U

If X

Fr', Y =

F](u) = 0, F,(v) ~ 0
F3 (u) = 0

(UI,lJ2, . .

,u,,) E R"

where F',. : R" --+ R, i = 1,2, ... ,n. A single iteration of the Newton method consists in solving the linear system

(6.4.3)

0 . ,>0<1"_""""'''''''10''-1

..... ,

and then setting

It nmy be observed that if F is an affine function, that is, F(x) :::: A(:r) b, A :::: (alj) is a S(juare matrix of size n; that is, A E An (n) and b E nn, then the iteration described above reduces to the solution of the linear system AUk+1 '" b. In that case, the method converges in a single iteration. We now look for (i) sufficient condit,ions which gllflrRntee th(~ exi.ste~n~(~ of a zero of the function F, <Uld (ii) an algorithm for approximating such an element 11.; that is, for constructing a sequence {Uk} of points of U such that

\Ve state below two theorems concerning the existence of a unique zero of F and state their corollaries for the existence of the unique zero of \7 F The extrema of F will exist at the zero of \7 F. Theorem 6.4.1 Let X be a Banach s/Jace, U an o/Jen subset of X, Y a nonnetl linear S/}(J.ce and F : U C X --t Y flifJeretlfiabIe over U. Suppose that there exist three constlltlts Cl', /3 ami ~I such that Q > 0 antI S,Auo) --= {" EX/II" - "0Il < oj <; U

(i)
sup sup
IIA;I(u)IIB[x,YI

< /3, Adu) :::: A k

B[X,ll

k?,O uES" ("0)

is bijective.

(ii)
sup
k?O"ES,,(1t~)

sup

IIP'(:r) - Adx')IIl:l[xYl

<

/3

'Y,

antI 'Y

< 1.

(iii)

Then the set[ltetlCe tIefinetI by

k>k'>O

(6.4.4)

.n". "'_'_""""'''''''10''-''

is etltirely contaitled within the ball and converges to a zero of F in S,,(uo)

which is 'Utli(JUe. Furtbenllore

< ,. II Uk _ U II - II", - "Oil, 1-,'

(6.4.5)

Theorem 6.4.2 Let X he a Banach S])(lce, U an open subset of X,F : V c X --t V, and V a normed space. Furthermore, let F be contimwilsly differetltiable over U. SU]J]JOse that u is a point of V such that F(u)
=:

0, A"", P(u) : X --t

-

J'

boundnilinear amI bijectiVe} ,and),

{ sup IIA k k?O

AII.u(x,l']

S II

til

.u(1',X]

< 1(2

Then there exists a closed ball, Sr(u), with Cetlter u arul rudi1l$ r SItch thal for every point Uo E Sr(U), the seqnence {uk} (/efineJi by k
1S

> 0,

(6.4.6)

conl.ame(l tTl Srlu), and Cf)nverges 1.0 a ]JOmt u, which IS the only zero of F in the ball Sr(u). FUrtheJYTlore, there exists a number")' such that

> O.

(6.4.7)

As a couscquem::e of Theorem 6.4.1, we get the following result:

Corollary 6.4.1 Let L' be an open sltbstd vf a Banach S])(J.ce X aud F : V C X -4 R which is twicf' (/ijJerentiable in the O]Jen set U. SUIJIJose that there are three con.~lants: 0:. B." .~1tch that Q > 0 awl S,,(uo) =: E

{v

Xlllv-uoll
awl

sa} C V, Ak(v) E S[X, X"]ll1ul bijective for every v E So:(u)

Then the seqltetlce {ud (/efitlC(/ by

Uk-t-1 "'" Uk - A;l (Uk' )F'(Uk),

k>k'>O - -

.n",

"'_'_""""'''''''10''-''

is cotltained in the ball Sa(UO) and cotwerges to a zero of F ' , say u, which is the otlly zero in this ball. Furthermore,

< II Uk _ UII - II"1 - "oil, l' '}.

As a consequence of Theorem 6.4.2, \\'e get the following result:
Corollary 6.4.2 Let L' be au allen sltbset of a Banach slmcr. X and F : U C X --+ R a flttlction which is twice difJeretltiable in U Moreover, let U be a lwint of U such that
F/(u) = 0, F"(u) E BIX,X*] and bqe.chve aml.\ <
l:l(x- ,xl

sup II A k

'

F"(u)lIuIX,X.J

$11 (F"(")) '\ I

2'

Then there ensts a closetl ball S,.(u) wtth center u and radius r > 0 such that, for every lwint uo E S,.(u), the sequence (ud tlefined by Uk+l = Uk - A k l F'(Uk), is containell in S,.(u) and Cotlverges to the lwint u, which is the only zero of F ' in the ball. Furthermore, Uk-!-I = Uk - A;:I(Uk)F'(Uk) converyes geometrically, namely, there exists a'Y such that 'Y < I arId lIuA'ull $ '/11", - ull, k > O.

Remark 6,4.1 (i) Let X Corollary 6.4.1 l..'l.kf' thf' form

R", the generalized Newton method of

k~kt~o,

(6.4.8)

where Ak(uk) are invertible matrices of order n, and V'F(ul;) denotes the gradient vector of the function F at the point Uk (( R")* is identified with R"). In particular, the original Newton method correi:ipondi:i to

k > 0,

6.4.8a

where the matrix \72 F(UI;) is Hessian of the function F at the point u. (ii) The special case, Ad'll",) = <,0-11, is known as the gmtlient meUlOd wiat fixed lmrometer. (iii) The spf'f'ial case, A,,(Uk') = -op;:l/, is called the gradient method with variable lJaram.efer.

0 . ,>0<1"_""""'''''''10''-1

..... ,

(iv) The special case, Ak(Uk') = -( 'P(Uk) -1/,

i~

called the gmtlient

method wiUt o]JUmalIJammeter, where the number 'P(Uk) (provided it exists) is determined from the condition
(6.4.9)

General Definition of the Gradient Method. Every iterative method for which the point Uk+1 is of the form
'PI; > U,

is called a gmtlient met/wtL [f 'PI; is fixed, it is called a gradient method witll fixed IJarameters, whereas if 'PA' is variable, it is cRlled a gnulietlt methO(I with variablt parameters. Theorem 6.4.:1 Let X = R'l awl the ftmctional F : X ---t R be elliIJtic, that is, there is a positive Clmstant a such that F(x) ~ allxll 2 for all x EX. Then the graflicut meth(l(l with uIJtimal parameter converges.

Remark 6.4.2 (i) The following properties of elliptic functiouals are quite useful (For details. \w refer to Ciarlet [1]):
(R) Let F : X ---t R (X is a Hilbert sp&:e, in pRlticular X = Rn) be strictly convex and coercive. then it sRtisfies the inequality
F(v) - F(u) 2: (\7F(u).v -

ul + ~IIV - uIl 2

for every u.v E X

(6.4.10)

(b) If F is t.wiCA! differentiable, then it is eIlipt.ic if and only if

(\7 2 F(u)w,w) >

allwW

for every wE X

(6.4.11)

(c) A quadratic functional F over R"

F(v) =

~(AV,v) -

(y,v), A is the n x n mRtrix and

(6.4.12)

A = At is elliptic if and only if

(\7 2 F(u)w, w) = (Aw, w)

> >'dlwll 2 for allu, w E

eigell\~.llue of

R",

where

>'.

denotes the smallest

A.

.n". "'_'_""""'''''''10''-''

- (y,v), A; R" -J' (R"r = R n . Since \7J(ud and \7J(Uk_H) are orthogonal and \7J(v) = Av - y, we have
(ii) Let J(v) ==

~(AV,v)

(\7 J(uk+d, \7 J(ukl) == (A(Uk - op(Uk)(Auk - y)) - y, AUk 2

Y) = o.

IIwkil where Wk = AUk - Y = \7J(uk). A (AWk,Wk) single iteration of the method is done as follows;
This implies that op(ukl ==

(i) calculate vector

WI;

= AUk - Y;

(H) calculate the number <p( Uk) = (A

(iii) r-alculate the vector

IIwdl'
Wk,Wk

); and

Theorem 6AA Let F ; R" -J' be a [tmctiorwl which There are two lJOsitive amstants Q and f3 such that

IS

tliffererltiable.

(i) (\7 F(v) - \7F(u),v - u) ~ allv -

uII 2 for all v, U E

nn and Q > 0,

Iii) II \7 F( 0) - \7 F( v) II < Plio - v II loreo',," v, v E R".

Furthennore, let there exist two numbers a and b SItch tJtat

20 o < a < <Pk < b < f32

for p.1Ie,"y k.

Then the gradient metbod with variable lJarumet.er amverges and the amvergence is geomeb'ic in tbe setlSe tbat there exists a amstant 'Y deperuling on 0:, p, a, b such that 1" < 1 anti lIuk - nil < ')'k lluo - uli.

Remm'k 6.4.3 (i) [f F is twice dill'eJ"entiable, then condition (ii) can also he written in the form sup 1IV'2 F(u)1I :S" B. I (ii) In the case of an elliptic quadratic functional F(v) == "2 (Av, v) -(y, v), one iteration of the method takes the form

and it followo from Theorem 6.4.4 that the method is cOllvergent if 0 < Q< <Pk :::; b ~ 2>',/>.;" where >'1 and >." are, respectively, th" least and the largest eigenvalues of the symmetric positive definite matrix A.

.n", '"'_._""""'''''''10''-''

Proof of Thcorell1 6.4.1.

First, we prove that for every integer k 2: I,

~ ~IIF(",_>lII,

II", - a,_>l1
lIuk - noll

< u eqUIvalently Uk E Su(uo)

~ ~lia,

IIF(u<l1l

-",->II

We apply the method of the finite induction for the proof. Let us show that the results are true for k:::: 1; that is,

lIu, - uoll < ~IIF(uo)1I lIa, - "oil ~ n, IIF(u')lI

Putting k :::: 0 in relation (6.4.4), we get

, - aoll iJ"a,
(6.4.13)

a, -Uo ~ -A,;'(ao)F(uo),

which implies that lIu] -uoll < I1I1F(uolll :$ 0(1--1') :$ a by the hypotJleses of the theorem. Further, from (6.4.13), we can write
F(ut} = F(ud - F(uo) - Ao(uo)(u] - uo)

By the Mean Value Theorem applied to the function u we have

~~

F(u) - .4. 0 (110)11,

IIF(u')lI < sup 11F'(a) - Ao(ao)lllIa,

t<ES,,(t<o)

by condition (ii) of the theorem. Let us assume that the desired results are true for the integer k = n - 1. Since Un - Un_I :::: - A;:;-~] ( U(n_l)' ) F( U,,_l), it follows that IlU., - un_lll :$ I1I1F('ll n _1 III which gives the first relation for k:::: n. Then we have

lIu" - a,,_>l1

~ IIA,~', (X(,,_>j' )F(a,,_dll

<

~IIF(a,,_,)1I
l'

~ ~ ~II",,_,

- u,,-,II

This implies that

lIu" - uoll

~ t."u. - ".->11 < {t.,.-,} lIu, - aoll

<
II'll] - Uoll
1 "

<a
,

bl' condition (iii)

which means that Un E Sa(UO)' For proof of the last relation, we wnte

By applying the f"lean Value 'Theorem to the function x(U(n_ll')U, we get

U -4

F(u) -

A(n_lj

and the last relation is established for n. Hence, these three relations are true for all integral '{'<tItles of k. \\le now prove the existence of a zero of the functional F in the ball 5 0 (110)' Sincp,

where {ud is a Cauchy sequence of pomts in the ball So(uo) which is a closed subspace of a complete metric space X (X, a Banach space). This implies that there exists a point u E So(uo) such that lim
" ..... 00

Uk

== U.

Since F is differentiable and therefore continuous, we get

which, in turn, implies F(u) == 0 by the first axiom of th~ norm. By taking the limit m ---t
00

in (6.4.14), \\'e find that lIuk -

ull

-/ lIul - uo!l 1 -,

is

the desired result concerning geometric convergence. Finally, we show that u is uuicjue. let v be another zero of F; that is, F(v) == O. Since F(u) == F(v) == 0,

B.

......

v - u == -A
, to<l',_"""",,,,,"l1>"-I

o (F(u) 1

F(v) - Ao(uo)(v - u)),

from which it follows that

IIv - "II <: IlAO'(uo)1I

which implies that
11

"ES,,(uo)

"'I'

IIF'(v) - Ao("o)lIl1v - "II < >llv - "II, I

as 'Y

< 1.

Proof of Theorell1 6.4.2. a and {3 such that

(i) First, we show the existence of constants

a> 0, So(u) ~ {x E X/llx

and sup sup
I;?OxES,,(l,j

-"II <: a)

C U,

(li.4.15)

111- A;l F' (x)1I <,' < 1.

(6.4.16)

For every integer k, we can write AI; = A +A-I (AI;- A)) with If A -I (AI;A)II < ), < 1 in vicw of a condition of the theorem. Thus, AI,; are isomorphisms from X onto Y and mOI"(.-. '()Vet,

(I

II A;'II ~ II(A(I+A-'(A,-AW'II
<: 11(1 +

A-'(A, - AW' IIIW'II <: I~A-;I

by Theown 2.6.3 and

11(/ + B)-' II <:

;-1---i;1I"8"'1I

This implies that

II/ A;' All

IIA;' A, - A;' All < IIA;'III1A, - All

fo,), <

-1'\ <: II A, III1A 'II IIA-' II ), <: I ), IIA 'II '

II/ - A;' All <:

Let 0 be snch that {3'

.\

),

={3'<i.

< {3' + 0 =

,-

< 1 This implies that

1"(,,)) II

II/ - A;' 1"(")11 <: III - A;' All + IIA;' (A -

B.

.... ,

, to<l',_"""",,,,,"l1>"-I

from which, (6.4.15) and (6.4.16) follow immediately keeping in mind the continuity of the derivath'e F' and the fact that A = F'(u). (ii) Let 110 be any point of the ball 5(u) and {Uk} be the sequence defined by Uk+l = Uk - A;l P(11k); each of these elements lies in 5(11). This implies that {11d is well defined. Since P(u) = 0, we haye
11k+l - 11 = Uk - A;l P(11k) - (11 - A;l P(u)).
Oy the t<,'Iean Value Theorem applied to the function. x -t x - A;I P(x)

shows that

"UA'+1 - ulI::;

sup
;reS,,(t,)

III -

A;l F'(x)JIII11k - 1111

< 1'lIuk - uli.

By (fl.4.lfll and continuing in this way, we get.

lIuk+l - 1111 ::; '1,k- 1 1l 111 - ull,

which is the geometric conyergente. This relation also implies that Uk -t 11 ask-tooas'1'<L (iii) The lI.ero of F, point u, is unique. For this, let v be another point such that F(v) = O. The sequenCf> {ud corresponding to 110 = v is a stationary sequence, since Ul = U(l - A0 1 F(U(l) = 110 and, on the other hand, it. conv"':,ges to the point 11 by the above discussion. This implies 11 = v. I For a proof of Theorem 6.4.3. we refer to Ciarlet II, pp. 300-301): here. Wt: prove Theorem 6.4.4. Proof of Theorem 6.4.4. In a gradiem method with variable parameter, we have Uk+l = Uk -If'k\7F(11k)' Since \7F(u) = 0 for a minima at u, we can write Uk+l - U = (Uk - u) - If'd \7 F(ud - \7 F(u)}. This implies that I[uk-t-l - ull 2 = [luk - ull 2 - 2lf'k < \7 F(ud - \7 F( u), Uk - 11 > +If'~I1\7 F(Uk) - \7P(u)11 2 < {I- 2G'lf'k.+ (32lf'I} IIUk - u1l 2 under the condition that ({Jk > O. If 20 o < G' ::; ({Jk ::; Ii < (P' then

andw

lI11k+l - ull ::; 1'lIuk - ull ::; 1'''+llIu - uoll,

where l' < 1 which depend:; on I convergence of {Uk}'
G',

a, band {3. This also implies the geometric

Q.

......

, to<l'._..

....,..",,"l1>"-I

6.4.2

Conjugate Gradient Method

The conjugate gradiem method deals with the mlllimizalion of the quadrauc functional on X:::: R"i that is,

J; vERn
0'

-Jo

~(Av,v)

- (b,v),

I J(v) = 2(Av,v) - (b,u),

where A is the n x n matrix. Starting wil.h an lIlitial arbitrary vector 'Uo, we set do:::: \7J(110). If \7J(U()):::: 0, the algorithm tenuinates. Otherwise, we define the number
1'0 ::::

("7J(vo),'1o) (Ado. do)

then the yector

111

is given by
111 :::: 110 -

1'odo,

Assuming that the vectors 111,d l , ... , Uk_I, (h-I,111; haye been constructed which assumes that the gradient vectors \7J(11(), 0::; k- I are all non-zero, one oCthe two situations will previlll: \7J(11I;) :::: 0 and the process terminates. or \7 J(11k) -# O. in which case we define the vector

e::;

d, = "7J(",) +

II~J(Uk-l )11'1k-l ' y l

II"7J(v')II'

then the numbers

1'"

and

111;+1

are giYen by
("7J(u,),d,) (A(h,dd

and

respectively. This elegant algorithm was developed by Hestenes and Stiefel In 1952. This method COIl\'rges in at most n iterations. The study of the conjugate gradient method for non-quadratic function on Rn into R began in sixties. Details of these methods and their comparative merits may be found in PO)flk [15J and other references [16, 171 We

",,,. "'_'_""""'''''''10''-''

present here the essential ingredients of these two best methods, namely, Fletcher-Reeves (FR) and Polak-Ribiere (PR). Let F . R" --+ R; we look for inf F( v) where F is twice differentiable. The point at which inf F(v) is attained will be denoted by arginf(:r.),
"ER"

"En"

Starting with an arbitrary "ector 110, one assumes the vectors UI, 112,., , ,Uk to have been constructed, which means that the gradient vectors Y'F(Ui), o <i < n - I, are non-zero. In such situations, either Y'F(11,,) = 0 and the algorithm terminates, or Y'P(11,,) f- 0, in which case, vectors Un+1 are defined (if they exist and fire unique) by the relations

U"-1-1 =

11" -

1'"d" and P(1J.,,+d

=.:

inf F(u" - pd,,),

'ER

the successive descent direct,ions d; being defined by the recurrence relation

do = Y'F(110)

J-'ilF(
(, 11,

('ilF(u,),'ilF(u,j-'ilF(u,_,.

IIY'F(u;_dIl 2

1:4-1,

1',

('ilF(u,), 'ilF(u,) - 'ilF(u, ,)

II'ilF(u,_dll'

is called the Polak-IUbiere formula and in this case the conjugate gradient method is called the Polak-RibiC1oe conjugate gradient method and one denotes 1'; hv 1>;R. The case

11'il F(u;) II' II'ilF(u,_dll'

is cfliled t.he Pletcher-Iiee1lCs formula and the corresponding method is called the Fletche1'-Reeves conjugate !J1'adient meUIOd. Such 1', is denoted by 1{R, It may be observed that the Polak-IUbiere conjugate gradient method is more efficient in practice.
PoJak-Ribiere conjugate w'adient algorithm:

Data, U(J E R" Step O. Set i =.: 0, do =.: Y'Pl110), and Ito = -doStep L Compute the step sizf' Ai =.: arg inf P( u, + All;), Step 2, ~lpdate: Set 11'-1-1 rli-l- I = V' F(11i+d l'f R =.: (di-l- 1 ' rl q . 1 - d,)

=.:

11i

+ >.;11;

11<411'

",,,. "'_'_""""'''''''10''-''

hHI == -di-t-I + 1{ fi h. Step 3. Replace i by i + 1 and go to Step I.

Polak-Reeves conjugate gradient algorithm:

Data. 'Uo E R" Step O. Set i == 0, do == V"F('Uo) and ho == Step l. Compute the step size Ai == arginfF(ui + Ah.)
.\<':0

-(10

Step 2, Update: Set rli-f-I == v P( u,+d

U.-1-l

==

Ui

+ A;h;

r.

Ffi

hi-l- I

== IlrliIFl == -(-4-1-1 + 1';Ffi

IIrl'-H Ir'

St.ep 3. Replace i by i: --\- I and go to Step L

6.5

ProblAms
If / E C[a,b] and

Pmblem 6.5.1

lb

j(t)h(t)rlt

== 0 for all hE CI[a,b]

with h(a) == h(b) == 0, t.hen prove that / ==

o.

{x,,}

Pmblem 6.5.2 Let X be a Hilbert space, !( a convex subset of X and a sequence in such that lim IIx,,1I == inf 11.1:11. Show th"t {x,,}

converges in X. Gi\'e an Illustrative example in R2 .

" .....=

:rEX

P7YJblem 6.5.3

Let K == {x==

(XI,X2,'.'

,x"l/i;x.
J{.

= I} be a sub-

set. of R". Find a \'ectur uf mini/nUn! Tlorm in

Pmblem 6.5.4 Let X be a normed space and K a subset. of X. An element. x E J( is called a best appmximation to an arbitrary element x E X if

d(x, 1<) ~ inf 1111 yEK

xii

IIx - xII

The approximation problem is a special type of optimization problem which deals with minimizing a translate of the norm function, or if x = 0, it deals

",,,. '.'_'_""""'''''''10''-''

with minimizing the norm function itself. Let J{ be a finite-dimensional closed subset of a normed space X. Show that every point of X has a best approximation.
Pmblem 6.5.5 inner product

Let X = C[-1f,1f] be an inner product space with the

(f,g)

f(x)g(x)dx,

and

J{

be the subspace spanned by the orthonormal set

{ .J2ii' .;IT cos x, J1f sin x, .

Find the best approximation to:
l. f(x) = x

II

,In

COS'IlX,

~ sin nx}.

2. f(x) = Ixl
Pt'Oblem 6.5.6

Show that for each

I E C[a,b], there exists a polynomial

P,,(t) of maximum degree n such that for every 9 E }, = SP;Ul{ lo(t), II (i),

... , f"(l)}. f,(l) = t'. on"" If(t) - Pn(t)1

a<t<b - -

< a<t<b If(t) - g(t)l on""

--

Pmblem 6.5.7 Let m > n > 0 and A = la.;J)' i = 1,2, ... ,m,i 1,2, ... ,n and 11 E R"'. Then write down the solution of optimilmtion problem for the function F(x) = II Ax - ull oyer R". Pt'Oblem 6.5.8 Let Ax = y, where A is an n x n matrix and:r <lnd yare elements of R U \Vrite down a sequence of approximate solutions of this equation and examine its convergence. Under what condition on A. this equation has necessarily the unique solution? Pmblem 6.5.9 Explain the concept of the steepest descent and apply it to study t,he optimization of thf' functional F defined on a Hilbert space H as follows:
P(x) = (A.x,x) - 2(y,x),

Q.

.... ,

, to<l'._..

....,..",,"l1>"-I

where A is a self-adjoint positive definite operate on H, x,1I E H. For any Xl E H, construct a sequence {x n } where
x'H-l

X"

+ (A z,,, z" ) 'n,

(z", z,,)

for appropriately cho.<;en z" and show that {x,,} com'ergf'S to Xo in H which is the unique solution of Ax = y. rllrth~rmore, show by defining

F(x) = (A(x-xo),x-xo)
that the rate of convergence satisfies

P7'Obiem 6.5.10 bundle concept. Pt'Oblem 6.5.11 tion. P7'Obiem 6.5.12

Explain the non-smooth optimill.ation problem and the

Discuss Newton's method for non-smooth optimizR-

Proye Equation (6.2.2) .

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

References
[IJ PG Ciarlet. Introduction to Numerical Linear Algebra and OptimizRtion. Cambridge: Cambridge Universit.y Press, 1989.

[21 FH Clarke, YuS Ledyaev, RJ Stern, PR Wolenski. Nonsmooth Analysis and Control Theory. Berlin, New York: Springer-Verlag, 1998.
[3] FH Clarke. Optimizfltion and Non-smooth Analysis. New York: John Wiley and Sons. 19R3. [41 An. COHIl, NIM Gould, Ph LT Toint. Trust-region methods. Philadelphia: SIAM, 2000. [5] RF Curtain, AJ Pritchard. Functional Analysis in Modern Applied tvlathemfltics. London, New York, San Francisco: Academic Press, 1977 [61 I EkeJand, R Temam. Convex Analysis and Variational Problems, Classics in Applied Mathematics. Philfldelphia: (first published in 1976, North I-Iolland and American Elsevier), SIAt-,'l, 1999.

[7] N[M Gould, PL Toint. SQP methods for large-scale nonlinear programming. In: MJP Powel, S Scholtes, 00:;. System Modeling and Optimization: Methods, Theory and Applications. Boston, Dordrectht, London. Kluwer Academic Publishers, 2000, pp 150-178.

[8] JB HJrlart-Urruty, C Lemankha1. Convex AnalySIS and Minlllllzation

Algorithms. Berlin-Heidelberg: Springer-Verlag, 1993.

[9] CT Kelley. Iterative Methods for Linear and Nonlinear Equations.

Philadelphia: SIAM, 1995

[10] DG Luenbergcr. Optimization by Vector Space Methods. New York:

",,,. '.'_'_""""'''''''10''-''

John Wiley, 1978. [II] NM Makela, P Neittaan maki. Nonsmooth Optimization. Singapore; World Scientific, 1992 [12] JJ rvlore, S.J. Wright. Optimization Software Guide. Philadelphia; SIAM, 1993. [13] H Nelln'l.f'rt, AH Siddiqi. Topics in Industrial Mathematics-Case Studies and Related Mathematical Methods. Boston, Dordrecht, London: Kluwer Academic Publishers, 2000.
[14 J J Outrata, 1'",1 Kocvra, J Zowe. Nonsmooth Approach to Optimization Problems with Equilibrium Constraints: Theory. Applications and Numerical Results. Boston, Dordrecht, Lundon: Kluwer Academic Publishf'rs, 1998. [15J E Polak. Optimization, Algorithms and Consistent Approximations. New York-Berlin: Springer, 1997.

[16, BT Polyak. Introduction to Optimization. New York; Optimization Software Inc., PlIblicfltions Division, 1987.

1l7] MJD Powell. Convergence properties of algorithms for nonlinear optimization. SIAtvl Review 28:487-500, 1986. [i8] RT Rockafellar. Com'ex Analysis. N.J.: Prmcewn Mathematical Series, Princeton University Press, 1970. [19] RT Rockafellar. The Theory of Subgradients and its Applications to Problems of Optimization: Convex and Non-convex Functions. Berlin; Helderman Verlag, 1981.
[20J AU SiddIqi, Functional Analysis wIth ApplicatIOns. New-Delhi, New York: Fourth Reprint, 1993. Tata McGraw Hill.

",,,. "'_'_""""'''''''10''-''

Chapter 7

Operator Equations and Variational Methods

7.1 Introduction

This chapler is devoted to formulation of real-world problems in terms of operator equations and their equivalence with variational problems. Existence and uniqueness of ~olutiolls of tmch lIlodels are pr~nted. Appruximation methods such as Galerkin and Ritz af(' discussed.

7.2

Boundary Value Problems

Let. 0 be a bounded or unbounded region of R" (in application, n = 1,2,3), r or 8n be its boundary, Land 5 be linear differential operators, u(x), :r = (Xl, X2, ,x,,) E R!' be a function on flU belonging to a Hilbert or Banach space of functions. <Ind ,((x) and g(x) be given functions of x. Then Lu(x) = f(x)
Su(x) = g(x)
in

(7.2.1) 17.2.2)

on L'

is called a linear boundary value problem (BVP) for u. u is an unknown

function which is called a solution. The boundary value problems of the type

LU=f(Xj,u,~,... )
Su=g(x,)
onf, are called rI01-uinem' boundmy value pmblems.

inn

(7.2.3) (7.2.4)

j=1,2,3, ...

",,,. "'_'_""""'''''''10''-''

We shall see that these problems can be expressed in the form of finding solution of the abstract variation problem; Find u E H (H is a Hilbert space) such that
a(1t, v) = P(v)

for all v E H,

(7.2.5)

where a(,) is a bilinear form with flppropriflte conditions find FE HV. Boundary value problems can also be written in the form of the operator equation
Au =
ti,

(7.2.6)

when: A is a linear or nonlinear operator on a Hilbert or Banach space into another one of such spaces. The La.x~rvlilgram L.emma provides existence and uniqueness of solutions of (7.2.5) and (7.2.6) under certain conditions. We present in Theorem 7.3.5, a more general result. Theorem 7.3.1 gives the equivalence of the problems gh'en by Equations (7.2.5) and (7.2.6) to the optimill.at.ion problem for the energy functional (Sect.ion 6.3.3).

Example 7.2.1

Consider t.he BVP

-L~1.1l =
U

2 (82U+ 8U) 8x
2 &J2

in fl

(7.2.7) (7.2.81

== 0

on

r,

where fl C R2 .

8'(-) In this example, L = .6.. = 8x 2

8(-)

8y2 =

Laplacian operator

S = T,

9 = O.

This is known as thf' Di1'1chlet bounda1'1j value p1"Oblem. The equation in (7.2.7) is called Poisson's equat.ion. By the classical solution ofthis BVP, we mean the function U(X,lI) which is continuous in t.he closed domain fl, sat.isfies (7.2.7) in the open domain fl and is equal to lI.ero 011 the boundary r. By assumption f E C(fl), thp solution u E C 2 (fl), thp space of continuous functions with continuous partial derivatives up to second-order inclusive, and equals zero on r. The set VA of these admissible functions u=Oonr}

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

forms a vector space. If the boundary condition (7.2.8) is nonhomogeneous, that is, u = g(x), g(x) =F 0, then VA is not a vector space. tn one-dimension, the boundary value problem, (7.2.7)-(7.2.8) takes the form

- dx 2 = I(x)
u(a) = u(6) = O.

<fu

in (a,b)

(7.2.9) (7.2.10)

Example 7.2.2

The BVP of the type

-.6.u = iJu

on on

(7.2.11) (7.2.12)

an =

r.

where :

denotes the derivative of u in tJle direction of outward normal t.o

the boundary,

r,

is called the linear' non-homogeneous Neumann BVP. The BVP

Example 7.2.3

-.6.u =
u=O
-

on

onr, on r2,

(7.'.13) (7.2.14) (7.2.15)

uu = g an

where r = r 1 U r 2 is known as the mixed BVP of Dirichlet and Neumann. Dirichlet and NeumaJlIl BVPs are of elliptic type.

Example 7.2.4

The BVP of the type

au + Lu = 1 at
Su

on Or = (0, T) x 0

(7.2.16) (7.2.17) (7.2.18)

=g

on [.1'

= (0, T) x r

u(O.x)='Uo

onO.

where 1 = I(l,x), g = g(t,x) and 'Uo = uo(x) is called the initial-boundary val1te problem 01 pambolic type. [f we choose L = -.6., S = I,g = 0 and 1 = 0, then we get the heat

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

equation

au -a - flu = t
11(0, x) = tiO'

on [O,T] x fi

(7.2.19) (7.2.20) (7.2.21)

1J=00n[0,Tl x afi

The one-dimensional heat equation wIth lIlitial-boundary conditIOns is gi ven os

ax2 O<x<o a" u(O, t) = ox (0, t.) = for t > 0 at.

lI(X, 0) :=::}{

all

0 211

for

<

x < O.

Example 7.2.5

Hyperbolic Equations with Initial-Boundary Conditions:

t

au_o (,)au +aax- , at

> 0, x
x E

E::

R, where a E R\ (OJ

(a) 11(0,X) = 1I0(X), (b) lI(l,O):=::lf'I(t)

n,
u(t., 0) = If'2(t)

ifa>O,

If a < 0

wherp If'\ and If'z are given functions.

(i i)
dllt,;J u(O. x)
:=::

>

0,

x E R,

(wave equation) with initial

111

1I0(X)

""d

au -a (O,X) = t
0,

(x)

(iii)

au at

a ox F(u)

== 0,

t>

x E

n,

where F(u)

18

a nonlinear

function of 1J, and u(O, x) = 1IotX).

A special case
alJ {hI -+,,-~O

at

ox

is Burger's equation .

......
Q,O'>o<I"_""""'''''''IO''-I

7.3
7.3.1

Operator Equations and Solvability Conditions

Equivalence of Operator Equation a,nd AJinimizatlOlJ Problem

LI. T be a posith-e operator on its oomain VT and y an element. of II Hilhert space fI, then the solution u of the operator equation

Tu = y

(7.3.11

can be shown to be the element at which the energy functional (Section 6.3.3) J attains it.s minimum. Conversely, if we can find the element 11 at which the energy functional altains its minimum, then u is the solution of (7.3.1). In fact, we have the following theorem:
Them'ern 7.3.1

Let T: V T C JI --+ ll, where fl M a Hi/be,"! space, be self-adjoint and positive operator on DT and let y E fl. Then the ennyy functional
1 J(lI) = '2(Tu,u} - (y,lI)

(7.3.2)
i:;

(lssumeJi its minimYtm value at x E VT if and only if x

(?J.J).

a sohttion oj

Proof. have

Let x be the solution of EquatIon (7.3.1). Then Tx = y and we

J(lI) = Z(Tu,u) - (Tx.lI)

= H{T(u-X),lJ)-{TX,u)].

Since T is self-adjolllt, we can write

J(lI) =

(T(v - x), u) - (Ttl, x)

+ (Tx, x) -

(Tx, x) J

=
=

~[{T(u-x),u)-{TU-Tx,X)-{Tx,x)J

~[(T(u-x),U-X)-{Tx,x)J
+ Z{T(ll - x),u - x).
1

= J(x)

Since T is positive, we have

(T(u - x),1J - x)

>0

for every

11

E Dr

= 0 if and only if u - x = 0 in Dr,

...... "'_'_""""'''''''10''-''

from which we conclude that

J(,,)

> J(x),

where the equality holds if and only if 11 == x. Inequality (7.3.3) implies that the quadratic functional J(11) assumes its minimum value at thp solution x E Dr of Equation (7.3.1); any othf'1' 11 E Dr makes J(u) larger than J(x). To prove the converse, let J(1J) assume its minimum value at x E Dr. This means that for arbitrary u E Dr, J(u) > J(x). In particular, for 11 == X + nv, v E Dr and 0: a real number, we have

J(x
By t.he definition of J,
WI'

+ au) "J(x),
+ nv), x + nv} - 2(y. x + nv).

have

2J(x + o:v) :::: (T(x

2J(x + o:v) == (Tx,x) +20:(Tx,v} +0:2(7v,v) - 20:\y, v) - 2(y,x). (UA)

Since x E DT and y E fl are fixed elements, it is clear from Equation (7.3.4) that for fixed v E Dr, J(x + o:v) is a quadratic function of 0:. Then J(x + o:v) has a minimum Dr at 0: == 0 (that is, J has a minimum at x) if the first derivative of J(x + av) with respect to 0: is zero at 0::::: O. We havf'
[ -ld J(x + aUl] 0=0 (0:

~0

or 2(Tx, v) - 2{y, v} == 0 or (Tx - y,v) = 0 for all v E Dr.

In particular, (Tx - y, Tx y) = 0 (choose v = Tx - y). This implies that Tx - y == 0 or Tx = y. Thus, we have proved that the element x E Dr that minimizes the energy functional is a solution of Equation (7.3.1) I

Remark 7.3.1 (i) Theorem 7.3.1 is of vital importance as it establishes all equivalence between the operator equation Au = y in Dr with the minimum of the energy (quadratic) functional. This variational formulation, thai. is, expressing thp operator pquation as thp problem of minimizing an energy funct.ional, facilitates the analysis, such as establishing existence

Q.

......

, to<l',_"""",,,,,"l1>"-I

and uniquen:8s results, and leads to the determination of approximate solu tions. Theorem 6.3.2 provides existence and uniqueness of the minimization problem of the energy functionals where T is associated with a coercive and symmetric bilinear form a(u, v) via Theorem 3.9.2. Hence. for such aT. Operator Equation (7.3.1) has a unique solutioll. Oi) By t.lw Lax-Milgram Lmma (Theorem 3.9.4), Operator Equation (7.3.1) has a unique solution if T is associated with a(,) by Theorem 3.9.2. (iii) Algorithms of Chapter 6 can be used to solve Equation (7.3.1).

7.3.2 Solvability Cunditiuns

Let. T be a lillPar operator on a domain D-r dense in a Hilbert space H. \Ve would like w know for what data y, we can wlve the nOIl-homogeneuus Equation (7.3.1), namely,

Tu=y
The existence of solution t.o (7.3.1) mainly depends on the solution of the homogeneous adjoint equation

(7.3.5)
Theorem 7.3.2 Let T be a closed bounded below linetlr opemtor" on flilbeft. space ll. Then Equation (7.3.1) has solutions if and only iJ
(t

R(T) ~ N(T")'-;

(7.3.6)

that is. the range of the operator' T i.~ equal to the orthogonal complement oj tlw null space oj tlu; adjmnt operntor To oj T (Definitions 2.4.1(8),
(3.3.1)(3) and (3.S.1)).

Proof. We first prove that the range of closed, bounded below operator is closed, that is, R(T) _ H(T). Let Yn E R(T) with y., --+ y and Tv" = Yn' We wish to show that y E R(T). Define 11" = v., Pv." where P is the projection operator from Dr onto N(T). Then 11., E Dr n N(T)l. and TUn = Yn' Since T is bounded below, we have

1111.. - umll < -IlT(lI" - um)1/ = -llYn - Ymll, c c

which shows that {u .. } is a Cauchy sequence in Dr and therefore u" --+ u in Ji Since T is closed, Ttl" --+ y and Un --+ 11 implies that 11 E V-r and TlI = y. Hence, R(T) is closed.

...... "'_'_""""'''''''10''--'

Now, we show that (7.3.6) is equivdlent to

(y,v)::::: 0

for all v E N(T*),

(7.3.7)

where y is data in (7.3.1). Since R(T) is closed, n(T).!.! ::::: R(T) ::::: N(T*).! or R(T)J. ::::: N(T*). This implies that {Tu, v)::::: (1I, T*v) ::::: 0 for all 11 E V r and v E fll(T*). This shows the equivalence of (7.3.6) and (7.3.7). Let y E R(T). Then there exists an element 11 E Vr such that TlI ::::: y. Since T hi bounded below, T hm; 11 continuous inverse T- 1 defined on its range, and therefore 11::::: r-ly. MorPf)ver, we haW'

To prove the oonvp.1"SP-, lp-t u::::: T-1y. Hence. Tu::::: y and (Tu,v) ::::: (y.v) for aJly v E neT). By the definition of TO, {T1J, v) :;::: (v.,Tv}::::: 0 for aJly v E N(T*). In other '\'ords, u E N(TO)J..

The Lax.-Milgram Lemma (Theorem 3.9.4) can be presented in the following form. Them'em 7.3.3 Let T : H --+ H be a linea,' bmmded opemtm' on a Hilbert spnce H into itself. Suppose tlUlt the7'f~ is n comtwflt a > 0 such that

that is, T is st''01lgly monotone (elliptic y E f/, the opemtor equation

01'

coercive). Then lor each given

uEH
lUIS

n unique solution.

We prove here the following generalization of the Lax-Milgram Lemma: Theorem 7.3.4 Let T : H ---t JIo be a linear" bounded opemtor 011 the real Hilbert space H. Suppose that T is stmngly monotone, that is, there is a constant a > 0 such that
(Tv.,lI)

Q . ,>0<1"_""""'''''''IO''-I

.... ,

> allull 2

1m> nil 11 E H .

Then, f01' each given

E JI*, the operator eqtudion

Tu =

I,

11

E 11

hn:> a lIniqlle .solution.

Proof. By the Riesz Representation Theorem (Theorem 3.7.1), there is a element w E 11 such that

(Ttl, v) = (w, v)

for all v E If,

and IIwil = IlTitll. It IS dear that (TlI, v), Tu E H*, denotes the value of the functional TlI E fl* at the point v. Let 511 = w. Then

115,,11 ~ IIT"II

IITllllnll

If we consider 5u for <1.1111, then 5 . fl --t JI is linear and bounded. Moreover, we have

(511,u) = (TtI,u)? lIull 2

for

aJ111

E H.

(5U,11) is reaJ as fl is a real Hilbert space. Again, by the Riesz Represen-

tation Theorem, there exists an element y E H such that

(/.v) = (y,v)
The operator equation TlI =

forallvEJI.

can be wliHen as

(T",v)
or equivalently

U,v)

for all v Elf;

(5", v) =

U, v)
I

for all v E 11.

511 =

f E II,

which has a unique solution by Theorem 7.3.3.

I
Lellllllil

The following geut'xalizations of the Lax-Milgram proved [15,pp.174-175]:

have been

Theorem 7.3.5 Let T : Dr ~ JI ----lo H be a linem opemt.m on the Hilbert space H over the field of real 01 complex numbers. Then 101' etldt y E 1-1, the opemt.or equation
Tu=V
tI

E fl

...... '.'_'_""""'''''''10''-''

hn:> at most one solution in case one of the following five conditions is s(ltisjied: (a) Strict monot.onicity (positivity): Re(Tll, u)

>0

for (lUll E Dr with u =F O.

(b) A pn071. estimate (stabilif.y):

.B1I1I11 < IITlIlJ

(c) Contractivity of (T - J): IITu - ul!

f07 all

E Dr and fixed .B > 0,

< 1I1J II

f07' all u E 'Dr with u ;f;. O.

(d) Monotone type: With respect to an oHle,' cone on ll,

(e) Duality: There exists an opemtor S : Ds <;; f/ --t 1-1 with R(S) = H and (Tu,v) = (lJ,SV) f07" all u E DT,ll E Ds,
COI'ollary 7.3.1

Let T : DT <;;:; H --t f/ be a linetU' operat07' on the Hilbert space H, where Dr and Dr' are dense in f/, Then the following two statement,s (I~ equivalent: (a) For each y E H, the equatIOn Tu = y has at most one solution, (b) R(T) is dense in lJ,. that is, R(T*) = f/

Equation Tu = y, when> T : DT <;;:; f/ --t H is an operat.or on the Hilbert space 11, lUIs at most one solution if the operator T: Dr <;;:; II --t H is strictly monotone (elliptic) (lnd Re(Tu-Tv,lI-v) > 0 1m' all u, v E Dr with u =F v.
Theorem 7.3.6

7,3.3

Existence Theorem for Nonlinear Operawrs

Theorem 7.3.7

Let II be (t Hilbe,t space and T (In operator on I1 mto itsell stlt.isfying the conditions

IITlI - Tvllu < >'Ilu - vllu

rr Lipschitz continuous)

(7.3.8)

and

{Til - Tv, u - v)u 2':

when~

'-diu -

vll~

(stnmgly monotone),

(7.3.9)

>. > 0,

1-'

> 0 and ..\ > 11-. Then the equation

(7.3.10)

has exactly oue solution for evef1/ y E fl.

P1'OQj. S,u:= U

Lt Y E fl, and to > O. Let the operator 5, b(' defined as (Tu - y). For any V'l v E lJ, we have

115,11 - S,v1l 2 := 1111 -

vIIi, - 'l(TlI- Tv,1l 2j.I.

v)

+ f,zIlTu -

TVII},

:5 (l Thus, if we choose
(;:2 ).2)1/2,

+ (2 >.2)1111

vll~f'

>

0 so that

< ~

and if ,\"e take

0: :=

(1 - 2J1,

then t.he operator 5, is a comraction with consta.nt 0: < 1. By the BaJlach contraction mapping theorem, there exists exactly one. solution of the Operator Equation (7.3.10). I

It may be observed that ThPOrf'IU 7.3.3 has been provoo in a general form by F.E. Browder (see, for example, [5, p. 243]) where Hilbert space

is replaced by reflexive Banach space, Lipschitz continuity is replaced by a weaker condition, namely, demicontinuity and monotonicity coercivity is replaced by strong monotonicity.

7.4

Existence of Solutions of DiridJiet and Neumann Boundary Value Problems

One-dimensional Di['iehlet P['oblern: The Dirichlet problem in onedimension is as follows' Find u such that
1/'=-j or

d2 u -dx2 ==j

O<x<l

u(O) == 11(1) = O.
A variational formulation (Equation (7.2.5)) of this problem is

a(u, v) == F(v)

for v E flJ(O, 1),

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

where a(u,v) =

1
1

u'(x)v'(x)dx and

Pte) = 1'V(X)/(X)<1T

= HJ(O.I)

(See Section 5.4.2. m

= 1.0 = (0, I), an = {O.l}).

, denote; the first derivative while" denOl.es the second derivative. Multiply both sidCh of th~ given equation by vex) and apply integration by parts and the boundary condition 1/(0) = u(l) = 0 to get

Verification.

which is the deslfed equation for v E [fJ(O,I). This equation has unique solution if a(,) is bilinear, bounded, and coercive and F is linear and bounded.
a(u + w. v) =

= a(u, v)
J

a(au,v)

l' =1
1

(u' + w')v'dx =

1
1

u'v'dx +

1
1

W'1J'dx

+ a(w, v)

au'v'dx=a !l/v'dx

= cW(lI,v)
1/2

la(lJ, v)1 <

1 lu'(:r.)v'(x)ldx :5 (111/(X)12dx) lIZ (lIIV'(X)IZdx)

(By the CSB).
by Hem",k

S Mllullllvil

5.4.30).

Thus, a(-,) is bilinp.ar ;lnd bounder!.

la(u,u)1 = 11IU'(X)12dx > allulllfJ'

a

>0

by the Poincn.re Inequality (see Remark 5.4.3 (i)).

Therefore, a(,) is coercive and by the Lax Milgram Lemma. the onedimensional Dirichlet problem has a unique solution. Since a(u,v) = a(v, u); that is, a(lI, v) is symmptric, the solution of this is equivalent tn the solution of the minimization problem for the energy functional J(u) =

~a(lI,u) -

F(u) =

1
1

(u(x))Zdx

-1

f(x)u(x)d:r:.

...... "'_'_""""'''''''10''-''

Algorithms of Chapter 6 can also be applied, if necessary, to find a solution to the minimization problem and consequently to the original problem. The n-dimensional Dirichlet Problem. Finding u defined on 0 such that
..6.u = I in 0 u=Oonr;

(7.4.1) (7.4.2)

i.e., finding the solution of the linear "'lIiptic BVP or Dirichlet problem is equivalent to determining the solution u of the variational problem
a(u,v)
~

L(v),

(7.4.3)

for all v E l1J (0),

f!u /Jv a(u,v) = ' " ~~dx, L... ..-. uX' uX' ;=1"
and

"1

(7.4.4)

L(v) = llvdX.

(7.4.5)

Ve7'ification. Let I E L2 (O), u E li 2(O). Multiplying Equation (7.4.1) by a function v E liJ(O) and by integrating the resulting equation. we have

-k
By Theorem 5.4.7, we have
-

..6.uv dx =

l Iv
x;

dx.

(7.4.6)

n..6.u1Idx= - "

"18' a L 1ax. {)x.

~
fl
n

~vdx 2

. .=1

fl

f!u d " 8" - -/Jv x - " v-ds. ~ r an 0=1

Since v E liMO), vir

= 'l'v = 0 and consequently

Q.

.... ,

, toOl',_"""",,,,,"l1>"-I

Therefore, we gel

1.

ro ..

6.uvdx =

"J&n&v L ro - - d x . ox- oxi=l..

(7.4.7)

By Equations (7.4.6) and (7.4.7), we obtain Equation (7.4.3) where a(',') and L are given by Equations (7.4.4) and (7.4.5), respectively. Existence of the sohltion of Equation (7.4.3): Since

a(u,v) = '\' n " " d x L uX I' uX t' .=1

= '"'

"1.&u&V
"

t=l

"1.&u&u = a(v,u), ~~dx nuX uX

a(, .) is symmetric.
a(u,u):::

"1.&u&u L ro ux. uX, ~~dx

1=1 ..

2':

kllltlllfl~(fl)

(by Theorem 5.4.8,

S("e

also Remark 5.4.3 ).

Thus, a(,) is coercive. We have

L(v]

+ V2)

= k

f(VI

+ v2)dx

= kfVldX+ kfV2dX=L(Vd+L(V2)

L()'v) =

1.

!(),v)dx = ),

1.

!v(dx)

= ),L(v),

where), is a scalar. Also,

by the CSB inequalit". Since

>I'

L,(rl) 11111 = (1.lfI'dx)

S k,

k>O ,

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

and

we get
IL(v)1

< kllvIlHJ(o}
By Theorem 3.9.4,

Thus, L is a bounded lincar functional on HJ(O). Equation (7.4.3) has a unique solution.

It can be showlI on the lines of Dirichlet boundary value problem that the solution tI of the BVP g1\"en in Example 7.2.2 is the solution of the variation problem
a(u, v)
~

L(v)

(7.4.8)

a(u,v) ~

L(v) =
and vice versa.

n1f!uOv L nUx, o_dx -,;;=1

!1Jdx

(foE,

(7.4.9)

gvdr.

7.5
7.5.1

Appmximation Method [or Operator Equations

CaJerkin fdetllUd

Let us consider the operator equation

Ttl = y
tI

E If

(7.5.1)

together with the Galel'kin

equation~

unEfl,

n;:::1,2, .. ,

(7.5.2)

Ii is a real separable infinite-dimensional Hilbert space, If" = span{wl,w2, .. ' ,wn }, {wIll is a basis in Ii and P,,: 1-(-+ H" is the orthogonal projection operator from H into U". Since POl is self-adjoint, Equation (7.5.2) is equivalent to

where

tI"

E l-f",

j = 1,2,3, ... ,n.

(7.5.3)

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

Definition 7.5.1 For gi ven y E 11, Equation (7.5.1) is called uniquely approximation-solvable if the following conclitions hold:
(a) Equation (7.5.1) ha<; a unique solution u. (b) There exists a number m such I hat, for all n ;::.: m, tht:' Galerkin Equation ((7.5.2) or (7.5.3)) has a unique solution Un' (c) The Galerkin method cOtl\wges, that i5, 1I'Ut1 -

ullu --+ 0 as n --+

00.

Theorem 7.5.1 Under the conditions mentioned above, for each y Eli, Equation (7.5.1) is uni(luely approximation-solvable in the case where the linear operat07' T : H -+ H sotisfies one of the following four properties:

(a) T =

(+

11511 <

S, whp.re S : l-f --+ l-f is linmr flnd k-wntmr:title, that 1. (n th15 rose, for n ;::.: m, the error est?mates

i.~,

holdl1. (b) T = (+ U, where U : H -+ H is linoor and compar:t (image of a bounded set A under L' is pnxompact, that lB, U(A) is compact), and Tu = 0 implies 'U = O. Under these conditions,

(7.5.5)
(c) 7' is linear, bounded, and strongly monotone (or coercive), that ill', (Tu, u) > 0:IIull 2 fo" all u E H and fIXed a > O. Under the hypotheses for 0: > 0,

all" - unll

l17'un- yll

(7.5.6)

(d) 7' = S + U. where S : H -+ Ii is linear continuous and coercive and U is linear and compact, and Tu = 0 l77lplies u = O. In cases (a) and (c), 111 = 1. In case (b), m is independent of y. We prove here cases (a) and (c) and refer to Zeidler [15} for the other cases.

OlPnll

Pmof.

(a) Since P"u" = u" for u" E 11" and IIP,ISIl < IISI1 < 1, = 1 by Theorem 3.4.1(3)). The following equations

+ Su

= y,

u E If,

(7.5.7)

Q.

......

, toOl'._......... ",,"l1>"-I

and (7.5.8) have unique solutions by the Banach Contraction Mappin}!; Theorem (Theorem 1.2.1). F\uthennore,

II(! + p"S)-'11 ~

I)p"S)' ~
.1:=0

L 11511'
k=O

By (7.5.7) and (7.5.8), (1 Hence,

+ P"S)(u -

'U,,) =

U -

p"u.

lIu - u"1I ~ (I -1I511-')lIu - P"ull ~ (1 -IISII-')d(u, p"u).

(c) For all u Eli, we hm'e Pnu =
U

and hence

(P"Tu,u) ~ (Tu, p"u)

> ollull'.

(7.5.9)

Thus, the operator PnT . Ii" --t fin is strongly monotone. By the Laxl\'lilgram Lemma, the two operator Equations (7.5.2) and (7.5.3) have unique solutions. If n > j, then it follows from (7.5.3) that

(Tu",Wj) = (y,Wj),
(TUn, u.,) = (y, 'Un).

(7.5. to) (7.5.11)

By (7.5.11),

This yields a pTlon estimate

cllu.. 11

lIyll,

that is, {Un} is bounded.

U
"

Let {u ll '} be a weakly convergent subsequcnCf' with 'Un' -->. V as n --t 00 (Theorem 4.4.7). By (7.5.10), (Tull"w) --t (y,w) as n --t 00 for all W E lin. Since H II is dense in Ii and {Tull} is bounded, we obtain

U
"

TUn

-->.

Y as n --t

00.

(Theorem 4.4.7).

...... "'_'_""""'''''''10''-''

Since T is linear and continuous

Tu",
->.

Tv as n --)

00.

(Theorem 4.4.1(i)).

Hence, Tv = y; that is, v = u. Since tlte weak limit u is the same for all weakly convergent subsequences of {u,,}, we get Un U as n --t 00. It follows from
--->.

allu" -

UJj2

< (T(u"
as n

- u),u" - u)
-

= (y, u,,) - (TUn, u) - (Tu, Un

--t 00;

u) --t U

that is

tI"

--t

tI

as

1l

--t 00.

Therefore,

ollu" - ull'
and Tu = y. Therefore,

IITu" - Tuliliun - ull,

allu" - ull < IITu" - yll.

7.5.2

R11,vleigIJ-llitz-Galerkln Metbod

The Rayleigh-Ritz-Galerkin method deals with the approximate solution of (7.5.1) in the fonn of a finite series
Um

L
J=!

Cjj --I-

rPo,

(7.5.12)

and its weak formulation (variation formulation), a(u, v) = F(v), where the coefficients cJ ' called the Rayleigh-Ritz-Galerkin coefficients, are chosen such that the abstract variational formulation a(v, w) = F(v) holds for v =;,i:=: 1.2..... m: that is,
i

== 1,2, ...

,1H.

(7.5.13)

Since a(, .) is bilinear, (7.5.13) takes the (orm

L: a(" ')', ~ F(;) j=1

OJ

a(" OJ,

(7.5.14)

Ac- b, -

(7.5.15)

..... ,

Q . ,>0<1"_""""'''''''IO''-I

where

is a matrix

and
C:::: [CI,CZ, ,Cm).

which represents a system of til, linear algebraic equations in m unknowns C;. The columns (and rows) of coefficient matrix A must be linearly indepeudent in order that the coefficient in (7.5.15) can be inverted. Thus, for symmetric bilinear forms, the Rayleigh-Ritz-Calerkin method can be viewed as one that seeks a solutiou of the fonn in Equation (7.5.12) in which the parameters are determined by minimizing the quadratic functiollal (energy fUllctioual) given by &Juatiou (6.3.10). After ~ubstituting
1 Um of Equation (7.5.12) for u into J(u) :::: 2"a(u,u) - F(u) and integrat-

ing the functIOnal over its domain, J(u) lx"Comes an ordinary functIOn of the parameters Cl, C2,. . . The neces.<;ary condition for the minimum of J(Cl.C2, .. em ) is that

8JC) ~ 8JC) ~ ... 8J() ~O

DCI {)C'l Bc",'

(7.5.l6)

Thisgi\es m linear algebraic equations ill m unknowns, cJ,j:::: 1, 2,..., 11i. It may be observed that (7.5.14) and (7.5.16) are the same ill the symmetric case while they differ in the non-symmetric case. III other words, we get the same c, 's by solving (7.5.14) and (7.!i.16) separately. In the nOll-symmetric case, we determine the 111 unknowns by solving the linear algebraic equations (matrix equations) (7.5.15). The choice of {j},j :::: 1,2, ... ,m is crucial and this should be the basis of the Hilbert spaCC under consideration.

7.6
7.6.1

Eigenvalue Problems
Eigenvalue of Bilinear Form

Let a(u,v) be a symmp-tric coercive and bOllndp-d bilinear form defined on the Hilbert space Ii <l.'Swdated with the opt:rator equation
T; H C 2(0) -+ 11, Tu:::: yin fl,

(7.6.1)

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

H = H"'(O). The problem of finding a number A and a nonzero functlOll that a(u,v) '),(u,v)L,(fl)

E H such

(7.6.2)

is called the eigenvalue problem of the bilinoor [onn a(u, v). The weak problem associated with the operator equation

Tu - AU = y
comprises finding u E H such that

(7.6.3)

(7.6.4)

7.6.2

Existence a1/(1 Uniquelleijij

III

Theorem 7.6.1

The fUlIctivlI u E Ii (7.6.3) if and only if

the weak Ilolution vf Problem

/7.6,5)

u-ATu=Ty

holds in Ii. The number A 18 on C1genooJue of the bilmoor fonn a(u, v) and I,he function u(x) is the corresponding eigenfunction if and only if
u- ATu=O,

ufO

(7.6.6)

holds

In

Ii.

Proof.

Let u(x) be a weak sollll.ion of Problem (7.6.4); that is,

a(u, v) = (y + AU, Vh~(fl)'

Then by the definition of operator T, we havf'

u = T(y + AU) or u - ATu = Ty.

To see the cOll\'erse, let u - ATu = Ty, then it is clear that u is the weak solution of (7.6.3). If Problem 7.6.2 has a nontrivial solution, it follows from the definition of T that

that is, Equation (7.6.6) holds

III

Hand \'ice versa.

",,,. "'_'_""""'''''''10''-''

In view of a well-known remit (see Rektorys [11] or Reddy [10] or Zeidler [15]), the symmetric, coerci\"e and bounded bilinear form a(u, v) has a count.able set of eigenvahleb, each of which being pooitive,

(7.6.7)

The corresponding orthogonal system of eigenfunctions flP.} consl.itutes a ba.<;is in the spao" HO'(O); hence in 11. Furthermore, Wf> haw"
(7.6.8)

For>' ::j;. ).,." 11 == 1,2,3, ... , Problem (7.6.4) has exactly one weak soluuon for every y E 2(0). If the eigenfunctions lPj(x) belong to the domain Dr of the operator T, then the relat.ion for e\wy v E H gives the classical eigenvalue problem
(7.6.9)

If WI'! choose {lPj} as tlw basis of 110 "(0). then for t.he N-paramet.rk Ritz soludon of Tu == Y, we write
N

UN (x)

==

L>jlPj,
j='

where cj =:: (Y,lPj)' Thus, eigenfunctions associated with the eigenvalue problem Tu == AU can be use<! to admntage in the Ritz method to find thl'! solut.ion of t.he equation

Tu == y.

7.7

Boundary Value Problems in Science and Technology

Example 7.7.1 (The Laplace Equation).

6u == 0,
(7.7.1)

...". '.'_._""""'''''''10''-''

where
tj2 '1J

6.u = {)X2

+ {)y2 + {)z2'

tP'1J

tP'1J

(7.7.2)

The operator 6. is known ru; the Laplace operator in dimension three. Equation (7.7.1) models the electrostatic potential in the absence of charges, the gravitational potential in the absence of mass, the equilibrium displacement of a membrane with a given displacement of its boundary, the \e1ocity potential for an inviscid, incompressible, irrotational homogeneous fluid in the absence of sources and sinks, the temperature in steady-state flow in the absence of sonrces and sinks, etc.

Example 7.7.2 (The Poisson Equation).

6.11=-f(x,lJ,z),

(7.7.3)

wherf' f is a given function. Equation (7.7.3) represents the electrostatic potential in the presence of charge, the gravitational potential in the presencf' of distributed matter, the equilibrium displacement of a membrane under distributed forces, the velocity potential for an inviscid, incompressible, irrotational homogeneous fluid in the presence of distributed sources or sinks, the steady-state temperature in the presence of thermal sources or sinks, etc.

Example 7.7.3 (The Non-homogeneous Wave Equation).

8'u

[Jt2 - 6.u

= - f(x, y. z).

(7.7.4)

This equation and its corresponding homogeneous fonn arise in a large number of physical situations. Some of these problems include the vibrating string, vibrating membrane, acoustic problems for the velocity potential for the fluid flow through which :,,(lund can be tHmslllitted, longitudiual vibrations of an elastic rod or beam, and both electric and magnetic fields in the absence of charge and dielectric.

Example 7.7.4 (Stokes Pmblem). The problem of finding solution of the following BVP is known as Stoke's Problem:
-lt6.u + 'Vp =

div u = 0

in 0 in 0 on

(7.7.5)

(7.7.6)
(7.7.7)

u = 0

r.

...... '.'_'_""""'''''''10''-''

This BVP represents the phenomenon of the motion of an incompressible viscous fluid in a domain 0, where u = (u], U2, ,un) is the velocity of the fluid, p denotes the pressure, / = (I], h, ... ,/,,) E the bod)' force per umt \'olume, and J-t is the \'isca:;ity
n

(L (n))" represents
2

Vp= gradr=Lax.e,
;:=]

ap

,=(O.O.... 1.0.... );

I.e., the i-th coordinate is 1, div u= " -8 ~ a",

n

.=]

X,

Example 1.1J; (The Navier-Stokes Equal.ion). the stationary flow of a viscous Newtonian fluid subjected to gravit)' loads in a bounded domain o of R3 is governed by the BVP

-r.6.u + ~ Ui . ox
;=1

ou

+ Vp

== /

in 0 in 0 in r = 00,

(7.7.8) (7.7.9) (7.7.10)

divu=O
u==Q

where U represenlS the velocity, p the pressure, / the bod)' force per unit volume and Vp, divu haw' the same meaning as in Example 7.7.4 for 71, = 3. r = ...!!:....- = II R. where R is called the Reynolds number. Here It is
dvp

the viscosity of the fluid, d, a length characterizing the domain O. v a characteristic velocit)' of the flow and p the density of the fluid.

Example 7.1.6 (llent Equation). The followmg equation govems the diffusion process or heat conduction to a reasonable approximation:

au

at

02 U ox2

XE(-l,I),

tE(O,oo).

(7.7.11)

This equation is known as the !leat cfJUation. The boundary conditions fOr this equation llIay take a variety of forms. For example, if the temperatures at the end points of a rod are given, we would have the BVP

ox U(l,t) = e- t,
,,(x,O) = 1

au

at

8 u -- on 0, 2
2

0 =

{(x, t)1 -

1<

x < 1,

< t < oo}

O:s;t<oo

Ixl $

1.

We assume that u(x,O) = uo(x) = l. The solution of this BVP. u(x, t), will provide the temperature distribution in the rod at time t. The heat equation has recentl,Y been applied for predicting apPl'Opriafp hp.f. for sharps (for dpl.ails, Sf> [7] and references therein).

Example 7.7.7 (The Teleympher's Equation).

(7.7.12) where a and b are constants. TillS equation arises in the study of propagation of electrical signals in a cable transmission line. Doth the clll'rent I and the volt.age V satisfy an equation of the fonn (7.7.12). This equation also arises in t.he propagation of pressUl"f' wa\'es in the study of pulsatile blood flow in arteries, and in one-dimensional random motion of bugs aJong a hedge.

Example 7.7.8 (The Inhomogeneous Helmholtz Equation).

(7.7.13) where ),. is a constant It is essentiall)' a time-dependent inhomogeneous wave Equation (7.7.4) with)" as the separation constant.

Example 7.7.9 (The Bihfi1T1lunic Wave Equatiun).

1 Ej21J; Cl 8t 2 = O.

t11jJ -

(7.7.14)

In elasticity theor)', the displacement of a thin elastic plate in small dbrations satisfies this equation. When 1/1 is independent of time t. (7.7.14) redll~ to the so-called biharmollic equation (7.7.15) This is the equilibrium equation for the distribution of stress in an elastic mediUlli satisfied by Airy's stress function . In fluid d)'namics, the equation is satisflpd by t.he stream function in an incompressible viscous fluid flow .

B.

.....

, toOl',_"""",,,,,"l1>"-I

Example 7.7.10 (The Timelndependent Schrodinger Equation in Quantum Mechanics).

I' -'-tl1j.J+(E-F)1j.J=O, 2m

(7.7.16)

where m is the mass of the particle whose wave function is 1/', Ii is the universal Planck's constant, F is the potential energy and E is a constant. If l! = 0, (7.7.16) reduces to the Helmholtz equation. We prove here the existence of the solution of Stokes equation and refer to Chipot [1], Dp.hnath and lvlikusinski [4], Quarteroni and Valli [8], R.eddy [9], Reddy [10], and Siddiqi [12] for variational formulation and existence of solutions of other boundar)' value problems. A comprehensive prcsentation of variat.ional formulation and existence of solution of parabolic equations including classical heat equat.ion is given in Chipot [I, Chapters II and 12J. Existence of the Solution of Stokes Equations. We have
-Iltlu+ grad p-

= 0

in u E 0 in u E 0 on r,
(Ul' U2, U3)

(7.7.17) (7.7.18) (7.7.19)

divu=O u=O

where f E L 2 (O) is the body force vector, u ::: vector, p is the pressure. and It is the viscosit)'. We introduce the following spaces:
D ~ {u E C;:'(fI)1 dlv u ~ OJ
H

is the velocity

= {u E HJ(O)

x HJ(O)/ dh' u

= O}

(7.7.20)

Q~ {PE L,(O)I llxlX~O}

The space H is eqU1pped with the inner product

{v, u)u =

t: In

grad

Vi'

grad

Ui dx ,

(7.7.21)

where n is the dimension of the domain U C IR". The weak formulation of Equations (7.7.17) (7.7.19) is obtiuned using the familiar procedure (Le., multiply each equation with a test function and appl)'ing Green's formula for integration by parts (Theorem 5.4.7).

.n",

"'_'_""""'''''''10''-''

We obtain for v E V,
(-I-I.6.u

+ grad p - J,

v> = 0

or /1. ~

grad u, . grad v,dx = (grad p, v)

= (1,v)

+ (1, v)
11

for every

E V.

As for v E V, we ha\'e div v = 0 and v = 0 on

a(v, u)

r, giving

= (v, f)

for ever)' v E V,

where
a(v, u) =

t lit
;=1

grad

Vi

grad u,dx.

(7.7.22)

We now have the following weak problem: find u E I-l such that

a(v, u)

(v, f)

(7.7.23)

holds for every v E 11. The proofthat the weak solution of Equation (7,7,23) is the classical solution of Equations (7.7.17) - (7.7.19) follows from the argument (see Temam

[14])
for every v E V. (7.7.24) This does f10t imply that -It.6.u - J = 0 because v is subjected to the constraint (because v E H) div v = O. Instead, Equation (7.7,24) implies that
-1-1.6.11 - / = -grad p,

(7.7.25)

because (necessary and sufficient)

(v, grad p)

= (div v,p) = 0

for every p E Q.

(7.7.26)

The bilinear fOHn in Equation (7.7.23) satisfies the conditions of the Laxl'vlilgram Lemma. The continuity follows from Equation (7.7.22) using the CBS inequality

la(v,u)1 =

l; L

It grad Vi' grad ujlix

""[t,1,
~

[g,ad

V'['dx] 'I'

[t, 1,

[g,ad

U'['dX] 'I'

,'lIvll H lIu[,"-

.n",

"'_'_""""'''''''10''-''

The F-ellipticit:y of a(',') follows from

la(v, v)1 = 1-1

,==1 n

grad v, . gr<ld 1Jidx

= ItUvll~,

;::: ollvlljJ

for

<::::

I.t. Thus. the variational problem (7.7.23) has a unique solution in

H.

7.8

Problems
Prove the existence of solut.ion of the following boundary

Problem 7.8.1 value problem

Jlu du --+-+u(x)=! 2 d.x dx

infO.11

( <Ill) d.x

~
",=0

(d")
dx

~ O.
",=1

Problem 7.8.2 Let a(u,v) be a coerch' and hounded bilinear form on [-[ = I-P"(fl) (Sobolev space of order m). Then prove t.he continuous dependence of t.he weak solution of the operator equation Tu = y, where T is induced by the given bilinear form, on the giwn data of the problem, that is, if t.he function changes slightly in the norm of ['2(0) to fj, then the weak solution u also changes slightly in the norm of J-lm(fl) to ii. rdore precise1.}'. prO\, that

Pmblem 7.8.:1 Apply t.he Haleigh-Hitz method to find the solution of the differ~ntial etjuation

-- = 2
dx

(f"ll

COS1rX

0< x

< I,

with (a) the Dirichlet boundar)' condition

and (b) t1w Neumann boundar)' condition ,,'(0) = "'(I) = O.

...... "'_'_""""'''''''10''-''

Examine whet.her a solut.ion of t.he following boundary value problem exists or not

Problem 7.8.4

Tu = f
where

inOcRn

=0 on 80,

Tu = ~

" L

8 [ 8x a.j ax.
J

au] + ao,

f,j=I'

7.8.,15 value problem

Probl~m

Let 0 C Rfl. Shuw that the Robin buundary condition

-~u+u

in 0,

E 2(0)

an
has a unique solution.

- + ern

8u

= 0

on 80, a:

> 0,

Problem 7.8.6

Prove that sup !lu - P.,ull

uEM

-~

0 as n -t

00

provided t.he

nonempt.y subset At of 11 is compact, where {P',} is a projection operator on a I-IiIbert space H into lin' a finite-dimensional subspace of H, dim Un = n. Let T be a bounded linear operator on a Banach space X into another Banach space Y. Suppose that t.he equation

Problem 7.8.7

Tu = y,

u E X

(7.8.1)

has an approximate solution, that is, there exist constants l( It E (0,1) such t.hat for each a E Y, there exists a lI(a) E X with

> 0

and

IITu(a) -

all

""all, lIu(a)1I < Kllall

11

Show that, for each h E F. t.he Equation (7.8.1) has a solution lIuli ~ K(I -1')lIbll

with

Problem 7.8.8

Let us consider t.he uperator equation

+S=y,

E X,

(7.8.2)

B.

......

, to<l',_"""",,,,,"l1>"-I

together with the corresponding approximation equation

Un

EX,,,

n= 1,2,3, ..

(7.8.3)

Let us make the following assumptions: (i) X n is a subspace of the Banach space X and dim.\n = X ... is a projection operator onto X n . (ii) The operators 5 : X and 1+5; X -+ X
-~ i~
Fl.,

Pn ; X -+

X and 5 n : ...\11 -~ X n are linear and bounded,

onto.

(iii) There exists a constant lin with d(Su. X ... ) ~

t3nllull

for all

u EX.

(iv) As n -~ 00, IlPnS - Snllxn -~ 0, II P., 1It3'1 -~ 0 and IIP"lId(y,X,,) -~ o for all y E X hold. Show that Equation (7.8.3) is uniquely approximation-sol\<lble and

II"" ~ "II ~ /(1 II P"S ~ S"lIs" + I!P"II II"" ~"II <

f(

(p" + dIU,X,,))

(I!P"S ~ 5,,11 x" + II P" II <1(", X,,) )"

where K is an appropriate constant. (This problem was studied by Kontvrovich).

Problem 7.8.9

Show that

lI6.V

= V'. (lIV'V) - (V'1I)' (V'v).

Problem 7.8.10 Let [-[ be a real Hilbert space and T a linear operator on 11. Prove that T is continuous and T- 1 eJl:ists if
II x ll=l

inC

((Tx, x) + IITxll) > O.

(This is a generaliyd version of the La.x-J'vlilgram Lemma proved by Jean Saint Ravmonrl in 1997).

B.

......

, to<l'._..

....,..",,"l1>"-I

References
[IJ M Chipot. Elements of Nonlinear Anal)'sis. Basel, Boston, Berlin: BirkhiJuser,2000. [21 RF Curtain, AJ Pritchard. F'llllctional Analysis Mathematics. Academic Press, 1977.
In

Modern Applied

[3] Il Dautray, JL Lions. lvlathematical Analysis and Numerical Methods for Science and Technology. Vol. 2 Functional and Variational Methods. Berlin, Heidelberg, New York: Springer, 1988. [4J L Debnath, P Mikunsmski. IntroductIOn to Hilbert Spaces with Applications. Sandiego, London, Boston: Academic Press, 1999.

[51 S FUCiks, A Knfller. Nonlinear Differential Equatio!l$. Amsterdam, Oxford, Nf'W York: Elsevier, Studies in Applied M~('hallics, 1980
[6] RD tvlilnc. Applied Functional Anal)'sis. An lntroductor.y Treatment. Boston, London: Pitman Advanced Publishing Program, 1980.

171

H Neunzert, AH Siddiqi. Topics in Industrial Mathematics-Case Studies and Related Mathematicall\'lethods. Boston, Dordrecht, London: Kluwer Academic Pllblishsers, 2000.

[8] A Quarteroni, A Valli. Numerical Approximation of Partial Differential Eqnfl.tions. Berlin, New York: Springer-Verlag, 1994. [9J BD Redd)'.lutroductor)' FUnctional Analysis with Applications to Boundary Value Problems and Finite Elements. New York, Heidelberg, Berlin: Springer, 1999. [IOJ IN Reddy. Apphed Functional Analysis and Variat.ional Methods in Engineering. New York: McGraw-Hili, 1986.

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

[11: K Hektorys. Variational Methods in Mathematics, Science and Engineering. Dordrecht, Boston: D. Reidel Publishinp; Co., 1980. [12J AH Siddiqi. Functional Anal)'sis with Applications. New Delhi, New York, wndon: ,lth print Tata McCraw-Hili Publishing Ltd., r993.
[13] B Smith, P Bjorstadt, \\' Gropp. Domain DecomposItion, Parallel l'vlultilevel l'vlethods for Elliplic Partial Differential Equations. Cambridge: Cambridge Unh'rsity Press. 1996.

[14] R Temam. Thoory and Numerical Analysis of the Navier-Stokes Equations. Amsterdam: North Holland, 1977. [Ui] E Zeidler. Nonlinear Functional AnaJysis and its Applications. New York, Heidelberg, Berlin: Springer-Verlag, 1990.

III A,

B . ,~,_ .._",,"l1>"-'

..... ,

Chapter 8

Finite Element and Boundary Element l'vIethods

8.1 Introduction

Finite element and boundary element methods are nowadays major numerical tools fol' different types of boundary \'alue problems (BVPs) and for studying part.ial different.ial equations (PDEs) modeling real-world problems. Functional anal)'sis plays a vital role in reducing the problem in a forln amenable to computer analysis. It is a basic tool for errol' estimation betwt:etl solutions of continuuus and discrete problems and convergence of solutions of the latter to the original problem. Thp finite element method is a general technique to build finite-dimensional spaces of a Hilbert space of some classes of functions, such as Sobolev spaces of different orders, and their subspaces, in order to apply the Ritz and Galerkin methods to a variational problem. The technique is based on ideas like (i) Division of the domain n in which the problem is pused in a set of simple subdomains. called elements-often these elements are triangles, quadrilaterab, tetrahedra, etc. (ii) A space H of functions defined on n is then approximated b)' appropriate functions defined on each subdomain with suitable matching conditions at interfaces. Usual!)', pol)'nomials or functions obt.ained from polynomials by a change of variables are IIsed for such an approximation. While applying t.he Rit.z and Galerkin method, one selects a basis which yields a sparse matrix equation for the discrete problem with a variational principle and a function space. Changing t.he variational principle and the spat'''' in which it is posed, leads to a different finite element approximation even if the solution of the original problem remains the same. Broadly speaking, finite element methods are nothing but approximations in Sobolev spaces. A systematic study of variational formulation of the boundary value prob-

.n",

'.'_'_""""'''''''10''-''

lems and their discretization began in the early seventies. In early 1950s, enf,rineer Arg}Tis started the study of certain techniques for structural analysis which are now known as the primitive finite element method. The work representing the beginning of finite element was contained in a paper of Turner, Clough, Martin and Topp (43], where endeavor was made for a local approximation of thp partial differential equations of linear elasticity by the usage of assembly strate/"ries, an essential ingredient of finite element method rn 1960, Clough termed these techniques as "finite element method". Between 1960 and 1980, several conferences were organized in different parts of the world, mainly by engineers, to understand the intricacies of the method. A paper by Zlamal [491 is considered as the first most signifiCilllt mathematical contribution in which analysis of interpolation properti(!s of a class of t.riangular elements and thf'ir applk.ation to the second- anci fourth-order linear elliptic boundary value problems is carned out. Valuable contributions of Ciarlet, Strang, Fix, Sd1Ultz, Birkhoff, Bramble <md Zlamal, Babuska, Aziz, Varga, Raviart, Lions, Glowinski, Nitsche, Brezzi have enriched the field. Proceedings of the conferences edited by Whiteman [47] and the book by Zienkiewicz and Cheung (48] have popularized the method among engineers and mathematicians alike. The Finite Element Handbook edited by Kardestuncer and Norrie [26] and the Handbook of Num~rit:i:l.1 Analysis ~dited by Ciarlet and Lions [20] pwvide contemporary literature. Wahlbin [441 presents somp of the current research work in this field, Tn short, there is no other approximation method which has had such a significant impact on the theory ann applications of numerical methods. It has been practically applied in every conceivable area of engineering, such as structural analysis, semiconductor devices, meteorology, flow through porous media, heat conduction, wave propagation, electromagnetism, environmental studies, sating sensors, geomechanics, biomechanics, aeromechanics and acoustics; to name a few, The finite element method is popular and attractive due to the following reasons: The method is based on weak formulation (variational formulation) of boundary and initial valup problems. This is a critical property because it provides a proper setting for the existence of even discontinuous solution to differential equations, for example, distributions, and also because the solution appeal'S in the integral of a quantity over a domain. The fact that the integral of a measurable function over an arbitrary domain can be expressed as the sum of integrals over an arbitrary collection of almost disjoint subdomains whose union is the orihoinal domain, is a very important. point, in t.his method. Dne to this fact, the analysis of a problem can be carntrl uut locally uver a subdumain, and by making the subdomain sufficiently small, polynomial functions of various degrees are sufficient for representing the local behavior of the solution. This property

.n",

"'_'_""""'''''''10''-''

can be exploited in every finite element program which allo\\'s us to focus the attention on a typical finite element domain and to find an approximation independent of the ultimate location of that element in the final mesh. The property stated above has important implications in physics and continuum mechanic~ dnd consequently, the physical laws will hold for every finil.e portion of the material. Some important features of the finite element me/hods are (I) arbitrary geometries, (2) unstructured meshes, (3) robustness, and (4) sound mathematical foundation Arbitrary geometries means that, in principle, the method can be applied to domains of arhitrary shapes with different boundar}" conditions. By unstructured meshes, we mean that, in principle, une can place finite elements anywhere from the complex cross-sections of biological tissues to the exterior of aircraft, to internal flows in turbo machinery, \\'ithout the use of a globally fixed coordinate frame. Robustness means that the scheme developed for assemblage after local approximation over individual elements is stable in appropriate norms and insensitive to singularities or distortions of the meshes (This property is not available in classical difference methods). The method has a WUIlU mathematical basis as convergence of an approximate "olution of the abstract variational problem (a more general form is variational inequality problem) and error estimation of the abstract form in a fairly general situation and their special cases have been systematically studied during the last two decades. These studies make it possible to lift the rumJysis of important engineering and physical problems above the traditional empiricism prevaJent in many numerical and experimental studies. The main objective of this chapter is to introduce basic ingredients of the mathematical foundation of the finite element method.
Boundary Element Method. The theory of integral equations is well known through the classical work of Fredholm, especially due to its relevance to the potentiaJ theory (Laplace type problems). The works of Kellog, Tricomi [42], Kupradze [27], J\[jkhlin [28, 29], etc. provides a rich literature on the subject. However, methods of integral equations were not popular with engineers until late seventies in spite of thf' invention of digital computers. Researchers at Southampton University may be credited for making a systematic endeavor in t.he

.n",

"'_'_""""'''''''10''-''

eighties and nineties to solve problems of different branches of engineering by modelling them through intf,rral equations on the boundary of a region (domain) and then by discretizing these models. This technique led to the development of the boundary clement method (also called the boundary integral method in the initial stages). Thus, the boundary element method (BEM) comprises transformation of the partial differential equation describing the behavior of an unknown inside and on the boundary of the domain into an integral equation relating to only boundary values, and then finding out the numerical solution. The combination of the finite element method is possible with the boundary element method where the integral equation on the boundary call be discretized through the finite element techniques treating the boundary as an element. Now-a-days, BEtI.'f is a powerful technique as it reduces t.he dimension of the problem by one as boundary integral equations and it permits for complex surface elements to define the external surface of the domain. One of the significant advantages of this method is that discontinuous elements can also be used and meshes of such elements can be refined in a simpler manner. This method has been applied in a variety of physical phenomena like tnmsient heat conduction, thermo-elasticity, contact problem~, free boundill"y problems, water waves, aerodynamics, elasto-plastic material behavior, electromagnetism, soil mechallics. ThNe is a vast literature published UII these topics in the last fifteen years which can be found in books by Brebbia [5-13], Antes and Panagiotopoulos [iI, Chen and Zhou [18J, and Hackbusch [241. The boundary element method is a rapidly expanding area of practical importance. \Ve illustrate here the basic ingredients of this method with examples.

8.2 Finite Element Method

8.2.1 Abstract Problem and Error Estimation
Let H be a Hilbert space and a(,) a bounded bilinear form on If x If into R. for each P E Ifo, the dual space of If (the space of all bounded linear functionals on fl) the variational problem is to find u E H such that
a(u, v) = F(v) for all v E H

(8.2.1)

By the Lax-Milgram Lemma (Theorem 3.9.4), the variational Equation (8.2.1) has a unique solution if n(,) is coercive or elliptic. The LaxMilgram LelHma (Theorem 3.9.4) mainly depends un the Riesz Representation Theorem concerning the representation of the elements of H' in terms of the inner product on H.

.n",

"'_'_""""'''''''10''-''

The procedure for finding a finite-dimensional sllbspac<> l-h of H such that there exists '/.1'11 E T-h satis(ying the equation

a(tl'h, Vh) = F(VII ) for all Vh E if"

(8.2.2)

is called the finite element method. Equation (8.2.1), is known as the abstmct variational problem and Equation (8.2.2) is called the approximate problem. If H h is not a subspace of H, the above method is known as the non-conformal finite method. Equation (8.2.2) is lluthil1g but a matrix equation of the form
AU = B.

(8.2.3)

when' f J =

(01, oz, 03, . __ ,0N(h)),

N(h) = dimension of if"

At = (a(w"wj))i,j

(8.2.4) (8.2.5) (8.2.6)

B = (F(wtl,F(wz), ... ,F(WN(1l)))

N(h)
1111

n,wl

;=0 t
N{h)
VI).

BjWj,

(8.2.7)

j=l

and Pj are real numbers, i,j = 1,2,3 , N(h). The dJoice of th", basis {will of fI", i = 1,2N(II), is of vital importance, namely, choose a basis of H" which makes A a sparse matrix so that the computing time is reasonably small. In the terminology of structural engineers, A and F(wj) are called the stiffness matrix and the load vector, respectively. If a(', .) is symmetric, then finding the solution of (8.2.1) is equivalent to finding a solution uf the optimization problem
0;

J(u) = inf J(v), vEIl

(8.2.8)

where J(v) = ~atv,v) - F(v) is called the energy functional. In this case, the finite clement method is nothing but the Rayleigh-llitz-Galerkin method. It is quite clear that (8.2.2) and the approximate problem for (8.2.8), namely, (8.2.9) where .l(v,,) = ~a(vh, v,,) - f(vI') have unique solutiolls.

Finding 1I11-u"II, where 11_ and Uh are the solutions of (8.2.1) and (8.2.2), respectively, is known as the envr estimation. The problem '/.I." -j. 11 as h -j. 0; that is, 1111'1< - 1dl --t 0 as It -j. 0 or n = 1; -j. 00 is known as the convergence problem.

Error Estimation. Theorem 8.2.1 (Cea's Lemma). dent of the subspace ff" such that

There extsts a constant C mdepen-

lIu -

11'1<1111 S C

1J"EI1,.

inC

lIu -

v"

1111"

= Cd(u, Th) ,

(8.2.10)

where (,' = ~f is independmt of fl", M i5 the constant associated with the c01dimity (bOlt7uledness) of ah') and a is the coercivity constant. 1f a(-,) is symmetric. then the deqree of approximation is improved. that is, we get
C=

! whir.h i.5 ie.55 than the con5tant in the non-5ymmd,'ie case.

Proof. By (8.2.1) and (8.2.2), we get a(l1,v) - a(ll",v,,) = F(v) F(v,,) and this b';ves a(ll,vh) - a(ll",vh) = 0, for v = v". By bilinearity oC a(, '), we get

(8.2.1I)
by replacing Vh by
Vh - Uh.

Since a(, .) is elliptic,

-

a(ll -

111<, U

111<)

2: O'llu -

11'1<11

0'

I
n

[a(u -

UI" U -

VI<)

+ a(u -

11'1<, VI< -

'/.I.,,)] > lIu -

2 1),1<11

Using (8.2.11), this becomes

.n",

'.'_'_""""'''''''10''-''

using boundedness of a(,); namely,

la(a,vll < Mllullllvll

This gives us

Iitl. - iI'hll

<C

""ell"

inf Ilu -

Villi

When the bilinear form ne.,) is symmf!tric, it leads to a remarkable interpretation of the approximate solution, namely, the approximate wlutiulllth is the projection of the exact solution tl over the subspace fI" with respect to the inner product a(,) (Induced by the bilinear form which is denoted by the bilinear form itself.) as a(tl.- UI" VI,) = 0 for all Vh E fill. Thus, we get a(t~-UI"tl.-tl'll) ==- inf a(u-vh,U-Vh)'
""EH..

By the properties of ellipticity and boundedness of a(, '), we get

Thus,

Thus, we have a smaller constant

(M Va as Ai 2':

(x.

RemU1'k 8.2.1 Inequality (8.2.10) shows that the problem of estimating the error !Iu-uhll is reduced to a problem in approximation theory, namely, the evaluation of the distance d(u, H h ) = inf Iltl,-vhll between a function

u Efland a subspace IJ" of H. Assuming conditions on 1', one can show that d(u, rl,,) < C(tl.)hJ3 for /3 > 0, where C(u) is independent of h and depelldent on u, and so Ilu - hhl!:::; C(tl.)h J3 In this case, we say that the order of convergence is /3 or, equivalently, we have an order O(hJ3 ), and we write IIt~ - iI'I,1I = O(h J3 ).

""ell"

...... "'_'_""""'''''''10''-''

Now, we prove a result concerning the abstract error estimation, known as the first Strang lemma due to Gilbert Stnmg (41]. In this result, the forms a"h') and FhU are not defined on the space fl, since the point values are not defined in general for fun~tions in the space 1-1 1 (0). Theorem 8.2.2 (First Strang Lemma), Let fl be a Hilbe,t space and RhO ils finite-llimensional subspace. Further, let a(,) be a bilinear bounded anll elliplic form on fl anll FE fl'. Assume Ihat 111< is the .~ol1t1ion of Il,e fotlowin9 approximate problem: Finll U" E Ih such Ihat
(8.l.12)

where a" (', .) is a bilinear anll bounded form definetl on HI< anll F" (.) is a boundetl tinear functional defined on H h. Then Ihere exists a constant C independent. of Ih sueh that

provtded ah h .) 1S ltnif07'mly I-h -eU'1)ttC, that 2 fJllVh 11 for all v" E H" and all h.

tS,

3 fJ

> U Sltch that. a" (Vh, Vh) >

It may be observed that although al',') <.md F/J) are not defined for all the elements of R, but Equation (8.2.12) has a unique solution under the given conditions.
PnJOf.

We have

by the triangular inequality of the norm and (8.2.13) By continuity of the bilinear form a(', '), (8.2.13) take; tlte rOfln
(:Jllt/'h - vl,1I

< a(U -

VI<, Uh - V,,)

+ {a(vh' U"

- Vh) - ah (Vh, t/'I< - Vh)}

B.

......

{Fh(U", - V,,) - F(tl,', - VI,)} ,

, to<l',_"""",,,,,"l1>"-I

I3I1uI1 -

vhll -:;

Ml!u- vhll

+ ja(Vh, Uh

- 1Jh) - all (VII, 111, - VI,) J

Iluh Utl'h

villi

+ eW,,',,,,ee",,-,"V",,,)_,-F"io(,"'""----'-v"',,,,) ( I
vhll
BUp

'/11 < 11' 1t -

Vh

II +

Inevll,wl<) -a/l{v/"wIJI

w"EIf"

II w hll

-/

sup
whEN"

IF,(w,) - F(w,)1

II whll

By putting the value of lIuJ, - villi in the first inequality and taking the infimum over Hh' we get the desired result. I

Now, we present a theorem on abstract error estimatIon known as the Second Strang Lemma due to Gilbert Strang [41]. In t.his lemma, Nh is not contained in the space ff The violation of the inclusion ff" C ff results from the use of finite elements that are not of class Co; that b, that are not continuous across adjacent finite elements.

Theorem 8.2.3 (Second Strang Lemma). Let u" be a solution of the following approximate problem (lliscn~te problem): Find Uh E HI< (a finite-dimensional space having the noml eqltivalent to the norm of ffl(O)) sltch thai,
(8.2.14)

where ahl','J is as in Theorem 8,2.2 and F E H~, Then there exists a constant C independent. of the subspace fh such that

where ff'l need not be a subspace of H = HI (U). Proof. Let Vh be an arbitrary element in the spuel' H h . Then in view of the uniform H,l-ellipticity and o-mtinuity of the bilinear forms ai, and of

.n",

"'_'_""""'''''''10''-''

the definition of the discrete problem, we may write

1311U'h

~vl,II;{

< ah(11'h

-Vt"U'h -VI,)
-

= a,,(u -Vh, 1Ih

Vh)

+ {F(uh

- VI,)

-a,,(u,uh - 1Ih)}, from whic:h we rlerluc:e

I-' 11." -

"II

VI.

II /1 < 1< 11 'fll _

VI,

II /1 + IF(w,, - VI,) - a,,(w,w/, - v,,)1 II Uh VI! II fI

sup
w"E\'1.

'fll < 1\ 11.-VI. II 1/ + _

IF(w/,) - a/,(w,w/,)I

II

W" II

II

Then the inequality of the theorem follows from the above inl::jUality and the triarll,,'1..Ilar inequalit)'

I
lWmark 8.2.2 (i) Theorem 8.2.2 is a generalization of Cea's lemma as a"(,,,) = a(,) and F"O = FO in the case of conformal finite element method (the case when T-f" C H). (ii) The problem (8.2.1) can be written in the form
Au =

I,

(8.2.15)

where A : l-l --t l-l bounded and linear. By Theorem 3.9.2, thl"e exists a bounded linear operator A on 1-1 into itself (1-1* = 1-1) if H is a real Hilbert space such that (Au, v) = a(u, v). By the Riesz theorem for each v, thl'e exists a unique f E H such that F(v) = (f,v). Thus, (8.2.1) takes the form (Au,v) = (f,v) which gives (8.2.15). Convergence Results. As a consequence of Theorem 8.2.1, we find that lIul. -ull --t 0 as Ii -+ 0; that is, the approximate solution UJ, converges to the exact solution of (8.2.1) if there exists a family {H,,} of subspaces of the space H such that for each u E H. inf lIu-vhll=Oash-+O.
'lJ"EII"

(8.2.16)

......
B.",to<l',_"""",,,,,"l1>"-I

If IIi/. - 11'hJl S CJ{' for 0: > 0 where C is a positive constant independent of 11 and '/.I'h and 11 is the characteristic length of an element, then 0: is called the rate of convergence. It may be observed that the convergence is related to the norm under consideration. say, L.-norm. energy norm (L 2 -norm) or Loa norm (ur Slip norm).
Corollary 8.2.1 Suppose there e:asts a dense subspace U of H and a mappinf} Th ; H --t fh such that lim IIv - Thvll = 0 V v E U. Then
,~O

Pmof. Let e > O. We choose v E U such that for C > 0, lIu-vll < e/2C (U is dense in 1-1) and It sufficiently small such that IIv - T"YJII S e/2 C (This choice is Vossible in \iew uf the hyvot.hesis). Then by Theorem 8.2.1, and in view of these relations

lIu-tlhll < C

w,EH"

inf

lIu-vIlIl
because T"V E HI,

< II"!! - Thvll,

S C [1111. -

vII + lit! -

TIlVU]

Ge
= e.

Ce

< 2C + 2C
Therefore,

I
There is a vast literature concerning convergence for special types of norms and spaces and special types of boundary and initial value problems. For interested readerfl, we refer t.o \Vahlbin (44]. Ciarlet [191, Ciarlet <.md Lions [20], Kardestuncer and Nome [26J.

8.2.2

lntermd Approximatiun of HI (D)

This problem is concerned with finding a finite-dimensional subspacf' H" of HI(O). For achieving this oujective, first, we define a triangulation.

.n",

'.'_'_""""'''''''10''-''

Definition 8.2.1
of triangles I. 0 =

Th

Let 0 C R2 be a polygonal domain. A finite collection satisfying the following conditions is called a triangulation:

U
KE7i,.

l\. l\ denotes a triangle with boundary sides.

3. J( n K I = a vertex or a side, i,e., if we mnsider two different triangles, their boundaries may have one ,rertex, common or one side common.

Rema7'k 8.2.3 Let P(K) be a function space defined on K E 7h such that P(K) C HI (K) (Solmlcv space of order 1 on K). Cenerally, P(}{) will be a space of polynomials or functions dOSf' to polynomials of some degree.
Theorem 8.2.4 Let co(n) be the space of continUOUS n~al-valued functions on 0 and Hh = {Vh E ('O(O)/vhIK E P{K), K E 71,}, whm~ vhlK denotes the 7l;striction of Vh on}{ and P( K) C HI (K). Then Hh C H1(O).

a (ul K). v.1 K is well defined as til J{ E HI (K) Moreover V; E L (n) as 2 a vii K = -8 (ul K) E L (K)" The theorem will he prnvPd if show that
OX;

Proof.

Let u E H h and

Vi

be a function defined on

n such

that v;l}{ =

x,

Wf'

Vi = a{)U E V (n)

x,

rz.
fll

E V'(n) implies tIlat u E H1(n) which, III turn, implies H h C For all cb E V(f2), we have

en) J.

(v,,)

1
fl

v,dx

I:
KET..

I.

v,dx

~ I:
~

KE7/,,"

La~(U/
,

J()<ixJx

KET..

I: [- Jr (UjJ()gx+ Jr=8K (u/J<)hllar] r K x,

."'" '"'_'_""""'''''''10''-''

by applying the generalized Creen's formula, where flf denotes the i-th component of the outer normal at r Therefore,

flf

The second term on the right-hand side of the above relation is zero as u is c-Ontinuous in fl, and if K 1 and [(z are two adjacent t.riangles, I.hen 2 I = ~11,K Therefore,

which implies that

v,

8& E D'(ll).

x,

I
RemOf'k 8.2.4

Let 11

nodes of the triangulation, P(I<) = PI (K) less than or pqual to J in x and y,

= KETh max

(diameter of [(), N(h)

= the number of

= space of polynomials of degree

l. It (',an be seen that H h C CO(fl),

2. The funCtions

lVi,

i = 1,2, ... ,N(II), defined by at the i-th node at other nodes

form a basis of Hh. 3. In view of (2) and Theorem 8.2.4, /-I h defined in this remark is a subspace of H1(fl) of dimension N(II).

8.2.3

Finite Elements

Definition 8.2.2 In R", a (nondegenerate) n-simplexis the c-Onvex hull [( of(n+l) pointsllj = (aiil~, ERn, which are called the vertices of the

.n",

'.'_'_""""'''''''10''-''

n-simplex, and which are such that the matrix

A~

a" a" a" an

a.,1 a,,2

al,n+1

a2,n+l
a",.,+1

(8.2.17)

is regular, i.e., (n+ 1) points aj are flat cont:uned in a hyperplane. In other words, [( is n-simplex if

Remark 8.2.5 I. 2-simplex is a triangle 2. 3-simplex is a tetrahedron.

The barycentric coordinates >"j = >"j(x), J ::; j 5 n + 1, of any point x E R", with respect tu the (n + J) points aj, are defined 1.0 hp- thp- uniqup- solution of thl' linear syst.em
Definition 8.2.3

n+'
j=1

L a;)>"j =
n+'
)=1

Xi,

(8.2.18)

LA, - I,

(8.2.19)

whose matrix is precisely the matrix A given in (8.2.17).

Remark 8.2.6 1. If J( is a triangle with vertices (1.,(12,(13 and (lij,j = J, 2, are the coordinates of (I" i = 1.,2,3, then for any x E R 2 , the lxlrycentric coordinates >"i(X), i = 1.2.3, ofx will be the unique solution of the lint!ur ::;ysttml

L A.al) =
.=1

Xj,

j = 1,2

LA' ~ \.
1=1

(8.2.20)

B . ,to<l"_""""''''">#1>''-I

..... ,

2. The baryc.entric coordinates are affine functions of XI, xz, .. . ,xn ; i.e., they belong to the space PI (I,he space of all polynomials of degree I).
A,:::= Lb;jxj +bin+i.
j=1

"

< i < n + 1,

(8.2.21)

where the matrix B = (b,j) is the imrerse of matrix A. 3. TllP barycpntrk or center of gravity of 1m n-simplex J( is I,hllt point of !< all of whose barycentric C'.oordinal.es are e<lual tOl / (n + 1).

Example 8.2.1 Let n = 2, then J( is a triangle. Let al,OZ,a3 be its vertic.es. The baryc.entric C'nordinates of al,a2,a3 are Al = (1,0,0), Az = (0,1,0) and A3 = (0,0, 1), respectively. The baryc.entric coordinates of the c.entroid G of K are (I/3, 1/3, 1/3). Remar'k 8.2.7 that
Using Cramer's rule, we determine from E<luation (8.2.20)
XI aZI a31 xz azz un

all a21 a31 a12 azz a32

J J area of the triangle xaZa3 area of the triangle alaza:!'

1

Similarly, area of the triangle alxa3 area of the triangle alaZa3 area of the triangle al azx area of the triangle aj aZa3 .
I. The e<luation of the side aZa3 in the baryc.entric w ... ordinates is ).1 = o. 2. The equation of the side al a3 in the baryc.elltric wordinates is )'z = o. 3. The (Iuation of the side a, az in the barycentric coordinates is A3 = O.

Remark 8.2.8

Let J( be a polyhedron in R", PK the spac.e of polynomials with dimension m and LK a set of distributions

Definition 8.2.4 (Ciarlet, 1975).

.n",

'.'_'_""""'''''''10''--'

with cardinality m. Then the triplex (J(, PI\, 2:K) is called a finite element

if

L
;

= { L; C V' Ii = I. 2.... ,
O:j

m}

is such that for a given

E fl, I

:S: i :S:

Tn,

the system of e<luations

forl<i<m - has a unique solution p E PK. The elements L., i = 1,2, ... , n arc ('ailed degrees of freedom of PI<.

Remat* 8.2.9 L. If J( is n-simplex, (J(, PI\ 2:K) is called the i'!implicial finite element 2. If J( is 2-slmplex, I.e., a triangle, then (J(, PI\, 2:1\) IS called a tnangtdm' finite element. 3. If J( is 3-simplex, i.e., a tetrahedron, the finite element (J(, PI\, 2:1\) is (',ailed tetmhedmL 4. If [( is a rectangle, then (J(, Pk , 2:K) is called a nxtangular finite element. Remark 8.2.10 1_ Many authors ('Insider 2: K as the set of values of p E PK at the veniees and middle points of the triangle and rectangle in the case of triangular and rectangular elements, respectively. 2. Generally, L; is the Dirac mass concentrated at the vertices and the middle point and 2:K may ('Insist of the Dirac mass or Dirac delt<t distribution and its derivative (For Dirac mass, see Examples 5.4.5 and 5.4.10). Remark 8.2.11 I. Very often, J( itself is called a finite element. 2 The triplex (J(,n,{a,}f==I)' where [( is a triangle, PI a space of polynumials of degree:S: 1 and 01,a2,a3 the vertices of [(, is C,8,1Ied the triangldaT finite element of type (f). 3. The triplex (J(, n, {aj }:=I)' where J( is a rectangle with sides parallel to the axes and 01,02, a3 and 04 are corners, is called a 1'Cctangulm' finite element of type (f). Example 8.2.2 (Finite Element of Degree 1). Let [( be a triangle, PK = PI (J() is (:jUal to t.he space of polynomials of degree less than or e<J11al

B . ,~._ .._",,"l1>"-'

..... ,

to 1 and the space generated by I,x,y == [l,x,yJ.

dimPI{==3

L
K

= {O",/a"

= 1, 2, ~ are vertices},

where oai is the Dirac mass

conc~ntrated at

the point ai. Then

(K. PK, L) is a finite element of degree 1.

K

Remark 8.2.12 I. L i = oa, is defined as oa,(p) = pea;), i = 1,2,3. It is a distribution. In order to show that L I (p) = a" J < i ::::; 3, has a unique solution, we need to show that peal) = ai, 1 < i < 3, p E PI (K) has a unique solution. This is equivalent to showing that p(a.) = 0 has a unique solution Since this e<luation has a solution, it suffices to show that it is unique. \Ve know that a polynomial 1) E PI (K) is c-Ompletely determine<\ if its values at three noncollinear points are given; so p(ad = 0 has a unique solution. 2. If A, are the barycentric coordinates,
Oa,(A J ) = AJ(a,) = O,j.
Hence, Aj, j = 1,2,3 forms a basis for PI (K) and if p E PI (K), then
3

P ~ I>(a;).\;.

,=1

Example 8.2.3 (Finite Element of DC9ree 2). Then

PI{ = Pz(K) = [J,x,y,xZ,xy,yZj

Let K be a triangle.

L
K

[0""0,,,,/1 ::::; i < 3,

1 <i< j < 3},

where ai denote the vertices of K and aij the middle points of the side aiaj' (K, I:K' PK) is a finite element of degree 2,

Example 8.2.4 (Finite Element of DC9ree 3).

dim PK = 10

Let K be a triangle,

PI{ = P3(K) = Span{J,x,y,x2,xy,y2,x3,xZy,xyZ,yZ}

L
K

= {O"" , 0"",,00128 /1

< i < 3, 1 <j S 3},

."'" "'_'_""""'''''''10''--'

where a. denote the vert,ices of J(, 0",,,,, is the centroid of J( and a;;)

~l4 + ~aj'

Then (J(, PK, I:K) is a finite element of degree 3. If 1)) E PK

j ::; m, is such that
111,

Definitton 8.2.5

::;

~.(Pj) ::::: O'l') < i < m, 1::; j <

L. E

E,
K

then {Pi J fonus a basis of PI{ and any p E PK can be written as

p~

L L.(p)]';.
;=1

{p)} is ('<1,1100 the sequence of basis functions of the finite element.

Definition 8.2.6 Let l K, PK, I:K) be a fimte element for each J( E Ti" where Ti, is a triangulation of a polygonal domain fl. Let =

L
K

UL
K

KETh

and
H h = {Vt,fVf,/J( E PK,
J( E

Ti,}.

\Ve say that t.he finite element method is oonforming if H II Otherwise, it is caIled nonconforming.

HI(fl).

8.3 Applications of the Finite Method in Solving Boulldmy Value Problems

In this section, we explain by examples how the finite element method is applied to resolve boundary value problems. The same procedure can be adopted to resolve other boundary ,a1ue problems of Chapter 7. Let us illustrate different steps in the finiw element method solution of the following one-dimensional two boundary problem:
Example 8.3.1

J'u -- + u <ix'
"(0)

= f(x)
~

<x <1

(8.3.1 ) (8.3.2)

uO) ~

u,

where f(x) is a C'.ontinuous function on {O, II .

B.

......

, to<l',_"""",,,,,"l1>"-I

We further assume that f is such that Equation (8.3.1) with (8.3.2) has a unique solution. Let H = {vii" is a continuous function on [0,11 and 1," is piec\\lse c.ontinuous and bounded on [0,11, and v(O) = v(l) = O}. Multiplying both sides of (8.3.1) by an a~bitrary function v E H and integrating the left-hand side by plUts, we get

1 C~;::
1

+uv) dx =

1
1

f(x)v(x)dx.

(8.3.3)

We can write (8.3.3) as

a(u,v) = F(v)
where

for every v E H,

(8.3.4)

a(u, v) =
and

r' 10

(dUdU dx d~'

+ uv ) <!x,

(8.3.5)

F(v) =

1
1

f(x)v(x\dx.

(8.3.6)

It can be seen that a(,) given by (8.3.5) is symmetric and bilinear form. It can be shown that finding the solution of (8.3.4) is equivalent to finding the solution of (8.3.1) and (8.3.2). Now, we discretize the prohlem in (8.3.4). We consider here H n = {vhl VII is continuous piecewise linear functions}, h = ~. Let 0 = Xo < Xl < X2 < ... < x" < X,,-,-l = 1 be a partition of the interval [0, I] into subintervals I) = [Xj_J,Xj] oflength It} = x) - X)_I ,j = 1,2, ... , n+ 1. With this paltition, H n is assodated with the set of all functions v(x) that are continuous on [0,1] linear on each subint.en'3l l),j = 1. 2, ... n + 1, and satisfy the boundary conditions v(O) = v( I) = 0, We define the basis function {iPl, iP'2,' " iPn} of Hn <lfi follows:

(,.) () {I
'P)
Xi

0 if

if i = j

if:i

(ii) iPj(x) is a c.ontinuous piecewise linear function.

iP)(x) can be eomputed explicitly to yield

.n",

'.'_'_""""'''''''10''-''

See Fibrure 8.3.1. Since 'PI. v>z,'f'3, .. , 'f'n are the basis funct.ions. any v E H n can be written as
n

v(x) ~

I>o",,(xj,
i=:= I

where V; = V(Xi).

It is dear that H" C H The discrete analogue (g.JA) reads: Find u" E H n such t.hat
a(Un,v) = F(v)
n

fur every v E H n .

(8.3.7)

Now, if we choose Un =

L o:;r,o;(x) and observe that Vluation (8.3.7) holds

;=1

for every function 'f'j(x), j =: 1,2,3,.

, n, we get n equations, namely

a (f;a;'f'i''f'J) =F('f'j)
By the linearity of a(, .), we get

foreveryj=:I,2,3, .. ,n.

Fisure 8.3.]

Basis Junction in Example 8.3.1

.n",

"'_'_""""'''''''10''-''

or

Lo,a((,O;,(,Oj)=F(l/Jj)
i=l

"

foreveryj=I,~,3>... ,n.

This can be written in the matrix form AS =

(r~),

where A = (aij) is a symmetric matrix given by

and

W"),

F(<p,).

The entries of the matrix A can be eomputed expliCItly. We first notice that
G.j

= aii = a(r,.o',iPJ) = U if

Ii - jl 2 2.

This holds due to the local support of (,O;(x). A direct computation ghres

n,

GjJ

o
b,

a, a,

(Fn ), (F"h

(8.3.9)

B.

.... ,

, to<l'._..

....,..",,"l1>"-I

h . In the special hj hHl 3 h) 6 case of uniform grid h) = h = ] In + 1, the matnx takes the form

where

aj

= -

+ -- + -

[h,

+ hH1 ] and b)

= --

+ ---.L.

o
1 L 4

Example 8 ..'/.2 problem

Let

liS

consider t.he Neumann homogeneous boundary in

tl.u+u=j ou = 0

n,

OCR' }

on

on I

(8.3.10)

The variational formulation of Equation (8.3.10) is lIS follows:

H ~ H'(O)
a{u,v) = = L(v) =

f{2 at/ov 10 ~OXiOXi

f'vdx.

+uv dx
(8.3.1l)

(\7u \7v + uv)dx

An internal approximalion of this problem is as follows:

Hh = {vh E

CO(n)/

't/KE

Th, Vh/T< E PI (l<)}.

(8.3.12)

This is the result of Theorem 8.2.4. A bm;is of JI h is given by the following relation:

1IJi(aj) = Oij,
N(h)

1 <i -:; N(h),

]::; j::::: N(h),

where al, a2, .. . ,aN{h) are the vertices of the triangulat.ion 7". We have
'r/ Vh E

lh, Vh

vh(a.)wi

<=1

The finite element solution of Equation (8.3.11) is equivalent to finding Uh such that
(8.3.13)

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

Since {Wi}:S,j is a basis of fh, the solution

Nth)
Uh

"(kW~"
~/k

(8.3.14) are the solutions of the

J'=I

of Equation (8.3.13) is such that the coefficlems following linear system:

N(h)

a(wl;,l/Jthk = L(wt}

for 1 ::;

e< N(h),

(8.3.15)

k=1

where (8.3.16)
and

(8.3.17) Thus, if we know the stiffness matrix a(w~",wt) and the load vector L(wd, "(k can be determined from Equation (8.3.15) and pUHing these values in Equatiun (8.3.14), the solution of Equation (8.3.13) can be calculated.

Practkal Method to Compute a(wj , w,). We have

a(u,v) =
=

(Vu Vv\- uv)dx

L r (Vll' Vv + uv)dx.
KET..

10

In the element

J(,

we can write

,
llo= 1

u(x) - " U m"II' >'0 (x) L..J K

V(X) = " L
p=l

llml.'j{>'p (x), K

where (m o K)a=1.2,3 denotes the thfef' vertices of the element K and >.~ (x) the associated barycentric coordinates. Then Equation (8.3.18) can be

...... '"'_'_""""'''''''10''-''

rewritten as

where
K ao{3 =

J.

(y>.K . y>.K a {3

+ ),K>.K)dx' o{3

The matrix A K -= (a:;'{3)I:'So9.1:'S{39 is called the element :stiffne:s:s matru of K.

8.4

Basic Ingredients of Boundmy Element Method

First, we discuss weighted residual methods, in verse problem and boundary solutions. The boundary element method is explained with the help of Laplace equation with Dirichlet and Neumann boundary conditions. For more details such as error estimation, coupling of finite clement and boundary element methods and applications of this method to parabolic and hyperbolic equations, we refer to {I, 4-13,18,21,24,37,40).

8.4.1

lVeighted Residuals l\JetlJOd

In Section 7.2, we have seen that a boundary \'<Ilue problem can be written in the form of an operator equation. Let us consider a boundary value problem

Tu= -finO 8u = gon r 1 Lu = Ii on r 2

(8.4.1 ) (8.4.2) (8.4.3)

where n c R2 , r = r l +r2 = boundary of the domain n, T : Dr <; H --t H is a bounded linear self-adjoint operator and H is a Sobolev space of order 1 oc 2 (H'(n) oc fl'(n)). (8.4.1)-(8.4.3) can be writtell as

Au=finO.
If v E D T is such that

(8.4.4)

(Av - f,1/1,,) = 0 for every k = 1,2,3, . .. ,

(8.4.5)

..... ,

Q . ,>0<1"_""""'''''''IO''-I

where {'l/Jk} is a basis in fl, then Av - f :::: 0 in H, that is, v is a solution of (8.4.4). Thus, finding a solution of (8.4.4) is equivalent to finding a solution of (8.4.&). This fact is the basis of the weighted residuals method. We observe that an element w is not nceesf;arily represented with respect to the basis {?fil}' Any basis {'Pd in "DT can be used to represent w, while the residual (8.4.6) where

WN ::::

L O:,'Pi
1='

(8.4.7)

and N is an arbitrary but tixed positive integer, is made orthogonal to the subspace spanned by 'l/Jk, namel)'
k:::: 1,2,3,.

(8.4.8)

Equation (8.4.8) is known by different names for different choices of 'l/Jk. All these methods are grouped into a class of methods known as the weightetl H:.sidual.s methOll.s. In the weighted residual method, we look for an appropriate solution WN of Equation (8.4.4) in the form of Equation (8.4.7) in which 0:, are to be determined by (8.4.8). This gives N equations for N unknowns 0'\,0'2, ... ,UN. II A is linear, then (8.4.8) takes tile form
N

Lo,(AI'",p,) = (f.,p,).
i=\

k=1.2.3..... N.

(8.4.9)

It may be observed that the requirement 'Pi E "DA means that 'Pi are differ-

entiable 2m times; A is a differentiable operator of order 2m, and satisfies the specified boundary conditions. For more details, we refer to Finlayson

[23J.
We take V;I as Dirac delta functions. Then (R, w)::::

jn

r RwdO:::: 0, where

are Dirac delta functions. In t.his case, till' weighted residuals method is called the collocation method (see, for example [13, 21, 34]).

Q.

......

, to<l',_"""",,,,,"l1>"-I

8.4.2

Inverse Problem and Boundary SoluUons

III the weighted residual methud, une looks for functiuns which satisfy the boundary conditions (or some of them) and arp approximate in tllP domain, that is, not satisfying exactly the governing equations. Conversely, one could propose using functions which identically satisfy the governing equation but only approximately satisfy the boundary conditions. To formulate these ideas clearly, we consider

T :::: D. == Laplace's operator

Sit==uonr 1
{)u
Lit::::

1".4lU) (8AlI ) (8.4.12) (8.4.13)

In
au on

an on r 2
~ b)wdO:::: O.

(D.u

Integrating by parts once, we have

r t(~" ~w +In")dn~ Jr qwdr r Jo k=l vx" vX"

where q :::: Integrating (8.4.14), we get

(8.4.14,

Jo

{ (.6.w)udn == { u ~w dr

Jr

Vii

Jr

{ qwdr

+ ( bwdn.

In

(8.4.15)

By introducing the boundary conditions Su :::: u :::: U on r

Lu
1

== q == q on f 2 ,

Equation (8.4.15) ca.n be written as

{ (.6.w)udn == { -

h,

u~w dr + {
vn

Jr2 Jrz

c:
vn

df
(8.4.16)

Jr,

{ qwdr - { qwd'

+ { bwdO.

Jo

By int.rndudllg the boundary mnditions in the above manner, we are making an approximation which can be demonstrated by integrating by parts Equation (8.4.H)) twice to retrieve the operator .6. on u. This yields

Jo

{ (.6.u - b)wdn == { (q - q)wdr - { (u _ u) ~w dr.

Q.

.... ,

Jr z

Jr,

(8.4.17)

, to<l'._..

....,..",,"l1>"-I

We observe that Equation (8.4.17) can be written in terms of three errors or residuals as (8.4.18) where

R = .o.li- b; R. = U -u,R2 = q - q,
For homogeneolls Laplace equation .o.u = 0 on fl, Equanon (8.4.17) takes the form

1
fI

(t.,,)wdll

1
r2

(q - q)wdr

-1 ("r,
(Ii -

u) ~v dr n

(8.4.19)

asb=O. As .o.u = 0, we have

to

satisfy

1
r2

(q - q)wdf =

1 it)::
r,

dr.

(8.4.20)

For u and w chosen as the same function, we obtain a method known as the method of Ttdftz, :SL'C for example [13]. By Green's theorem, (8.4.21) where r = f l + f 2 By choosing u and w as the same function, we have .o.u = .o.w = 0 and can write w = ou. In view of this (8.4.21) becomes

1
r

8" 8udr = n..

vI/.

1 86"
['

U - dr.

8n

(8.4.22)

ApplYlIlg boundary conditions (8.4.22) takes the form

1
r,
8.4.3

q8udr

1
r?

qoudr =

1f't

it

86"'18J" dr. m ell + u On

/.

r2

(8.4.231

Buundary Element MetllOd

Boundary element sdlemes are generally based on inverse relationship l8.4.16). For the weighting function, one uses a :set of basis functions which eliminates the domain integrals and reduces the problem to evaluating integrals on boundary. TllP following steps are involved in solving a boundary value problem by the boundary element method:

...... '.'_'_""""'''''''10''-''

(i) Conversion of the problem into the boundary integral equation. (ii) Discretization of the boundary r into a series of elements over which the pot.ential and its normal derivatives are supposed to varyaccording to interpolatio!l functions. These elements could be straight lines, cirl'lIlar areas, parabolas, ptc (iii) By the collocation method, t.he discretized equation is applied to a number of particular nodes within each element where values of the potential and its normal derivatives are associated. (iv) EvaluatlOTI uf the integrals over each element by nurmally usiug a numerical quadrature scheme. (v) To derive a system of linear algebraic equations imposing the prescribed boundary conditions and to find its solution by direct or iterative methods. (vi) DeterrninatlUIl of values of the functiull u at the internal puints uf domain under consideration. We illustrate these steps with the aid of the following example: Let us consider the boundary value problem

",,(x) ~ 0 in n ll(X) :: Ii 011 r l { an

on

(8.4.24)

= q(x) on

r2 ,

where r = r 1 -r r 2 ,r = boundary of the region n c R 2 . The weighted residual equation (8.4.17) takes the following form for this boundary valul'! problem:

r r in "n(x),,(x,y)dn(x) ~ ir - Jr, (,,(x) r

'(

(q(x) -q)n'(x,y) dI'(x)

(8.4.25)

n(x))q'(x,y) drtx) ,

where uC) is interpreted as the weighting function and

q x,y -

) _ o,,'(x,y) iJn(x)

(8.4.26)

Q.

.... ,

, to<l',_"""",,,,,"l1>"-I

Integrating by parts (8.4.25) with respect to Xi, we have

i :~:~~

a~:~~;f)

dO(x) = - [ , q(x)u(x,y)dr(x)

_Jr, q(X)iLO (x, y)dr(x) r - Jr, I"(x) - "(x)] q'(x,y)dr(x) , r

(8.4.27)

where i = 1,2 and Einstein'f; summation convention is followed for repeated indices. Using integration by parts once more, we get

In

6,,,'(x.y)u(x)dll(x)

-l +l

q(u)u'(x.y) d

r(X)}

(8.4.28)

tt(x)q' (x. y)df(x) ,

keeping in mind that r = r l + r 2 . Recalling the following properties of the Dirac delta function o(x, y):

o(x,y)

~ 0, ;[

In

00,

if X = y

X" Y

(8.4.29)

u(x)o(x,y)dfl(x) = ,,(y),

assuming u'(x,y) to be the fundamental solution of two-dimensional Poisson's equation: namely,

6,u'(x,y) = -2 rr o(x,y),

(8.4.30)

and putting the value of tlu'(x,y) from (8.4.31) into (8.4.29), we have
2rr u(y)

u(x)q'(x,y)dr(x) =

i I

q(x),,'(x,y)dr(x).

(8.4.31)

Considering the puint y to be on the boundary and accounting for the jump of the left hand, (8.4.32) yields tlw integral equation un the boundary Qf the given domain 0 (boundary integral equation)
c(y)u(y)

+[

iL(X) q{x.y)dr{x) =

q(XhLO(X,y)dnx).

(8.4.32)

Remark 8.4.1 (i) If we are looking for a solution of the Neumann bounrlary problem, then the right hand side of (8.4.32) if; known and for the desired solution, we have 1.0 solve a Fredholm equation of the f;econd kind.

...... '.'_'_""""'''''''10''-''

(ii) If we are interested in solving a Dirichlet boundary problem, the values of u(x) are prescribed throughout r and the problem reduces to solving a Fredholm equat.ion of first kind in unknown q(x) = &n(x) , normal derivativc. (iii) The solution of a Cauchy (mixed) boundary problem will require solution of a mixed integral equation for the unknown boundary data. The integral equation given in (8.4.:12) can be discretized into a large numucr of elellients. Thc uUUlH.1ary i:; divided into :;cgIllcnts of constant. and linear forms as shown in Figures 8.4.1 and 8.4.2.

au

(i)
Figure l:S.4.1 Constant boundary elements

ror the constant. element. case, the boundary is discretized into N elements. Let N) belong to r j and N 2 to f 2 , where the values of u and q are takcn to be constant on each element and equal to the value at the midnode of the element. \Ve observe that in each element t.he value of one of the two variables u or q is known. Equation (8.4.32) can be writtcn in the form
qUi

u q*(lr =

quO

dr,

(8.4.33)

wherc

Ii'

Q.

.... ,

~ log( ~).
2rr r

Here we have chosen x.(y) =

Ui

and c(y) = 4'

, toOl',_",,,,,,,,,,,,"lO"-I

(ii)
Figure 8.4.2 Linear boundary elements

(8.4.33) can be discretized as follows:

C,lt,

+L
)=1

Ir

r,

uq' d

j=l

( L Jr, u*qdr

(8.4.34)

For a constant element, the bouudary element. IS always smooth; hence the Ci coefficient is identically equal to ~. f j , denote the length of element j. (8.4.34) represents in discret.e form the relationship between node i at which the fundamental solution is applied and all the j elements, including the ca.,,!'! i = j, on the houndary. The values of 1t and q inside t.he integrals in equation (8.4.34) are constants at one element and, consequently, we have
(8.4.35)
Let fl.) =

Jr,

q*

dr, and G.)

Jr,

t/ dr.

(8.4.36)

(A symbol involving i and j is used 1.0 indicate that the integrals ( q' rl

Jr,

relate the i-th node with the element j over which the integral is taken) .

Q.

.... ,

, toOl',_"""",,,,,"l1>"-I

Equation (8.4.35) takes the form

2u , + L

N _
Hi,Uj

L
,=1

G i , qj.

(8.4.37)

j=l

Here, the lIuegraJs in (8.4.36) are simple and can be evaluated analytically but, in general, numerical techniques will be employed. Let us define

H"
H ij =

ifi

, Hi' + 2"
N

(8.4.38)

ifi=j,

then (8.4.37) can be written as

N
j=l

LHijUj = L
j=l

G;j qj,

whieh can be written in the form of matrix equation as

AX = F,

(8.4.39)

where X is the vector of unknown u's and q's (Either u's or q's will be known and X has N components), and A is a matrix of order N Thus, all the values of potentials and fluxes on the boundary are knuwn and une could calculate tllP vaJues of potentials and fluxes at any interior point using
Uj

l
=

qu d

-l
N
,=1

q*

dr.

(8.4.40)

Equation (8.4.40) represents the integral relationship between an internal point, the boundary values of U and q and its discretized form is
N
U.

LQ,G1j - Lu,H ij
j=1

(8.4.41)

The values of internal fluxes can bf' determined by differentiating (8.4.40) which gi ves us

:~ = -ll
where
XI

q U - <If

Qq'

ax/

(8.4.42)

are the coordinates, l = 1,2 .

Q.

......

, to<l',_"""",,,,,"l1>"-I

Remark 8.4.2 (i) The values of u(x) at any internal point of the domain are determined by equation (8.4.32). (ii) The values of derivatives of u, at any internal point y with Cartesian coordinates x;(y),i == 1.2 can be determined from the equation
8u(1/) ~ _'_ { q(x) 8u"(x, 11) dr(x) _ 8x,(y) 20'1f ir 8x,(y)

ir

r u(x)8q"(x,y) d r(X)} .
8x,(y)
(8.4.43)

It is dear that (8.4.43) is obtaim..od by differentiating (8.4.32) under ap. propriate conditions. (iii) A significant advantage of the boundary element method is the relaxation of the condition that the boundary surface is smooth (Lyapunov), that is. it can be used for surfaces having corners or edges. Remark 8.4.3 H;i and e'i can be evaluated by simple Gauss quadrature values for all elements (except one node under consideration) as follows:
Hi,

Jr,

(8.4.44)

and
(8.4.45)

where Ii is the element length and Wk is the weight associated with the numerical integration point k.
Example 8.4.1 Solve the followlIlg boundary value problem by the boundary element method:

d'u dx'1 +u
u(O)
~

== -x in n, n == (0,1) }
~

(8.4.46)

u(l)

O.

Solution. get

Since Liw == -x '1 d

J'w

+ wand b == -x.

br Equation (8.4.16). we .

,(J'W) 1 "'d2 +
OX

w 1Ldx +

l'
0

xwdx

+ [qw];;::;;b - [dW] .~, == o. udxx=o

(8.4.47)

...... "'_'_""""'''''''10''-''

Very often, in boundary element method, the fundamental solution is chosen as the weighting function w. This solution will be indicated as an a.<;terisk, that is, w , to emphasize its special character and is the solution
of

(Pw

dx 2 + w :::: OJ,

(8.4.48)

where OJ indicates a Dirac delta function which is different from zero:lt the point i of coordinate {. The Dirac delta function is such tlmt

r' dx2 10 (<1'w

+w.. ) udx:::: 10 O;'l.Ulx::::Ui.

r'

(8.4.49)

Ln view of this and taking into account u :::: 0, we get

'1..1,. =: -

10 xw 0d x - [qw 0j,,"=l "",,,0'

r'

(8.4.50)

The function w which satisfies

10

r'

dW]'=' (<1'' +h+X) wdx-r [(U-U)dx x=o =0

&1'2

(8.4.51)

1 or (8.4.49) is w" :::: "2 sin T, where l' :::: Ix - {I \Ve can substitute the value of w" into Equation (8.4.50) to obtain a system of equations (one at x:::: 0; the other at x:::: I) fTOm whidl the two values of q at x =: 0, x :::: 1 can be found. That is,
qo:::: - . - - J.q,:::: -.--1.

1 sml

cos 1 sml

Note that Equation (8.4.50) can he USf>d t.o calculate the values of the u function at any internal point. If we choose ( to be at the midpoint of our internal domain, we obtain the value of u at x :::: as

,,(1) ~ _'2 10 xsin (~-x) dx- ~f' r' 2 2 2

.
- ql Sill

xsin

1/2

(x-~) dx 2

"2 + flo S1II "2'

I I

in which

,. _
{

--x
2

Q . ,~,_ . ._",,"l1>"-'

..... ,

-- 2 . -

forO<x< '2 1 for "2 < x < I.

Thus, we obtain
11

(I)
2
-

= --f(cosl-l)/smlJslll--2 2 2

. I 1('--sm-1) .
2 2

-~(~+sin~~coo~)
= 0.06974691

which

IS

the same as the exact solutIOn, that is,

" ( ' ) = sin

sin I

" 1 = 0,069746964.

8.5

Problems

Problem 8.5.1 Illustrate the differcnt aspects orthe finite element method through the one-dimensional boundary value problem
- d,x2 =1,

<1'"

0< x -

< -

I,

with u(O) = 0,

u'(O) = O.

Probrem 8.5.2 Discuss a finite element solution of the followmg boundary value problem:

--1 2
<X

cP11

+ 11(X)

= j(:c),

O<x<1

11(0)=11(1) =0. Problem 8.5.3 Discuss thp error between thf> exact and finitp element solution of the following differential equation:

- dx 2 = 4,
,,(0)
~

<I'u

0<x
~

<

u(J)

O.

Problem 8.5.4 Let 0 =:co < XI < X2 < ... < X n = J be a partition of [0, 1] and V the vector space of functions v such that.

P)

EC"([O, In,

..,,,. '.'_'_""""'''''''10''--'

(ii) v/[x'_l,x,l is a linear polynomial, i:::: 1, ... ,n and

(iii) vIOl :::: O. Let H:::: {v E L 2 (O,I)/

it (~:)
E

dx <

00

and v(O)::::

OJ.

Let for each Let Show

UJ

i :::: 1,2,.
interpolant that

,n. li', be defined as li'; (x j) :::: 8'3 :::: l{ ronecker delta.

VI

E V fOI"
:0::::

COlO, l' be defined as

1-"1 ::::

" L v(xdli'j.
;=1

for all u E H, where h:::: 1ll?X (Xj dT IS:,~" is the imerpolant of U and G is independent of h and u.

lIu -

U/II

Gh

([2u

-2

x.-d,

Problem 8.5.5 Let H and V be two Hilbert spaces. Let P denote the problem of finding u E H such that

A(u.v) :::: F(v) for all v E F.

where A : H x V --+ Rand F E V, dual space of F. Prove that P has a unique solution u E H provided there exist constants Q' and (j such that
(a)

IA(u,v)1 < iJllullllvll

A(u, v)
"EV

foc all u E H and v E ~';

u E H;

(b) sup

II II

.."0

V V

> olluJiH for all

(c) sup A(t/,v)

"EH

> 0 for all v E V, v- O.

:0::::

F'urtherillore,

lIul1H

IWllv
a

Problem 8.5.6 Formulate a theorem analogous to Theorem 8.2.1 for the bilinear form considered in Problem 8.5.5. Problem 8.5.7 Find the error between the exact solution and a finite element solution of the following boundary value problem:

- - (I+x)dx dx u(O) :::: 0,

d[

dUJ

~O,

O<x<l
~1.
x=1

Q . ,~._ .._",,"l1>"-'

......

dU] [(1+x)dx

Pmblem 8.5.8

Solve the following BVP:

-6u = 2(3' + y)in the square with vertices (0.0),(1.0),(1.1).(0.1). where the boundary cUllditiuns are
u(O,y)

= y2,

u(x,O)

= x 2, u(l,y) = 1 02 u 8 2u 8 x '+8112 =4

y, u(x, 1)

= 1- x.

Problem 8.5.9

Solve the Poisson equation

with boundary conditions u element. mpthod.

= 0 un r(x = 1, 11 = l) by the boundary

...... "'_'_""""'''''''10''-''

References
(11 H Antes, PP Panagiotopoulos. The Bonndary llltegral Approach to Static and Dynamic Contact problem~. Bo~ton, Basel: Birkhau~er, 1992. [2] JH Argyris. Energy theorems and structural analysis. Aircraft Gngg

26:317-356,383-387.391.1954.
[31 Babuskii, AI< Aziz. Surve)1 lecture on mathematical foundations of the finite element method. In: AI< Aziz, ed. Mathematical Foundations of the Finite Element Method with Applications to Partial Differential Equations. New York: Academic Press, 1972, pp 3-359.

[41 PI< Banerj'f'. The Boundary Element Methods in Engineering. London: tvIcGraw-Hill, 1991. [5] CA Brebbia. The Boundary Element Methods for Engineers. London: Pen tech Press, 1978. [61 CA Brebbia. Topics in Boundm"y Element Research. VoL 1. Berlin, New York: Springer, 1984.
[7] CA Brebbia. Topics in Boundary Element Research. Vol. 2, New York, London: Springp.r, 19&'). [8] CA Brebbia. (00.) Boundary Element X. Berlin: Springer Verlag, 1988. [9] CA Brebbia. (ed.) Boundary Elf'lllenl. Technology VI. London: Elsevier (Appl. Science), 1991. [10] CA Brebbia, GS Gipson. teds.) Boundary Elements XIII. Berlin: Springer-Verlag, 1991.

...... "'_'_""""'''''''10''-''

[l1J CA Brebbia, M Tanaka, T Hanna. (eds.) Boundary elements XII. Berlin: Springer Verlag, J 990 [12J CA Brf'bbia, S Walker. Boundary Element. l'P.chniqnp.!\ in Engineering. London: Newncs-Butterworths, 1980.

[131 CA Brebbia, JCF Telles, LC Wrobel. Boundary Element Techniques.

Iledin, Heidelberg: Springer Verlag, 1984 (Theory and Applications in Engineering).

[141 SC Brenner, LR Scott. The Mathematical Theory of Finite Element

Methods. New York, Berlin: Springer Verlag, 1994.

\151

F Brczzi, MFortin. Mixed and Hvbrid Finite Element rvlethods. New York: Springer Verlag, 1991

[J61 MVK Chari, PP Silvester. (eds.) Finite Elements in Electrical and

Magnetic field problems. Chichester: John Wiley and Sons, 1980. [!7J G Chavent, J Jaffre. Mathematical Models and Finite Elements for Reservoir Simulal.ion. Am~t.f'rdam: North Holland, 1986 [18J G Chen, J Zhou. Boundary Element lvlethods. Cambridge: Academic Press, 1992. [19J PG Ciarlet. The Finite Element Methods for Elliptic Problems. Amsterdam: North Holland, 1978. (20] PG Ciar[et, JL Lions. (eds.) Hand Hook of Numerical AnalysisFinite Element Methods. Amsterdam: Elsevier Science Publishers, Vol 2, 1991.

[21/ R Dautray, J Lions. Mathematical Analysis and Numerical Methods

for Science and Technology. Vo[s. 1,2,3,4,5,6, Berlin, Heidelberg, New York: Springer-Verlag, 1990-1995. [22] RS Falk. Error \--'l;timates for the approximation of class of variation inequalilies. Mati. Comp 28:963-971, 1974

[231 BA Finlayson. The Method of Weighted Residuals and Variational

Principles. New York: Academic Press. 1972.

",,,. '.'_'_""""'''''''10''-''

[21J W Haekbusch. Integral Equations, Theory and Numerical Treatment

Basel, Boston, Berlin: Birkhauser, 1995. [25] J Jin. The Finite Element tvlethod in Electromagnetics. New York: John Wiley and Sons, Inc., 1993

[26J H Kardestuncer, DB Norrie. Finite Element HMdbook. New York:

McGraw Hill Book Company, 1987. [27] VD Kupradze. Potential Methods in the Theory of Elasticity. Jerusalem. Israel Scientific Publ., 1968. [281 SG Mikhlin. Integral Equations. Oxford: Pergamon Press, 1957. [29] SG Mikhlin. Approximate Solutions of Differential and Integral Equations. Oxford: pp.rgamon Prpss, 1965. [301 H Neunzert, AH Siddiqi. Topics in Industrial Mathematics-Casf' Studies and Related Mathematical Methods. Boston-DordrechtLondon: Kluwer Academic Publishers, 2000. [311 JT Oden. Finite elements: An introduction. In: PG Ciarlet, JL Lions, eds. Handbook of Numelical Analysis, Vol.II, Finite Element Methods (Part I). Amsterdam: Elsevier Science Publishers, 1991, pp 3-15.

[32J A Qualteruui, A Valli. Numerical Approximation of Partial Differential Bwations. New York, Heidelberg, Berlin: Springer-Verlag, 1994. [331 IN Reddy. An introduction to the Finite Element tvlethod. New York, London: McGraw-Hili, International Editions. 1985.

[31: IN Reddy. Applied Functional Analysis and Variational Methods in

Engineering. New York: McGraw-Hill, 1985.

[35J K Rektorys. V<'lriational Techniques. Dordrecht. Boston: D. Reidel

Publishing Company, 1980.

[36] S HjasMOW. Approxunation of Boundary Elements Matrices. "lndustrial Mathematics" , Lecture Notes Workshop on New Delhi, February 23-26. 1999 (Coordinator AH Siddiqi) .

Q.

......

, toOl'._..

....,..",,"l1>"-I

[37J AH Schaltz, V Thome, WL Wendland. Mathematical Theory of Finite and Boundary Finite Element Methods. Basel, Boston, Berlin: Birkhauser-DMV Seminar l30nd 15, 1990. [38J AfT Siddiqi. Functional Analysis with ApplicatIOns. New Delhi: 4th plint, Tata McGraw-Hill, 1986. [391 RP Silvester, llN Ferrari. Finite Elements for Electrical Engineers. Cambridge: Cambridge University Press. 1983 (edition 1990 (editioll U).

n.

[401 E Stein, \VL Wendland. (eds.) Finite Element and Boundary Elemem Techniques from Mathemtical and Engineering Point of View, WienNew York: Springer-Verlag, 1988. [41] G Strang. Variational crimes in thp finitp. p.lement method. In AK Aziz, ed. The Mathematical Foundations of the Fmite Element Method with Applications to Partial Differential Equations. New York: Academic Press, 1972, pp 689-710. [42/ F Tricomi. Integral E~]uations. London, New York: Interscience Pub!. 1957, lleprint New York: Dover Publ, 1985. (43 MJ Turner, mv Clough, HC Martin, LJ Topp. Stiffness a.nd deflection analysis of complex structures. J Aero Sci 23:805-823, 1956 [441 WB Wahl bin. Supercullvergeuce in Galerkin Finite Elenlent Methods. Lecture Notes in Mathematics. Bonn, Heidelberg: Springer Verlag, Vol. 1605, 1995. [45] It Wait, All tvlitchell. Finite Element Analysis and Applications. Chichester, New York: John Wiley and Sons, 1985 [46] XH Wang. Finite Element Methods for Non-linear Optical Wave guides. Advances in Non-linear Optics, VoL 2, Amsterdam: Gordon and Bread) Publishers, 1995. [47J J Whiteman. The Mathematici'; of Finite Elements and Applications, I, II, In. Pmc. Conf, Brunei University, Academic Press, 1973, 1976, 1979, London: Academic Press, Harcourt Brace Jovanovich Publishers, 1990.

",,,. '.'_'_""""'''''''10''-''

[481 OC Zienkiewicz, YK Cheung. The Finite Element Method in Structural and Continuum Mechanics. New York: McCraw-Hill, 1967.
[49) M Zlamal. On the finite element method Numer Math L2:394-409. 1968.

[50J E Zidlcr. Nonlinear Functional Analysis. Vol. II A and B. New York, Berlin: Springer-Verlag, 1990.

Q.

......

, to<l',_"""",,,,,"l1>"-I

Chapter 9

Variational Inequalities and Applications

9.1 Motivation and Historical Remarks

In tillS chapter, ",-e discuss fundamental results of varianonal inequalities including recent work of J.L. Lions on parallel al&'Orithms of evolution variational inequalities. Modelling, discretizalion, algorithms and visualization of solutions are vital components of Industrial Mathematics. In olden days, the knowledge of calculus, matrices, and differential equations was thought to be sufficient for studying modeling of problems of physics, engineering, economics, business and trade. However, in the last couple of decades several new areas of mathematics have been developed knowledge of which has become inevitable to understand and analyze properly the problems encountered in dbciplines like designing and manufacturing cars, aircraft, spa<:enaft, railway engines and tracks, different organs of robot for hazardous industries, dams, heavy machines, life fatigue, image proce.'ising revolutionizing, electronic and computer industries; composite materials, economic and financial systems. In fact, motivation for invention of new topics in mathematics has been provided by the problems of these and many other areas. The theory of variational inequalities having the life of about three decades is one of such branches of mathematics which has been applied in the above mentioned areas. This theory has provided solutions to several problems of physics and engineering earlier thought to be inaccessible. The beauty of this theory lies in the fact that the unknown part uf an elastic body which is in contact with a rigid support or rigid punch h~comes an intrinsic part of the solution and no special devices are needed to locate it. It models free boundary value problems.

...... "'_'_""""'''''''10''-''

9.1.1

Contact Problem (Signorini Problem)

In virtually ~very structural and mechanical system, ther~ romps a situation in which one deformable body comes in contact with another body. The contact of one body with another is, in essence, how loads are distributed into a structure and is the mechanism, whereby structures are supported to sustain loads. Therefore the nature or character of this contact may play a fundamental role in the behavior of the structure namely its deformation. motion and the distribution of stresses, etc. The general problem of the equilibrium of a linear elastic body in cuntact with a rigid frictionless body was proposed by Signorini in 1933 who himself presented a more comprehensive account of this theory in 1959 when he considered that the base is a rigid body and one looks for deformation in the elastic body kept on it. The first rigorous analysis of a class of Signorini problems was presented by F'ichera in 1963. He formulated the problem in the form of an inequality involving an operator (bilinear form) and a functional, and proved the existence of a solution in a convex subset of a Hilbert space. Simultaneously. St.ampacchia presented a I.h(>()t1, of variational illequalities i1l\'olving non-symmetric bilinear form and published the pioneering paper jointly with Lions inI967. In 1972, Dm'aut and Lions wrote a book modeling important phenomena of physics and engineering in ternls of variational inequalities which became the main source for future applied research in this area. In t.he seventies, several generalizations of variational inequalities were introduced by Mosco and Bensoussan, details of most of these results, specially from application point of view can be found in Mosco [22J, Bensoussan and Lions [2, 3J, llaiocchi and Capelo [I] and Kinderlehrer and Stampacchia [IS]. Kinderlehrer and Stampacchia's book comprising fundamental results and applications to free boundary problems of lubrication, filtration of a liquid through porous membrane, deflection of an elastic beam and Stefan problem appeared in 19S0. Applications of finite element methods for numerical solutions of variational inequalities were presented in book form by Glowinski. Lions and Tremolieres [14\ and Glowinski [13]. A detailed study of solid mechanics through variational inequalities was fin;t presented by Jayme Mason [15]. Applications of the theory of variational inequalities to the dam problem was systematically presented by Baiocchi and Capelo [11. During the same period, the book of Chipot entitled Variational Inequalities and Flow in Porous Media appeared which presented the obstacle and the dam problems. An exhaustive monograph on the study of contact problems in elasticit.y through variational inequalities and the finite element methods (discretization, error estimation and cunvergence of approximate solution to the variat.ional solution of the problem) was written by Kikuchi and Oden [171.

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

9.1.2

Variational Inequalities in Social, Financial and l\lanagement Sciences

In 1980, Cottle, I<annardian and Lions observed that the theory of \a.riatiollal inetlUalitie~ indudes, as a ~pedal case, the well-known theory of complementarity problem of operations research which includes problems of programming (linear and quadratic pro&'Tamming) as special cases. This provided motivation for studying variational inequalities in finite-dimensional space R". The Stampacchia School of Majorana centre, Eriche, Italy has organized many conferences and most of the v,'Ork in this area can be found in the publications edited by Giannessi. Some of the references are given in Giannessi [11, 121. ft was also observed dunng the same penod that the equilibnum con~ ditions of the traffic management problem actually had the structure of a variational inequality problem. Stella Deformas identified network equilibrium conditions with a variational inequality problem. This has opened up new horizons for mathematical modeling, analysis and efficient computation of more general equilibrium systems than ever before. r"[oroover, since equilibrium thoory was the central theme to economics and gante theory as well as tu other social ~ciences, variational inequality theory provided a mechanism by which relationships between operations research and other disciplines could be established. Stella and her coworker Anna Nagurney developed a general multi network equilibrium model with elastic deman(L., and asymmetric interactions, fommlated the equilibrium conditions as a variRtionai inequality problem and introduced a projection method for com~ puting tile equilibrium pattern. Their work ranged from a variety of traffic uetwork equilibriulil 1l10del~, spatial price equilibrium mudels, and general economic equilibrium problems with production. In short, the last decade has witnessed the successful usage of the variational inequalities theory in the setting of R" for the formulation of quali~ tative analysis, and computation of various perfectly and imperfectly competitive equilibrium problems, e.g, see Nagul'lley et a1. [24,341. Dupuis and Nagurney [91 introduced a class of projected dynanlical systems (PDS) for studying thf> dynamical evolution of conlpetitivf> systems underlying many equilibrium problems. These systems are characterized by a polyhedral constraint set, and their stationary points <tre the solutions to the corresponding variational inequality problems (VIP). Existence, uniqueness and iterative methods have been studied in the recent past along with applications, see e.g, Nagurney and Zhang [24). In recent past f>volution variational inequalities have been applied to study American option pricing [29, 33).

...... "'_'_""""'''''''10''-''

9.2
9.2.1

Variational Inequalities and Their Relationship wi th Other Problems

Classes of Fariational Inequalities

Let 1111 denote the norm induced by (".) and let (',.) denot .. the duality bet\..-een Hand H*, where H' is the dual of H. In case of the real Hilbert space (., .) = (.,'). Let J( be a dosed convex subset of H. Assume that T, S, A are nonlinear operators on H into H* and a(,) is bilinear form on H. Let F be a fixed element of H*. The following forms of variational inequalities are used to model problems of different areas:

Let H be a f{!ul Hilbert spa<:e with the inner product

h,},

Modell

[Starnpacchia-Lions]. Find u E

J(

such that

a(u,v-u) 2 (F,v-u) forallvE K,

where a(, .) is a bilinear form on H
X

(9.2.1)

H into H.
J(

Model 2 [Stampacchia-Hartman-BrowderJ. Find u E

such that

(Tu,v -u) :?: (F,v - u} for a1l v E K

Model 3 [Duvaut-Lions]. Pind u E I< such that
a(u,v - u) + j(v) - j(u)

(9.2.2)

> (F,v - u)

for all v E J(,

(9.2.3)

where

jO

is an appropriate real valllOO function.

Model 4. Find u E K such that

(Tu,v-u)::::: (Au. v -u) for all v E
f(

(9.2.4)

Model 5. Find u E [( such that

(Tu, v - u)

+ (u,v)

- (u, u)::::: (Au, v - u) for all vEl<,

(9.2.5)

where h') : H x H ---t R is an appropriate function. Model 6 [Bensoussan-LionsJ. Find u E Ju),

(Tu,v-u)
f(;

fI ---+ 21/ such that

(9.2.6)

> (Fu, v -u)

for all v E J((u).

",,,. "'_'_""""'''''''10''-''

This model is known as the quasi-variational inequality. In padicular, T can be chosen as {Ttt, v} = a(tt, v) and K(u) as K(u) = m(tt) + K, m : H ~ H is a nonlinear map, or [{(u) = {v E Hju ~ Mv,M; H ~ H, where ~ L., an ordering of H}. Model 7 Find u E [{(u) such that

(Tu, v - u)

+ l(u, v)

- (u,u)

> (Att, v - u)

for all v E K(u),

(9.2.7)

where (".) and [{(u) are the same as in Model 5 and Mvdel6, respectively. Model 8. Find u E I..:'(u) such that

(Ttt. Sv - Su) ::::: (Au. Su - Sv) for all Sv E 1(.

Stt E K

(9.2.8)

It may be observed that the variational inequalities in Models 1 to 8 are called the elliptic va7'iational inequalities. Model 9 (Parabolic Variational lnequalities) Find 11(t), where u ; [0, T] ---7 Hand [0, T) is a time interval, < T < 00 such that

(at'V -u) +a(u,v-u) &

> {F,v -u} = F(v -u)'v'v E K,

and

}
(9.2.9)

t E (O,T),u(t) E K for almost all t.

Here, FE L 2 (0,T,H) = {F; [O,TI

f

~ HO

I F(t)

E Ld, a Hilbert space.

" -8 = 0,we get tee11"IptlC \'anatlOna1. h .. ror 8u lIlequa1Ity.

Model 10 (Rate-independent evolution variational inequality). Find tt ; [0, TI ---+ H such that

(~:t, v _ u) + a(u, v -

tt')

+ j(v) -

j(u' ) ::::: F(v - tt') for v E H.

(9.2.10)

For information concerning more general form of inequalities like vectorvalued variational inequalities, simultaneous \ariational inequalities, generalized quasi-variational inequalities and implicit \'ariational problems, we refer to Siddiqi, Manchanda and Kocvara [29, 3OJ, Brokate and Siddiqi [51, and Giannessi [12J .

..... ,

Q . ,>ooI"_""""'''''''IO''-I

9.2.2

Formulation of a Few Problems in Terms of Variational Inequalities

R be a differentiable real-ntlued function defined on the closed interval of R. We indicate here that finding the point Xo E J = la, bl such
that

1. Minimization of a Single-valued Real}""'unction: Let f; [a, b] -----t

f(xo) ~ f(x) for all x E J; that is, f(xo) = inf f(x), . E J

is equivalent to finding the solution of a variational inequality. WE' know that (a) if a

< Xo < b, then

1'(xu) =

(b) if Xo = a, then 1'(xo)

? 0; and (c) if Xo:= b, then 1'(xo) < o.

From this, we see that

(f'(xo),x - xo)

> 0 for all x

[a,bl,xo E [a,b);

thaI is, Xo is a solution of Modell, where F :::: O,a(u,v - u) (f'(u), v - u) :::: (Bu, v - u), B :::: l' : R -----t n lineal' or Xo is solution of Model 2, where F = 0 and T = 1'.
2. Minimization of a Function of

Variables: Let f be a differentiable real valued function defined on the closed convex set K of R". We shall show that finding the minima of f, that, is searching the point Xo E K such that f(xo) = j~Lf(x), that is, f(xo) < f(x),
11

for all x E l( is equivalent to finding the point Xo E K, which is a solution of MadeliaI' Model 2. Let Xo E K be a point where the minimum is achieved and let x E K. Sin~ K is convex, the segment (1 - f.)xo + tx = Xo + t(x - xo),O < t < I, lies in K. /(t) = f(xo + t(x - xo),O < t < 1, attains its minimum at t = 0 as in (1). 1"(0)
=:

(grad f(xo), (x - xo) ? 0 for any x E

= \1f(x) =

8J ( ~'~""'~ ,x VXI VX2 vX"

Of

Of)

1(,

where gradf

= (XI,X", . . ,x,,) E IC This

shows that, Xo E J( is a solution of tht=> variational inequality, (gradf(xo), (x - xo? 0 for any x E 1(, or Xo solves Model 2 where

",,,. "'_'_""""'''''''10''-''

u = xo, v = x, T = grad 1 = \71, F = O. Thus, if 1 is a convex function and u E K is a solution of variational inequalit.y (Model 2) for T = grad 1 = \11, then
f(x)
~
vEK

inf f(v)

3. Consid(lr UlP following boundary value problf'm (BVP) - dx' ~ f(x) on (0,1)
u(O)

d'u

(i)
(H)

> 0;

u(l)

>0

du (-f ),~" <x

<0

(iii)

(9.2.111

du (dx)X=1 ;::.: 0 dv u(x)dx = O,x E {O, I}

(v).

Let H = HI(O, 1),1< = {u E fll(O,l)Jtt{O) ;::.: O,u(l) > OJ. It can be observed thaI. if J( is a cone with vertex 0 in Modell (the convex set [( is caned cone with vertex at O. if for every pair of elements V,tV E K,v -t- 'tV E A",tv E K holds for every t ;::.: 0), then (9.2.1) is equivalent to the pair of relations
a(u.v) = (Bu. v} ;::.: (F.v) a(u,u) = (Bu,u) = (F,u).
(9.2.12)
(9.2.13)

(B is a bounded linear operator associated with a(-,) given in the beginning of Section 7.4). To verify this, let Q(u) = B(u) - F; so (9.2.12) and (9.2.13) reduce to

(Qu, v) 2 0;

(Qu, u)

(9.2.14)

Subtracting the second from the first, (Qu,'iJ - u) > 0 for all v E K. Now if we choose v = 2u, which is permissible since K is a cone, we obtain (Qu,u) > 0, By choosing v = 0, we have (Qu,u) < O. By combining these two inequalities, we obtain (Qu, u) = 0 and, finally adding this to (Qu, v - u) > 0, we &'et the desired result. \Ve show now that Model I, where Hand [( are as chosen above,
a(u.v) = (Bu, v) =

Q.

......

'dudv x and (F,v) = --d o dxdx

l'

l(x)v(x)dx

, to<l',_"""",,,,,"l1>"-I

is equivalent to (llVP) (9.2.11). Modell takes the form

{' dx dx dx (' J du [dV - dU] dx > J J(x)(v(x) o o

u(xdx

(9.2.15)

for all vtx) E Ie It can be checked that [( is a cone with vertex 0 and so by the above remark, (9.2.15) can be rewritten as

io

{'dudv dx dx ([x

> io j(x)v(x)dx
J(x)u(x)dx

r'

(9.2.16) (9.2.17)

l' (:)' l
dx =

Assume tlUlt the solution u of (9.2.16) is smoother than stnct.ly required for it to be in HI(O,I), for example u E C 2 (0, 1) and 11 E Cg"(O, 1). Integrating by parts the left-hand side of (9.2.16), we find

10
>

1 - : v(x)dx
1

+ [( ~:) x=1

v(l) - ( : ) x=o V(O)] dx

(9.2.18)

j(x)v(x)dx

Since v E [( n CO'(O, 1) and (9.2.16) also hold for (-v), we conclude that

t io

dudv (I dx dx dx < j(x)v(x)dx.

io

(9.~.19)

This together with (9.2.16) implies

, 1

dudv --dx:::: o dxdx

l'
0

j(x)v(x)dx.

(9.2.20)

An integration by parts here yields (9.2.17) and, as v E Cg"(O.I); that is, v(O) :::: v(I), the boundary term ''8nishes and we obtain

~~ :::: lex) in (0,1).

In view of (9.2.17), (9.2.18) gives us

(d")
dx

x:o::l

v(l) -

(d")

dx x=o v(O)

> 0, for u, vEl(.

(9.2.21 )

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

dU) (-d > 0 x (BVP) 9.2.11 (iv. On the other hand, by choosing vel) = O. and . dx ~ 0 or (d") .,=u S; 0 v(O) = I . (9.2.21), we obtam - (dU) dx
ff we choCJ.'* v(1) = I, and v(O) = 0 in (9.2.21), we &'et
",,,,,I
In
",=0

which is (BVP) 9.2.11 (iii). Since tt E 1<, we have (BVP) 9.2.11 (ii). (BVP) 9.2.11) (v) is obt.ained in a ~imilar way from (9.2.17). Thus if u is a solution of (9.2.15) then it is the solution of (DVP) 9.2.11 (i-v). If u is a solution of (BVP) 9.2.11 (i--v), then multiplying both sides of (BVP) 9.2.11(i) by vex) and inte&'Tating by parts and using other conditions, \ve get (9.2.16) and (9.2.18). This implies that u is a solution of (9.2.15).

4. Complementarity Problem and Variational Inequality: The following problem is known as the non-linear r complementarit.y problem (NCP): Find u E R" such that u : : -: 0, F(u) : : -: 0 and u and F(u) are orthogonal, that is, (u, F(u) :::: 0, where F: R" ---+ R". The NCP has been used as a general framework for quadratic programming, linear complementarity problems, mathematical programming and some equilibrium problems. For details, we refer to Cottle, Pang and Store [81.
It can be easily checked that u is a solution to the NCP if and only if it is a solution of the variational inequality: Find u E such that (F(u), v - u} > 0 for all v E where {v:::: (Vi> Vz, .. , v,,) E R"/Vi > 0, i:::: 1,2, .... n}.

R+,

R+ : :

R+

Proof fG-, so

Let u Le a ~olution to the NCP. Then (F(u), v} ::::-: 0 't:/ v E

(F(u),v - u) = (F(u),v) - (F(u),u) :::: (F(u),v)

>0

because (F(u),u):::: O.

Thus u is a solution of the variational inequality. To prove the converse, let u be a solution of the variational inequality. Then v :::: U + Ci, ; :::: (0,0, ... ,1,0, ... ) (1 in the i-th place) is an element of R~'_, so 0 5 (F(u),u + tot - u) :::: (F(u),;) :::: Fi(u) or F(u) E R+-. TllU~, since v:::: 0 E R+, (F(u),n) 5 O. But u,F(lI) E R~_ implies that (F(u),u) > O. Hence, (F(u),u) :::: 0; that is u is a I solution of the NCP.

Q.

......

, to<l',_"""",,,,,"l1>"-I

5. Rigid Punch Problem: The \'ariational inequality approach for studying the contact problems is well established in the methodology see, for example, Kikuchi and Oden [17], Duvaut and Lions [101 and Brokate and Siddiqi [41 for an extended discussion of the rigid punch problem. In the rigid punch problem, one considers a material body deformed by another body (the pundl rigid hody of definite shape). If we consider a specific situation as shown i.n Figure (9.2.L) for the iJlilial and equilibrium configurations, then the deformation u in the elastic body is a solution of the variational inequalit.y: Find u E [( SUdl that
a(u,v-u)
~

(F,v-u)

'VvE [(

where a(, .) is the usual bilinear form of linear elasticity on a suitable Hilbert space H,
a(u,v} =
_

l
L

a;,klf.fj(Ulf.kdv}dx,

~,j(t.I) = 2"(J,Uj

+ DjU;),
(2 C

u = (il,y) = (Ul,U2,y) E fl.

The linear elastif' material body occupies at. rest the domain

R2

, . ,
!
r

i"xl

I !
!y
!

"0
fo

ro

fo

Figure 9.2.1

Rigid punch

Q.

......

and is kept fixed at a part roof its boundary. The punch has one
, to<l',_"""",,,,,"l1>"-I

degree of freedom, namely, the motion in the vert.ical direction, and presses (rom above with a fixed force resulting from its own weight onto the material body. Frictionless contact is assumed. Here, the unknown (the deformation) is u = (U1>Uz,Y), where u.: 0 ---+ R2 and y E R is the vertical displacement in the downward direction of the rigid punch and
(F, u)

In fju,dx + Py,

where P

> 0 corresponds to

the weight of the pUllch.

The constraint set J( represent::, a lineari7..ed form uf the non-penetration conditions (i.e., the condition that thc punch is always above thf' elastic body). A general form is

Here, P" is the part of the upper boundary of 0, where the contact may possibly occur. The function denotes the vcrtical distance (01 some modification of it) of the punch from the elastic body in the initial wnfiguratiull and Ow functions ail i3 lU'~ connected to Ow ullit normal of the boundary of the punch or of the elastic body Various forms are possible; for example,

+ Z < (x), 1];(X)Vj(X) +'}2(X)Z < 'f2(X)(X), Vz(x) + Z::; (x).

1], (x)v,

(x)

6. Reasonable Price System: Let AI = {p = (PI,Pz, ... ,Pn) E ROl: o ::; p, < I} and let N be a compact, convex, nonempty set in ROl. Suppose there are n commodities C 1 , C z , G3 , , Gil' Let Pi be the price of G'j with 0 < l)j < 1. We assign a Sf't J(p) t::::; N to p.ach price vector P EM. q E K(p) with q = (QI,/}"2, ... ,qll) means that th~ difference between the supply and demand for Gj is equal to q,. The number
(p, q) ~

" L p,q,
i=1

is, therefore, equal to the difference between the value of the commodities which are supplied to the market and the value of the commodities which are demanded by the market. Thus.

Q.

......

(p,q)
, to<l',_"""",,,,,"l1>"-I

0,

q E ](1')

is the mathematical fommlalion of the situation "supply equals demand". This ideal situation cannot always be realized. We are therefore satisfied with a weaker condition. First, it is rea.sonable to assume that
{p.q}

> O.

for all pE lvi, q EK(p).

This is the so-called the law of Walras. Roughly, it means that. we only consider economical situations with a supply excess. We call (j5, ij) with ij E K(p) a Walms equilibrium if and only if the following holds. 0 < (p, q) < (p, lj) for all p E Ai. Broadly speaking, this means that the '~d.lue difference between the supply and demand becomes minimal. The vector p is called the equilibrium price system. The fundamental problem of mathematical economics is to find conditions for the supply excess map !( which ensures the existence of the Walras equilibrium. This is equivalent to finding the solution of Model 6, where H := R", AI = unit cube, {Tu, v} = (u, v) inner product in R". Moscu (22] has given a theorem dealing with th~ solutioll of this model

9.3

Elliptic VariatiolJalllJequalities

Let H be a Hilbert space and K a non-empty dosed convex subset of H. Further, let a(,) be a bilinear form on H x H into Rand F E H" which is identified with element, say F", or y of If, by the Reisz representation theorem. Then the Modell (variational inequality problem: VIP, for short) takes the form: Find u E [( such that

a(u,v - u)
0'

> F(v - u):= {y,v - u} for all v E [(

(9.3.1 )

(Au, v - u)

> F(v -

u) = (F, v - u) = {y, v - u},

~

(9.3.2)

whe"e A, H ---> W, (Au, v) ~a(u,v),IIAIl in l\,todel 2, we find u E K such that

{Tu,v - u}

lIali.
[(,

> (F,v - u) "Iv E

(9.3.3)

where T ; H - - t H" ur mtu itself in the c~ of real Hilbert space. [n Section 9.3.], we shall study the existence of solution of (9.3.1) when a(',') is Ixmnded and coercive bilinear form (not necessarily symmetric),

...... "'_'_""""'''''''10''-''

More precisely, we present the celebrated Lions-Stampacchia theorem. Prior to this study, it will be proved that, for the bilinear bounded coercive and symmetric bilinear form, the optimization problem for the energy functional is equivalent to a variational inequality problem, that. is, "Find u E [( SUdl that J(u)
~

J(v) 'r/v E IC where J(v) = 2u(v.v) - F(v) is equivalent to

(9.3.1)". Section 9.3.2 is devoted to the existence theorem for the VIP (9.3.3) along with the Wtz-Galerkin and Penalty methods.

9.3.1

LionsStampacchia Thoorem

For a symmetric bilinear form, the Lions-StampacdJia theorem follows (rom Theorem 6.3.2 in view of the following theorem:

Theorem 9.3.1

Lei J(v) = ~o(V, v) - F(v). Jf of-,) is bilinear bounded coe,'cive and symmetric then the problem of optimization, (P) fm' J() and (VIP) are equivalent. That is to say "Find u E J{ such that J(u) < J(v)'r/v E [{" holds ~f uud unly if (9.3.1) huldij, namely, there exists u E [( such that a(u,v - u) > F(v -u) 'r/v E 1(.
Pronf

Let (9.3.1) hold for all?) E 1<, t.hen we have

1 J(v) = J(u +v - u) = 2u(u

+v-

U,U

+11- u) - F(u+v -u)

U,V -

1 1 :::} J(u + v - u) = 20(u, u) + 20(t' 1

1 u) + '20(u, v - u)

+2u(v- u,u) - F(u) - F(v - u) :::} Jlu

1

+ v - ul = J(u) + [o(u, v - u) - F(v - ul]

+ 2"0(v-u,v-u).
Since a(,) is coercive, the third term is greater than or equal to 0 and the second term is also non-negative by the assumption that (9.3.1) holds. Therefore, J(u) ~ J(v)'r/v E f(. For the converse, let there exist u E J{ such that J(u) ~ J(v)'r/v E K. This implies tJlat
t ..... O

. J(u I1m

+ f(v - u)) t

J(u)

> , -

till" Gautcax derivative of J in the direction of (v - u) is greater than or equal to 0, that is, DJ(u)(v - u) > O.
01'

...". "'_'_""""'''''''10''-''

By Solved Problem 5.6.3 and Theorem 5.2.3,

DJ(td(v - ttl::: DJ(u)(v - ttl ::: a(u.v - tt) - F(v - u) ora(u.v-u) ~ F(v-tt) 'r/vEJ<

>0

Hence, (9.3.1) holds.

Theorem 9.3.2 If a(',') is bilinear bounded and coercive, then VIP (9.3.1) has a Unique solution. F'urthermore, the mapping F -----t u of H* into H is Lipschitz continuous, that is, if UI and Uz are solutions corre.sponding to Fl and Fz of (9.3.1) then !lUI - uzl! < tllFl - Fzllu.
P1'Oof.

(i) Lipschitz continuity: Let UI and Uz be two solutions.

Then (a)
a(ul,v-Ul) ~ Fl(V-UI) and a(uz, IJ
-

(b)

u:J ~ Fz(v -

(V2):::

(F'}., v - Uo2).

Put \' == ttz in (a), then we get a(u\, u2 - ttl) > F I (tt2 - ttd or a( -Ul, ttz uJ) < -FI(UZ-UJ) by linearity ofa(,) and F I (). PUL v::: ttl in (b) then we get a(u2,ul - U2) > Fz(ul - U2) or a(uz,u2 - ud < Fz(uz - ud by linearity of a(,) and F2(')' These two inequalities imply a(ul - U2, u\ uz) :5: (FI - F2,UI U2). Since a(o, 0) is coercive and FI - Fz is bounded,

(ii) Uniqueness of Solution: Solution is unique for ::: O:::} Ul ::: U2,:::} solution is unique.

FI

:::

Fz,

lIu\ -u211

(iii) Existence and Uniqueness: We have (Au, v}::: a(u,v) by Corollary 3.9.1, and F(v) ::: (y,v), by the lliesz representation theorem. VIP <=} (Au,v - u) ~ (y,v - u) 'r/ v E J( <=} (-pAu,v - u) :5: p(-Y,v - u) for any J-I > Oor (u-p(Au-y)-u,v-u) :5:0'r/VE [( orllnd U E [(such that PK(u - p(Au - y ::: U (see Theorem 3.5.2). [('t. T p : H ---4 H riellned b~'

T,(v)

Q . ,>0<1"_""""'''''''IO''-I

......

PK(v-p(Av-y)). We show that T p has a unique fixed point, sa,}'

tI,

whidl is the solution

o[VIP (9.3.1).

IIT,(v,) - T,(v,JII ~ IIPK(V, - p(Av, - y)) - PK(v, - p(Av, - y))11 < lI(v\ - p(Avl - y)) - (V2 - p(A'll2 - Y))1I 2 (By Theorem 3.5.3) ::: lI(vI -v:'d - pA(Vt -V2)U = IIVI -1).211 -2pa(V2 -Vl,1J2 -vd + p2 II A(V2 - V.) 11 2 (By Remark 3.4.2.(1))

< (I - 2(>0 + P'IIAII')II(", - v,)I1'

If 0 < P <

II~~I"

then IIT,(v,) . T,(v,)II' <

/lll(v, - v,)II',

0<

/l <

1.

By Theorem 1.2.1, T p has a unique fixed point which is the solution of (9.3.1). I

Remm'k 9.3.1 The proof of the above theorem gives a natural algoritllm for sol"inp; the variational inequality since v --+ PK(V - p(Av - y)) is a 2" contraction mapping for 0 < P < II A 11 2 ' Hence, we can use the following
algorithm to find u; Let uO E H, arbitrarily given, then for n > 0 assuming that u" is known, define U,,+I by U"+ I := PK(U ll - p(Au"-y). Then un --4 u strongly in H, where u is a solution of the VIP. In practice, it is not easy to calculate y and A except the case H = J-I 0 If a(', .) is symmetric, then dJ(u) = Y(u) = Au - y and hence 11"+1 = PK(u" - p(J'(u"}), where J is the energy functional (Theorem 9.3.1). This method is known as the gradicnt-projcction method with constant step p.

9.3.2

Variatwnal inequalities for Monotone Operators

An operator T : J( -----+ J-I' or, in particular, H is called Morwtonf if {Tu - Tv, u - v} ~ O\lu, v E [C The monotone mapping T is called strictly monotonc if (Tu - Tv, u -v) = 0 implies u = v. T is caBed coe,"cive if there exists Vo E J( such that

(Tv, v - vo)

Ilvlll/

--+

00

as

IIvllfi --+ 00.

Lemma 9.3.1 (Minty Lemma). Let K be a noncmpty closed and convex subset of a flilbe,-t space If and lct T : K ---+ H' be monotone and continuous and F E H'. Then the elemcnt tI E [( is the solution of tht; variational incquality

(Tu,'t. - u)

> (F,v - u} \Iv E

(9.3.4)

Q.",~,_.._",,"l1>"-'

.... ,

if and only if

(Tv. v - u)
Proof.

(F, v - u)

"Iv E K

(9.3.5)

Let (9.3.4) hold. We have

'r/v E
J(

(Tv,v -u) == (Tu, v - u) + (Tv -Tu,v - u)

(9.3.6)

by keeping in mind Tv == Tu - Tu + Tv = Tu + Tv - Tu. Since T is monotone (Tv - Tu, v - u) > 0 and therefore (9.3.6) gives us (Tv, v - u) > (Tu" v - u). This inequality together with (9.3.4) implies (9.3.5). To prove the converse, let (9.3.5) hold for every v E [(. In particular, choose v ::::: (I - t)u + tw where w E J( is arbitrary and t E (0,11. Theil v - u::::: t(w - u) and v::::: u + t(w - u). By (9.3.5), we have
(Tv. v - u) ~ (T(u

+ t(w -

ulJ,tlw - ul) ~ t(T(u+ tlw - ul). (w - ul)

> (F,v-u}:::::(F,t(w-u):::::t(F,w-u)
For t > 0, we have

(T(u...L. t(w - u,w - u)

(F, W -1!).

(9.3.7)
continuit~,

lly taking limit as t ---t 0 in (9.3.7) we get (9.3.4), by the and the fact that w is arbitrary. I

of T

As a consequence of the Minty lemma. we get the following: result: Theorem 9.3.3 Under the conditwns of the Minty lemma, the set of solutions of the variational inequality (9.3.3) is convex and closed. See Problem 9.7.3. Theorem 9.3.4 Let K be a dosed and convex subset of a Hilbert space H and T a continuous, monotone. bounded and coercive operator defined on K mto H. Then (9.3.3) has a solution for evny F E H. Fuyther1llOfe, ~f in addition, T is strictly monotone, then VIP (9.3.3) has a unique solution. Theorem 9.3.4 is a consequence of Theorem 9.3.7 As we have seen in Section 3.5, the projection of an element x E H onto the set K, is characterized by the variational inequality (9.3.8)

Q.

......

, tl',_"""",,,,,"l1>"-I

This is evident from Figure (9.3.1) The left-hand side of (9.3.8) is just the cosine of the angle 0: between Jines connecting the point PK(X) with the point x and with the arbitrary point n y E K (see Figure 9.3.1) respectively, and we have that coso: ~ 0 as 0: > 2 necessarily holds. Conversely, for every other point 'J! E J(, x' ::F P,,(x), there exists a point y' E f( such that the angle between the lines connecting the point 'J! with the points x and y' is less then ;

1/-...._- x'
J(

Figure 9.3.1

Geometrical intCTpretation of charactCTizatioli of projection in terms of Hlriational inequality

Theorem 9.3.5 Let K be a non-empty closed convex subset oj a Hilbert space 11 and T an operator on K into fl,F E H\a > O. Then u is a solution oj thfO Variational Inequality (9.3.3) - Model 2 iJ and only iJ
u ~ P(u - a(Tu - F));

(9.3.9)
tI UJ

thal is, u is the projection oj the point (u - o:(Tu - F)) or point oj the opnntm" S 9iven by th(' equation
Su
~

thto fixed

Pdu - a(Tu - F)).

(9.3.10)

P1'OOf. Let w == u - o:(Tu - F). By (9.3.8), (9.3.9) is equimJent to the inequality: For u E K, (u - a(Tu - F) - U,v - u) ~ 0 "Iv E [(,

",,,. '"'_'_""""'''''''10''-''

or-a(Tll-F,f'-ll)

<

O'r/vE J(, or (Tu,v-u)

>

(F,1'-U)'r/vE K..IS

>

(J.

Thus, u is t.he solution of the Variat.ional Inequality (9.:t3).

Theorem 9.3.6 Let, in addition to thc hypothesis of Thcorem (9.3.5), the operator T satisfy the followin.fJ conditions: There exist constants J-t > 0 and A > 0 .~uch thal

liT" and
(Tv - Tw, v - w)

Twll < ~lIv

- wll
"Iv, w E [(.

19.3.111

:?:

Allv -

w1l 2 ,

(9.3.12)

Then the,'c exists a unique solution u E [{ of the Variational lncqualily

(9.3.3) for every F E fl. If a is so chosen that 0 < a <

2~
~

and if Uo is an

arbitm,y elnlUmt of

f{,

then
U

= lim

11---'"00

u."

whcre
11"+1

= PK(U.,

a(Tu'l - F, n

:=

0, 1,2, . ..

(9.3.13)

Proof. In view of Theorem 9.3.5, it is sufficient to show that the operator S defined by (9.3.10) is a contraction operator. By (9.3.11) and (9.3.12), we get

liS" - 5wll'

~ IIh(v - a(Tv - F)) - PK(W - a(Tw ~ ~

F))II'

II (v -

a(Tv - F)) - (w - a(Tw -

F))II',
wI"
Tw1l 2,

by Theorem 3.5.3(a)

lI(v -

w) - a(T" - Tw)1I ~ IIv -

- 2o:(v - tv, TtJ

Tw)

+ o:211Tv -

by Remark 3.2.4(1) and properties of the inner product,

2aAliv - wll 2 + a211211v _ = (I - 2aA + a 2J-t2)lIv _ w1l 2 .

<

IIv -

wll 2

wll 2

If 0 .

2), . . < a < ""2' tIoperator ' a contractIOn operator Wit I1 constant Ie S IS

JI - 2aA + a2rt2 = ~ < 1. Thus, the theorem follows by Theorem 1.2.1 I and Problem 1.4.17.

I'

...... "'_'_""""'''''''10''--'

The Ritz-Galerkin Methods: The Rit'l.rGalerkin Methods deal WitJl finding a finite-dimensional subspace H II of separable Hilbert space H and an element U E f{ n H", where f( i- , closed convex subset of H such that
(Tu",v -u,,)

> (F,v-

ulI)'iVE Hand eadl FE H.

The solution of Uw VI (9.3.3) is obtained as the weak limit of the sequence

{UII}'
The Penalty Method: Let J be a functional defined on a separable Hilbert space H, [( =I- ~ H. The penalty method deals with the conditions such that finding the solution of the optimization problem for ,J 011 K is equivalent to solving (9.3.3). Instead of finding u E [( such that J(u) S J(v) 'r/ v E K, we look for U. E H at which a new functional J e : H --+ R attains a minimum, that is,

0'

'Ve writp herp

J.(v)

J(v)

+ -(v),c > 0, c

where is a functional on H for which ( v) :::: 0, Vv E K, (v) > 0 V v (j. [(. Under apPl'Opriatc conditions, U c ----' u E J{ as --+ O. is called the penalty functional Let T be an operator 01' H into H* and a bound~J continuous munotone operator {3 exists which also maps H into 11* such that
{3v =0 if and only if v E [(

(9.3.14)

Then {j is f:a1led t he penalty operator with respect to J(. Let arbitrary and let us look for the point U E ff such tJ13t

>

0 be

(9.3.15)

TUe

Q.

......

J +, e= -{3u

F.

, to<l',_"""",,,,,"l1>"-I

Thus, Variational Inequality (9.3.3) on K is replaced by Variational Equation (9.3.15) on H. The ba<;ic idea is to convert a problem of variational inequality into one of \-ariational equation. Theorem 9.3.7 IJet H,T, K be as in Theorem 9.3.4 and let f3 be the penalty operator corTtsponding to the set K and 0 E K. 'J11Cn tI,el'C exists a solution u~ of (9.3.15) fort> 0 and a SClJ1lellce {tn} such that t n --t 0 and tI!~ converges weakly to the solution u of vaf"iational inequality (9.3.3). 1f. in llIlilition, T is strictly monotone, then VIP (9.3.3) has a unique solution.

Proof
I. Uniquenes..<.;: \Ve first prove that for strictly monotone operatorH,

the solution is unique. Let is,

"lit

and "liz be two solutions of (9.3.3), that

(TUt,V -UI)
(TU';~,v

> (F,V-UI)VV
~

E J(

(9.3.16) (9.3.17)

-uz)

(F,v -uz)Vv E I<.

Putting v = tlz in (9.3.16) and v .....: til in (9.3.17) and adding the resulting inequalities we get {Tu! - Ttlz, Uz - til} ~ O. Consequently. {Tul - Tuz, til - uz} < O. Since T is strictly monotone UI = Uz must hold. 2. Existence of Solution of (9.3.15): Since T and f3 are continuous and monotone for a given E

> O. the operator SE

= T

(~) B is

continuous and monotone. The operator S~ is also coercive because (9.3.14) implies that f30 = 0, and furthermore, we have V v E K

(S,v,v) = (Tv,v) + -({lv,v)

= (Tv, v)

+ -({Iv - {l0, v - 0) -> (Tv, v)

1 as -(f3v - (30, v - 0)

> O. This implies that

(Tv, v)
--J. 00

(S,v, v)

Ilvlllf > IIvllu

as

II II
v
If --J. 00.

Since SE satisfies all the conditiom, of Theorem 7.3.7, we find that there exists u~ E H such thnt SE1lE = F, for each F E H. This means that (9.3.15) holds for all v E H, and tiE is a solution of (9.3.15) .

Q . ,>0<1"_""""'''''''IO''-I

.....

3.

u~

-" Uo E K, u E K: \Ve now show that For c > 0, we have

(Tu"u,) J1u~lI"

{u~ ..

} is a bounded sequence.

(Tu"u,) ~ (5,u"u,) ~ (F,u,) ~

or

IWllllu.!1

<

II II

F ,,-.

This implies that {UnE > O} is bounded, otherwise, T is non-coercive, a contradiction of the hypothesis. By Theorem 4.4.7, there exists a sequence {cn}, Cn > O,E n --t O. such that the sequence of elements u~n' n E N converges weakly to :>ulIle element Uo E H. Now, we show that Uo E 1\. Since (3 is monotone

(9.3.18)

By (9.3.15), (3u"" = c.,(F - Tu",.). Since T is houllfled {Tu~n} if> bounded and thus (3t1".. --t 0 for n --t 00. By taking linut in (9.3.18), we get

20 Vv E H. By choosing v = uo+tw, where t. > 0 and w E H; we get V-Uo am] ((3(uo + tw),w) > O. By the cont.inuity of the operator
{(3v,v -tlo)
follows that

= tw
(3, it

((3uo,w) 2 0 for every wE H(by letting t

The inequality also holds for -wand so

0+).

(9.3.19)

((3uo, w) :$ 0 for every w E H.

(9.3.20)

(9.3.19) and (9.3.20) imply that ((3110' w) = 0 for each w E H. This implies that (3uo = 0 and so by (9.3.14) Uo E K
4. Uo is the solution of the Variational Inequality (9.3.3): (9.3.14)

and (9.3.15) and the monotonicity of the operator T and (3 imply that

(Tv - F,v - u E ) = (Tv ..

=

+ (Tut;.. - F,v -u~.. ) 1 (Tv-Tut; " ,V-UE~) + cn -((3v-(3u~~,v-u~ ) .. .. ..

Tu,,~,v

- tlc .. )

20 "Iv E K.
0'

by (9.3.18) and monotonicity ofT

(9.3.21)

(Tv,v-uo) 2 (F,v-tlo)

By (9.3.21) and the l\1int)1 lemma, we find that Uo is a solution of (9.3.3). This proves the theorem.

",,,. "'_'_""""'''''''10''--'

9.4

Finite Element Methods for Variational Inequalities

[n this ~ction we present results concerning convergence of the solution of the discretized variational inequality to the solution of continuous form and the f'rror estimation hetwff'1l these I.wo solutions. [n Section 9.4.1. results ... concerning 1 1ode1 1 are presented whereas a concrete case is discussed ill Section 9.4.2.

9.4.1

Convergence and Error Estimation

Let H be a real Hilbert space, K a non-empty cla>ed and convex subset of Hand a(,) bilinear bOllllflerl and coercive form on Hand F E H', We are interested in tile approximation or discretization of the Varintional lnequality (9.3.1), namely
a(v.,v-u) 2:F(v-ujVVEK, uEI(

(9.4.1)

As shown earlier, there exists a bounded linear operator A : H ---t H such that (Au, v) = a(u, 11) and IIAII = 110 II. Let {Hh} h he a sequence of finitedimensional subspaces of H, where It is a given parameter converging to O. Furthermore Jet {Khh'v' h be a sequence of closed convex subsets of H U(h may not be a subset of K) satisfying the following conditions:
1. If {Vh}h is such that Vh E Kil Vh and {vhh is bounded in H, then

the weak limit points of {Vh} belong to K"

2. There exist X ~ H,X

=: J(

and

"h:

X-

K h such that

Remaf'k 9.4.1

(i) If }(h

~ [(,

then (I) is tnvially satisfied as

J(

is weakly closed.

(ii) nhKh ~ K. (iii) A variant of condition (2) is as follows: 3 a subset X ~ H such that X =: J( and rh : X --+ K h with the property that for each v E X, 3 ho =: ho(v) with ThV E K h for all It 5 ho(v). and lim ""hv-vll" =: o.
"~O

Approximate P('Oblem

Ph:

Find

Uh

J<h

such that

(9.4.2)

...... '"'_._""""'''''''10''-''

it is clear that Variational Inequalities (9.4.1) and (9.4.2) have a unique solution under the hypothesis. Here, we present two theorems, oue related to the convergence of {Uh} as h -t 0 with respect to the norm of H; that is, Iluh - ulIlI ---+ 0 as It ---+ 0, where U and Uh are the solutions of (9.4.1) and (9.4.2), respectiveJy; and the other provides the order or the upper bound of lI1lh - ullli' It may also be observed that (9.4.2) will be a system of matrix inequalities which ar~ to be evaluated in practical ~ituatioll~. [t may be noted that if 0(',') is also symmetric, then tile solution of (Ph) is equivalent to solving the following progTamming problem:

Problem P;': Fino Uh E K h such that

J(1lh)
='

inf J(VII)
"J,

(9.4.3)

Eli.""

1 where J(Vh) = 20(1Jh' Vh)

(F, Vh).

Theorem 9.4.1 are satisjietl.

We have lim lJu ,~O

ut,1I

= 0, where the above conditions

P7'OOf.

The proof is divided into three parts:

1. Priori estimates of {1lh}

2. Weak convergence of UII and

3. Strong convergence.

1. Estimates for {Uh}: We show that 3 constants C 1 and C2 such that

lI1lh112 ~ Cdl1lhll

C2 V It.

Since {1lh} is the solution of Ph (9.4.2), we have

a(1lIl,Vh -UII)

2:: F(Vh -1lIl) VVh

K II

or or

O(UII, VII) 0(11'h,UII)

> O(UII.Uh) + F(Vh) - F(1lh)

<

a(uh' Vh) - F(Vil - UII),

by lin~arity of a(,) and F()

0'

011",,11'

IIAllllv,lIl1vhll + 11F1I(lIvhll + lI"hll) VVh E K h , by applying

~ a(",.v,) ~

coercivity of a{, '),

boundedn~ss of

Q . ,~._ .._",,"l1>"-'

......

ano A induced by a(,) .

Let Vo E X and Vh '-- rhVO E Kh. By condition 2 on K h, we haH' 1'hVO --+ Vo strongly in H and hence IIvhll uniformly bounded by constant m,

where c, = -

2 Weak convergence or {Uh}: {Uh} if; uniformly bounded. ~ {Uh} has a weakly convergent subsequence, say {Uh,}, such that Uh, ....... u in H (by condition 1 on {KlIlh, U E K) by Theorem 4.4.7. Now, we show that U is a solution of (VIP). We have a(uh;>uh,) < a(Uh"vh,}-F(Vh,-Uh,)'v'Vh, EJ(h,. Let v E X and Vh. = rh,V. Then this equation takes the form a(uh;> 'Uh,) S a(uh;> rh, v)- F(1'h, V-Uh,)' Since '1.l'I.; ....... u' and rh,1I --+ V as hi --+ 0, taking the limit in this inf'quality, 'Iff' get
limh~'.io a(uh"uh,)

"

(mllAfi + II FID

and Cz

-IIFII => lIuhll <: c ~ h.

"

S a(u',v)-F(v-u')

'v'vE X.

(9.4.4)

On the other hand, 0 < a( Uh, -11' ,'Uh, -u') < a(Uh" Uh, )-a (Uh" u)a(11' , 11h,) +a( u', u) or a('Uh" u )+ll( u , 11h,) -a(11' ,11') < a(Uh" Uh,). B~' taking the limit. we get

(9.4.5)
From (9.4.4) and (9.4.5), we gel
a(11', u) S I\~~~f a(11h" 'Uh,)

<

a(u', v) - r(v - 11') 'rtv E x.

(9.4.6)
Therefore, a(u', v - u) > F(v - UO) 'rt,' E X, U E K. SinCf' X is dense in K and a(,) , F are continuous by the hypotheses; from this inequality, we obtain

Since conditions of Theorem 9.3.2 are satisfied, solution u must be equal to u. Hence, U is the only limit point of {uhh in the weak topology of H, and th~r~for~ {Uh}h converges w~akly to 'U. 3. St.rong Convergenc.e: SinN' a(,) is cOPfcive, we gf't.

<alll1h-uUZ <a(uh-u,Uh-11) } = a(uh,uh) - a(uh,11) - a(u,11,,) + a(u,'U) .

(9.4.7)

Q.

......

, to<l',_"""",,,,,"l1>"-I

Since rhV E J<h for any v E X, we have

(9.4.8)

From (9.4.7) and (9.4.8) and keeping in mind lim 1111 =

,~o

11

weakly in

H and lim 1'hV = v strongl,v in H. we get

,~O

o <0:
,~o

lim inf lI11h - 1111 2 So: lim sup lI11h < a(11,v - u) - F(v -11)

ull 2

(9.4.9)
=

=> lim lIuh - ull 2 = 0, by density and continuity and letting

11

11

in

(9.4.9). Therefore, 11h COfl\wges st.rongly to u.

I
Theorem 9.4.2 Assume flial (Au-Fy) E H, where A is defined through a(,) and Fy by F (Theorem 3.7.1). Then there exist:> C independent of the subspace H h of H awl a nonempty closed convex subset K h of H such that

(9.4.10)

Corollary 9.4.1 11K = H, then Au - Fy = 0, $0 that with flie dlOice of K h :::: H h the envr estimate (9,4.10) reduces to the well-known estimate known as the Cea's Let/una (Theorem 8.2. I). [t IIIay be noted that K h fl\:'Cd lIot be a subset of K.

Proof.
0:1)U-Uh1!2

s:

a(u-uh,U-u/.)=a(u,u)+a(11h,uh)

(9.4.11) 'v'VEJ<, a(u,u) < a(u,v)+F(u-v) by (9.4.1) (9.4.12) 'v'Vh E f{h, a(11h,uh) a(uh, Vh) + F(Uh - Vh) by (9.4.2). (9.4.13)

-a(u,11h) - a(11h,11)

s:

..,... "'_'_""""'''''''10''-''

Therefore we have, for all v E i( and for aJI Vh E K h by (9.4.11), (9.4.12) and (9.4.13),

all1l-1lhIl Z

s: a(1l,v -Uh) + a(1lh,Vh -u) + F(1l + a(uh

U,

v)

+ F(1lh

- Vh)

= a{u,V-1lh) - F{V-1lh) +a(u,vh -u) - F(Vh -u) Vh - u) = (F - A1l, U-Vh)+ (F - Au, 1lh -v) +a(u-1lh, U-Vh)'

Thus, we have, for all v E K and for all Vh E Kh,

allu - uhll' < IIF - Auliliu - vii + Mllu Since

+ IIF - Aulllluh Uh IllIu - vhll

vhll

lIu -

Uh 1I111l

Vh II

s h ~ 1I1l -

1lh IF

~ 1I1l - Vh liZ), we get

This implies th~ error eslirnatioll (9.4.10).

9.4.2

Error Estimation in Conaete Cases

Approximation in one-dimension: Let us consider the variational inequality: Find U E K

1
0(1l, v)

1 du dv d1l a dx(dx - dx)dx

l'

a f(v-u)dxVvEJ(,

(9.4.14)

whee. H = HJ(O,I) = (v E H' (0, 1)lv(O) = v(l) = 0),

K -

{v E HI

I:~ I <:

I a.e. in (0,1))

1 d'l.illv a elx dx dx, F(v) =

l'

a f(x)v(x)dx, f E Lz(O, 1).

Let N be a positive integer and 11 = ~. Let Xi = ih for J = 0,1, ... , Nand Ci = [X._hX;], i = 1,2,. ,N Let H,! = {Vh E C[O, IJ/ Vh(O) = vh(l) = 0, vh/ciEPI,i=I,2, ... ,N} Then
K h = KnH h = {VhEHhl !Vh(Xi)-Vh{X1_I)

Q.

......

< h, fori=
, to<l',_"""",,,,,"l1>"-I

1,2, ... ,N}.

The approximation problem (Ph) takes the form

1 dUh dVh dUh - ( - - -)dx 2: odxdx dx

l'
0

f(Vh - uh)dx,

(9.4.15)

Theorem !J.4.3 ''CSIJectively, then

Let u arul u" be fIte solutwns oj {9.4.14} llwl (9.4.15),

lIu P'"OOf.

uhll" ~ 0(")

for /

L,(O.l).

Since Uh E KI> C K, from (9.4.14) we have

a(u, Uh - u)

?:

1\
1

J(U" - u)dx.

(9.4.16)

Aoding (9.4.15) and (9.4.16), we obtam

a(Uh U,

Uh - u)

< a(vh

- U, Uh - tI)

a(u, Vh - u)

J(Vh - u)dx, VVh E J(h,

which,

In

turn, Implies that

-1
Since

-111 Uh - U[I' 2
1

111 < - Vh
2

- UII' +

l' -

odxdxdx

du (dVh --

dU) - -

dx

(9.4.17)

f(Vh - u)dx

'r/Vh E /(".

\ 1

E J( n HZ(O. 1), we obtain

du d - -(1Jh - u)dx = o dx dx

1\
0

(flu ..1-Z (v" - u)dx

u;(;

< II -zlIl'2I1vh liT

(flu

uIlL2

But we have

II :~~
Therefore, (9.4.17) becomes

L,

< II/ilL,

19.4.181

(9.4.19)

Q.

.... ,

, to<l',_"""",,,,,"l1>"-I

Let v E J(, tilen the linear interpolation is oefined by

and we haw'
_ _ -d (ThV )/ c. - v(x;) - V(Xi_l) - -I

dx

liv -dx . h "'_I dx

1'"

(9.4.20)

Hence, we obtain

~~(rhV)

::; I, since :

<I,a.c., in (0,1) .

Thus, rhV E K h . Let us replace Vh by 7hU in (9.4.19), then (9.4.21) From (9.4.18) and by well known approximation results, we get
IIrhtl- ullu IIrhu -

<

Chilullll~(o,l) ::; Chllfll1..2

(9.4.22) (9.4.23)

uIl1.. 2 (fl) $ Ch.2l1ull,F(o,lj ::; Ch 2 11fU1..2'

where C denotes constanb inoependent of u ano h. By (9.4.21)- (9.4.23), we get lIuh - ull,., = 0(11). I

9.5
9.5.1

Evolutinn Va.riational [nequa.lities and Parallel Algorithms

Solution of Evolution Variational ImxJllalities

Variational inequalities of evolution were introduced by Lions and Stampacchia (see, for p;xa.mple, Lions [211 for updatl'd references). Let l'" and H be Hilbert spaces such that

v C H, v' dense in H,

V --+ H being continuous.

(fL5.1)

We identify H with its dual so that if \'. denotes the dual of V, then

v c He F.

(9.5.2)

,n". "'_'_"_"""10"-"

We shall use the following notations: Here, L 2 (0, Tj H) oenote the space of all measurable functions u : [0, T] -+ H, which is a Hilbert space with respect to the innCT product
(U,Vh'2(O,T;Hj

T {U(t),V(t))l/dt.

(9.5.3)

If H" is the topological dual of H, then the dual of 2(0, T; H) = L 2 (0, T: H*) for any Hilbert space H. HI,2(0, T; H) (HI,I (0, T; H) will denote the space of all those elements of L 2 (0, T; H) (L1(0,Tj H) such that their oistributiona! derivlltiVf~ D, also belong to L2 (0. T: H) (L I (0, T;H). We also have a set K c V such that
K is a closed convex subsel. of V.

(9.5.4)

We do not restrict ourselve'> generally (it suffices assuming that

LO

make a translation) by

o E 1<.
Let

(9.5.5)

f be given such that f E ,(0, T; V').

(9.5.6)

We now consider a bilinear form

(u, ii) -+ a(u,fi) which is continuous on F x a(u, ii) is symmetric or not, a(u,u);?:olluIlL 0>0 'v'UEV,

V}

(9.5.7)

where we denote by

lIuliv

the norm of u in V. We look for u such that

u E L 2 (0, Tj V) n Loo(O, Tj H), u(t) E J( a.e.,

(

~: ' ii - '11) + a('11, ii ~

u)

> (/, ii - u) 'v'ii E K

(9.5.8)

u(O)

The solution has to be t.hought of as being a weak solution of (9.5.8), otherwise the condition u(O) = 0 in (9.5.8) is somewhat ambiguous. This condition becomes precise if we ado the condil ion
';;; E

L,(O,T;V'),

(9.5.9)

",,,. '.'_'_""""'''''''10''-''

but this condition can be too restrictive. We can introduce weak solutions in the following form: We consider smooth functions {i such that

~E ~:(o,T:V), ~ E L2(O.T:V
u E h for a.e.t, u(O) =

l}

o.

(9.5.10)

Then, if u satisfies (9.5.7) and is supposed to be smooth enough, we have (wp- wriw (u, u) instead of (u, 11,),,)

.Io l(~~,u- u) +a(u,u - u) T

(f,u u) -

U)] elt

~.L [(~;,u - u) +a(u,u + l'r(fJ(U{)~lJ),U_U)dt.

The last term equals

(f,u -U)]

dt

T[(:,U-U/ +a(u,t1-u) - (f,u-U)] elt > 0, .fo

u E L 2 (0. T: F), u(t) E
J(

~lIi'(T) - u(T)II~,

(since u(O) = 0, il(O) = 0), so tJ1at

(9.5.11)

for all u satisfying (9.5.10). We then define a weak solution of (9.5.8) as a function u such that
a.e.
(9.5.12)

and whidl satisfies (9.5.11) for all u satisfying (9.5.10). Thp. following exist.p.l1ce and ~pproximation result.s are wP,1l known:

Theorem 9.5.1 If the bilinear> fom~ is coen:ive, symmetric and bounded. 1.h101I the evulutiun vaflalioMI tuequali/.y giVen by (9.5.8) has a unique solution.

Proof. putting
W

Ul

Uniqueness of solution. let Ut and u" !1f' two solutions. By U = Uj and U2 in (9.5.8), adding these ine<jualities and setting - U2 in the resultant ine<lualit.r, we obtain

-(w'(l.j, wet)) - a(w(t), wet))

By coercivity, this ine<luaJity gives us

> o.

2dt1w(tjl +ollw(t)1I

I Ii

< 0,

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

1d 2dtl'U1(tW S;

o.
I

This implie:. that w(t,) = 0 or

til

U2.

:F'or existence see, fOr example. Duvall! and Lions 110].

9.5.2
J(i:

Decomposition Method aDd ParaJleJ Algorithms

We introduce N couples of Hilbert spaces

V; C Hi C \Ii&
[(j

Vi

and Hi, and N convex sets

(9.5.13) (9.5.14)

i = 1,2, ... ,N

C \1;,

[{i

closed convex subset of \I;, nonempt}'.

'Ve are given linear operators r l such that

r, E L(H, H,) n L(\'; I.,)}

Ti

maps ]{ into K f

i=1,2, ... ,N.

(9.5.15)

We are also given a family of Hilbert spaces H ij stich that

Hi] = H j ;

Vi, j E [1,2, ... 1\1,

(9.5.16)

and a family of operators

"ij

such that
(9.5.17)

The following hypotheses are made:

(9.5.18)
If N e1emenls
Uf

are given such that

u. E

[{i

V i,j,

(9.5.19)

then there exists u E [{ stich that

U,

TtU,

and mon:t)W:r

lJulli: ::; c ([,: Ilu.IJ~;) .

(9.5.20)

The h.rJ)Othesis

Ki = V. for a subset of[I,2,3, ... ,IVj,

(9.5.21)

Q.

.... ,

, to<l',_"""",,,,,"l1>"-I

is perfectly acceptable!

We now proceed with the decomposition of the problem. We introduce the following bilinear forms: Ci(U" it,) is continuous, symmetric on Hi x Hi and it satisfies

(9.5.22)

ai(11i,U,) is continuous, symmetric or not, on a;(u.,u,) > ,o.1I1,II~,

We assume that
N

\I;

\ii, and it Hatisfies

(9.5.23)

,oi > 0,

Vu, E V;,

Lc.(r,u.riul = (U.1J)II.
;=]

VU.iiEH,

(9.5.24)

La;(r,u""ti) = a(u,ti),
;=]

VU,iiEV.
aH

(9.5.25)
follows: We

Finally, we assume that. the function f is also decomposed are given functions /; E .z(D, Ti; ~a) such that
N

~)j;,r,u) ~ (J,f4 ~ (J,i,)

i=]

VUE\!.

(9.5.26)

We are now ready to introduce the decomposed approximation. \Ve look for functions U; (i = 1. 2,3.... , N) such that
Ci

(~~i ,ti, E Lz(D, Tj

IIi) + ai(u" U, - u,) + ~ ~ (rj,11 , ,

VUiEK"
Ui(t)
l~),

rijUj,

"J' (ti'i

- Ui)

1/,~

~(f"it,-u)
Ui

[(i

a.e., 11;(0) = D.

(9.5.27) (9.5.28)

Remm'k 9.5.1 (i) It may be remarked that each of the variational inequalities in (9.5.27) has to be thought of in its weak formulation as introduced above. (ii) In (9.5.27), (' is positive and small. Thp. corrf'sponding term in t.his equation is a penalt.r term. (iii) In the examples, IIrJili is a Hparse matrix.
Theorem 9.5.2

The set of (decomposed) \!arialionallnequalities (9.5.27)(9.5.28) admif<s a unique solution u, = ui (i = 1,2,3, ... N), Pu"the,', as

...... "'_'_""""'''''''10''--'

. -i>

0, one has

U:
and

-i>

11; in 2(0, T; Vi) weakly

(9.5.29)

when!

'1.1

is the solution

0/ (9.5.8)

(weak/onn (9.5.11)) .

PJ'(JOj. Step 1: A priori Estimates. We can assume, without loss of generalit.r, that 0 E K;. Therefore, taking fl; = 0 in (9.5.27) is allowed (for a complete proof, the technical details are much more complicated. One has to work first on approximatiolls of (9.5.27), by using (other) penaJty arb'Uments; see the bibliographical references. This simplification gives (we write Vi instead of vi for the time being)
(i
~

1..... N),

(9.5.30)

where

x, =
\Ve can write
Xi

L
;

(rj;u; - r.Juj, rJiu;)If'J .

(9.5.31)

~L
j

II r ji u , -

rij

u,lIif" + ~ L llr;ivill~('J - ~ L Ihj u.II~/'J

j j

(9.5.32) But one easily verifies that

LXi = ~ L
;

IhiUi -

1,J

r;juJJI~{'J'
(0,0, of

(9.5.33)
(9.5.30), and by

Therefore, by integration in t, in the interval summing in i, using (9.5.33), we obtain

(9.5.34)

Step 2: It foHowH from (9.5.22), (9.5.23) and (9.5.34) that, as c -+ 0 (and we now UHe the notation u:),

ui

remainH in a bounded set of

L2 (O,T;V;)

n Loo(O,T; fl;),

uHf,)

E I<"

(9.5.35)

~(rj,u:

- Tijuj) remains in a bounded

(9.5.361

Therefore, we ran extract a subsequence, still denoted b,r

u:=, such that

(9.5.37)

u: -+ Ui in L2(0, T; 11;) weakly,

and, by virtue of (9.6.36), we have Notice that we have not used the fact that

1Ii(t) E

I<~.,

(9.5.38)

u: remains in a bounded set of

(9.5.39)

Loo(O,Tj Ho). It follows from (9.5.37), (9.5.38), and the hypothesis (9.5.20) that
U;

= ",11.,

u.(t) E [( a.e.,

u. E L 2 (O,T; V).

It remains to show that u. = u is the solution of 19.5.8) or (9.5.111.

Step 3. We use the weak formulation. To avoid slight technical difficulties, we further weaken (9.5,H), by writing it

[ [( ~:,u - u) + a(u,u -u) - (f,u - U)] <It >

for all {/ satiHfying (9.5.10). We introduce Ui such that

o}

(9.5.40)

U, E L2(0, T; Vi), u,(t,) E I<i for a.e. I"

~~' . E L2(O, T; \I;.), }

u;(O) = 0,

(9.5.41)

and we replace (9,5.27) by its (very) weak form

, 0

'r[ (Ou,. vi,

Ci

~,lI,-ui

,) +0, (..

Ui,U,-U.

')
(9.5.42)

+~ ~ (TJOU;

- TiJUJ' Tj;(U; -

u: Hi;] lIt

: : IT
n

(f;, Ui - uDdt.

.."" "'_'_""""'''''''10''-''

Let us now assume that

iii = rilf,
If E L 2(O, T; VA),

~~

E 2(0, T; \ 'A),

<pet) E K. }
(9.5.43)

Since rjirilf = rijrJlf, the!

tel1llS

in (9.5.42) drop out so that

(9.5.441

We can pass to the limit in

in (9.5.44). Because of (9.5.39), we obtain

.
2:

'r

[Ci (:t(rilf),rilf-r,u.) +Qi(rilf,rilf-riU.)] df.

7

(9.5.45)
(/"

rilf - ,,u. )dt.

Summing (9.5.45) in i and using (9.5.24) - (9.5.26), we obtain

1 [(~~'lf-1l.)+a(lf'lf-u*)-(f'lf-U*)]l1t2:0
HO that (by uniqueness) u, = u.

,-

Parallel Algorithm. We introduce the time step At and a semi-diHcretization. We denote by 1; (what. we hope is) an approximation of u,(nAt). We then define by

u:'

CO

' +- L-. rjiUi - riJU j -1 ,rji (Ui - U," " ., > (f" ,lti _ i
e

1" , , ("

Ui -ll i

" At n-' ,Ui-Ui

~"

+a,(ui,u,-Ui)
" U, )

nA

V fl, E

/{i,

u," E l' \,
where

(n=1.2 .... ),
(9.5.46)

f." =

I [,,61 Ilt

. (n-l)LH

f,(t)dt,

U, -

-0 .

(9.5.47)

Remark 9.5.2 The algorithm (9.5.46) is paraJlel. Badl ui' is computed through the solution of stational]' \-ariational inequaJity. Once the uj'-1 are computed, in the computation of u~, only those j such that rij f:. 0 are used. Let Ub now show the sf.abilily of Ihe a[907'itllm. n.eplacing u. b.r 0 in (9.5.46), we obtain
(9.5.48)

where
19.5.49)

We observe that
_ 1 (n .. -1 -26tCjU;-U;

+ 26t (C,Ui -Ci (,,-I I (") Ui

(9.5.50) so that
"""'

,,=1
where

'" (n !J.t' ) ".,

Ui

Ci

" U.

1 ("') 1, = 26i c; u. -I- 261.<';""

(9.5.51)

(9.5.52)

Wp ObHervp next. that.

-n . = '

', ~ Irjiu, 2 " I"

j

rijU j

""/1' II,)

I + '2

"II rjiUi"/1 ~
j

If"

- '1"11rijU ""11' 2~ lJ'J'

j

(9.5.53)

If we define

F" =
then

~ L Ihi u i'II
,. ,j

If'J '

(9.5.54)

LX;' =
;

Z" -I- y" _ y.,-I,

(9.5.55)

Q.

..... ,

, toOl'._..

""""",,"l1>"-I

where

Z" = ! '"' Ilr)l~. t" 2L

i.j

_ ,.

'J

_un-III' J II"

(9.5.56)

Consequently, by summing (9.5.48) in i and in n, we obtain

_1_ ' " e;("m) 26t L
;

+ 26tL~im+ LLai(1~:')

!y'" + - L , ,

I m
,,=1

,,=1

(9.5.57)

Z" S; L
,,=1

L(f;',ui'),
i

where notations CO(Ui,Ui) = Ci(Ui) and a;(ui,Uj) = a,(u.) are used. But

(/:'. u;) S;

~01 (un + ~ lit:' II~:

so that (9.5.57), after multiplying by 26t, gives

LCi(U;")
i

+ L~im+6tLLai(u;)
i
'1=1

+ -y'" + -- L z"

26t

26t m

"

S; )"6t

n=1

,,=1

L Ilfi'lli, S; 11; .

(9.5.58)

hcuce stability follows. Here, '" and 11 are constants.

Remark 9.5.3
(9.5.46).

(i) Other time-discretization schemes could be used in

(ii) An open problem. The previous methods do nOt apply (at least
without new ideas) for nonlocaI constraints; for example, for variational ine<jualities of t.he t.ype

(a;;,

it - u) 11 t- a(u, it - u)

+ j(it) -

j(u) ;::: (f, it - u), }

(9.5.59)

'r/ ii E V, u(t) E V,

u(O) = 0,

where l/ = 1l~(!1), II = L~i!1), and where, for instance,

j(u)

~ (in 1,,"I'd")

1/2

(9.5.6()

Q.

......

(iii) Onp can p.xtend the prCHent method to some quasi-variational ine<lualities.
, to<l'._......... ",,"l1>"-I

(iv) The examples of decomposition given here correspond to decomposition of domains. Other possibilities can be envisioned, e.g., multi-Galerkin methods, using replica equ<l.tions (in our case, replica variational ine<jualities).

9.6

Obstacle Problem

In t.hifl flection, we discuss ohst.ade problem in one and two dimensions. 'Ve have seen in Section 9.2, how a one-dimensional obstacle problem expressed as a boundary vaJue problem can be written in the form of a variational inC<juality. We briefly mention how a two-dimensional obstacle problem can be modeled in t.he form of a variational inequality.

9.6.1

Obstacle Problem

Let UH consider a body A C R'l, which we shaJl call the obstacle, and two points P l and Pz not belonging to A (see Figure 9.6.1); let us connect PI and P2 by a weightless elastic string whose points can not penetrate A. Finding the shape assumed b.r the string is known as the one-dimensional obstacle problem.

x
Figure 9.6.1 Obstacle problem

We consider a system of Cartesian ax!OS OXV with respect to which PI and Pz have coordinates (0,0) and (O,t) and the lower part of the boundary of the obstacle A is a Cartesian curve of e<juation y = 1f;(x). By experience we know that if y = u(:l:) is the shape assumed by the string, then
ufO) ~ ute) ~

o.

(9.6.1)

Q.

......

, to<l'._..

....,..",,"l1>"-I

Since the Htring connects Hand P2, u(xl because the .string
doc~

< w(x)

(".6.2)

not penetrate the obstacle. u"(x) > 0

(9.6.3)

because the string being elastic and weighlless must assume a convex shape. and
u(x)

<

tjJ(x)

===}

v"(:!;) = 0;

(9.6.4)

that IS, the strmg take:,; a linear shape wheJ'e it d0e8 not touch the obstacle. Since the string tendH to the shape with the minimum possible length (in particuhu:, in the absence of an obstacle this would be f), (9.6.1). (9.6.2), (9.6.3) and (9.6.4) are equhalent to (9.6.1), (9.6.2), (9.6.3) and
[u(x) - .p(x)]u"(x) ~ O.

(9.6.5)

A physicist may prefer to formulate the obstacle problem by saying that the configuration that the st.ring takes is such as to minimize the energy of the system. In the system made up of the string and the obstacle, given that the only energy involved is that due tu elastic deformation (the string being weightless) and HO by the principle of minimum energy, the configuration u taken by the string is that which minimizes the energy of the elastic deformation
E(v) =

2 "

it

v'(x)2dx

(9.6.6)

under given conditions. Let f{ = {v E flJ(O,f)J vex) < tjJ(x) for all x E (0, f)}, and assume that tjJ E HI (0. f) Hatisfies th~ conditions. maxtjJ > 0 n .p(0) < 0, 'P(t)
(9.6.7)

< 0

(9.6.8)

The following formulations of the obstacle problem are e<luivalent:

Problem 9.6.1

Find u E [( Huch that E(u) $ E(v) Vv E T<.

Problem 9.6.2
HJ(O,f), K
a8

Model 1 where a(u, v)

above.

Q . ,~,_ .._",,"l1>"-'

..... ,

Problem 9.6.3 Find u E lfJ(O, t) such that 11(0) =:: 11(f) =:: 0, u(x) < ~(x), utI :? 0 in the sense of distribution and u" =:: 0 whenever u < 'Ij;.

9.6.2

Membrane Problem

(ECluilibrium of an Elastic Membrane Lying over an Obstacle). This problem eonsists of finding the equilibrium posit.ion of lUI elantic membrane, with tension 7, which

passes through a clrrve f; that is, the boundary of an open set the horizontal plane of the coordinates (XI, X2), is subjected to the action of a vertical force of density C =:: T j,

n of

must lie over an obstadH which is represented by a fWIction" : O-----t

n.
(see Figure 9.6.2). This physical situation is represented by the following bowldary value prohlem:

-..6.u > finn

u? xinn
(-..6.n - f)(u - X) in 0 u=::Oonf, u=::x onr o
;:;-=;:;-onr, un un where

au

ax

r'

is the interface between the sets {x E 0

11(X)

{x E

is the normal derivative operator along un ro. ro is an unknown of the problem. Finding a solution to this boundary value problem is <juivalent to solving Modell where 11 =:: HJ(O), J< =:: {v E 11J(O)/v? x, a.e. in O}

nI

>

X(x)} and

u(x) =-- X(x)} and

au av a(u, v) =:: ~1 n-~dx L..J

;=1
fl
UX 1 UX 1

(F, v)

In fvdx.

.n",

"'_'_""""'''''''10''-''

wlkllQwn contact zone

_-

Figure 9.6.2

Equilibrium of e1l.1stic string

As an ilIustl'alioll, we verify that the following bowHlary vaJue problem:

-L\lI+tt:::: -L8:;+1I:::: fonOeR'l,

(0 is boundary subset of R1.)
11

, a'

.=1

'

>

au 0,;:;un

(9.6.9)

> Oonan

u - :::: Donon
is (.-'(juivaJent to solving rvloJeI 1, where 11 11J(O), j{ {v E 1 (0.)/ (restriction u On 11 u/fJU 2': OJ, 11/00 is understood in the sense

au an

an)

of traces.
a(tI, v) ::::

(F, v)

In (t, g~ ::. + Uv) dx In f(x)v(x)dx.

>

Since [{ is a cone, the argument of Section 9.2 could be applied. By the Green's theorem (in place of integration by parts), we have

10

r (_ t ~u) + 10. un r ~uvd'

i=l UXi

in

r fvdx.

B.

.... ,

, to<l',_"""",,,,,"l1>"-I

Following the arguments of Section 9.2, it can be shown that { 0 vds 2':

811

;::: 0 Oil an. Tim", we have (BVP)(9.6.9). If 11 is a solution of (9.6.9), then Oil the lines of till"' Section 7-4, WI"' can fihow t.hat 11 is a solution of Modell
O. However 0 for all v E K which impliL-'S that

vlan ;:::

811 0n

io n

9.7 Pmblems
Problem 9.7.1

Prove th'" existence of solution of rvlodel 3. Find

u(O)
u(x)
11" 11"
U

Problem 9.7.2

E llJ (0, 2) such that

u(2)
~

S ,p(x) > 0 in the sense of distribution == 0 whenever u < 1/1.

Probl~m

9.7.3

Give a proof for Thl.'Orem 9.3.3.

Show that the variational inequahty

Problem 9.7.4

a(u.v-uF(v-u)forall vEl(
has a unique solution where

a(u,v)

=:::

1
(l

ov aa dx k=l
~ 0"
L..J
XI" Xk

J( =:::

{v E l1J(O)/v > 0 a.e. in o}

11 = llri (0).

Problem 9.7Jj
Problem 9.7.6 Theorem 9.3.2.

Discuss the existence of solution of Model 4 Obtain the Lax-Milgram Lemma (Theorem 3.9.4) from

B.

..... ,

, toOl',_"""",,,,,"l1>"-I

Problem 9.7.7 Problem 9.7.8

Sketch the proof of the existence of solution of rvlodel 9. Sketch the proof of the solution of Model 6.
~

Problem 9.7.9 Discuss the properlies of the mapping J u is a solution of Model 9 corresponding to FE H.
Probl~m

F -+ u where

9. 7.10 Discu~~ a paraJl(d algorithm for the following boundary value problem on the lines of Section 9.5.2
l' = HJ(!1) c II ~ L,(n)
J(

= {v/v

(u, v) =
u(u,v) =

In l

> 0 in O.v E V}

uv dx
\7t/. \7?J (Lx

Problem 9.7.11 Discuss the existence of solution of the variational inequality problem: Find u E 2(0, T; lJ) n L<x>(O, T; H), u(t) E [( a.e. such that
(i) "(,,(t),v-a'(I))

~ (F,v-a'(t))

(iD a(u'(t), v - u'(t) :,::-: (F.v - u'{t,.

Problem 9.7.12
J(

Let 0

(0,1) and F(v)

el

dx,

c > 0,

= {v E HJ(O)/lv'l

< 1 a.e. on O}. Write down the explicit form of

VvEJ<
u E J<.

the varialional inequality

a(u,v-uF(v-u)

References
[II C Baiocchi, A CapeJa. Variational and (luasivariatiollal lnequalilies Application to Free Boundary Problems. Chichester: John Wiley and
SolIi'>. 19R4

[2] A Bensoussan, .IL Lions. Applications of Variational Inequalities in Stochastic ControL Amsterdam: North Holland Puhl Co., 1982. [31 A Bensollssan, .IL Lions. Lmpulse Control and Quasi variational InequalitiCH. Paris: Gauthier-Villar:5, 1987.
[4J Drokate, AH Siddlql.Sensltlvity in the rigid punch problem. Advances in Mathematical Sciences and Applications, Gakkotosho. Tokyo 2:445-456. 1993.
]'v[

[5] M Brakale, AH Siddiqi (eels.) Functional Analysis with Current Aj)plications to Science, Technology and Industry. London: Pitman Research Notes in Mathematics, Vol. 377, Longman, 1998. [61 M Chiput Variatiullal In~(lUalilie; ami Flow in Porous tvledia. New York: Springer Verlag, 1984. [7] M Chipol. Elements of Nonlinear Analysis. Basel, Boston, Berlin, Birkhauser Verlag, 2000. [8] RW Cottle, .IS Pang, HE Store. The Linear Complimentarity Problems. New York: Academic Publishers,1992. [91 P Dupis, A Nagurney. Dynamical systems and variational inequalities. Annals of Operations Researdl 44:9-42, 1993. [lOJ G Duvaut, JL Lions. Inequalities in Mechanics and PhysIcs. Berlin, New York: Springer-Verlag, 1976.

...... '.'_'_""""'''''''10''-''

[11] F Giannessi Complementarity systems and some applications in the fields structw-ed engineering and of equilibrium on a network. In: AI-I Siddiqi, ed. Recent Developments in Applicable Mathematics. Delhi, Madras, Banglore: Macmillan India Limited, 1994, pp 46-74.
[12] F GiaJlllessi. (cds.) Vector Variational Inequalities and Vector Equilibria, Mathematical Theories. Boston, Dordrecht, London: Kluwcr Academic Publishers, 2000. [13] R Glowinski. Numerical Methods for Nonlinear Variational Problems. Heidplberg: Springer Verlag, Compnt.ational PhysirB Seril'l. Springer Verlag, 1984. [14J R Glowmski, JL Lions, R Trcmolieres. Numerical Analysis of Vanational Inequalities. Amst(~rdam: North l-loJland Publ Co., 1981. [15) M Jayme. tvlethods of F'ulldional Allalysis for Applicatiun ill Solid tvlechanics. Amsterdam, New York: Elsevier, 1985.

(16J AH Jiang, L Qi. A new non-smooth equations approach to nonlinear complementaritv problems. SIAM J Control Opt 35:178-193, 1997.
[17/ N Kikuchi, JT Oden. Contact problems in elasticity: A study of variational inequalities and finite element methods. Philadelphia: SIAM, 1988.

[181 0 Kinderlehrer, G Stampacdlia. An Introduction to Variational Inequalities. Nf>w York: Academic Prf>Ss. 19RO
[19J \1 r(o~vara, JV Outrata. On a class of quasivariational inequalities. Optimizalion tvlethods and Software. 5:275-295. 1995.

[20J M KOl,;vara. J Zowe_ An iterativf> two step algorithm for linear complementarity problems. Num Math 68:95-106, 1994.
(21J .IL Lions. Parallel Algorithms for the Solution of Variational InequaJitie:;. Interfaces and Free Boundaries. Oxford: Oxford University Press, 1999.

[221 U tvlosco. Some introductory remarks on implicit variatiOll<'ll problems. In: AI-I Siddiqi, ed. Recent Developments in Applicable Math-

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

ematics. DeJhi, Madras, Banglore: Macmillan India Limited, 1994

[23J A Nagurney. Network Economics, A Variational Approach. Boston, tv1assachusetts: ((Juwer, Academic Publishers. 1993. [24] A Nagurney, D Zhang. Projected Dynamical Systems and Variational Inequalities with Applications. Dordrecht: Kluwer Academic Press, 1995. [25] J Outrata, tv1 (ol,;vara., M Zow~. Nonsmuoth Approach to Optimization Problems with Equilibrium Constraints: Theory, A.pplications and Numerical Results. Boston-Dordrecht-London: Kluwer Academic Publishers, 1999. [26] L Prigozhin. On the Bean critical state model in superconductivity. Europ J Appl t>,'1ath. 7:23-247. 1996. [27] L Prigozhin. Variational model of sandpile growth. Euro J Applied Math, 7:225-235, 1996. [28] B Rodrigue. Obstacle Problems in Mathematical Physics. Amsterdam: North Holland Publ Co., 1987. [29] AI-j Siddiqi, f' Manchanda, M KOI;\~d.ra. All iteratl\'e two-step algorithm for American option pricing. IMA Journal of Business and Industry, 11:71-84.2000.

[30] AH Siddiqi. Certain current developments in \'ariational inequalities. In: Lau, Twiddle, eds. Topological Vector Algebras, Algebras and related areas. London: Pitman Research Notes in Mathematics Series, Longman, 1994, pp 219-238.
[31] AH Siddiqi. Introduc:l.ion to variational inequalities. mathematical models in terms of operators. In: AH Siddiqi, ed Recent Developments in Applicable Mathematics. New Delhi, Madras, Banglore, Macmillan India Limited, 1994. [32J .J Sokolowski, JP Zolesio. Introduction 1.0 Shape OptimiZ<'ltion, Shape Sensitivity Analysis. Berlin: Springer Verlag, 1992.

[33] P Wilmott, J Dewynne, S Howison. OptIOn Pricing. Mathematical

B.

......

, to<l',_"""",,,,,"l1>"-I

~I'lodels

and Computation. Oxford: Oxford Financial Press, 1993.

[341 D Zhang, A Nagurne)'. On the stability of projected dynamical systems. JOlll"nal of Optimization Theory and Applications 85:97-]24, 1995.

Chapter 10

Wavelet Theory
10.1

1ncroduction

Wavelet. theory is the outcome of a multi-disciplinary endeavor that brought together mathematicians, physicists and engineers. This relationship created a flow of ideas that goes well beyond the construction of new bases or transforms. A formal history of wavelets began in t.he early 1980s. In 1982, Jean Morlet, a I-te.llch geophysicist, introduced the concept of a "wavelet" , meaning small wave, and studied wavelet transform as a new tool for seismic signal anaJysis. Immediately, Alex Grossmann, a fund} theoretical physicist, studied inverse formula for the wavelet transform. In 1984, the joint collaboration of Morlel and Grossmann yielded a detailed mathematical study of the continuous wavelet transforms and their various applications, of course, without realizing that similar results had already been obtained 20 to 50 years earlier by Calderon, Littlewood, Paley and Franklin. However, the rediscovery of the old concepts provided a new method for decomposing a function or signal. In many applications, especially in the time-frequency analysis of a signal, the standard Fow-ier transform analysis is inadequate because Fburier transform of t.he signal does not contain any local information. It is a major drawback of the Fomier transform since it ignores the idea of fn~quencies changing with time or, equivalently, the notion of finding tll(' fn~quency spectrum of a signal locally in time. As a realization of this weakness, as far back as 1946, Dennis Gabor first introduced the windowed-Fourier transform (short-time Fourier transform), commonly known as the Gabor transform, using a Gaussian distribution function as the window fWlction. The Gabor transform suffered from some algorithmic difficulties which were resolved by Henrie rvla1var in 1987. rvlalvaJ" introdul..1...-'d a new wavelet called Mah'ar wavelet, which is more effective and superior to Morlet-Grossmann and Gabor wavelets.

.n",

"'_'_""""'''''''10''-''

In 1985, Yves Meyer accidentally found the existing literature of wavelets. I-laving the knowledge of the Calderon-Zygmund operators and the Littlewood-Paley theory, he was able to lay the mathematical foundation of wavelet theory. The first major achievement is due to Daubechies, Grossman and tvleyers for constructing a painless non-orthogonal expansion in J986 During 1985-86, the collaboration of Meyer and Lemarie yielded the cullstruetiun of smooth orthonormal wavelet bases 011 RamI R". The aJlticipation of tvleyer and Mallat that the orthogonal wavelet bases could be constructed systematically from a general formalism led to the invention of the multiresolution analysis in 1989. It was also Mallat who constructed the wavelet decomposition and rCOnstruction algorithms, applying multiresolution analysis. Subsequently, Battle and Lemarie independently proposed the construction of spline orthogonal wa\elets with exponential decay. Inspired by the work of Meyer, in 1988, Daubechies made a remarkable contribution to wavelet theory by constructing families of compactly supported orthonormal wavelets with sollie degree of smoothness. I~er paper stimulated the study of wavelets and their diverse applications. This work p..xplainoo the connection between continuous wavelet on R, and the disCrete values on Z and Z"; the latter has been found very useful for digital image processing. Ronald R. Coifman and Yves Me,ver constructed a huge library of wavelets of various duration, oscillation and uther behavior. With a clear algorithm developed by Coihnan and Victor Wickerhauser, it. bprame possible 1.0 do very rapidly computerized sf'arches through an enormous range of signal representations in order to quickly find the most economical transcription of measured data [5, 301. This development allowed, for example, the FBI to compress a finger print data base of 200 terabytes into a less than 20 terabytes. saving millions of dollars in transmission and storage costs.
As anticipated during the course of a seminar in the Summer of 1989 at the Kaiserslautern Uni\'ersity, Germany; Coifman, Meyer and Wickerhausr introduced thl'! conrppt of '\vavelf't packets'. TIll' wavelet theory is a refinement. of Fourier analysis which enables to simplify the description of a cumbersome function in terms of a small number of coefficients. In wavelet anaJysis, there are fewer coefficients compared to the classical Fouri(~r anaJ.ysis. The remaimng sections of tillS chapter are devoted to Continuous and Discrete Wavelets (Section 10.2), Multiresolution Analysis and Wavelets Decomposition and Reconstruction (Section 10.3), Wavelets and Smoothness of Functions (Section lOA), Compactly supported Wavelets (Sectioll 10.5), Wavelf'1. Packets (Section 10.6), Problems (Section 10.7), and References.

Q.

......

, to<l',_"""",,,,,"l1>"-I

10.2 Continuous and Discretp Wavelet Transforms

10.2.1 Continuous Wavelet Transforms

Definition 10.2.1 A function ti' E Lz(R) is called a wavelet if it has zero average on t -00,00); that is,

I:
Remark 10.2.1
c,;) = 211" JR

~(t)d' ~ O.

(10.2.1)

If'lj; E Lz(R) n L. (R) satisfying

r 1v,(w)I'
Iwl
dw

< 00,

(10.2.2)

where ;j;(w) is the Fourier transform of 'lj.!, then (10.2.1) is satisfied. The condition (JO.2.2) is called the wavelet admissibility cc.lflditiOfI.

w-Hx>

Verification. By the Riemann-Lebesque theorem (Theorem P.lO), lim ;j; (w) = 0 and the Fbmier trausfol'ln is continuou~ which implies that

o ~ V, (0) ~

I:

>j,(t)dt.

The following lemma gives a method to construct a variety of wavelets:

Lemma 10.2.1 Let IIJ be fWflzem n-times (n > L) dilJef'efltiable lion sitch that If?(''' E Lz(R). Thp.ll

JUtlC-

is a wavelet.

Proof.

Prom the property of the FOurier transform (Corollary F.l),

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

I';'(wll ~ Iwl' 1\3(wll Then

Hence, (10.2.2) holds whIch implies that 1/J is a wavelet in view of Remark 10.2.1. I

Lemma 10.2.2 Let 0 #- V! E L1(R) n L2(R) with f3 In Ixl 1/J (x) dx < 00 for a (J > 1/2. The'l 1/J is a wavelet.

In !/J(t)

o and

Without loss of generaJity, we may assume that 1/2 < (J < L. From this, it follows that I + Ixl l1 > (1 + Ixl)P and Ixl)iJlvJ(x)ldx <

Proof.

1(1+

00.

We know that the function .p (x) :::: [:roo 'Ij;(t)dt is differentiable almost everywhere and If" (x) :::: 1/.1 (xj. Pol'
X ~

0, we find

I,,(xll <
If T

(I

+1 in (I + III) " 1,p(Illdl Ixl)" (

(10.2.4)

>

0, the zero mean value of 1,') implies that If' (x) = -

00

1/7 (t) dt

This shows that (JO.2.4) holds for all x E R, and therefore If' E L2 (R) Since tp' :::: 1/J E 2 (R), by Lemma LO.2. L, 1/J is a wavelet. I Corollary 10.2.1 Fm" every flOfl-zem elemeflt 'lj; of suppmt, the following statements are e({Uivalefll:
~(R)

with compact

Q.

.... ,

, toOl',_"""",,,,,"l1>"-I

(a) The fUllctio11

'l/J is a wavelet.

(b) ReiatiOfI (10.2.2) is satisfied.

Proof.
(a)
=}

(b)

=}

(a) (Remark HJ.2.1)

R

(b) : Let. V! be

wRvelet. t.hat is, VJ E L"l(R) and has compact support,

00

~ 'l/J(t)dt == O.
is,

Since

'l/J E

L"l(R), and

'l/J

1'l/J(x)1

E L)(R); that

In
In particular,

Ixl P 11j) (x II dt <

for all (J 2': 0.

for all fJ By Lemma 10.2.2, 'l/J is a wavelet.

> 2".

Examples of Wavelet
Example 10.2.1 (Haar' Wavelet).
I

Let

O"Sx<~
~<x<l

,,(x) =

-1
0

otherwise

It is dear that

and 1!' (x) has the compact suPPOrt

10, 1] . It can be checked

Sl!l4

that

VJ (w) == - - (
~

. W)'
w/4

v'2ii

c-'('~-")/2,

and

rOC) LOC)

0(W)I"l dw ==
lw]

rOC) Isin ~ 1dw < 00. 1r LOC) Iwl

8
4

",,,. "'_'_""""'''''''10''-''

V! (x) defined above is known as the Haar wavelet in honor of the Hungarian
mathematician Alfred Haar who studied it in J909. The Haar wavelet is discontinuous at x = O,~, 1.

-1

Figure 10.2.1

I-Iaar w8",;let

Example 10.2.2 (Mexican Hat Wavelet). the equat.ion

The function defined by

is known as the Mexican hat wavelet. !p(x) satisfies equation (10.2.3) of Lemma 10.2.1, namely,
'Ij;(x) = _ dx'l

r.P

c-x2/'l = (1_x'l)e-z2/'l.

It is clear that the Mexican hat wavelet has no discontinuity. .,p(x) is a wavelet by Lemma 10.2.1.

Example 10.2.3 (Gabor Wavelet). A Gabor wavelet, with width parameter wand frequency parameter v. is defined as

VJ (x) = w- I /'le~",(x/w)2 c1'l1rtJz/w.

...... "'_'_""""'''''''10''-''

It is a complex-valued function. Its real part 'l/Jn (x) and imaginary part are given by t/Jn (x) VJI
(X)
0::::

'l/Jl

w- 1/ 2 e- ro (x/w)2 cGi(27rv:rjw)

0::::

w- 1/ 2C- ro {Z/W)2 sin (27rvxjw).

The following theorem can be used to generate new wavelets:

Theorem 10.2.1

Let'l/J be a wavelet and r.p a bounded ifltegmble function, then the COflllOlllli(Jf1 Junction V! * r.p is a 11Ial/elet. PfYJOf.
Since

,"ve have VJ * r.p E

00 /
_CQ

~(R).

Moreover,

IM(Wll'dW __ /00
Iwl
=
_CQ

LI,):_(W"l",-(w-Lll' dw
Iwl

b:,-' Theorem F 13

r00 !,):(wll' I,,(wll'dw 1-00 Iwl

00 (wJI' :s: [00 (I,): Iwl dw) suplr;?(wJI 2 00 . <

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

Therefore, 'l/J * !.p is a wavelet.

I
2 ,

Example 10.2.4 B.y convolving the Hanr wavelet with !.p(x) = e-:l: get the function represented in F'iw;re LO.2.3.

we

It is interesting to observe that the set of non-zero wavelets with compact support is a dense subset of L 2 (R) ; that is, we have the following theorem: Theorem 10.2.2 A

Let

= {1/) E L'1 (R) ! 1/J -f;. 0, 1/J has compact support awl

In

I/; (t) dt

o} ,

thetl A is a dense subset of lr~ (R).

Proof.

Let hE '1 (R), then

h E L:J. (R).
0

Let he be defined by

h ( ) ~ {h(W) Iwi >c

'" w

lwj < E

Then, for every

E,

he satisfies (LO.2.2) and so by Corollary 10.2.1, he is a

wavelet. Since by Theorem F15,

J-

('
e

Ih(w)1 , dw --t O. as
~

lIhll1..2 = Ilhll1..2' it follows that Uh", - hll1 2=

E '"-4

O. This means that every function h in '). can

be considered as a limit of a sequence of wavelets and hence A is dense in L,(Rl. I

Definition 10.2.2 (Continuous Wavelet Transform,). The continuous wavelet transform 1~ of a function J E L'). (R) with respect to the wavelet t1J is defined as
[(a,b) = T.[(a,b) =

lal

_'I' (

~(t-b) J/(t)>/' -a- dt,

(10.2.5)

where a E R \ {O}, b E Rand 'l/J denotes complex conjugate.

It is clear that for a real-mlued function, 'l/J = considered in this chapter.

1/). Ileal wavelets are often

...... "'_'_""""'''''''10''-''

u'----\I

Figure 10.2.2

Mexican hat wavelet

0.2

Figure 10.2.3

Convolution of I-Iaar wavelet and !p(x) = e- r

'fi..,.-.I""""''''''''''''_''-'

..... ,

Remark 10.2.2
0) If we consider 1ba.b (t) as a family of functions given by ( 10.2.6) where '1/' is a fixed function, often called mother' wavelet, then (10.2.5) can be wl;tten as

T1/J (a, b) = (f, 'lpn,b) = the inner product of 1 with

'lpn,b.

(ii) Sinee the wavelet transform is eJ..--prl<ssoo as the inner product of 1 with !/Ja,b (t), it is linear. The propertk>s mentioned below can be checked by using the properties of the inner product given in Section 3.2.1. Let'lj; and l' be wavelets and let 1,9 E 2 (R) . Then the following relations hold:
1. 1',;, (a!

+ f39J (a, b)

= aTw! (a, b)

+ /3T",9 (a, b)

for an.y a, (3 E R,

2. T,J>{Scf)(a,b) = l'/(a,b-c), where Se is a translation operator defined by ScI (t) ~ I (t - c).

3. T of' (Del) (a, b) = (1 1.jC) T",I (al c, blc), where c is a positive number and Dc is the dilation operat.or defined by Dcf (t) = (l/c) /(ilc) ,
4. T",rp (a, b) = T1b (I la, -bla) , a
5. T'a"'+f3 (a, b) = a T",I (a, b)

1= 0,

+ f311 (a, b) , for any scalar a, (3,

wherl< A is definl<d by A'Ij;(t) = l/J(-J.),

6. TA1/; AI(a,b) =

T'~.f(a,-b)

7. (TSc,p(/)(a, b)) ~ T./(a, b + cal,

8. (TD,.,J)(a,b) = 1/,;c(T.J)(ac,b), c> O. Remark 10.2.3 It may be observed that the wavelet transform T".f (a, b) is a function of the scale or frequency a and the spatial position or time b. The plane defined by the ....a riables (a, b) is called the scale-space or tUlle/1'equeflcy platle. T1/)I(a,b) measures the variation of 1 in a neighborhood of b. Fbr a compactly supported wa....elet, the value of T",I (a,b) dependt> upon the wt.luf' of 1 in a neighborhood of b of size proportional to the scale a.

Q . ,..-I"_"""""""IO"-I

.... ,

At small scales, T",f(a, b) provides localized information such as localized regularity of f(x). The local regularity of a function (or signal) is often measured with Lipshitz exponents. The global and local Lipschitz regularity can be characterized by the asymptotic decay of wa\'e1el, transformation at small scales. FOr example, if f is differentiable at b, T f (a,b) has the order a 3 / 2 a" n ---4 O. For morp d~tails, Sf' [8J and Section 10.4. Now, Wp, prove Parseval's formula for wavelet transforms, isometry of t.he continuous wavelet transform and its inversion formula.

Theorem 10.2.3 (ParseV'.u" Formula for \Vavelet Transforms). Let'lj; E L 2 (R) satisfy (If}.i.i), that is, V! be a Wime/ct. Then, fm' any f, g E L 2 (Il), the following forTnula holds:

I (1.9)=Cop

1= 1=
-00 -0Cl

dbda (Twf)(a.b)(T,.g)(a.b)-,
a

(10.2.7)

Proof.
F.15),
WP

By the Parseval's formula for the FOurier transformh (Theorem have

(T"f) (a. b) = (I, ,po,b)

= (I. <I'o.b)
=

i:
~

j(x)

101 1 / 2 eibx.J; (ax)dx

= (2n)"2 F{lal,,2 [(x)';;(ax)) (-b),

and

(Twg)(a,b) =
=

1: i:

9 (t)la l-"'<J;(':b)df
9(Y)lall/2e-ibllw

(~) dt

= (2n)"2 F {lal"2g(x)';;(ax) }(-b) .

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

Then

r= r = d b d a J_=J_= (T.n (a,b) (T,.g) (a,b)7

~ 2.

L:L:
oo oo

l' {fIx) J, (ax) }

l-b)1':::r{~!J(~x):",j,7(a~x") (-b) d:'" }

= 211" r= r= I (x) g(x) 'IjJ (ax)

L L
r=

-1-

I' dx-;;, <fa

~ 2. J_= ,p(ax)

1-

by Parse-val's fonnula for Fourier transform (Theorem F'15),

I'da r= -;; J_= f(x)g(x)dx

by F'ubini's theorem (Theorem 0.8),

2.

r= 1J,(wx)I' dw(j, iil 1Iwl

00

= c",(J, g), by Parseval's formula for F'ourier transform (Theorem F'.J 5).

I
Theorem 10.2.4 (CalderoIJ, Grossman, Morlet). Let Vi E 2 (R) satislyi1l9 (10.2.2). Thell, lor arlY I E ~ (Il), the lollowirl9 relatio7lS hold:

(Inver'sion fonnula)

f (t) ~ -1
Clio

j= j= (T"n (a. b) lal-'/'" ('- - b) db,.. da

-00 -00

(10.2.8)

(I.'wmetf'Y)

(10.2.9)

(UU!. .9)

mll

Iw

1117'iltC71 (I,~

(10.2.10)

...... "'_'_""""'''''''10''-''

 Proof.

For any 9 E Lz(R), we have

c,;(f, g)

(T.I, T.g),

1- co l-

r~

(':0
00

dbda T,::f(a,b)T'fig(a,lI) aZ '

=
=

I:I: roo roo roo

(I: I:

by Theorem 10.2.3,

Tv../(a,

b)I[":"':g::(t-::).p~"=,,:;:(t~) lit d/~~(J

dh d a -

L oo L oo l- oo T'fif(a,b)1/Ja,b(t)
by Fubini's theorem,

aZ g(t) dt,

Tf(a, b)Wa,b lll;J,:a

,9)
(10.2.11)

/ roo roo dlula ) \c,;,J- 1- 1-0: Tw!(a,b)1}Ja.b--;;z,g

00

=0 for aJI 9 E Lz(R}.

By H..emark 3.2.1(7), we get

Cri-J -

L oo 1-

roo

r
00

Ttb !(a.lI)tI-'a,b--;;Z = 0

dhda

and thus we have (10.2.8).

We now prove (10.2.9). Since Fourier transform in /) of (T'fif) (~, b) is (w + () (w) , Theorem F.15 concerning isometry of the Fourier transform applied to the right-hand side of (10.2.9) gives

;p

-1

/00 /00 I(T./I(P)I'dl>'-;- ~ -ew /00 - /00 V(w+O,ft(w)!,dwdl;. c,;,

II; 1 1
-00 -00 ( -00

Col'

-00

By the Fuhini theorem and Theon;m F.15,

wp

get

-!.
c,p

1-ex
I

roo V(w H)I' dI; ~ 11111'

which proves (10.2.9).

[t lIIay be remarked that Thoorem 10.2.4 was first proved by CaJderon ill 1964 in Mathematica Studia, a journal started by the proponent of Functional Analysis Stefan Banach, without the knowledge of the concept of

wavelets in explicit form. Grossman and l'vIorlet rediscovered it in the context of wavelet in their paper of 1984 without the knowledp;e of Calderon's result.

10.2.2 Discrete ""aveJet Transform and lvavelet Series

In practical applications, especially those Illvolving fast algorithms, the continuous wavelet transfonn can only be computed on a discrete grid of point (am bn ) ,n E Z. The important issue is the choice of this sampling so that it contains all the information on the function f. For a wavelct,w, we can define

ti}""n(t)=(J~/2tb(aot-IJQm).
where Do > 1 and 1J0 questions are
I.

n.mE Z.

> 0 axe fixed pnrameters. For such a family, two vital

Does the sequence {(f, Wm,n)} m,nEZ completely characterize the fune. lion I? . Is it
pu~ible

11.

to recuver

f fWIII this sequence itl a stable manlier?

These questions are closely related to the concept of frames which we discuss briefly in an abstract setting.

Defimtion 10.2.3 (Plumes). A sequence {l/!,.} in a Hilbert space 1I is called a frame if there exist positive constants A and B such that

AIIIII' < LIU,'I'n)I' < BIIIII'

0=1

for all fEll.

(10.2.12)

The constants A and B nre called frame bounds. [f A = B, then ('([uality holds in (10.2.12). [n this ca.<.:p, the frame is often called the tight frame. [t may be observed that the frame is an 01tJIOn01mal basis if and only if A = B = 1.

Definition 10.2.4 (Plume Operator). Let {l/!n} be a frame in a Hilbert space H Then the operator F from H into 6 defined as

is called a frame operuwr.

0 .,_,_......,..",,"10"-1

..... ,

It may be proved that the frame operator F is linear, invertiblf' and

bounded. Consider the adjoint. operator F* of a frame operat.or F associated wit.h the frame {lpn}' For arbit.rary {an} E 2, we have

=
n=l

= ~ (~>"",",f)
n=1

Therefore, the adjoint uperator F* of a frame operator F has t.he fonu

F' (la") ~
S;nce

La""'n.
n=1

(10.2.13)

=,

L: IU,,,,")I

IIFfll

(F'FI,f), (10.2.12) c'", be exp,essed as

Al - F*F< BI, < -

(10.2.14)

where I is the idemity operator and < is an ordering introduced in Definition 3.8.2. From this, it is clem' that FO F h8.'3 a bounded inverse. Theorem 10.2.5 Let {lpn} be a frame with frame bounds A and Band F the associatedjl-ame opemtor. Letlpn = (F*F)-llpn. Then {rPn} i~ a frame with bounds liB and 11A. The sequence {I{Jn}
IS

called the dual frame of the {lpn}.

PwoJ.

By Theorem 3.8.1(8), we have

(F'F)-' ~
Hence,

((F'p)-'r

and t.hen

= = n"f, IU,{",n))!' ~ n"f, 1(F'p)-' I,,,,") /'

~ IIF(F'F) 'III'
~ (F(F'F)-' I,F(F'F)-' f)
~ F' F)-' l,f).

Now in view of (lO.2.14), it can be verified that

!/ -< (F* F)-l < !J. B - A

This implies thaI

(10.2.15)

~ II/II' < n~, 1(1, (ii,,}) I' ~ } II/II'

Thus, {$n} is the dual frame of {lPn} with bounds The sequence {$n} is called t.he dual frame. Verification of (lO.2.15); It follows from the following results.
1.

and

~.

If T is an invertible posit.ivfl operator on a Hilbclt space, then its inverse T- 1 is positive. T is a positive operator on a Hilbert space 1I such that cd
0:

II. ]f

{3T for some 0 <

< {3, then r/:<:=:: T- 1 <

T <

J,

Lemma 10.2.3 Let F be a frame opemtor a.ssociated with the frame {lPn} and F the frame operator associated with its dual frame {$n}. Then

pOP=[=poF.
P7'OOJ.
Since
F(F'F)-' / ~ ((FF)-'
~

/.op,,))

~ {(I.ii,,)l

F/,

we have

and

...... '.'_'''''''''''''''''''''10''-''

Theorem 10.2.6 Let {IPn} be a frame in a Hilbert space Hand {rlin} the dual frame. Then

f
f
for any f E I-l.

~ ~

L L
00

00

(f, 'Pn)ipn
(f,ipn)'Pn

(10.2.16) (10.2.17)

n=1

n=l

P7'OOJ. Let F be the frame operator 8.'3sociated with frame {IPn} and F the frame operator associated with the dual frame {rlin} . Since I = p. F by Lemma 10.2.3, for any f E H, we have
_ _ 00

f = F' F f = po ((f,'Pn)) = L (f, 'Pn)ipn,

n=l

by (10.2.13).

Hence, we arrive at (10.2.16). The proof of (10.2.17) is similar.

Rem01'k 10.2.4 takes the form

If the frame is tight, then rlin = A-IIPn; so (10.2.17)

[f the frame is all orthonormal b8.'3is, then

=,

00

(f, 'Pn)'Pn.

Definition 10.2.5 (Ricsz Basis). ,\ sequence of vectors {IPn} in a Hilbert space I-l is called a Riesz basis if the following conditions are satisfied

1. There exist constants A and B, 0

< A < B such that < B 11011 ,

A 1I01[

<
'/2

L
nEN

On'Pn

wherelloll= ( L:lonl2 ) nEN

,0=(0I,02,03,"',On,"')'

2. [{ IPn} 1= H; that is, /-I is spanlled by {IPn} .

......

0",_,_"""",,,,,"l1>"-I

A sequence {It'n} in I-l satis(ying (1) is called Riesz sequence. It may be observed that a Reisz basis is a special case of frames and an orthonormal basis is a particular case of a Reisz basis, where A == B == I. Such cases can be obtained for the particular choice ao == 2 and IJ(j == 1, m == j qand 11. ::::: k; j hat is 1/Jj,k (t) = 2jf21/J (2}t - k) .

Definition 10.2.6 A function functiolls 1P}.k (t) defined by

1P

E L 2 (R) is a wavelet. if the family of

>P", (t) ~ 2'1'>p (2't -

k),
III

(10.2,18)

where J and k are arbitrary integers, is an OIthononnal basis space L2 (R) .

the Hilbert

It may be noted that the admissibility condition (10.2.2) is a necessary condition under which 1Pj,k (t); that is, 1Pj,dt) == 2jj2 1P (2 j t - k) is afnune, in general and an orthonormal basis, in particulal. For more discussion one can see [8, 11J.

Definition 10,2,7 Wavelet coefficients of a function f E ~ (R), denoted by d.."k. are defined as the inner product of f with 'l/;j,k (t); that

".
(10,2.19) The series

L L (f, V'", (tl)>p", (t)

jEZ kEZ

(10,2.20)

is called the wavelet series of f. The expression

f =

L L(f,>p", (toj""
}EZ kEZ

(t)

is called the wavelet representation of f.

Rem01'k 10.2.5
1.
'l/;j,k

(t) is lIlore suited for representing finer details of a signal ac; it oscillates rapidly. The wavelet nJefficients d},k llleasure the alllount of fluctuation about the point t ::::: 2- j k with a frequency detennined by the dilation index j.

2. It is interesting to observe that

dj,k = Toj,f (2- i ,k2- J ) = Wavelet transform of f with wavelet 1j;

at thp point (TJ,kTJ). We conclude this section by a characterization of Lipschitz < 1 in terms of the wavelet coefficients.
0'

class, 0

<

0:

ThcorcmlO.2.7 IE Lipn; thatis.lj{x)-j(y)1 0' < 1 if and only if

< [(Ix-yl"'.u<

where

J{

is a positive constant, 1j; is smooth and welllocoJized.

Proof. Let us prove this assertion First, suppose that f is uniformly Co.. Then

Idj,kl

In f

(x) Wj,k (x) (Ix

i" JI' < 1< { Ix _k2-ilo. 2 - in (I+lx-k2

< J(2-(~+o.}j

r (I(x)-[(k2
,

j))>I'j,,(x)dx d.x

11)2

i" sumption.

as { j(kTJ)1j;j(x)dx vanishes and 11j;(x)1 Conversely, if IdJ,kl S T<2-U+o.)i, then

I[(x) - [(1/)1

,,-,,1=, in view of the as(I + lxI)'

>1'",(1/)1

<: J( I: I:T( !+o), 1>I';,,(x) ; k

Let J be such that 2-) ~

Ix -

YI < 2- J + 1 Using the Mean Value Theorem,

tf!j.k

LLT
i'5.J
k

( k+o.)J IWi,k (x) -

(Y)I
I

<

0 .,_,_......,..",,"10"-1

......

I: I: [(TO;'" Ix - 111 sup ( (1 + Ix -1k2 J9

k

;1)" (1

+ 11I-k2

'I)') .

The sum in k is bounded by a constant independent of j, and the sum in j is bounded by T<2(1-ojJ Ix - yl ::; l( Ix _ ylO . FUrthermore,

L LT(!+o)j )1J;j,d x )
j>J k

1J;j,dy) I

<:

LLT
j>J k

( !,o)j (I,pj,,Ix)

I + 1'''''dy)1!

by the given propertv of ,p.

Theorem 10.2.7 is about global smoot.hness. Using localization of the w3velets, a simi18.l result for pointwise smoothness of f will be proved in Section lOA.

10.3 Multiresolution Analysis. Wavelets Decomposition and Reconstruction

1O.:l. 1 MuJUresoJuUOlI Annlysis (MRA)
The concept of multiresolution analysis (MRA) is known 8.'3 the heart of wavelet theory.

Definition 10.3.1 (Mnlti1'solution Analysis, Mallat, 1989).

multiresolutioll analysis is a sequence

p.;}

A of subs paces of L 2 (R) such that.

(i) ... C \'_1 C Vo C

(ii) span
(iii)

~'1

C'"

U \Ij
jEZ

= L2 (R),

n Ij ~ (OJ,
jEZ

(iv) f(x) E \-j if and only if f

(2- j x)

E "1- 0 ,

(v) f(x) E Vo if and only if f(x - m) E \/0 for all m E Z, and (vi) there exists a function lj) E Fo called scaling function, such that the system {lj)(t - m)}'''EZ is an orth~JllOlllLaJ b8.'3is ill Vo.

0 .,_,_......,..",,"10"-1

......

For the sake of convenience we shall consider real functions illy stated.

l{)

unless explic-

Definition 10.3.2 (a) Translation operator. Given a real number II, the translation operator T h acting on functions of R is defined by
T, (I) (x)
~

(x - h).

(b) Dilation operator. GivelJ an mteger J, we define the dyadic dilation operator Jj acting on function:; defined on R by the formula .

J, (I) (x) ~

I (2'x).

It may be observed that (i) one can define dilation by any real number, not only by 2J (ii) T h and JJ are invertible and T h- 1 = T_ h and JJ- 1 = J_j (iii) Th and 2J /1. Jj arc isometries on L2 (R).

Rerum'k 10.3.1
a. Conditions (i) to (iii) of Definition 10.3.1 mean that every fundion in L2 (R) can be approximated by elements of the subspaces \/~, and as j approaches 00, the precision of approximation increases. b. Conditions (iv) and (v) express the invatiance of the system of subspaces {~} with respect to the translation and dilation operators. These conditions can be expressed in terms of Th and .Ij as (iv), FJ = Lj (V for all j E Z

o)

(v), \/0 = Tn (Vo) for all n E Z. (vi)' Since.ls and Tn are isometries, (vi)

{2

CaJl j 2 Jx / l{)

(2

be rephrased for each J E Z that the - k)} kEZ is an orthonormal basis in \tj.

system

(c) Condition (v) follows from Condition (vi). (d) If Wt: define the dilation operator Da , a > 0, &:> D,,!(x) = a 1/ 2 !(ax), translation operator as abo"," for 11 E R, modulation operator E c as

Ecflx) = e 2wx lex}, !lx) E L l or L2 on R, c E R,

then the following results cau be easily verified: (i) D"T,/(x) ~ a'/2/("" - b) (ii) D"ToI(x) ~ T"_,,D"/(x)

(iii) (f, D.g)

(D._, [, g)

(iv) (f,ng) = (L,[,g)

(v) (D.[, Dog) = (f,g)

(v;) (T,[, T,g) = (f,g). (vii) TbEcf(x} = e-Z"-lb<:EcTt,f(x}.

(viii) (f,E,g) = (E_,[,g).

The following theorems describe tht basic properties of an MIlA whose proofs can be found in any of the standard references 011 wavelet theory (see, for example, [31]). Theorem 10.3.1

Lellj) E L 2 (R)

saii~fy

(i) {ip (t - m)}",ez is a Riesz sequence itl 2 (R),

(ii) tp(x/2) =

L
kEZ

ak

.p(x- k) amverges on Lz(R),

(iii) iP(() is continuous at 0 and 4'(0) =f. O.

Then the spaces lj = spon{"P(2J x-k)}kEZ Mihj E Z form an MRA.

Theorem 10.3.2 Let Pi]} be an MRA Mih a scalmg functwn tp E Yo. The function t/J E Wo =- Ifl e Vo (Wo EEl \;0 :::: VI) is a wavelet if and only if

(10.3.1)
for some 2r.-periodic function v (~) such that Iv l) I = 1 a. e., where m"" (,) ::: 1 2" a"c- n(. Each .meh wavelet t/; has the pmpcrty that SP(1fl {"Pi }kEZ,j<S =

nez \'8 for even) s E Z.

Remark lU.3.2

I. For a given MRA {VJ } in L 2 (R) with the scaling functionl/J, a wavelet is obtained in the manner described below; it is called the wavelet associated with the MRA {tj}. Let the subspace W j of L2(R) be defined by the condition

VJ EB lFJ

= l':i+l' \t} ..L WJ 'T/ j.

Since Jj is an isometry, from Definition 10.3.1, Jj(Vl) = ltJ+l. Thus, 1/)+1 = J) (Va EEl 11'0) = Jj (Va) EEl J j (~Fo) = V)

EEl JJ (lFo)

V;." =
This gives

EEl
j~",+1

IV)_

W) = J; (Wo ) for all J E Z.

(10.3.2)

Prom c.:onditions (i) tu (iii) uf Definition JO.3.1, we obtain an orthogonal decomposition

L, (R) ~
=

Ell L
jEZ

W, ~ W,

Ell w, Ell .. Ell W"

(10.3.3)

EEl
)EZ

J1').

We need to find a function t/J E W o such that {t/J (t - m) }mEZ is an orthonormal basis in \Va. AllY sndl function is a wavelet: it follows fmm (10.3.2) and (10.3.3).
2. Let iP be a scaling function of the MRA {\!j} , then

v,(x) = Lu.,(-lriP(2x+n+l),
nEZ

(10.3.4)

where
Un

f:

<p (x/2) <p (X - n).

(10.3.5)

is a wavelet. In fact, assuming that inteb'"r translates of iP generate an orthonormal basis for Va, wavelet t/J can be constructed as follows: By conditions (i) and (ii) of MRA, there exists Cn such that (see Section 10.4.3)

<p(x) ~ Lc,,<p(2x-n).
rlEZ

Then, (x) is J!,iven by

t/J(x) =

L
nEZ

(-I)c n+liP(2x+n).

.n",

"'_'_""""'''''''10''-''

3 It should be noted that the convention of increasing subspaces {\!j} used here is not universal, as many experts like Daubechies and MaHat use exactly the opposite convention by choosing decreasing sequence {~~}, where (iv) is replaced by f (x) E \'] if and only if f (2 j x) E 1'0 and \.j EEl ~1!j = Vj +I , \'1 .1 BTj is replaced by \'] EEl Wj = \'j-l' \'1 .1 ll'j. Broadly speaking, in the convention of increasing subspaces adapted by i\'leyer, the functions in \'] scale like 2- j whereas in the convention of decreasing sllbspaces followed by Daubechies aJld Mallat, they scale like 2j

4. With a t>ffiuoth wavelet 1/1, we can assodate an J\[RA. More precisely, let 1/1 be an 2 (R) function such that {2 j / 2 1/1(2J :r - k),j E Z, k E Z} , is an ONB of ~ (R) . Is "I/J the mother wavelet of an MRA? For any 1/1 the answer is no. However, under mild regularity conditions, the answer is yes (for details, see references in [10, p.45]).

10.3.2 Decomposjtion a1ld Reconstruction Algorit1Jms

Decomposition Algorithm Let Cj,k and dJ'~' denote, respectively, the scaling and wavelet coefficients for j l:Uld k E Z defined by

CJ.~' =
and

In f

(x)

r.pJ,k

(x) ([x,

(1O.3.G)

(10.3.7)

where
r.pj.k

(x) = 2i/2 r.p (2 J x - k)

1/'j,~' (x) =

2J / 2 1jJ (2J x - k).

r.p (x) and 1/1 (x) are, respectively, the scaling function (often called the father wavelet) and the wavelet (mother wavelet) .

0 . ,>0<1"_""""'''''''10''-1

..... ,

Since Vii," (x) = 2j(2 Vi (2 i x - k), there exists hf such that

ViJ,~' (x) =
= = =

L h/2J/2ViI,f(2 i x - k) L hf 2U +1)/2'f' (2J+I X feZ feZ feZ

2k -

)
(10.3.8)

L h/ViJ+ (x) L hf-2~'Vij+l,( tx)I ,+2" fez

Substituting this V'dlue into (10.3.6). we get

ej,/.: =
=

in

ff

(x)

fez

L hf-2/.:'f'j+l,f (x)

L h'_2~' in f (x) 'f'J+I,/ lX) dx f = L ht

feZ
2k Cj+l,f,

fez

0'

CJ,~' =
I

L h/-2kCJT I,f.
iez

(10.3.9)

Since lIo C \II every 'P E V also satisfies 'P E Fl. Sinc~ {'PI,,,, k E Z} is an orthonormal basis for ti then> exists a seqUeIH'P {h~.} such that
I

'f' (x) =

L hk'Pl,k (x) ,
kez
III

(10.3.10) the form

and that the sequence element may be written

The two-scale relatwlls/np (10.3.10), relating functions with differing scaling factors, is also knowll as the dilation etj1wtion or the refinement etj1talion. It can be seen that for the Haar basis

hk = {

0, otherwise .

h' k=O,l

(1O.3.11 )

B . ,t><l',_""""",,"l1>"-I

......

dJ-k+I,.

d j _ I ,.

Figure 10.3.1

"
<-

" "CJ-k+I,.

<

<-

" "CJ -2,

""<

" "CJ-L

<

Cj,.

Schematic reprcscntation of the decomposition algorithm

If lp is a scaling function of an MRA, then the mother wavelet 1p and father wavelet 'f' are related by the relation

'if' (x)

L
~'EZ

(-1/' IL~'+I'f'l,~' (x) .

(10.3.12)

Substituting the value of 'if' (x) from (10.3.12) in (10.3.7), we obtain

dj,k

L (_1)[
lEZ

h-l+'U'+lcj+J,l.

(10.3.13)

Given scaling coefficients at any level ), all lower-level scaling functIOn coefficients for i < j can be computed recursively using (10.3.9) and all lower-level wavelet coefficients (i < j) can be computed from the scaling function coefficients applying (10.3.13). Defining Cj,. and dj . to represent the sets scaling function and wavelet coefficients at level j, respectively, this decomposition algorithm is representeft schematically in Fi!,'lIrf' 10.3.1: The arrows represent the decomposition: Cj_2; and dj - 2 ; can be computed using only the coefficients Cj-l; for example. It may be observed that algorithms in (10.3.9) and (10.3.13) share an interesting feature, that is, in either equation, if the dilation index k is increased by one, the indices of the {hf} are all offset by two. This means that if there are only finitely many non-zero elements in the sequence {II,}, then applying the de('.omposition algorithm to a set, of non-zero scaling coefficients at level j+ 1 will yield only half as many nOll-zero scaling coefficients at level j. In a similar manner, there will only be half as many non-zero wavelet coefficients at level j. Thus, computation by the decomposition algorithm yields fewer coefficient at each le\'C1. ivlallat named the decomposition algorithm as the pymmid algorithm while Daubechies called it the cascade algorithm. This property is the key to the fast algorithm discussed in Section 10.3.4.

......
B."to<l',_"""",,,,,"l1>"-I

d,.
c

dj+l'

d/+ k _ 1

,..

'\.

'.
~

".
CJ +".-l,

Cj+I,.

--.

Cj+k,.

Figure 10.3.2

SChematic representation of the reconstruction all!;oriLhm

--

'.

...

Reconstruction Algorithm
Let {opi,d kEZ and {1/JJ,dkEZ be gencmted, respectively, by the father wavelet 'P and the mother wavelet 1/1; that is, "Pj,~' (x) == 2i/21f! (2i x - k) and 1/JJ.~. (x) == 2jf21/J (2 i x - k) form the orthonormal basis, respectively, of t} and n'~ of a given MRA for each k Further. let
Q2k

== ('Pl,O, ,",O,k),

a2k-l

= ('PI ,I, 'PO,I.:)

bz~ = ('PI,O,'l/JO,k), bn - I = ('Pl,l,'l/Jo,d,

Cj,k

L
feZ

aU_kCj_l,t

+ bU_J;dj_I,t.

(10.3.14)

0'
(10.3.15)

Verification of Equation ( 10.3.15).

'PI ,0 (xl =
and
'Pt,l

L (a2e.po,~. (x) + bz"WO,k (x).

~'EZ

(1O.3.1G)

(x) = L(a2~'-I'PO,~. (x) +b2k - 1'/fro,k(X).

kEZ

(10.3.17)

By (10.3.16) and (10.3.17), ....oe can write a similar expression for any

1.f!1,k'

First, we derive the formula for even k

<P1,1

(x) =

<P1.0

(x-~)
(a2flf'o,~+t (x) + ~f1,bo,~+f (x)
+ bU_ktiJn.1 (x).
I1U.3.18)

= =

~~a21~'( (x-~) +~(~,f (x-~)

fEZ
lEZ

= L(a2f-~"iPo,f(x)

A similar result holds for odd k. For k, only the even-indexed (odd-indexed) eleruellts of the ~'QUClKX."l; {ar} and {btl are accessed. On similar lines, an expression relating each scaling funct.ion 1.pj,k to scaling functions and wavelets at level j - I can be derived, that is, we have
1.pj,k

(x) =

L (a2f-~"lf'J-I,( +
fEZ

!>:tt-k'ljJJ-l,f).

(10.3.19)

Putting this value of 1.pj,k lx) in (1O.3.G), we get (10.3.]4); substitutin.c: the values of aU-k and b2t-~" in terms of It", ....oe get (10.3.15). I
Remark 10.3.3
L The scaling function coefficients at any level can be computed from only one set of Im\'-Ievel scaling function coefficients and all the intermediate wavelet coefficients by applying (10.3.15) recursively.
2. The two-scale sequence {hd completely characterizes each wavelet basis.

:t

For wavelets with compHct. support, the sequences considered above will contain only fimtely many non-zeru elements. For wavelets with support on all of R, each sequence element h~. is, in general, non-zero, but elements decay exponentially as Ikl --+ 00.

10.3.3 Connection with Signal Prucessing

The main idea of signal processing is to describe real-life signals (functions) be it for computation, compression ur ullderstandiug. More precisely, the tasks of signal processing arc analysis and diagnostics, coding, quantization and compression, transmission or storage, synthesis and reconstruction.

...", '"'_'_""""'''''''10''--'

Very often, the study of climatic variations and global warming are used as an illustrative example. For a lucid introduction and updated literature, we refer to [2-8, 10, 11, 16-18,21-24,26-30]. Let us assume that a signal f(t) is defined for all t E R and has finite energy, that is, If (01 2 df < 00; in other words, f E 2 (R). The

i:

objective of signal processing is to transmit or store/analyze this signal using some finite device. The whole function f is given by the totality of its values at points of R but instead of transmitting or storing the entire f, we will be looking for the situation where only a finite number of values will give the desired result. If {'{),,} is an ONB in ~ (R), then we can write

t=

L
nEN

< t, 'Pn > 'Pn

Thus, instead of transmitting the function f, it suffices to transmit the sequence of coefficients {(I, '{),,)} and let the recipient sum the series himself. Uthe ONB {'{),,} is given by a compactly support.ed wavelet {1,b;J} (Definition 10.2.6 or wavelet associated with an MRA), then. in view of Remark 10.3.3, we achieve our objective. Now. we express decomposition and reconstruction algorithms in terms of a concept of the signal processing called the filtering processing. A filter can be considered as an operator on 2 into itself. Thus, applying a filter to a signal of 2 (discrete form of 2 (R) results in another signal. [I. may be remarked that the term filter is coined from its real-life uses. We know that a filter is used in a laboratory either to purify a liquid from solid impurities (if the liquid is of interest) or to remove a solid from a suspension in a liquid (if the solid is of interest), our filter applied to a pure signal contaminated with nOIse might attempt either to Isolate the pure signal or to extract the noise, depending on our primary interest, signal or noise. As we have seen in the previous subsection, applying the decomposition algorithm involves a down-sampling operation. The operation of filtering, called subband filtering, which we consider here, operates exactly as the decomposition algorithm, namely, applying a subband filter to a signal yields a signal with length half that of the original signal. In practice. we deal only with sigllals having a finite number of nOll-zero terms; usually, we consider an even number of non-zero terms. In general, a filter A is defined by

Ah

L at-2kfr,
lEZ

(10.3.20)

where f = (/"12",, ,f","') {Afd E 2; that is, the filtering process consists of a discrete convolution of the filter sequence with the signal. Let

B.

..... ,

, to<l'._..

....,..",,"l1>"-I

[-{ be a filter defined by the relation (10.3.21 ) which corresponds

to

(10.3.9), where j is replaced by J - 'I; that is,

Cj_l,k

L,,!t(-u,Cj,f..

lEZ

Applying the filter H m-umes, we b'et

If '" Cj,. Cj_m . '

(10.3.22)

Let G be another filter defined by

GCj .

dj_I,.,

(10.3.23)

which corresponds to (10.3.13) where j is replaced by j - 1; that i!5,

dj_I,~ =

L(-I)'ILI+2k+ICJ,f.
(eZ

(10.3.24)

By (10.3.21) and (10.3.23), we b'et d j _ m,. = GHm-Icj... (1U.3.25) This means that thf' wavelet coefficients at arty lower level can be computed from scaling function coefficients at level j. Thus, the decomposition algorithm has been written in terms of filter Hand G. In engineering literature, H is called the low-pass filter while G is called lIi911-11O.SS filter. Both are examples of quadrature mirror filters of signal processing. Broadly speaking, low-pass filters correspond to the averaging operations which yield trend while high-pass filters correspond to differencing and signify difference 01' fluctuation.
Example 10.3.1 Let us consider the Haar wavelet in which 11 0 = hi 1/-12 Let I ~ U" h'/,,'" ,/" ... ) ~ Ud E I,. Then

HI:]"
where

1 = .j2(/Zk -r

hk-d, and

GI ~ Ii,
where

Ii

1 = .j2 (12k -

12k-d Choose I =

(4,G, 10, 12,8,6,5.5). Then

HI: (5-12.11 -12, 7-12, 5-12)

GI ~ (--12,--12,--12,0)

B.

......

, to<l',_"""",,,,,"l1>"-I

10.3.4

The Fast Wavelet Trallsform Algorithm

It is well known that the Cooley and Tukey fast f-ourier transform (algorithm) of data has a reduced computational cost of only 0 (n log n), a very significant achievement, see for example [21]. The main idea is to reduce the number of computations made by recursively computing the discrete Fourier transform of subsets of the data. This is achieved by reordering the data subsets so as to take advantage of some redundancies in the usual discrete Fourier transform algorithm. Along similar lines, we could take appropriate wavelet transforms by replacing the function f in the definition of wavplpf. cocffidf'nl,s hyan estimatp such as

where

9 (1,) = X~.,

k- 1<

J.

<k

= 0,
This idea could be cients as well:
lISed

otherwise.

to estimate the following scaling function coeffi-

C),k::::

i:
n
(=1

g(x)l/!j,~'

tx)dx

-:::::: LX(l/!1'~' (x). Let us start with a set of high-level scaling function coefficients, and assume that we have only a finite number of coefficients. This assumption is reasonable, as any signal f E 2 (R) must have rapid decay in both directions; so dj,k can be neglected for large Ikl. Rescale the original function, if necessary, so that the scaling function coefficients at level m are given by Cm,o,... , cm,n-]. Computing the st:aling functiull and wavelet coefficients at level m - 1 is aceomplisllf'.-\ via Equations (10.3.9) and (10.3.13). As observed earlier, the {hd sequence used in these calculations has only finite number of non-zero values if the wavelets are compactly supported; otherwise h k values decay exponentially; so it can be approximated by finite number of terms. In either case, let K denote the number of non-zero terms used in the sequence, possibly truncated Computing a single coefficient at level m - 1 according to either (10.3.9) or (10.3.13) would take at most J( operations If scaling function coefficients em,k for k {O,'" , n - 1} an> set to zero, they give exactly n non-zero coefficients at level m, then the

B . ,to<l',_""""",,"l1>"-I

......

number of non-zero scaling function coefficients at level m - 'I would be at most

[;] +

[~l +2.

where [xJ denotes the greatest integer less than or equal to x . . The total number of non-zero scaling function coefficients at level m - L is approximately ~. and the total number of operations required to compute the one-level-down wavelet and scaling function coefficients is approximately n 2f{ . 2' Let nl be the number of non-zero scaling function coefficients at level m - L Applying the decomposition again requires no more than

operations to compute the scaling function coefficients, and no more than the same number of operations to compute the wavelet coefficients. There will be approximately nl /2 or n/4 non-zero scaling function coefficients at level m - 2, and the computation will require approximately 2f{ . nJ/2 or 2f{ n/4 operations. Continuing in this manner, we find that the total number of operations requirerl to do all decompositions is approximatply
2f{ (n

+ n + n + ...)
4 R

= O(n).

Thus, the fast wavelet transform algorithm described above is faster than the fast Fourier transform algorithm.

10.4

Wavelets and Smoothness of Functions

In this section, we will discuss a relationship between smoothness of function::. and properties of wavelets.

lOA.l

Lipsdlitz Class 8nd

~Va.velets

Definition 10.4.1 (Vanishing Moments). have n vanishing moments if

A wavelet 'IjJ is said to

0 . ,>0<1"_""""'''''''10''-1

......

i:

t"1j;(t)lit = 0 for 0 <k

< n.

(10.4.1)

A wavelet 1/J has n vanishing moments and is COl with derivatives that have a fast. decay means t.hat. for any 0 < k < nand mEN, t.here exist.s Aim . - such t.hat.
V, E R, '"

AI", ' I (k' ()[$ 1 + It!'"

(10.4.2)

Definition 1l1.4.2 (Pointwise Lipschitz Regularity). A function J is pointwise Lipschit.z 0, a 2: 0, at v if t.here exist.s /( > 0 and a polynomial
PH of degree

m = [01 such t.hat

If (I) -

p,

(t)1 < f( I' - vi" ,V I. E

R.

(10.4.3)
~

'l'hcol'cm 10.4.1 (.Taffard, 1991). 111 E L2 (R) is Lipschitz 0, a at v, tJw'j there ('-Xisls a positive cons/,ant A-f such that
IT.pJ(a.b)I </lla u-rl/2

(1+

b:V 0),"1 {a,b)ER+xR,

( 10.4.4)

Conmnely, if () < n is not an integer and there exist, M and 0' < () such that
IT\'>J (a, b) I < M a Cf+ I / 2

(1 + I

b: v

Cf') , 'V

(a, b)

R+

R,

(10.4.5)

then

is Lipschitz a at v.

pfYJOJ. Since J is Lipschit.z a at. v, t.here exists a polynomial p" of degree [a] < n and a posit.ive constant K such that

If (I) -

p, (/)1 $ f(

II. -

vl o .

A wawlet. wit.h n vanishing moment.s is ort.hogonal to polynomials of degree n -1 (cle<\r from Definit.ion IOA.l). Since () < n, t.he pol,ynomial p" has degree at. most. n ~ 1. Wit.h the change of variable t' = (t - u) / s, we check that.

ITp, (a.b)1 =

(00 I J-oo p, (I) Fa'" ('.-b) dt = o. -a-

( 10.4.6)

In view of (IOA.G), we get.

IT.f(a,b)1 =
~

1 (I-a-b) d' (00 i-co /\'I t - vi 0 va 1/J ('-b) -a(00 1-00 (/(I)-p,(I Fa'"
I

dL

.n". "'_'_""""'''''''10''-''

The change of variable x :::: - - yields

,- b a

IT,of(a,b)1
U

~,;a [ : Klax-<-b-vl"I,p(x)ldx.
1<J'(x)ldx"lb-vl" [:I'I'(X)ldx)

Siace la + &I" < 2" (lal + 1&1"),

U IT,,f(a,b)1 < K2 ,;a(a" [:Ixl"
:0=;

b_ v Mao+! / 2 ( 1+ -a- ") ,

where Al is a constant and that is the desned remit (10.4.4). Let us now prove the converse. Equation (10.2.8) shows that decomposed as

can be

f (t) ~
with

L
J=-OC

l>;(')

(1004.7)

(1004.8)

Let .6.j be its k-th order derivative. In order to prove that f is Lipschitz a at v, we shall approximate f with a pol,ynomial that generalizes the Taylor polynomial
p" (t) = ( ;
[UJ (

j];<x D.~k) (v)

oc

~!v

)'

(10.4.9)

If f is n times different.iable at v, t.heIl Pv corresponds to the Taylor poly-

!lamial but this mar not be t.rue. \Ve shall first prove that

= L:

t:.~k) (v) is

j=-oo

finit.e by getting ujJJJer bounds on 1.6.~k) (t)I. To simplif,y the notation, let K be a generic constant which ma,Y change vallie from one line to the Ilf!xt hut that does not depend on j and f. The hypothesis (10.4.5) and the asymptotic decay condition (10.4.2) imply that

1l>,(I)I<-"-1=r""J<a"(,+b-VU')

/(1= 2U _='

_<;<)

12'
(1

1 + [(I. - b)

Moo

/a[m a2
(1

da dh
)

+ 2'

b-v

0') 1+1(1

b)/"'I mV

I db

0.4.10

B.

.... ,

, to<l'._..

....,..",,"l1>"-I

Since

Ib -

vr'

< 20"

(Ib -

tin'

-l-It - v() , t.Ilf'

change of mriable fI

2- j (b - t) gives

It> (1.)1 -< 1<2' J

Choosing m = a' -+- 2 yields

j=

-00

1+ Ib'lo' + Iv _1)/2'1' . db'. 1 + IlJl m

1 (01 'S

) 6

f(20. j

(1 + ~ to'') .
v
T

(10.4.11)

The same rleriv;lt.ions applied to the rlenvat.ivei'i of 6j (t) give

16rl (01 'S K2{0.-klJ (1-t- Iv 2~ t 0') 'V k < [a]

At. I, = v, it. follows t.hat.

1.

(1OA.12)

16rJ (v)! < /(2(0-k}j 'V k < [a]. (10.4.13) This guarantees a fast. dee..1.y of 16~k) (v)1 when 2 goes to zero, because a
j

is not. an int.eger so a

> [aJ.

At. large scales 21 , since

ITl.'>f (0, b)1 'S IIfllll'J';lI

wit.h t.he change of YaJ'iable, b' = t - b in (10.4.8), we have a

and hence 16j (v)1 'S K2-{k-H /2Jj, which t.oget.her with (10.4.13) proves t.hat. t.he polynomial p defined in (I0A.9) has finite coefficients. Wit.h (lOA. i), we compute

If (I) -

p,

(1.)1

L (t>; (I) - L t>; (v) (I ~!v)

J=-'lC

00

[0.

k=O

The sum over scales is divided IIll;WO at. 2J such that. 2 J ? It - vi > 2J - t . For j ? J, we can uS(' t.he classical Tarlor t.heorem t.o bound t.he Taylor C).:pansion of 6j as follows:
J

~
'S

L
1=J

00

t>; (I) -

Lt>; (v) (I. ~!v)

k=O

[oj

f:';

~ (I. -

v)lol+'

([oj + I)!

B.

......

h~~2f)1

101+'

D. j

)1
(h .

, to<l',_"""",,,,,"l1>"-I

Inserting (10.4.12) yields

I <
j{

It -

vl[ol+!

L
j=J

2- j ([01..Ll-0)

v-t 2J

0'

and since 2J ;:: II. - vi > 2J - 1 , we get I Let us now consider the case j < .1.
II

< K Iv _ tl O
(t _ /'

~ L

J-I

[01
1<=0

t>,(t)- Lt>l(v) . k!v

i=-oo

and since 2J 2': It -iJ[ 2': 2J - 1 . we get II < K 111 - tlO . This proves that Ifm - Pv(tll < Klv - W', hence f is Lipschitz () at v. I

10.4.2 Approximation a,nd Detail Opera.tors

First of all, we discuss approximation and detail of Uw Ha..1.r wavelet.
op~raton;

in th~ context

Definition 10.4.3

(a) For each pair of integers, j,k, let

(j.,~ [Tik,T'(k+l)).

The collection of all such intervals is called the family of dyadic intenJals. (b) A dyadic st(~P j1mcf.ion is fI st:f!p function f(x) with the property that for some integer j,f{x) is conSLant on all dyadic intervals Ii,"" for any integer k. For any inlerval I, a dyadic step function on I is a dyadic step function that is supported on I. (c) Given a dyadic interval at scale j, 1 k, " we write Ii." = I;,k U li.k' where Il." and lJ.k are dyadic intervals at scale j + 1, to denote the left half find right 113lf of the interval Ii,k. In fact. , Il." = 1, + 1 ,2k and 1;,k = 1 + 12 1<+1.

Definition 10.4.4 (a) Let lj'{x) = X[O,I), and for each j,k E Z define 'Pj,k(;;) = 2i12 lj'{2j x - k) = D2 ,T,,'P(x). The collection {'Pn-dX)}J,kEZ is called the sysfem of Ham" scaling functions at scale j.

.n", "'_'_""""'''''''10''--'

(b) Let 1/J(X) = t[O,I/2)(X) - X[i/2,tj(X), and for each j,k E Z define 1/Jj,k(X) = 2J / 21/J(2'x - k) = D 2,Tk1/J{x). The collection {1/'j,k(X)}j,kEZ is called the Ham' wavelet syslnn at sc..lle j. (c) For each j E Z, we define the appmximal,w71 Ofwmt07' Pj for functions J(:J:) E L 2 and for Ha..1.r scaling system by

P,/(x) ~

"fJ/,'Pi,k>v",k'
k

(10.4.14)

(d) For each j E Z, we define the approximation space span {'PJ,k (x)} liE z

Vi

hy

Vi

Remark 10.4.1 (i) In view of discussion in Example 3.6.2, P,f{x) is the function in 1~ best approximating f(x) in the L 2 sense. (ii) Since lI"k{X) = 2J / 2 );),.,,{x),
(f,'Pj.!J({Jj.dx) =

(21 !

fU)dt)

XI,."lx).

I ,.k

Thus, Pjf(x) is the average value of f{x) on Ij,k' Due to this reason, one may consider the function Pjf(x) as containing the features of f{x) at resolution or scale 2- j . Lemma 10,4,1 (a) Fm' j E Z, Pj is linear 01Jemtor on L2 (b) P, is idempotent, that is, Pi = Pj. (c) Given irlf.e!Jers J,j' wtth j < j', and 9(X) E \II' Pj.g(x) ~ g(x). (d) Given j E Z and f(x) E L 2,

IIPi /IlL, < II/I1L,.

Proof of Lemma 10.4.]. (a) and (b) follow from Example 3.6.2 and (c) from (b); we prove here (d). Since {({J"k(x)hEZ is an orthonormal system (Lemma 1004.6), by Thooff'm 3.6..'l(C),

II P j/lll, ~ LIU,'Pi.k)l'
k

~LI2i1'/, l(t)dtl'. II ',.k

By the Cauchv-Schwmtz inequality (Holder's inequality for for p = 2).

2i /'

B.

......

I(t)dt

~U

. 2'dt) U.I/(t)I'di)

I/(t)I'dt.

, t><l',_"""",,,,,"l1>"-I

Therefore,

II P ;fIll, ~

L /,
Ie
[),k

If(t)I'dt = /, If(tJI'dt = IIflll,

R

I
Lemma 10.4.2 port. Then
,~OO

Let

f
~

be continuous functwn

071

R will. compact

8111)-

(a) Hm IIP,f - fll"

(h)
,~OO

o.

Hm IIP,fll"

u.

Proof of (a). Suppose t.hat. f(x) is supported on [_21'.',21'.'] for some integer N. By Problem 10.7.13, there is an intell;er p, and a function g(x) E FJ such that.

IIf - 91100
If J

xER

,up If(x) - 9(x)1 <

.J21\ +J

>

1), t.hen by Lenlllla 1O.4.1(c), PJg(x) = g(x) and by Minkowski's

inequality and Lemma 1O.4.1(d),

II P - fll" < II P - Pjgll" + 11P,9 - gil" + 119 - fll" ,f ,f = II P - 9JII" + 119 - filL, ,(f
~

2119 - fll,,

(i)

Since

By (i) and (ii), we get the desired result. Part. (b) follows from the fact t.hat. f has compact support on Rand tvlinkowski's inequalit.y. I

B.

......

, to<l'._..

....,..",,"l1>"-I

Definition 10.4.5 Lz(R) into R, by

For &1.ch j E Z, we define the detail olJemtor QJ on

O;/(x)

PH,/(x) - P,/(x).

(10.4.15)

The proof of I he following lemma is on t,he lines of the previous lemma:

Lemma 10.4.3

(aJ The del,ail operator Qj on Lz(R) is linea7>

(b) Qj is idemlJOtent (c) lj 9(X) E H'j and ij j' is an intege,' with /

:F j

then

(d) G;v,n j E Z, and I(x) E L,(R),

110,/11"

< 11111".

Lemma 10.4.4 Given j E Z, and a cont,tnltoltsjltnclwn j(x) wi/.h compact sltppm>l, on R,

O;/(x) ~
where the sum is fini/.e.

'2)1,"'",)"",,, ,

(10.4.16)

Proof of Lemma 10.4.4. Let j E Z be given and let j(x) have compact support on R. Consider Qjj(x) for x E 11k We observe that
2i+ 1 / , Fj+lj(X) =

21+ 1 / ,

, " ,

..
..

j{f)dt j(O&, if x E

Ij,,,,

/'

and that

Pd(x) = 2J

/,
fJ,k

j(t)dt if x E

Ij,k'

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

For x E IJ,I;
Qjf(x) = Pj-Hf(x) - Pjf(xl

~ 2' (2/, ~ 2' (/,

It

I' ;.1

f(')dt - /,

I' 1.k

f(t)dt. - /,

,~

f(')d')

J.k

f(t)dt. - /,

J.k

1'-

f(t.)d')

j.k

= 2)f2 (j, 1/J),k~,

by the definition of th~ Haar wavelet, and
011

Ij,k

Since

21/'2
1J;J,l;(X) = Q,f(x) ~ {
02 i12

t ifxE/),k if x E 1;'1; otherwise

(I,';'",),p",.

I
Remark 10.4.2 (i) As we have seen that in passing from f(x) to Pjf(x), the behavior of f(x) on the interval Ij," is reduced to a single number, the average value of f(x) on '),1; is lost. In this sense, Pjf(x) C<ln he thought of as a blurred version of f(x) at scale 2- j , that is, the details in f(x) of size smaller than 2-) arc invisible in the approximation of Pi/(x), but feature<> of size larger than 2- j are still discernible in Fj(x). (ii) For each j E Z, the wavelet space W j is given by
1Vj = Sp;lIl{1/Jj,I;(X)}"EZ.

Since h'Jj,k(X) hEZ is an orthonormal system on in light. of Lemma 10.4.4 and Example 3.6.2, Qjf(x) is the function in W j best approximating f(x) in the L 2 sense. In light of the interpretation of Pd(x) as the blurred version of f(x) at scale 2- j , we can interpret Qjf(x) 'loS containing those features of f(x) that are of size smaller than 2- j but larger than 2- j - 1 In other words, Q)f(x) ha.':i thoSf' details invisible to the approximation p)f(x) but visible to the approximation Fj+lf(x) .

n,

0 . ,>0<1"_"""""""10"-1

..... ,

Approximation and Detail Operators for an Arbitrary Scaling Function. Let us now consider lj'{x) and 1,I;(x), respectively, general scaling and wavelet functions of an !VI RA {V,}.

Definition 10.4.6

For each j, k E Z, let

Pj/(x) ~ L(f,\Pi,,)\Pj,,(x)

(10.4.17) (10.4,18)

Q,/(x) ~ P,+, I(x) - P,/(x),

{lj'j,dX)}j,kEZ is an ort.honormal basis for

ltj [Lemma 10.4.61.

Mtppmt on

Lemma 10.4.5

Fm' all contmuous f(x) hamng

C011l1JGcl

n,

la)

,~OO

lim

IIPjl - IIIL,

(10.4.19) (10.4.20)

Ib)

1--->-0:>

lim

IIP,/IIL, ~ O.

Proof of Lemma 10.4.5. (a) Let f; > O. By Definition lO.3.1(ii), there exists p E Z a.nd g(x) E such that IIi - gl1L2 < t/2. By definition 1O.3.1(i), g(x) E V, and Pj!l(x) = g(x) for all j ;:::.: p. Thus,

tv

III -

Pj/IlL, ~

III - 9 + Pig ~ Pj/IlL, < III - gilL, + IIPj(f - g)IIL, < 2111 - gilL, < <,

by rvlinkowski's and Bessel's inequalit.y. SinCf' this inequality holds for all j

> p, we get. .lim UPji - ill/<2

,~OO

= O.

(b) Let i(x) be supported on I-M, All and let. l > O. By the orthonormality of {IPJ,k(X)}, and applying the Cauchy-Schwarz (HOlder's inequality

."", "'_'_""""'''''''10''-''

for p = 2) and Minkowski inequalities,

II P;/IIL

L(I,,,,,,,)
k
L~

~ L

'" 1(1,,,,;,,)1' ,
M '

L i-Itt f(x)2 U'",(2'x r

k

k)dx
M

<

L (i-Itt I/(X)I',[,') 2' ( LM 1",(2'x - k)I'dX) r r

k

~ 11/111, ~ ["M-' 1",(x)I'dx.

We need to show t.hat.

2j !ll-k

J.H~~ ~ [2;M-k 1'P(x)1 2 dx = O.

For this let
.

2J M-1e

> 0 and choose

f(

so large that

Therefore, if 2J M < 1{2, t.hen

."", "'_'_""""'''''''10''-''

2] At{-k

Since for each k E Z,

lim
J-4-oo

J-23 M-k

Icp(x)1 2 dx

= 0,

Since

> 0 was arbitrary, the result follows.

Scaling and Wavelet Filters

10.4.3

In this subsectIOn, we prove exIstence of scaling filters, present a construction of wavelet with a given scaling filter and pro e certain proper ies of scaling and wavelet filters.
Theorem 10.4.2 Let t.p be a scaling function of the l\.fRA, {Vj}. Then the'f" exists (l; sequence h k E 2 such that
J(X) =

L hk 21 / J(2x 2

k)

(10.4.2 )

is

(l;

junction in L 2 (R). MO'f"'over. we may write

tjJ(t) = 1nlp(tj2)cjJ(tj2).
(10.4.22)

1)Jhere

(10.4.23) Vve reqUITe the folloWing lemnla m the proof:

Lemma 10.4.6 For each j E Z, {lfJ,dx)L,A:EZ given in Section 10.3. is or) orthonormal basis.

Pruofof Lemma 10.4.6. Siul:elfo,k E \10 for all k, DefinitionlO.3.1(iv) implies that D 'l1 !.r-"o,k(X) E Vj for all k. Also, since {lfloAx)} is an orthonormal sequence of translates, Remark 10.3.1 (d) (v) implies that {<po.A" Iflo,m}

= {D 'l1 !fO,k' D 'l1 1fl0,m} = (IflJ,A"

Ifli,m)

= Ok_""

Hence, {Ifli.klx)} is an orthonormal sequence. Given j(x) E VJ , D 2- 1j(X) E Vo so that by Definition 1O.3.1(v) and Remark 1O.3.1(d),

D,_,jlxj = L(D,-,j(x),'Po,,(x))<po,dx)

= L(f,D,,<po,,)<po,,(x),
Applying D 'l1 to both sides of this quation, we obtain

j(x) = D"D,_,j(x)

= D 2, L,{f, D 'l'<P0,k)<po,k(X)
=

, LU' D2;<P0,k)D2J'P0,k(X) ,

= L(f,'Pj,,)'Pj,,(x),

This proves the theorem since {lfJ,k(X)} is an orthonormal sequence and every element of \.~ is Its linear combination. I Proof of Theorem 10.4.2. Smce Ifl E Fo ~ l'i, and since by Lemma 10.4.6, {lfl,k(X),.,EZ is an orthonormal basis for VI,
If(X) = L,(<p,<PI,d21/'l<p(2x - k).

Thus, (10.4.21) holds with hI; = {Ifl, Ifll ,1;), which is in 1?2 by Theorem 3.6.2 By taking the Fourier transform of both sides uf (10.4.21), we get (10.4.22).

I
Definition 10.4.7 The sequence {lid in Theorem 10.4.2 is called t.he scaling filter associated with the scaling function Ifl of M RA Vi. The

'fi"twl<.. . ~........._"-'

,"m

function m",,(t) defined by (10.4.23) is called the auxiliary function associated with <p(x). Equation (10.4.21) which is nothing but Equation (10,3.10) is called thf' refinement or' two-scale difJen!nce (llilation) equation. 9" = (-I)"hl_k is called the wavelet filter.
ThCOl'cm 10.4.3 {h k 1 and {911 1 be,

Let {l'j} be an MRA with scaling function <p(x), let n~spectivel", tile scaling anll wavelet filte,.. Let 'ljJ(x) = LYII2 1/ 2 <p(2x - k).

(10.4.24)

Then {1,b"II(X)}),kEZ is a wavelet orthonormal basis on R, We refer to [29] for Its proof.
Theorcm 10.4.4 fiUer gil, then

Let {\I;} be an 111 RA with scaling [dter' h k and wavelet

(i)

Lhn ~ h,
k
n

(iii)

L
k
k

hllh k _ 2n = L
,

gllgll-'2n = on,

(iv) LgA.hk-2fl = 0 fm' aff nEZ,

(iv) L
k

h m - 2k h "_2k

+ Lgm-2I1gn-'2k
k

= O,,-m'

We require the following lemma in the proof.

Lemma 10.4.7 Let <p(x) be a scaling function of an AI RA, PI)} belonging to L] (R)nL 2 (R); and letTjJ be the associated wavelet defined by (10.4.24)

.n",

"'_'_""""'''''''10''-''

and'ljJ also belong to L1(R). Then

j"
(i;)

r <p(x)dx

= I

(10.4.25)
(10.4.26)

h
..

'(x)dx = 0

(iii) liv)

<i'(n) --" 0 for all intcgers n

f:- 0

(10.4.27) (10.4.28)

I>(x +n) ~ I.

Proof of (i). Let I(x) be gwen such that IIfllL2 = I, j(t) is continuous and supported in [-0,01,0 > o. It can be verified that

rf'J,k(t) = 2-jf2<i'(2-it)C-2:dkTl/ } (By using properties of t.he Fourier I.nlnsform).

By Parseval's formula (Theorem F15)

Since {2-i/2 e-2Il"ikT 1t hEZ is a complete orthonormal system on the interval [-2i - l , 2i - l l, therefore as long as 2-; > 0, the above sum is the sum of the squares of the Fourier coefficients of the period 2J extension of the function j(t)<i'(2- i t). By the Plallcherel Theorem (Theorem FIS), we have

Since <p E LI(R), <i'(t) is continuous on R by the Riemann-Lebesgue theorem (Theorem F.IO). It follows that
,~=

.lim ~(Tit) = ~(O)

uniformly on [-0,01,

By taking the limit under the integral sign, we conclude that

.n",

'.'_'_""""'''''''10''-''

Since .lim

,~=

IIPj/ll"'2

:=

111111',2,
~ lim

II/I1L

,~=

II P;/IIl,

lim
)400

f" 1-

I/(t)l',;(2-'t)I'dt

~ 1';(0)1'1: [j(tll'dt
~
Hence,

1';(0)1' II/I1L

1.p(OW :=

1, and since If'(x} E LI!

1';(0)1 ~

j"

r <p(x)dx

~ 1.

I
Proof of (ii). Since ljJ(t) := m<;>(tj2)(jJ(tj2) and .p(0) #- 0 as we have seen above, 7n",,(O) := 1. By taking the Fourier transform of both sides in Equation (10.4.24), we get
(10.4.29)
where

(10.4.30) (m"" is given by tJUA.23). Thus, we can write ;Pet) := e- 2 ':I(t/2+1/2)m<p(t/2 by the orthollormality of {Tk<P(X)},

+ 1/2)ojJ(t/2)

and since

Im",(t)1 2 + Im",,(t + 1/2)1 2 := 1,

(see for example [311),

m",(l/2) := 0, and I.enee ;P(O) ::::: O. Therefore, we have the desired result as ,,(x) E L,(R). I
Proof of (iii). Step I: First, we prove that the sequence {Tng(x)}, where g(x) E L'AR) and l~l is as in Definition 1O.3.2(a) for It := n, is orthonormal if and only if for all t E R
( 10.4.31)
n

.n",

'"'_'_""""'''''''10''-''

We observe that

if and only if (y.TkY) = 6... By Parseval's formula,

In

g(t)g(t

By thl! uniql1l!lIess of Fourier series

if and only if

LIii(t+nW =
n

1 for all

t E R.

Step 2: In view of Step 1,

L 1\O(t. + n)l2 =
n

1 for all t E R.

In particular. which implies

L 1\O(nW =
n

1 by choosing t

= O.

By part (i). \0(0)

=0

L: 1-"(n)I' = o.
,.,t.o

Hencp-, \O(n) = 0 for n Proof of (iv).

f:. O.

We observe that
WI"

L cp(x + n) E 1 [0,1) and has period

n

L By parts (i) and (iii),

have

'fi"twl<.. . ~........._"-'

......

\0(0) = 1 and <,e(k) = 0 for all integers k

f:. U.

'The only function with period 1 and Fourier coefficients equal to Ok is the function that is identically Ion (0, 1). Therefore, we get Equation (1004.28).

Proof of Theorem 10.4.4.

r.p(x)dx

#-

Proof of (i).

By Lemma lOo4.7(i),

so that

r r.p(x)dx = Jr Lhn2 }n
Il
'l

1 2 / 1.p(2x

- n)dl - n)dx

= =

Lh" r 2
" Jn

1 2 / 1.p(2x

L h'l2- 1/

In

V'(x)dx.

Canceling the nonzero factor

In

V'(x)dx from both sides, we get

"
Proof of (ii). By Lf'Juma JOo4.7(ii). 1n'ljJ(Xh]x = 0 so that

In
=

w(x)dx.

~
=

/, L 9,,2"\'(2x R "
2

n)dx

L9n h21 / V'(2X-n)dx ~ L.,,,T'" r <p(x)dx J

I'

il

;, L 9n
n

as

in

<p(x)dx

~ 1.

.n",

"'_'_""""'''''''10''-''

Thus,

L 9n ..

.=::

O. Since {If'o.,,(x)} and {lfll'tl(X)} are orthonormal systems

Proof of (iii). on R, we havp

6" =
=

~ lfl(x)lfl(x ~L

n)

hklfll,k(X)

L h mlfll,,,,(X -

n)dx

~ 2::2:>'"'''-''' in \I""(X)\I'" ... (x)dx r n ",

=

L hk h k_2.,.

Therefore, h k hl-_ 2 " = Ii.,. The above argument gi\'es us thaL

is an orthonormal system of R. Proof of (iv). argument yields Since (1/Jo,,,(x), lflo,,,,(X)) = 0 for all n,m E Z, the above

L 9k h k-2n = O.
Proof of (v). Since for fIny signal (sequence) Co,,, ,
Co,n

= LC1,kh,,-Zk +
k

L d l,k9"-2k'
k

where
Cl,k

d] ,k

L CO,mh"'~2k ... L c{),m9m-2k, ...

m

(see also Section 10.3.2)

it follows that
C{)," =

L L cO,m h m_2k h ,,_Zk + L L cO,m9m-2kg"-2k

k k
m

= L

. c{),m , ..

(L

h m _ 2k h"_2k

+ L9"'-2k9"-Zk) ,

."", "'_'_""""'''''''10''-''

Hence, we must have

Lh ,
I
10.4.4

m _ 2k h,l_2k

+L

gm-2kg,,-2k

= 8,,_m'

Approximation by MHA Associated Projections

The main objective of this section is to discuss relationship between smoothness of functioll~ measured by modulus of continuity and properties of wavelet expansions.
Definition 10.4.8 Let I Sp< 00 and 8 > 0, let

I
~

be a real-valued function defined on R. For ,up II/(x) - I(x - h)II,, o<l h l9

w,,(f;o)

( 10.4.32)

w ,,(f; 8) is called the p-m()(lufus oj cmlfinuity oj I and we s<'ty that I has a p-lIlodulus of continuity if w,,(f;8) is finite for some 8> 0 (equivalently, for all 8). The set of all functions having a p-modulus of continuity is denoted by J.F,,(R)
Remark 10.4.3 (i) For each I and p the function w,,(f:o) is an increasing function uf O. (ii) w,,(f;o) -4 0 as 0 -4 n if either I E L",l :5: P < 00 or I is continuous and has compact support. For any positive integer Tn, w ,,(f; mo) :5:

mw,,(f;o).
(iii) For each 0 > lJ,

w,(f;o) S 211/I1L,.
(iv) If lim s-Iw,,(f; s) = 0, then
HO

(10.4.33)

(10.4.34)

Conversely, if w,,(f;8) = 0, then I is a constant function. (v) For translation and dilation operator (Definition 1O.3.2(a) and (b, (a)
(b)

w(T,I;o) ~ w,,(f;o) w,,( Jal; 0) = 2- a/l'w,,(j; 2G o)

(10.4.35) (10.4.36)

B.

..... ,

, t><l',_"""",,,,,"l1>"-I

J satisfies Holder's condition with exponent 0, coo, c > 0 constant.

(vi)

0<

0'

:s:

1 if w=(J j 0) <

(vii) For each 6 > 0, w p (j;6) is a semi-norm on lFp(R), that is

w,(ol + (1g;o)
and
w p (j;6) =

s lalw,(f;o) + 1(1lw,(g;o)
and only if J = constant.

(10.4.37)

if

(10.4.38)

With each AI RA, {{f;} following equation:

W~

ran assodatf' projL-'Ctions Pj defined by the

P,/(x) =

1:
'cZ

l(t)2'<p(2't, 2'x)dt,

(10.4.39)

where
<p(t,x) =

L <p(t -

k)<p(x - k),
t.h~

(10.4.40)

and Ifl(X) is a real scaling function satisfving

condit.ions
(l0.4.41) (10.4.42) (10.4.43)

I:
1<p(t,x)1
0''

1<p(x)1 S G(I + Ixlr",(1 > 3 1<p'(x)1 < Gil + Ixl)-P,(1 > 3

<p(x)dx = 1.

It may be observed that (10.4.43) is automatically satisfied in view of (10.4.41) and Lemma 1O.4.7(i). It follow:, from (l 0.4.4]) that

< C L (1 + It _ kl)B (1 + Ix _ kl)"'

'cZ

( 10.4.44)

(10.4.45)

From (10.4.43), it follows that

1:

<p(x, t)dt = J.

( 10.4.46)

...... "'_'_""""'''''''10''-''

Theorem 10.4.5

(Jackson's inequality).
E

The,-e exists a constant C

E

such that lor any I

W,,(R)

IIf - P,fll" < CWp(f;Z-i)

1):OS: 00 (IOAA7) takes the form

M oil j

Z.

(10.4.47)

It may be observed that in view of Remark IOA.3(iii) for

E L p , 1 :os:

(10.4.48) Let (10.4.47) hold for j = 0 and some constant C. Then in view of the relations

Proof.

IIJ,fll, ~ 2-/'lifllL. PiJ.. = J,.Pj _ r , where J s is the dilation operator (Definition 10.3.2(b
and (10.4.36), we obtain

IIf - pifll,. ~ 2-'/'IIJ-if PoJ-,fll,. i < fl) = Cm p " (f'Z-i) _ C2- /l'w " (J -J ,
so it suffices to consider the case j = O. From (10.4.39) and (10.4.46), we catl write

f(x) - Pof(x)
From (10.4.45), we gel

~ i:1f(X) -

1(L)]",(L,x)dL.

III -

Po/lI~. ~

/= < C/= (/=

= _= /
-00

_=If(x) - 1(L)]",(l,xjdt dx

I/(x) - I(/)Id')" &

1 + It

-00

~ c/=
Writing {3-1 = a+b with a,b

(/= If(x)(1 -+I(X+u)ldU)" dx. _= _= lull" ,

> 0, al) > 1)+1
and bq

xiI'

> 1 where

and applying Holder's inequality to the inside integral, we get

-+ - = 1 l' q

IIf - Po III < c/=

v -

I < C _ool+u (

1= luD"" + = I I)"p /= /
1/(") - I(x
(1 +
-00 00
-00

u)I' du

(/=

-00

du )'/' dx 1+ lujb<J

I/(x) - I(x + u)I"&du

< C _ool+u"P wp(f; luD'du. ( I I) /

...... "'_'_""""'''''''10''-''

By splitting the last integral into two parts and estimating each part separately as follows, we get the desired result:

[11 wp(f, IuD'" (1 +~:I)a", < CWp(J; 1)'"

and

L oo

r-'

-l-

/,= wp(j; IuDI' (1 +du luDo"

1

< 2 /, w,,(f;u)" (1 +u)a" I

du

< -

tiu U (J)P I+uo C/,=" W , , ; I ( )

I
00

(Remark 1O.4.3(ii)

$ Cw,,(j: I)'" / , I

u"du (l + 'U )"1'

$ Cw,,(j; 1)".
The last inequality holds from choice of a. The em;e p = 00 rl:'(luire:s the stalldanl lllodificatiom;.

10.5
10.5.1

Compactly Supported Wavelets

Daubechies \VaveJets

Daubechies (see for details [7,81 and Pollen in [3]) has constructed, for an BJ'bitrary integer N, all orthonormal basis for L 2 (R) of the form

2J1 '1 W(2 J x - k), j,k E Z

having the following properties: The support of WN i:, contained in [-N + 1,N] To emphasize this pomt, 'I./J is often denoted by'l/!N.

I:

1./1;.' (x)clx

I:

XVI;.' (x)dx

I:

N X 1./'N \x)dx=O. (10.5.1)

1./iN (x) has -yN continuous clerivatiws, where the

positive constant -y is about 1/5.

(lU.5.2)

In fact, we have the followmg theorem: Theorem 10.::;.1 (Dnubechies). The1-e exists a constallt [{ tjuch that for each N = 2,3,'" , thef'(; exists all MRA with the scali1l9 fU1lctioll r.p anll an associated wavelet 'I./J StICh that

B.

..... ,

, to<l'._..

....,..",,"l1>"-I

1. r.p (x) and 1/J (x) belong to eN.

2. r.p (x) ani.l1/J (x) an~ compactly sttppo1"ted and both S1tw r.p and sUP}J 'I/J (x) an~ contained in [-J<N, /{N] 3.

i:

1/JN (x) dx =

i:

xl/!N (x) dx = ' .... =

i:

N x t!'N (x) dx = O.

We refer to [8] fOI" a proof of the theorem. Here, we present aconstl"uction of the Dfmbechies scaling function and WH.vdp.t on [0,3] dup to Pollen (for details, see [31]).

Theorem 10.5.2
fi'f/9,

Let D =

U
JEZ

D j when; D j = {k2- j /k E Z}

(D

1S

that is, sums, dilJery;nce and pmduet of elements of D are also in D. It is a dense subset of R). Then tJler'e exists a unique function r.p ; D --t R having the following pmpertics; I'(x) ~ al'(2x) + (l-b)I'(2x - I) } + (I - a)I'(2x - 2) +bI"(2x -3)
(10.5.3) (10.5.4) (10.5.5)

L
kEZ

I'(k) = 1

o.p(d) = 0 ifd < 0 mod> 3,

wher"e a =
I

+ V3 b
4 '

1-

V3

-- < b < 0).

4

(it is clea," that

1 <a<land 2

Theorem 10.5.3 The function r.p defined m Tlwofy;tIt 10.5.2 extends tu a continuous function on R which we also denote by r.p. This continuous function r.p has the following pmperties:

f:

I'(x)dx= 1,

(10.5.6)

and if k = 0

;!k"O'

(10.5.7)

......

In olher words, o.p

15

a scolmg fllnctwn.

B."to<l',_"""",,,,,"l1>"-I

Theorem 10.:;.4

The function -0/ defined ll5

,p (x)

-b<p(2x)

+ (I

- a) <p(2x - 1) - (1 - b)<p(2x - 2)

+ a<p (2x -

3) (10.5.8)

sat1sfies the following conditions:

'"P/',p(x)

c [0,31

i:
Lemma 10.:;.1

r= J_= ,,(x)'i'(x-k)dx~
,p (x - k) <p(x) dx =

{l
0

if /';=0 if kfO

(10.5.9) (10.5.10)

fa' all k E Z.

To prove Theorem 10.5.3, we need the following lemmas (D is Throrem 10,5.2);

<is

in

For every XED, we have

L<P(x-k)=1.
kEZ

(10.5.11)

am'.l

,,(3-,13 ~ 2

+ k) )O(x - k)

x.

(10.".12)

Lemma 10.5.2

If xED and 0 $" x$" 1, then

2rp(X)T)O(X+ I) 2<p(x+2) +)O(x+ I)

X+

1 T ,13
2

(10.5.13) (10.5.14)

= -x+ 3-,13 2
-I + ,13
2 .

<p(x)-<p(x+2)=x+

(10.5.15)

...... '.'_'_""""'''''''10''-''

Lemma 10.5.3

For 0 <x < 1 and XED, the following X ,0(0+2 ) r r,p (

~

n~lations

hoUl:

a<p(x)

I+X) 2+.)3 2 = br,p(x) +ax + 4

.)3 r,p ( 2+X) =ur,p(I-rx)+bx+4"" 2 r,p(3;X) =ar,p(1 r,p (

<p

+x)-ax+~
3-2.)3 4

4+X)
2

=rup(2+x)-bx+

5+X) ~b<p(2+x). 2
N

Lemma 10.5.4

Suppose that m ({) =

L
k=M

ake-ik~ is a trigonometric

1m (01'

1m + rr)I'

~ 1 JM all, E R m (0) = 1
m({)

(10.5.16)

-#

f01'{ E [-;,

;J,

(10.5.17) (10.5.18)

Then the infinite pHH.luct

0)- IIm(Tj,)
j=l

(10.5.19)

converyes almost uniformly. The fU1lCtiun B (~) is thus continuOM, Mon;over', it belongs to L'1 (R). The funetwn r,p gwen by

rp

the support con/.ained in [M,N] ani.l is a scaling function of an MRA. In particular', {r,p (x - k)} ~s an OND 1n L'1 (R). The function t/J (x) i.lefined by
,p(x)~2

~ v2rr

B({) has

L
k=Al

(-I)'a,<p(2x+k+l)

(10.5.20)

M~N-I N-Af-Ij

'

.,.." "'_'_""""'''''''10''-''

We give au outline of the proofs of Theorems 10.5.3 and 10.5.4 and refer to [31] for more details and proofs of Lemmas. *Proof of Theorem 10.5.3. Let K be a nonlinear operator acting on the space of functions on R. Let us define l( (f) for x E [0, II by the following set of conditions;

I((f) ( o+X) ~ a/(x) 2

l(n(';X)
I((f) ( I((f)

= b/(x)+ox+ 2+
a/()

V3

I((f) ( 2+X) = 2

V3 + x) + bx+ 4

3+ 2

x) =b/(I+x)-ax+.I
~a/(2Tx)-bxT

2 ( 4TX)

3-2V3 4

l((nC;X) =b/(2+x)
I((f)(y) = 0 1m' y [0,31,
1 3 5 This definition gives two values of l( (f) at t1w points 0, 2' 1, 2,2, 2,3. Let us denote by f.{JJ,j = 0, 1,2, , the continuous, piecewise linear function on R which on D j equals f.{J. The function K(f.{J,) is well defirwd at each jJoint and in fact, K (f.{Jj) = f.{J,-..I. Let x E [0,31 and j > 0, then we immediately get from the definition of f( (f),
1',+, (x) - I'i (x) = I( (I'i) (x) - 1(1',_,) (x) } = "(I', (y) - I'i-' (U)),

(10.5.21)
l(

where 1/ = U ur 1/ = band y E R is a point depending on x. Since o for x 1- [0,3] aud max (Ial ,Ibl) = a, from (10.5.21), we get
1If.{Jj+l - f.{J,II"""

(f) (x) =

< a lIf.{Jj

f.{Jj-ll1"",,;

so by induction, we get

Since lIf.{Jl -!foil""" is finite, the seqnence {<Pi} converg~ uUiformly to a continuous function which is denoted by f.{J. This proves the first part of the theorem.

.n",

'.'_'_""""'''''''10''-''

We know that (JO.5.11) holds for all x E R. Since Stipp r.p C [0,3] for each x E R, there are at most thret> nonzero terms in the snm r.p (x - k) . For

kEZ

a positive integer At, let

F,,(x)~

" L
A=-M

<p(x-k).

From (10.5.4), we conclude that

IFM (x)1

< C for some constant C,

and
PM (x) == 1

if

Ixl < M -

=0 iflxl>M+3.
Thus. for every int.eger M,
2(M -3)-12C

I:

F" (x)dx < 2(M -3)

+ 12C.

(10.5.22)

From the definit.ion of PM, we also conclude that

I:

F" (x)dx

~ (2M + I)

I:

<p (x) dx.

(10.5.23)

Since (10.5.22) and (10.5.23) hold for every positiw> integer AI, making /\J --t 00, we obtain (10.5.6). To prow (JO.5.7), let
LA =

i:

r.p (x) r.p (x - k) llx

(JO.5.24)

Since

SUP1)

r.p

C [U, 3], we find that

L k =Ofor

Ikl >3.

(10.5.25)

It is dear that

(10.5.26) By a change of variable. we see that. for any l. m. n E Z.

1 J-oo ..p (2x - m) r.p (2x - 2e - 71) llx = 2L21+n-m.

r=

(10.5.27)

...... '.'_'_""""'''''''10''-''

Substituting value of <p (x) given by (10.5.23) into (10.5.24) for = 0,1,2 and applying (10.5.25) and (10.5.26), we obtain the followinp; equations

(a(l-a)+b(l-b))L o

n,

(l-ab)L, +(a(l-a)+b(l-b))L, (a (I - a) + b (I - b)) Lo + (1 - b) L, +(I_b)2 +(J _a)2 +b2 2

)L

2L2 = abL 1

+ (a (1 - aJ + b (J
- b) ~ O.

- b)) L2.

From given values of a and b (see Theorem 10.5.2), we have

a (1 - a)

+ b (J

(10.5.28)

Thus, the above system of equations becomes

0= (J -ab)L 1
2Ll ::: (1 - ab) L]

+ ((1

- b)2

+ (1

- a)2

+ b2) L 2
i- O. By (10.5.6),

2D2 = abL 1.
This implies that L 1 = L2 = O. Thus. Lie = 0 fol' k (10.5.4) and (10.5.7), we can compute
J

~
~

1= <p(x)dx~ 1=
-00
-(Xl

<p(x)' L<P(x-k)dx
kEZ

LL, ~ L . o
kEZ

This proves (10.5.7).

Proof of Theorem 10.5.4. Supp 'IjJ(x) C [0,3] follows immediately from (10.5.8) and the fact that supp <p C [0,3]. To obtain (10.5.9), substitute 1,b(x) given by (10.5.8) into the left-hand side of (10.5.9). We find (10.5.9) using (JO.5.27), (10.5.7), (10.5.28) and values of a and b. To obtain (10.5.10), we proceed similm:ly, but we substitute both 'IjJ(x) given by (10.5.8) and .;(x) given by (10.5.3) into the left-hand side of (10.5.10). It follows directly from (10.5.9) and (10.5.10) that {2 ij2 'IjJ (2i t - k) }jEZ,kZ is orthonormal.

10.5.2

ApproximatIOn by Family of Daubecllies Wavelets

Let P1l denote the orthogonal projection of L2 (R) onto Vn and let Q" denote the orthogonal projection of L 2 (R) onto W1l (Recall L2 (R) = VnEl ElTVj = W k , for any n E Z).

EB
l

J~'l

.n",

"'_'_""""'''''''10''-''

.For every integer N > 1, Daubechies constructed a pair of functions l/' and 1/) that aJ'e, respectively, X[O,l} and the Haar function for N == 1 and that generalize these functions for N > 1. The construction is on the following lines. Step 1: Construct a finite sequence ho,h t ,'" ,h2l'.. ~1 satisfying the conditions

Lh ,

k hL-.-2m

== Olll for every integer m,

( 10.5.29) (10.5.30) (10.5.31)

L'" ,.
,
whenever

V2,

Lgkk>rl = 0,

O<m<N-l, wheregl;==(-l)

hl-le.

It can be observed that (10.5.29) and (10.5.30) imply (10.5.31) for m = O.

Step 2: Construct the trigonometric polynomial

mo (y) =

1no

(y) by
(10.5.32)

J2 L

hke

iky

Step 3: Construct the scaling function l/' so that its Fourier transform ij5 satisfies

i3 (y) ~

(I/,rz;;:) II mo (T'Y) .
A'?,I

(10.5.33)

Step 4: Construct the wavelet function 1/; by

>P(x)~ LgW(2x-k).
k

(10.5.34)

B.

.... ,

For N > 1, l/'(x) and tjJ(x) define sets {l/'n,d, {t/l... ,d and subspaces of
, to<l',_"""",,,,,"l1>"-I

tvlRA V;, and IVn of L 2 (R) having the following additional properties:
tpol,k

= LhJ - 2k tpn+l" and 1/J",k = L9J-2ktpn+l,J }

j

,(10.5.35)

for any integer n.

,upp 'Pn.' ~ [2- nk" Tn (k + 2N - 1)] , } Stipp 'l}in,k = [2- 01 (k + I - N) ,2-01 (k + N) ]

/

(10.5.36)

tpj,~' (x) xmdx = 0 for all mt.egers j

<m < N
I,

and k }

(10.5.37)
(10.5.38)

and any intep;er 0

tpj,k and 1/Jj.~' E CJ.(n) = Lip.\ (n)

with exponent .\ (n), when' .\ (2) = 2 - log2 (I + v'3) '"" .5500. .\(3) ....., 1.087833,.\(4) '::;' 1.617926, and .\(N) '"" .3485N for large N. Graphs of the Daubechies scaling functions and wavelet functions for 2 < N < 4 are given in Figure 10.5.1. ForN=2,

Theorem 10,:;.5 a J( > 0 such that

If / E

C~(R), tp

and 1/J E if'" (R), then the,'C

exi~ts

11/ -

POI (f)lI u ... (Rl < KT,,(N-m j ,

VOl

(10.5.39)

who'C P'l(f) Ilenotes the o,thogonal projectioil. of L'1(R) on the (IN](';T of Dnu/J('.chips walldel1/J.

and N is

For proof, see Problem J J. 7.J.

Coifman Wavelets [coiflets]

Coifman wavelets are similar to Daubechie:; wavelets in that they have maximal number of vanishing moments; however in Coifman wavelets, til(' vanishing moments are equally distributed between the scaling function

B.

......

, toOl'._..

....,..",,"l1>"-I

and the wavelet function. These are very useful for numerical solutions of partial differential equations as they have very Rood order of approximation (see [21] for a comprehensive account).

Definition 10.5.1 (Coifmon Wovekts 01' Coiftets). An urthonormal wavelet system with compact support is called a Coifman wavelet system oj degn; N if the moments of associated scaling function tp and wavelet 1/J satisfy the conditions
Moml (J) =

::Iltp(x)dx = 1 if l = O.

(10.5.10)

Momc(tp}= !xltp(X)dX=O for l=1,2,3,,N Momiel/!)= !Xit;'J(x) dx =0 if

(10.5.41) (10.5.42)

f=O,I,2,3.,N.

It may be observed that (10.5.40), (10.5.41) and (10.5.42) are equivalent to the following conditions:
L(2k/h2k =L(2k
~.

+ 1)'hn +1 =O,

(or

=1,2,"

,N

( 10.5.43) (10.5.44)

LhH = Lh ,

~.

2 k+l

= 1,

where Ilk is the scaling filter of tp(x). Lell1ll1a 10.5.5 dcgt'ee N, then

Let tp (x) be a continuous Coifman scaling function of

L(X - k)' ",(x - k) = 0',10'-/= 0,1,2"

, N,

(10.5.45)

when: we choose 00 = 1.

Proof. We prove it by the principle of finite induction. Let (10.5.45) hold for l < n. where 0 ::; n ::; N - 1. and define

1 (x)

= L(x- k)"+' ",(x - k).

k

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

-......

'-.J

'

.:/:. ! . .. :. I :, . , ;1 ,

.'/ .\

,/ \ :!
\

ii

! \

.,

.. .

.. . .. i . . .

, \
Figul"e 10.5.1

,!

,, ,

.... , Q."to<l',_"""",,,,,"l1>"-I

Scaling functions (-) and wavelet functiolls (... ) (Top: N = 2, middle N = 3, bottom N = 4)

Then,

1 (x)

is well defined and continuous; moreo\'er,

2"+' I (xl ~ 2"+' L((x - k)n+' ,,(x - k)

k

= 2 M1 L
k

L(hm (x - kt+
m

lp
1

(2x - 2k - m))

= 2"+1 L =

L(h.-2dx - kt+ lp(2x - i))

i

L
,
,

L(lh-2k (2x - i
k

+i

- 2k)"+1

lp

(2x - i))

L L L (h,_" (n; J )
I;

(2x - i)'(i - 2k)n+ H

xlp(2x -

i))

~ LL ((
I

n; 1) (L(i I;

2k)"+'-P h, -2k)

x (2x - i)P lp(2x - i)).

Applying (10.5.43) and (10.5.44),

L (i k

2kt+ I - P "._21; =

O,,+I-p.

Thus,

2n +. I (x) ~

(2x - i)"+' ,,(2x - i) ~ 1(2x).

Since I(x) is continuous and

1
1

(x) dx = 0, it follows that

I(x) ~ L(X- k)"+' ,,(x-k) ~o.

k

It is clear that the result is true for n =

e:: : 1

Theorem 10.5.6 (Tian and Wells .Jr., 1997). Fm' an mt!lOgorwl Coilman wavelet system 01 degree N with scaling function lp (x), let {hk} be finite. Fm' 1 (x) E eN (R) having compact support, define
S(j)

(jl (x) =

Tjj2

L! (~) lpj,1; (x)

kEZ

'r/ j E:t,

(10.5.46)

...". "'_'_""""'''''''10''-''

then
(10.5.47)

where C depends only on / (x) and rp (x) . Proof_

By the Taylor expansion of / at the point x, we have

k 1 ( -. ) ~ 2)

N-' (

L q=O

I k I k -I"'(x) ( ~ - x ) ') + _/(N) (a,) ( ~ - x ) q! 2) N! 2)

for some
1<
Q ::;

Ok

on the line segmem Joining x and

~.

By Lemma 10.5.5., for

N, we get

and

L rp"k (x) = 2 L rp (21x 1/ 2

kEZ kEZ

k)

2J / 2.

Assume supp(J) C [-J<,/<], and supp(r,o) C [-J<,J<] for some positive number J<, then

L
Ikl~(2'+l)K

(~

-x)q /(q) (X)rpJ.k (x) = 0, for 1 <Q< N,

L
Ikl~(2'-H)1\

1 (x) 'I';,,(x) ~ 2'1' 1 (x),

and

5' (I) (x)

2-'1'

L
kEZ

1 (;')

'1'),' (x)

~ 2- U '

L
Ikl~(2'+I)K

1 (;') 'I';,,(x).

Q.

.....

, to<l',_"""",,,,,"l1>"-I

Putting tlw Taylor expansion of ! at x, we get

Thus,

SJ. (f)

(x)- ! (x) =

2- J /2

L
Ikl:O:;(2)+l)K

( N!!{N) (Ok) (2i -x , k)N

t.;)J,k

(x) ) ,

and

III Ix)
=

sj (f) (xl ilL,

2- J /2

L
Ikl:O:;(2'+I)K

(~r!!(N) (0,,) (~ _ x) N 'Pi,")

L2

2- J (I/2+N)

N!

L:
Ikl:O:;(2:i+I)K

(/,NI(ak)(k_y)N<p(y_k))
L2

where we make thf> suhstitut.ion y = 2J x. Lp.t

Yk (y) ~ IINI (Uk) (k - y)N <p (y - k),

then 9k (y) hus compact support [-K + k, [( -l- k], and Ilg" (Y)II L2 is unifonnly bounded, 119" (y)U/' 2 ~ C, where C depends Oil! (x) and t.;) (x). This

I
Definition 1U.5.2

5 J (J) (x) defined by equation (10.5.46) is caUed the wavelet sampling approximation of the function f (x) at the level j.

Corollary 10.5.2

Let

Pl (t) =

L ({ f (xl ty)J.k (xj dX) tpJ.k (xl

kEZ

in

Under' the hypoUleses of Them'em 10.5.6,

wher"e A is a positive constant which depends ordy on factor It".

and the sel/ling

Prvof.

Bv Theorem 3.3.2, we have

II! -

5i

IIJII;, ~ II! -

Pi

(f)II;, + IlPi (f) - s' (f)II~, I

110.5.49)

(10.5.48) follows from (10.5.47) and (10.5.49).

RemUf'k 10.5.1

It may be observed that orthogonality of Coiflets has not been used in t.he proof of Theorem 10.5.6. This means that (10.5.48)

...". "'_'_""""'''''''10''-''

holds also for non-orthogonal wavelets. More precisely, we ha\'e the following theorem: Theorem 10.5.7([24]). the following conditions:

Suppose {hk} is a finite liller>. Let {hk) sabsfy

L
kEZ

(2k)" It'lk =

L (2k
kEZ

-l-

1)9 h2k+1 = 0, (10.5.50) (10.5.51)

forq= 1,2,3"" ,N

L
~'EZ

1I2 k =

L
kEZ

h 2 k+1 = 1.

Define

Po (y) =
and ,,(y) =

~L
EEZ

h"e'k Y ,

IT F, (2-'y)
j=l

If t.p (x) E 2 (R), then for any function suppo,"t

f (x)

eN (R)

with compact
(10.5.52)

whcn:,\ depends on

and the sequence {hd, and 51 (f) (x) is the wavelet

sampling appr-oxUlwtion.

We can measure the approximation error in HI (R) norm, H 2 (R) norm, or even higher norms, depending on how smooth the scaling function t.p (x) is. In fact, we have the following theorem [24]: Theorem 10.5.8 Fm" t.p(x) E en (R), 0::; n::; N, under'the conditwns of Tltem-etn 10,5.7, the inequality
IIfIX) sj

(/lIx)II,," ~ i.2- j (N-n)

( 10.5.53)

holds, wher-e).. depends on

and fit,,}.

Remark 10.5.2 Existence of Coifman wavelets of degree N, say N = 9 can be proved by using a fundamental result of Kantorovich (see, for

'D,_.. . ~.._~ __,

......

example, [24, p. 216-218]). It is dear from definition that thl' degree 0 orthogonal Coifman wa\'elet system is exactly the same as the Haar wavelet system {Ito = 1, h] = I} . The Coifman wavelet system of degree N = 1, 2,3, etc. can be computed. Interested readers may find details in [241.

10.6

Wavelet Packets

Wavelet packets are generalizations of wavelets. They are organi7.ed into collectiuns, and each collection is an orthonormal basis for L 2 ( n). This makes possible comparison of the advantages and disadvantages of the various possible decompositions of a signal in these orthonormal bases and selection of the optimal collection of wavelet packets for representing the given signal. The Walsh system is the simplest example of wavelet packets.

Definition 10.6.1 (Wavelet Packets). sequences of 2 such that

Let {Itt} and {gt} be two

(10.6.1)

Lh
"EZ

n _ 2 k h ,,_2

= Ok_l

L
nEZ

1," ~

j2

(10.6.2) (10.6.3)

Furthermore, let rp(x) be a continuous and compactly supported realvalued function R that solves the equation
I"(O)I"(x) ~ 2'1'

L ,

h'l"(2x - k)

( 10.6.4)

with rp(O) = 1. Let t/J(x) be an associated function defined by

1}!(x) = 2 1/ 2 Lgkrp(2x - k).

(10.G.5)

A family or functions w" E Lz(R), n = 0, I, 2, ... defined recursively from rp and 'l); as follows, is called the wavelet packet

m
(ii)

wo(k) ~ I"(x),w, (x) ~ ,pix)

W2"(X) = 21 / 2 LhkWn(2X - kl

(10.6.6)

(iii)

1A.':!"+J(x) = 2 1/ 2 L9kW,,(2x - k)

Q.

.... ,

, to<l'._..

....,..",,"l1>"-I

in the case of wavelets, Ifl(X) and 'J,b(x) are often called, respectively, father and mother wavelets.
Ai;

It has been proved that {wn(x - k)}. k E Z. is an orthonormal basis of Lz(fl) for all n > 0 where

wn(x-k)=

~Lhk-z;wzn(;-i)+~L9k-z;W:!n+l(~-i)
2 2

(10.6.7)

I(x) =

= = L L
fl'"
-CX)

C.I.kW,,(X -

k),

110.6.8 )

k=-CX)

wher/"
(10.6.9)

is called the wavelet packet series and en k are called wavelet packet coeffidents of /. The Walsh s)'stem {Wn}~o is defined recursively on [O,IJ by Wo(x) = Xro,o(x) and Wz,,+<>(x) = W,,(2x) + (-lt~Vn(2x - I), 0' = 0, I; n =
0, 1, .. , .

This is an emerging area; for a comprehensive account, we refer to \Vickerhauser [301 and Meyer [W].

10.7 Problems
Problem 10.7.1 is a wavelet. Problem 10.7.2
Let 'J,b, and'lh be two wavelets, then show that 'J,b, +'Ih

Prove that Haar function is a wavelet.

Problem 10.7.3 Prove that the characteristic function X/o, 1/ of the dosed interval [0, 1] is the scaling function of the Haar function. Problem 10.7.4 Compute the wavelet transform of / (x) = sin x for the Baar and the Mexican hat wavelet.

..,.., "'_'_""""'''''''10''-''

Problem 10.7.5 Haar wa\'elet.

Obtain data of any kind and analyze them with the

Probkm 10.7.6 Describe the procedure for constructing mother wa\'elets from a father wavelet. Probkm 10.7.7
Show that B-spline is a wavelet.

Problem 10.7.8 Explain the concept of the multiresolution with the help of the signal given in Example 10.3.1. Problem 10.7.9 Let lhf(x) = ftx- h), Daf(x) = a l / z f(ax), EcI(x) = eZ",icx f(x), a,b,c E R, a> O. Then show that for f,9 E z
(a) DnTbf(x) = a l / z f(ax -

bl

(b) Don/(x) ~ 1~-"Do/(x)

(e) (J,Dog)

(Do-,I,g)

(d) (J,n,,) ~ (T_bl,g)

(e) (J, DoT,g)

~ ~

(T_bDo-,I,g) (J,g)

(f) (Dol,Dog)
(g) (T,I, Tbg)
~

(J,g)

(h) Tt,Ecf(x) = cZ".;bcEcTl,!(x)

Problem 10.7.10 Ch'en iO,I4J,Jl,k j E: Z, with either io -=F il or kO-=F k j , then show either
(a) i)"k! n ljo.ko = ,

(b) lJ"k, ~ flo,ko

""" "'_'_""""'''''''10''--'

Problem 10.7.1 J

Show that the Haar system on R is an orthonormal

system. Show that for each Integer N 2: 0, the scale N Haar system on [0,1] is a complete orthonormal system on [0,11.
Problem 10.7.12

Given I(x) continuous on [0,1], and f > 0, there is NEZ, and a scale N dyadic function y(x) supported in [0,1] such that I/(x) - g(:I:) 1 < to, for all x E (0,1]; that is, supl/(x) - g(x)1 < f..
Problem 10.7.13

Problem 10.7.14

Let the scaling function lp(x) be gh-cn by lp(x)

sin JTX ==. Then find the corresponding \vavelet 1/1.

'x

Prove t.hat if {i,,} is a tight frame in a Hilbert spae H with frame bound A, then
Problem 10.7.15

A(f. y) ~

L: (f, InW.,y)
..=1

00

for all f,y E J-I.

Problem 10.7.16

Let lp(xJ be defined by

Draw the graph of the wavelet obtained by taking the convolution of the Haar wavelet. with op(x). Prove that {w n (-, -k)}kEZ, :$ n < 2J , where {w n (.,.) denotes a family of wavelet packets, is all orthonormal basis of L 2 (R).
Problem 10.7.17

...... "'_'_""""'''''''10''--'

References
[1] JS Byrnes, JL Byrnes, KA Hargreaves, Kd Berry. (cds.), Wavelet and their Applications, NATO ASJ Series, Dordrecht. Boston. London: Khm'er Academic Publishers, 1994 (21 A Chambolle, RA DeVore, NY Lee, B Lucier. Nonlinear wavelet Image processing: Variational problems, compression, and noise removal Lhrough \vavelet shrinkage. rEEE Trans Image Processing, 7:319-335. 1998.

[31 CK Chui. (cd.) Wavelets: A 'I\ltonal in Theory and Applications. Boston, San Diego, Nev,- York. Academic Press, 1992.
(4] A Cohen WaveJet Methods in Numerical Analysis. hi PC Ciarlet,.IL Lions (eds.) Handbook of Numerical Analysis, Vol. vn, Amstf'rdam: Elsevier Science, 2000, pp. 417-710.

(5j RR Coifman, MV Wickerhauser. Entropy based algorithms for best basis selection. IEEE Trans Inform Theory 38;713-718,1992.
[6] W Dahmen. WaveleLS and multiscale methods for operat.or ffluations. Acta Numerica 6;55-228,1997. (71 I Daubechie:,. Orthonormal bases of compactly supported ,vavelets, Commun Pure Appl Math 4;909-996. 19S8. (81 I Daubechies. Ten Lectures on \Vavelets. Philadelphia: SIA~'l, 1992. [9] R GIO\vinski, W Lawton, M ful,vachol, E Tenebaum. Wavelet Solutions of Linear and Nonlinear Elliptic, Parabolic and Hyperbolic Problems in One Space Dimension. In: R Glowinski, A Lichnewski, eds. Pro ceedings of the 9th International Conference on Computer Methods

...... "'_'_""""'''''''10''-''

in Applied Sciences and Engineering, Philadelphia: SIAM, 1990, pp 55-120. (10] W HardIe, G Kerkyacharian, D Picard A Tsybakov. Wavelets, Appruxinmtions, and Statistical ApplicatiulIS, New York, Berlin: Springer, 1998. [11] E Hernandez, G Weiss. A First Course on 1996.
Wa~-elets,

CRC Press, Inc.,

[12] SKelly, rviA Kon, LA and fulphael. Poinhvise convergence of \vavelet expansions. J Funct Anal 126:102-138,1994. (13J M Kobyashi. (ed.) \Vavelets and their Applications. Case Studies. Philadelphia: SIAM, 1998. [14] J Kovacevic, I Daubechies. (eds.) Special Issue on wawlets, Proc IEEE, 84:507-614, 1996. [15] AK Loui", P Maass, A Reider. Wavelets Theory and Applications. Chichester. New York: John Wile)' & Sons. 1997. [16] S Mallat. A \Vavelet Tour of Signal Processing. 2nd ed. London: Academic Press, 1999. (17] P Manchanda, A Mukheimer, AH Siddiqi. Pointwise convergence of wawlet m:pansion m;sociatoo with dilation matrix,. Applicable Anal ysis 76(3-4):pp.301-308, 2000. [18] Y Meyer. Wavelets and Operators, Cambridge; Cambridge University Press, 1992. (19] Y Me)ler. \Vavelets Algorithms & Applications. Philadelphia: SIAM, 1993. [20] Y Meyer. Wavelets. Vibrations and Scalings, CRM, Monograph Series Vol. 9. Providence, Rhude Island: American Mathematical Society, 1998. [21] H Neunzert, AH Siddiqi. Topics in Industrial Mathematics-Case Studies and Mathematical Methods. Dordrecht, Boston, London:

0 . ,<1"_""""'''''''10''-1

......

Klu,...w Academic Publishers, 2000.

[22J RT Ogden. Essential \Vavelets for Statistical Applications and Data Analysis. Boston. BMel, Bprlin: Berkhauser. 1997. (23J DB Percival, AT Walden. Wawlet Methods for Time Series Analysis. Cambridge Series in Statistical and Probabilistic Series. CRmbrid~e; Cambridge University Pres, 2000. (24J HL Resnikoff, RO \Valls, Jr. \Va"elet Analysis, the Scalable Structure of Information. New York, Berlin, Heidelberg: Springer, 1998.
[251 G Strang, T Nguyen. \Vavelets and Filter Banks. \Vellesle:y: Wellesle:yCl1mbridge Press. 1996.

126J M Vetterli, J Kovacevic. Wavelets and Subbl1nd Coding. Englewood Cliffs: Prentice Hall, 1995
[271 JS Walker. Fourier analyis and wavelet analysis. Notices of the Americal Math Soc 44:658-670, 1997. [281 JS Walker. A Primer on Wavelets and their Scientific Applications. Boca fulton, London, New York: Chapman & Hail/eRe, 1999.

[29] DF Walnut. All Introduction to \Vavelet Analysis, Boston, Basel, Berlin: Birkhauser, 2002 130J MV Wickerhauser. Adapted Wavelet Analysis from Theory to Software. Boston: MA Peters, 1994.

[31) P Wojtaszczyk. A Mathematical Introduction to Wavelets London

Mathematical Society, Student Text Vol. 37. Cambridge: Cambridge Uniwrsity Press, 1997.

.n",

"'_'_""""'''''''10''-''

Chapter 11

Wavelet Method for Partial Differential Equations and Image Processing

11.1

Introduction

In the recent past, there has been an amazing growth of applications of wavelet methods [1-7, 12, 13, 15, 19, 20, 23, 24, 27, 29J in such diverse fields as image and signal processing, meterology, computer vision, seismology, turbulence, biomedical engineering, stock market predictions, time series analysis, pattern recognition, water marking technology aud numerical solutions of partial differential equations. In this chapter, we present an introduction of applications of wavelets to the solution of partial differential equations (PDEs) and signal processing. For numerical treatment ofPDEs by Galerkin methuds, the realistic cases usually require trial spaces of very large dimension \vhich amounts to the task of sol ving systems of equations of correspondingly large size. Wavelets offer a remedy for different kinds of obstructions encountered in applying Galerkin methods. It has been shO\vn that the stiffness matrix relative to wavelet bases is sufficiently close to sparse matrix. Therefore, efficient sparse solvers can be used \vithout loss of accuracy. These are consequences of the following results: (i) Weighted sequence norms of wavelet expansion coefficients are equivalent in a certain range, depending on the regularity of the wavelets to Soholf'v nOnllS. (ii) For a wide class of operawrs, their represent.ation in the wavelet. basis is nearly diagonal.

B.

.....

, to<l'._..

....,..",,"l1>"-I

(iii) The vanishing moments of wawlets remove the smooth part of a function. As discussed in Section lO.3.3, ft signal is a function of one variable belonging to L 2 (R) and signal processing has a close connection with wavelet analysis. An image is t.reated as a funr:t.ion f defined on the unit square Q = (0,1] x [0,1]. We shall see here that image processmg is closely linked with wavelet analysis. The concept of wft\'elets in dimension 2 is relevant for this discussion. Let '.p be a scaling function and 1}J(x) be the corresponding mother wavelet, then the three functions

1}J1 (x,y) = 1}J(:v)1}J(y) ,p,(x,y) ~ ,,;(x),,(y) ,,;,(x,y) ~ ,,(x),,;(y)

form, by t.ranslation and dilation, an orthonormal basis for L 2 (R2); that is,

m = 1,2,3 is an orthonormal basis for L 2(R2). Therefore, each f E L 2 (R 2 ) can be represented as

L
j,kEZ

dj,k-j,k,

(11.1.1)

where 1/) is anyone of the three 1/1", (x, y) and

11.2
11.2.1

Wavelet Methods in Partial Differential mid Integral Equations

Introduction

Jt has been establishL-'d beyond doubt that \vftvelet methods provide efficient algorithms to solve partial differential equations in the sense that (a) Qualit.y of approximation, that is, the set of approximation A to which belongs the computed solution must be close to the exact solution u, Mo)'(' precisely, in mathematical terms
inf d(u,v) L
"EA

(tI.2.1)

on",

"'_'_""""'''''''10''-''

< means very small) and A needs to be small enough to allow the computation of the numerical solution. (b) The algorithm needs to be fast, that is, less time-consuming. It may be observed that in (lrder to reduce time of computation, the algorithm needs to select the minimal set of approximations at each step so that computed solution remains close to the exact solution This point is called adaptivity, which means no unnecessary quantity is computed. Some properties of wavelets are important for building adaptive algorithms. for example, if the solution of the partial differential equation we wish to compute is smooth ill some regions, only a few wavelet coefficients will be needed to get a good approximation of the solution in those regions. Practically, only the wavelet coefficients of low frequencies wavelets whose supports are in these regions will be needed. On the other hand, the greatest coefficients (in absolute value) will be localized near the singularitics; this allows us to define and implement easily criteria of adaptivity through time evaluation.

11.2.2 General Procedure

General Procedure I: Let U be a tinite- dimensional subspace of a Sobolev space H in which we look for a weak solution of a partial differential equation (PDE). Then as \ve have seen in Chapter 7. the differential equation reduces to a matrix elluation in U. A solution of the matrix equation is an approximate solution to the partial differential equation and we shall try to find that solution of the matrix equation and in fact, that matrix for which the distance or metric bet\veen the solutions of the partial differential equation and the associated matrix equation is minimum, More specifically, let f! be a bounded open set in R 2 with Lipschitz boundary, that is, the boundary af! is a Lipschitz function. Let
U; ~

{J E X;/supp Inn" J.

Here, X j =:. Vj EEl \!j \vhich is spanned by the products CPj,k (X)cpj,k (y). for k, RE Z where as usual CPj,k (x) == 2jf2cp (2i x - k), CPJ,k (y) == 2J / 2cp (2i y - k) , where cp(x) is a scaling function. Since D is bounded U, is a finite-dimensional subspace of Vj EEl \ii; Uj is the Galerkin approximation space in this context. [f a Sobolev space HI (f!) is considered, then Vj c HI (D) and hence, \ve can use elements

.n",

"'_'_""""'''''''10''-''

of U) to represent approximations to a solution of a. differential equation which are in HI (0) It may be noted that if we use the wavelet 1/) associated with scaling function lp. we obtain a system
AJUJ = F J ,

,vhere UJ is the coordm,ue vector of UJ in the basis ("Pi,k(X)}, Pi = (I, W>'I>'1<J and AJ = ({AW" , W>.) )I>.I.I/I<J is the associated stiffness matrix ,vith the operator A (see Section 8.2). One can choose H = HJ (D), A = -~ or A = f - ~ or .-I = ~(~) = ~2.

Example 11.2.1 (Wavelet-based Galef'kin Method). We shall consider here a. simple example to illustrate the applicatiol1 of wavelets in solving differential equations. It is clear from the MRA that any function in L2 (R) can be approximated arbitrarily well (in the L2 (R) norm) by the piece-wise constant functions from \0 provided j is large enough. \0 is the space of piece-wise constant L2 (H) functions with breaking at the dyadic integer:,; k . 2-), j, k E Z. We consider the following Dirichlet boundary value problem:

-""(x)

+ Cu(x)
u(O)

~ f(x),
~

x E 11 ~ (0, I)}
~

"II)

O.

(11.2.2)

with C > 0, a constant, f E HI (n) and solve it for u = u(x). We apply the variational method of approximation (Galerkin method) for solving equation (11.2.2). As the existence of the solution of \'ariational problem is guaranteed only in a complete space, so ,ve take the Sobotev space HI (n) The idea is to solve variational equation on a finitedimensional subspace of HI (n). In variational form, the solution u E HI (0) of the above equation satisfies

In

(u'v'

+ uv) =

In

Iv.

/11.2.3)

for all?! E HI(O). To approximate u by the Galerkm's method, we choose a finite-dimensional subspace of HI (0) which is a space spanned by wavelets defined on the interval [0,11. We have already discussed the wavelet bases tPj,k, Wj,k and spaces lj, W J generated by them respectively in Chapter 10 For getting numerical solution of (11.2.2). we choose a positive value m and approximate u by an element U m E V", that satisfies

(u;"'v'

+ umv)dx =

In f(x)vdx,

v E V""

(11.2.4)

B.

......

, to<l'._..

...,..",,"l1>"-I

where

Urn

can be \vritten as
m-I

POll m

QkUm'

(11.2.5)

k=O

Pk is the projection from /-fl(n) onto Vk and Qk-I

fore,
m-12'-\

Co o,o

L L

dj,k 1/Jj.k

(11.2.6)

,=0 1,,=0

Here, <Po,o is identically 'one' on n, Co is the average of Urn on n. We have used a "multi resolution" approach, Le., we han~ written a successive coarser and coarser approximation to Urn' Therefore, for all practical purposes, u'" can be approximated to an arbitn1.ry precision by a linear combination of \vavelets. \Ve need now to determine d"k. Equation (11.2.6) together with (11.2.4) gives the folio\\"ing:
"'-12'-]

L L

dj,k

j==O k=O

(j,kj' ,k'

+ j,k,.,k')dx

(11.2.7)

(l

for k'

0"

.,2/ - 1 and j' = 0, ,rn - 1 or, more precisely,

AV = P,

where A is the stiffness matrix relative to the wavelet basis functions 'lj;j,k and V, F are corresponding vectors U = (dj,k), Ik = (f, j,k).
Example 11.2.2

Consider the follo""ing Neumann problem:

(11.2.8) (11.2.9)

B.

......

Proceeding on the lines of Section 7.4, it can be seen that (11.2.8) and
, t><l',_"""",,,,,"l1>"-I

(11.2.9) call be written in the following form. Find u E U such that fJ.(U,V) :::: (v) a(u,v) =

for all v E U, ,,,,here U = HI (0,1)

(11.2.10)

(v) =

, 1 II
-I-

Jt o

a(x) dlUdlV
(XtX

io

r {3(x)uvdx
ti,

~I (x)

d" -vdx for all dx

v E V,
(11.2.11)

fvdx+dv(l) Tl:V(O) for all v E U.

By the Lax-Milgram Lemma (Theorem 3.9.4), EquatIOn (11.2.10) has a unique solution if

0< ao

< a (x) < aM

::;

a.e. on (0,1)

(11.2.12) (11.2.13) (11.2.14)

13M a.e. on (0,1) l' E &;, (0, 1), lI"IIIL 2(o.1) < min (ao,/1o).

0<130 ::; (3 (x)

Let r.p (x) be a scaling function Rnd let us consider the Daubecbies wavelet for N = 3 (see Section 10.5.2). Let n be any integer and V.., be as in Section 10.5.2. Let V..,(O, I) denote the set of all elements of Vn restricted to (0, 1). Thooff'tn 10.5.5, every function in HI (0,1) can be approximat.ed very closely by an element in Vn (0, 1) for sufficiently IRJ'ge n; hence

(U o Vo (0, I)) - H' (0, 1) .

(11.2.15)

Therefore, the fmmly of subspaces Vn (0, 1) of HI (U, 1) is very appropriate for Galerkin solution of the Neumann problems (11.2.8) and (11.2.9) formulated as given in (11.2.10) - (11.2.11). The Galerkin formulation of this problem: For any integer n,find Un E Vn (0, 1) :>uch that
a (u."v) = L(v) for all v E Ff] (0.1).

(11.2.16)

Since (0,1) is bounded, ~.;., (0,1) is finite-dimensional and is a dosed subspace of HI (0, 1). This implies that 01.2.16) hAs a unique solution In view of Corollary 8.2. t and (11.2.15),

lim lIu,,-ull=OinJ-{I(O.1).
o~oo

(11.2.17)

For any integer n, the solution u" of approximate problem (11.:.!.16J can be represented as
ti"

, = L

ti",klpn.k-2N+l,

(11.2.18)

ko=c I

...... "'_'_""""'''''''10''-''

where p = 2 n + 2N - 2 and 111, U2,'" ,up E R, where functions are considered to be restricted to (0, I). This yields the following system of linear equations in p unknowns:

La (lPn,k-2N +
~'=l

I,

lP",J-2N+I) l1 n,k = L (rpn,j-2N+ I )

(11.2.19)

for every j = 1,2"" , p. This can be written in matrix form as

AU-P, -

lJ 1.2.20)

where A =aiJ,
ai,J =a(lPn,j-2N+l,,,cn,i-2N+I),

(11.2.21) (11.2.22)

F = (Ii),

f.

= L (lPn,i-2N+I),

and
U = (l1 n ,i)'

(11.2.23)

In the case where the bilinear form a(,) defined by (11.2.10) is elliptic on HI (0,1), the abo\'e matrix A is positive, implying the existence of a unique solution of (11.2.20). lfthe bilinear form a, (".) is symmetric. then (11.2.20) can be ::.olved by the conjugate gradient method.

Remark 11,2,1 The evaluation of ai,j and f. gh'en by (11.2.21) and (11.2.22), whidl are requirecl to compute the solution of the matrix equation (11.2.20), can be performed using well-known IlUmeflcai quadrature methods.

11.2.3

IHisr:ellallp.ous Examples

Example 11,2.3 Parabolic Problems We discuss in this exallljJle jJroblems of the type

au at + Al1+ 8(u) =
u(,O)~,,"(),

(11.2.24) (11.2.25)

where A is an elliptic operator and 8 is a possibly nonlinear function of u 01' a first deri\,ative of u,

."", "'_'_""""'''''''10''-''

Special cases arc the heat equation, the viscous Burgers equation and the Korteweg-de- Vries equatioll.

Heat Equation:

flu

&t =

a2 u +/ 8x2

for

t>Oall(lxE(O,I)

( 11.2.26) (11.2.27) (11.2.28)

u (0, t)

iJu = " (i, t) , ax (0, t) =

a" (1, t) ax

u(x.O) = uo(xJ.

Viscous Burgers Equation: For

/.I

> 0,

au (x, t) = a'u + u (x, t) 8x (x, t) , } a" 8t ax

/.I

for t

>

<
and

(11.2.29) (11.2.30) (11.2.31 )

< I,

u(x,O) =vo(x)

Reaction Diffusion Equation:

p>O,IJ>O.
with appropriate boundary conditIons.

Korteweg-de Vries Equation

f}l.l

au EfJ u 8t +ou 8x +13 8X3 =0

(11.2.32)

with appropriate boundary conditions. We consider here the solution of (11.2.26) - (11.2.28). Let /(t) and u(t) denote, respectively, thl:' functions . --t / (x, l) and x --+ u (x, t). It is assumed that, for almost e\'ery l > O. / (ll E \I. (dual of \I = HI (R)) and
u(t)ElI.

A variational formulation is obtained by multiplying Equation (11.2.26) by v E J-lJ (0,1) and integrating by parts with respect to x. This yield:,

8x'v at 8x (iJu) + 1r' auav dx = (I,V) , for e\'er)' v E Ho, (0,1).

0

(11.2.33)

...... "'_'_""""'''''''10''-''

Here, (.,.) denotes the duality pairing between F* and V which reduces to the inner product h,) of ~ when both arguments are in L z . It may be obsen'ed that outside [0, 11, the functions v E HJ (0, 1) are defined by periodicity with period l. Let N > 3, n ~ 1, .and Vn = lin (0, 1) be the subspace of V = HI (0,1). Approximate problem of (11.2.26)-(11.2.28): Find l1n (t) satisfying for almost every t > 0,

{'a.," ('&u,,( dv r' t 10 -----at (t) vdx + 10 fit t) dx dx = 10 In () vdx,

for all v E v~" u,,(O) =
110,..,

(11.2.34) (1l.2.35)

where 110,,, is the L z projection Pn (110) of the initial data 110 on \/n and where In (t) = P,; (J (l)) is the projection of f (t) on V,: identified with Vol in the sense of Definition 2.3.4; indeed I" (t) is the unique element of Vol satisfying

1
1

In (t) vdx = (f (t) , v) for all v E

11;,.

The problem described by (11.3.34) and (11.3.35) is equivalent to a system of first order ordinary differential equations obtained by substituting v in (11.2.34) with elements of a basis in H" (0, 1) The system is equiwl.lent to the following initial value problem in H" (0, 1)

&u" m+A"u..
Un

I" fort> 0
110,n,

(11.2.36) (11.2.37)

(0) =

where A" =

APn is the approximat.ion of A = - 8x z ' The prohlem (11.2.:16)-(11.2.:17) has thp following dospd form solution:
11,.,

P~

a'

(t) =

c(-tA.,)lI<J, ..

it

c-A,,(t-a} In(s)ds

(11.2.38)

COIl\,pntional numerical schemps are now obtainpd by expanding the evolution operator e-A,,(t-s). For example, the Taylor expansion for Un (t + llt) by the following quadratic polynomial in III giw'S us
U" (I

+ :>1) ;

Un (I) + :>1 Ifn (I) - An"" (I)] } .5:>1 [afn (I) fal - An Un (II - Anu" (I) J.

( 11.2.39)

B . ,~,_ .._",,"l1>"-'

......

Any such approximation

U

to e~6.tA~ provides

(n6.t) '" u" = "uo +

L
i=l

n-.

(11.2.40)

as a discrete counter part of (11.2.38). [u = Uk(t) ~ u(kTi, t))~~'Ol , f =

Udt)):~o', It) ~ f(kP.t)]. The simplest examples are = (I - ~I A,,) -1 (1 + ~' An) and they correspond to the Crank-Nicholson scheme. Suppose we are interested in longtime solutions of the heat equation. This requires high powers of c. In particular, I~he powers 2( can be obtained by repeated squaring. Setting
2~'-1

Sm =

2m,

Cm =

1 f, and noting that

i=O
2>n_l

i = 1+ +2 (J +E.) + 4 (! ++2 + 3) +. +:1:"1-1

i=O

(J + + ... + 2m - 1 _ I) ,

(11.2.41)

the following algol"lthm approxllnates the solution at dme t = 2m6.t after m steps.
AlgorithluI1.2.1: SetSo=,Co=f. Fort=I,2, ',m:

5; =5;_1'
C; = (l+S;-1)Ci - 1 Then U(2~) = Smu(O)+Cm is an approximate solution of (11.2.36) -( 11.2.37) at times 2 m6.t. Transform Algorithm 11.2.1 in such a way that the 5, become sparse (within some tolerance). For this we exploit the fact that wavelet representatium; uf CaideronZygmund operator T defined Leluw (and their powen are nearly sparse:

(Tf,g)

In In K(x,y)f(1I)9(x)dydx,
0>0.
J{

where T is continuous on 2 (R) and

0 IK(x,yJIS 1 I'X;<Y' x-]I

IJ{ (x,]I)

J{

(x',y)1

+ IJ{ (y,x) -

(y,x')1

< C Ix -

0 . ,<1"_""""'''''''10''-1

......

Ix-yl

x:~,p

8> 0,

Ix - x'I

<

Ix -

yl /2 and f and 9 have disjoint compact support.

Example 11.!?.4 Integral Operators and Wavelets Let T be defined by

T f(x) =

[(1-

x)yf(y)dy

=
where

'

l'

(I-1I)xf(1I)d1l

K(x,y)f(y)dy,

/(x,1I)

(I - x)1I

if 0 < y:<=:; x

= (l-y)x
The wavelet coefficients

if:r<y< I.

are zero if If!J,k and t/'J' ,k' ha\'e dlSJomt supports, provided that 'ljJj,~" or Wj' ,k' is orthogonal to polynomials of degree L For j > j', k > k', and Wj,k belonging to Lip a which is orthogonal to polynomials of degree n + 2 with n;::: a -1, we get
Imj,k,j,k'i =

1
1

TWj,dx)wj',k'(X) dx 9Ik,.,(sUPP(T\i>"k))'

< II T Wj,J;(x)IIL I 9~nt" IIwj',k' -

where p... is the space of polynomials of degree less than or equal to n, or Imj.kJ,k,j :<=:; 2-(o:+5/21IJ-i' 1 2{J+i'}. 2-

Example 11.2.5 Hilbert Transform and Wavelets The Hilbert transform of a function f(x), denoted by Ii f(x) is defined

'"
Hf(x)

= lim -I
,---to
11"

f(t) 1 f(x - t) rlt --dt = lim Ix-tl?' x - t ,---to 11" Itl2:' t

1.

provided that the Inuit exists in some sense. It is often written as

I Ii f(x) = , p.v. -

.,,,,, '"'_'_""""'''''''10''"-'

where p.\'. means principal wtlue, that is, p.v.

in

{j(x)dx = Jim
(--->0+

in \(_.,.)

j(x)dx.

Let H{j,k},{l,m} = (HWi,m,1h.~')' see Definition 10.2.6 for 'lj.Ij,k(X). We show by using the Haar wa\'elet that HU,k), (l', m) exhibit a decay with increasing distance of scales. Let Wbe Haar wavelet and
(lPn.o,1Pj"d =

11jJj.k(X)dX =U,

k=U, ... ,'}/ -1,j

> U.

Suppose now that. 2-f (m + 1) < 2- j k, and f > j, i.e., the support of Wt,m and Wj,~' are disjoint. Then, by the abo\'e relation and Taylor's expansion around 11 = 2- l m, one obtains
,->('+" 1IVi{j,k},{l,.,,)1 = /, 2-'k

{/,'-'('+') (
2-tm X

for some Yt,." in the support [2- t m, 2- t (m -t- 1) of Wt,.". Repeating the same argument, one can subtract a constant in x which yields
JrIH{j,kj,(i,mjl =

1/,

:rt{m+tJ{j,2-J{k+I}((V_2-Jm)
2-'k

2-t",

(x

]ll.m)

2 -

(1I- 2 - tm 2 (2 Jk Yl..,,)

W",(x)dx }Wf,m(yjdlll
On account of Taylor's expansion around x = 2- j k, the factor in front of 1/Jj,A;(x) can be written as -2(y- 2- l m)(x- 2- j k)/(XJ ,k -Yt,m)'\ where Xj,k is some point in the support [2- l k, 2- j (k + 1)1 of the wavelet 1/Jj,k. Noting that

a straightforward estimate provides

Jr IH {l,k},lm,i} I _

<2 -(,,+j)3/~12-jk -

2-f

1-3 _
-

Ik

z-Ij-ll! 2i lml 3 '

(11.2.42)

B . ,toOl',_""""",,"l1>"-I

......

11.2.4

Error Estimation Using Wavelet Basis

In this section, we pl'O\'e results concerning the elliptic equatIOn with Neumann boundary condition which indicate that order of apPl'Oximation is impl'O\'ed if Coifman or Daubechies wavelets are used as basis functions. Consider the elliptic equation
-~u+u =j, in 0 C

R2

(11.2.43)

with the Neumann boundal"y condition

&n == 9, on

8"

ao,

(11.2.44)

where n is the unit out.ward normal \"ecLOr of ao. If u is a solution of (11.2.43) and (11.2.44) and if h is a test function in Ii I (0) , then multiplying (11.2.43) by h and integration by parts over 0, one has from (11.2.44) that

In

vu\7hdxdy

In

uhdxdy::::

In jh([xdy + 100 9 hds .

(IL2.45)

Soh'ing (11.2.43) and (11.2.44) is equivalent to finding u E Ii I (0) so that (11.2.45) is satisfied for all h E Ii I (0) . Let ui be a solution (11.2.45) where u j E V j C HI (fl) and (11.2.45) is satisfied for all hE Vj. Let N "2': 2 be the degree of the Coifman wavelet" Theorem 11.2.1

Let 1/ be a sollition to (11.2.49) (md (11.2.44), then

< '2~j(N~') , I u-u'II I/l(Ol-1\

where), depends on til/; di(lmeter 010., un fIl, and 071 the maxi11lu11l11l0dlilus of the jint and second df'nvatives of u_

Proof.

Wp first. show that. if 11 E Vj, thpn

Ilu - n' II II

{fI}

~ lIu -

vlll/l{fI}'

(11.2.46)

To pl'O\'e this, let v E V" W = u' - v. Then, since u and ul both satisfy (11.2.45) for h E V j and W E Vj,

In (vu - \7u') . vwdxdy + L(u - u

i ) wdxdy = O.

(11.2.47)

Therefore, by (11.2.47) and the Cauchy-Schwart.<r..--Bunyakowski inequality,

Ilu uJII~'(n)

::=

::=

kl k

'\7U- '\7ui l dxdy

(u -

i )2dxd]l

('\7u - '\7ui ). ('\7u - '\7v - '\7w) dxdy

+ k(U-u')(u-v-WldXdY
::=

k +k

('\7u - '\7u') ('\7u - '\7v) dxdy (u - u i ) (u - v) dxdll,

(11.2.48)

If u = u i , (11.2.46) is evident; otherwise (11.2.48) implies (11.2.46). Suppose now that v = Si (u) is an approximation of u given by Theorem 10.5.6, then Si (u) E V r The pruof follows frolll the foJlowing 2-dimcllsional analogue of Theorem 10.5.6. "Let f E C 2 (IT) , where fl is open and bounded in R 2 . If

S'(f)(x)~
then

L
(p,q)En

!(P,q)'Pj"(x)<p,,,(y) , (x,y) En;

and, more generally,

for some constant

>.."

Corollary 11.2.1 Letu be a solutio7l to (11.2.43) a7ld (11.2.44), and let u i be the wavelet Galef-ki7l solulio7l to the following equatio7l:

1n

{'\7u'\7hdxdy+ {1Idxd]l= {phdxdy+ { g'hds,

1n

1n

1&11

(11.2.49)

.n",

"'_'_""""'''''''10''-''

whnc Ii and gi an~, respectively, W(lvelet appHJximalion 107' onJe" j. Then

and 9 01
(11.2.50)

II

tI-U

'II

'2-,CN-'l , 1/ 1 (0)-1\

<

when!). depends on the diameter olfl, on N, and on fhe maximum modltlus 01 the first derivatives 0111.

Pmof.

From (11.2.45) ;mel (11.2.47),

J ).

WP

ohtain
i)

In (\7u - \711 \7wdxdy + In (u - u wdxdy In (I-Ii) wdxdy + 100 19 - rI) wds.

(11.2.51)

Therefore, similar to the derivation of (11.2.48), we have

Ilv - v' II:" Co> < Ilv - v' I H' 10 >IIv - vII H' (01
+ (I(I-f') (u'-v)ldxdy+ ( 1(9-,1) (u'-v)I'ls.

10

1&0

(11.2.52)

Since
111.2.53)

where an is Lipschitz continuous and), depends only on the dimension of the underlying space and fl. From (11.2.52), (11.2.53), putting estimates of f - fi<md 9 - g; from the 2-dimensional analogue of Theorem 10.5.6 mentioned above, replacing v by if}, and applying the Callchy-SehwartzBunyakowski mequality, we get (11.2.50). I It may be observed that Theorem 11.2.1 holds w'ue for many other wtwelcts including Daubechies wavelet.

11.3

Introduction to Signal and Image Processing

Broadly speaking, signals are treated as functions of one variable and images are represented by functions of two variables. The study of these concepts is directly related to the emerging area of information technology. Functional analysis in general and w3\'elet methods in particular provide basic tools to make systematic mmlysis of problems related to signals and

B.

..... ,

, to<l',_"""",,,,,"l1>"-I

images. There exists a vast literature on this theme but we present here an introductory discussion of applications of frame wavelets and orthonormal wavelets in the removal or suppression of noise from a signal and wavelet methods for image processing. Noise in a signal is that part which lacks structme and pollutes the signal. Removal or suppression of this part is an important task in order to get pure form of the signal. This process is often called denoising. This theme is presented in Sections 11.4 and 11.5. One looks at the im~e _compression problem as one of approximating I by another function I(J represents compressed image). The object of a compression algorithm (procedure or method), an important ingredient of image processing, is to represent certain classes of pictures with less information than was used to represent original pictures. For a 10Sbless algorithm, the original and compress&] images will be the same, and the errol' between them will be zero. Very often a.lgorithrn:;, where original and compressed images are different, are studied. This kind of algorithms are of vital importance for minimizing storing space in a computer disc or diskettes or minimizing transmission time from one source to the other by sacrificing a bit of the quality of the image obtained after decompression of the compressed image. [n mathematical language, it means that algorithms are related to the following basic problems.

(0 [n what metric

01'

norm the error

III - ]11 be measured?

(ii) How should onp measurp the efficiency of algorithms?

(iii) For which pictures does an algorithm gi\'e good results?

(i\') h there an optimal level of compression that cannot be exceeded within a given class of compression algorithms and pictures? (v) What are near-optimal algorithms?
L p norm, 0

< p < 00, and Besov norm ha\'e been used to study these prob-

lems. Usually] is represented by a wa\'elet series (10.2.20) 01' its truncated version or series associated with scaling function 'P namely L L cJ'~''Pj,k(X) [see &wation (10.3.6) for C"k] or its partial sum of certain order. Theorem 10.2.7, Decomposition and Reconstruction Algorithms of Mal(at [Section 10.3.2]' Theorems 10.4.1, 10.5.5, 10.5.6, Corollary 10.5.2, and Theorem 10.5.8 are important results in the area of image compression. [n Section 11.6, Besov space and lineal' and nonlinear image compression are introduced. Readers interested in indepth study of this field are referred to [1,2,4,17,19,24,29,31] and references therein.

.n",

"'_'_""""'''''''10''-''

11.4
11.4.1

Representation of Signals by Frames

Functional Analytic Formulation

We discuss here representation of signals 011 a Hilbert space If and L 2 (R) in particular in the context of frames (orthonormal wavelets are special type of frames). Let T : If --7 2 be an operator defined by (11.4.1) where {w n } is a frame in l-f. T is called the frame rcprcscntatwn or frame discretization operatol. T : 2 --+ l-f defined by ( 11.4.2) where {a,,} E lz is the adjoint ofT. Let an operator 5 associated with the frame w" be defined on If as

Sf =
It can be easily checked that

L(f, wn )llI".
"

( 11.4.3)

Sf ~ L(f,w")w,, ~ TTf
n

S~TT.

(11.4.4)

S is called the fmmc operator. It may be called a generalized frame operator to avoid confusion with Definition 10.2.4.

Let {w,,} be a frame for a Hilbert space II with fram~ r~pres~ntation operator T. The frame C07TeIatIOn operator is defined asR=TT-.

Definition 11..4.1

Remar'k 11.4.1 (i) The frame representation operator T is injective (one-to-one). (Ii) T(l-f) is closed. (iii) T' is surjective (onto).
Definition 11.4.2
Let R -1 be the inverse of R, then
R1
=.

R- 1 PT(Hj,

(11.4.5)

Q . ,~,_ .._",,"l1>"-'

.....

where Pl'(fij denotes the orthogonal projection operator on to the range of T, namely T( H), is called a pseudo-inverse of frame correlation R.

Remar'k 11.4.2

(i) The frame correlation has matrix representation

R ~ (Hm . ) ~ (wm,w.)).

(11.4.6)
I

(ii) R maps T(II) bijectively to itself. TIlis implies that R (iii) R= Pr{fi)R= RPT(H)
(iv) R is self-adjoint.
(v)R~O.

exists.

(vi) U A and B are frame bounds (Definition 1O.2.3), then (a) A = 'IRI"-1 (b) B ~ IIRII.

Remark 11.,4.3

It can be verified that

RtR

I RR

PT(H)

(11.4.7)

11.4.2 1temtive Reconstruction

Let I be an arbitrary element of a Hilbert space II and let {wn } be a frame for H with frame bounds A and B, frame representation T, and frame correlation R. \Ve describe here an iterative method for the recovery of a signal I from its frame representation T f. The iterative method generates a sequence {cn } in T(H) that converges to acE T(H) such that I = TOr.. Moreover, the sequence comrerges at an exponential rate.

Algorithm 11.4.1 (Frame Algorithm). Let Co = TI Set do = O. For ), = 2/(A -I- B) definp dn and gn as
dn+l =

dn + (f - )'R)"co

gn = ),Tc".

Then (a) Iimg" =

(bl

I, and
an, whece u ~

119. - 111/11/11 < B/k

III -

)'RIIT(lf)

< 1.

The following lemma is required in the proof of (a) and (b). Lemma 11.4.1

For), = 2/(A -r B), the followmg relatwns hold

Q.

.....

, toOl'._.. _",,"l1>"-I

00

Ii) 1 ~

L)l- AS)'(AS)I
)",0

III-'\RIIT{lI)~

wp
cETUI)

1(1- AR)c,c)1 ()
c. c

<max{l1-AAI,I'-ABII<

1.

(iii)

='

>,

L T* (f j=O

00

)"RFTf. where T*c ::::

L>nWn for c :::: {en}

k

Proof of (i).

Since

<

B-A

A+B

<

1,

so that by the

Neumann expansIon,

S-' ~ A+B;:O A+B S)i 2 ~(lt

Proof of (ii). It cau be verified that

(See Theorem 2.6.3),

where I is the identity operator. By applying this to Sf, we get the desired result.

1- AB < (1- AR)c,c) < I _ AA.

(c,c) -

Since 1- ).,R is self-adjoint and 11 - ).,AI ::::

II -

ABI ::::

B-A
A+B

< 1,

we get

the desired result keeping in mind Theorem 2.4.2.

Proof of (iii)

Since (T9. c) :::: (9, T"c) and

(Tg, c) = LCn(g,Wn)
:::: (9,

L c"w,,).
( 11.4.8)

Because of (i) and relation (11.4.4). it is sufficient to prove

00 00

A LT'(I- AR)'TI ~ L(f - ATT)'IAT'T)f.

1'=0 )=0

This is pro\'ed by the principle of induction. It is true for j = 0 as terms on both sides are the same.

'D"twl<.. . ~........._"-'

.'""

ut it be true for j = m; that is,

(11.4.9)

Then, applying (I 1.4.9), we obtain

AT'(I- \R),"HTI ~ W'(I- AR)'"TI - W'(J - AR)'" ARTI ~ A(I- W'T)i(l - W'T)T'T I = ).,(l- )"TT)m+ITTf.
Thus, (11.4.9) is true for m + L Hence, (11.4.8) is true.
PI'oof of Algorltlllll 11.4.]

(a) By tnduction argument, we have

Lemma 11A.I(iii) implies that lim 9,.. = f.

" ..... <Xl

(b) We can write

IIg" - III

= lI(g"+1 - g,,)

-l- (g"+2

- g,,+.)

+ (g"+3

~ g,,+2)

-l-II

sL
k?:,..

IIgH> - 9<11

L
k>"

IIAT'(/ -AR)'TIII

< AS

(~, Ok) 11/11

ABII/II
by Lemma IIA.I(iiJ and using

C a) ~n
B

< A 0'' 11111,

relationships between frame bounds,

IITII, liT II and 11111.

Q.

..... ,

, to<l'._..

....,..",,"l1>"-I

11.5
11.5.1

Noise Removal from Signals

Introduction

Noise in signals is an unwanted and a problematic element, for example coc.k-pit noise in an aircraft or thennal noise in a weak r:tdia transmission. The nOIse in a signal is that pan which lacks coherence or .structure. [n audio signals noise can be readily identified with incoherent hissing sound. The main objective of this section is to present functional analytic methods for the suppression of noise. For more comprehensiw account, one may see Teolis [28]. \Vith respect to frame or wavelet representations of analog signals, there are certain domain!> in which noise may perturb a signal, nameJy th'" (analog) signal domain and the (discrete) coefficient domain. ut these two domains be specified as a Hilbert space H ~ ~(R), and the signal domain's image under T; that is, T(H) ~ 2, where T is the frame representation operator for the frame W n . A signal is said to be coherent with resped to fl. set of functions if its inner product repre~ntatioll with I~pect to that set is SucCll1ctive in the sense that relatively few coefficients in the representation domain have large magnitudes (see Definition 11.5.2) and Remark 11.5.1(iii)). In the contrary case, the signal will be called incoherent. Let f. E H be coherent with respect to the frame {wn } and let f be a noise-corrupted version of f.; that is,

f
where
'U

f.

-t

(1

11,

(11.5.1)
(1

is noncoherent with respect to

n and

is known as flOlse level. Let {w,,} be a frame for threshold operator

Definition 11.5.1 (Th,"eshold Operator).

If and let a~ E l"l and 0, called threshold, F{j = F{j,a' is defined on T( If) into l2 by

> O. A >0

an, (F '~QJ" = { 0,

la~1

otherwise

(11.5.2)

where T is the frame representation operator of {wn }.

It may be observed that a threshold elemnet. of T(H) is in general not in T(lf). It cal! be verified that F{j is a linear continuous uperator and 11F..'011 = 1. Often, the threshold operator F{j,a for a E T(l-f) is written as F/j.!, where a is given by the unique coefficient sequence satisfying f = T' a.

Definition 11.5.2 Let {wn } be a frame for the Hilbert space If with the frame representation operator T. Let!. E H be a fixed element. Coherellce functional, denoted by Coh<5, is a mapping which maps! E lf to

IiFM.Till'

IITill'

E [0.00).

Coherence distlibution, denoted by Coh<5!. is a function of Dgiwll by

(11.5.3)

Remar'k 11.5.1 (i) As a function D, the coherence disuibution describes how the energy in a o-truncated representation decays. (ii) A reconstruction procedure that starts from the tmncated sequence F<5,fT! hives

(11.5.4)

The operator T'R I F<5,/ is used to recover coherent portion (pure signal) and suppress the incoherent portion (noise) (see Algorithm 1I.5.J). (iii) Thresholding of the discrete representation of TIP! where Tv is the frame representation operator for wavelet frame 1/). Let

(T f)n.m :::: (j, 'lj)".m) == (j., t/~n,m) + (1 (tt, 'I,I-'",m),

where

(11.5.5)

W",m is associated with t/J as in Equation (10.2.18).

By the property of coherence, (j., tPn,m) must be large and (u, '/In,m) must be small. Thus, for an appropriate choice of 0, the contribution of noise will be nullified while the pure signal will be preserved. The iterative Algorithm 11.5.1 is the technique for the removal 01" suppression of noise.

11.5.2

Model 'llJd Algorithm

Figure 11.5.1 presems essential ingredients of the Noises Suppression Processing l'vlodel. Step 1: (Pure signal !.). A pure signal, !. E l-f is input into the model. Step 2: (Noise u). Noise u is mixed with f. and represents an incoherent disturbance with respect to wavelet system {t/Jn,nd. Step 3: (Wavelet Representation T.,,). The observed polluted signal f is transformed to the wavelet domain to yieJd the wavelet coefficients (j, 'I/!",,,,) :::: T UI d",m'

Threshold

~f

p,

f.

'-'/

Wavelet
r'l.:plC8CnWIOIl

Reconstl1lction

1-+.

/l.lalik AI

Figure 11.5.1

Noise suppression processing model

Step 4: (Threshold Operator Fo). Thn.'Sholding b performed so as tu eliminate coefficients of small magnitude and to preserve coefficients having large magnitudes. In this case, we choose a* = a in Definition 11.5.1 to get
' _ (F Ua )m,n -

{a m" O. _.

lam,,,1 > 8
otherwise

where a E T(H) ;; (2, Step 5: (Masking Operator 114). Mask operat.or AI is defined by
(Ma)",, =

{~~""

if(m,n)EQ otherwise,

where Q ;; N x N is fixed but arbitrary. It is clear that the concept of mask operator is more general than that of threshold operator. Step 6: (Reconstruction T4ntl,b)' Threshold coefficients are used to construct a noise removal version via an appropriate reconstruction algorithm (Algorithm 11.5,1)
RemGr,k 11.5.2 (i). For all 0 > O. 0::; Coh,If::; I (bounded ness) (ii) 01 < 02 implies CohoJ ~ Coho~f (Mollotonicity decreasing) (iii) (a) lim Cohof = I (Closed range)
,~O

lim Cohof = O. o-'IIT!lI"" (iv) Coho(af) = CohColol-l)/ plex). (b)

(Scaling) for all scalar a (real or

COIII-

Q,.,~._ .._",,"l1>"-'

......

Them'ern 11.5.1 For a signaL f. E l-f, let {w,,} be a fmme for l-f with represefltation operator T, Fame cOfTelation R, and frame bounds A and

B. Then for aUo > 0

IIf. - /;11' < B [ IIf.II' - A I

Col.f.],

where h = TR t PO,f. T f. and Coho!. is the coherence distribution for the Fame {w n }
P,oof. By (11.4.7) T flJ = TT" R t FO,f. T f. = PT(H)FO,f. T f .

Since T is a frame representation for all 9 E H,

..1119[1'
In particular,
cho~ing 9

IITfIl' < BII911'.

= f. - h yields

Allf. - /;1[' ~ IIT(f. - /;)11' ~ liT f. - T /;11' ~ IITf. - p'Ul) FM Tf. II' ~ IlPrrH\(l- F;,J.)Tf.II' ~ IlI'm') II . II II - FM JTf.II' ~ 11(/ - F',j.)Tf.II' ~ [1 - Coh,f.IIITf.II' < [1- Coh,f.]Bllf,II'.
This gives the desired result.

It may be observed that while making above calculation, we have used linearity ofT at (I. - JO), T f. = PT(1l)Tf. and IIPT(1f) II = 1 and properties of norm, frame bounds and coherence distribution (I1(/ - FlJ,/.)T f.1I < [I - CoIW. ] liT f.lll
Corollary 11.5.1 wavelet, then

If the fmme {w,,} in Thoorem 11.5.1 is o,thonormal

IIf. - /;11'

IIf.II'

~ 1- Co",f,

Proof. In this case, A = B, T(H) = Pr(1l) = [ and so we have the relult.

t Jo j. - hlf

= IIT(I. -

h)1I 2

and

Now, we modify Algorithm J 1.4.1 so as to raise the problem of initialization with a coefficient sequence outside the range T( H). We may recall that Algorithm 11.4.1 deals with the reconstruction of a signal from its frame representation. However, this does not converge for arbitrary initial data. We now present the modified algorithm. Let (wnJ be a frame for a Hilbert space lf with frame representation T, correlation R, and bounds A and B. Suppose c E lz is the polluted frame representation of a !>ignal f E H. Set Co = ill and do = O. If >. = V2j(A + B) and dn and On arc defined as

Algorithm 11.5.1

d"+1 = dOl

+ (I -

(>.Rr.!)"c{),

Otl = >.2T*d",
then (a)

"-'00

lim y" =

I,

= T*R'c, and = 111- (>.R)211

(b) Ifft - g,,11

< !lIan , where M < 00 and a

< 1.

We need the following lemma in the proof of Algorithlll 11.5.L

Lemma ]1.5.1

Let>. = hj(A+B), thw R t = lim >.2 '"'(l_(>.R)2)k R.

,,---.oc

"

10=(1

Proof. Choose \f - >.R and HI - fl 2 3.10.4. Then

TUf) in Solved Example

and
cO'{ H) ,lIcll= I

,nr

lI>'fuli > 0_

since by Remark 11.4.2(ii), R is I-Ion T(H) and IIcli = 1. By virtue of Solved Example 3.10.4, we get

111- (,\R)'II-qu) < 1

which, in turn. implies that the series),2

on T(H). ,",lith

L(f - p.R)'l)k conver~s to R-2

10=0

00

c an arbitrary element from

00

2 and Co = RC

A'

L(I - (AR)')'",
k=()

converges to (R-Z)RC as Co E T(H). Thus,

00

A'L(/ - (AR)')' R
10=0

is the pseudo-tnverse If! of R.

Pl'Oofof Algor"ithm 11.5.1. As in the proof of Algorithm 11.4.1, the principle of induction is used to show that for all 1l.

This implies that

(b) We have

IIg,,+, - g,,11 ~ IIW'(/ - (AR)')"el, S A'IITllnIT - ('\R)'II)"lIllili

<
where M' =).,2 8 3 / 2 11cll <
CXJ

AI' 0" ,

i';ince

c E 2.

Therefore,

11/1 - 911

L
k>"

IIJia k = Al' 10'''0: = AI0".

11.6

Wavelet Methods for [mage Processing

Function spaces, particularly a generalization of Sobolev space called Besov space, is \'ery appropriate for image compression [4]. Tn the next subsection, an introduction of Beso\' space is gI\'en.

..".. '"'_'_""""'''''''10''-''

11.6.1

Besov Space

We inuoduce here the notion of Besov space and equivalence of the Besov norm with a norm defined by wavelet coefficients. For any h E R2, we define
!1:J(x) ~ f(x) !1:.J(x) ~ !1(!1~)f(x) ~ f(x !1U(x) ~ !1(!1f(x)) ~ f(x

+ h) -

f(x) 2f(x + h)

+ 2h) -

+ f(x)

Ll~+l f(x) = Ll~f(x + h) - Ll~f(x),

k =,1,2,3, ... ,

Now, we definf> the r-th modulus of continuity in L 1' as

(11.6.1 )

where Irh = {x E fjX+Th E I. f = [0, tJ x [0.11}. 0 <p< 00: with usual change to an essential supremum when p = 00. Civen a > 0, 0 < P < 00, 0 < q < 00, choose l' E Z with q > 0: > T - 1. Then the space B~r(Lp( f)), called Besov space, consists of those functions f for which the norm IIflln;,r(L~,(l) defined by when q<oo
(11.6.2)

and

Remark 11.6.1 (a) IrO < P < lor 0 < q < 1, then 1I'lIn;'''(L~(l)) does not satisfy the triangle inequality, However, there exists a constant C such that for all f,9 E B;,r(Lp(f)),

(11.6.3)

(b) Since, for a.ny l' > a, 1" > 1', IIflln;,r(L p (1) and IIfIl B ;."(L,,(I)) are equivalent norms, we define the Besov space B~(L1'(f)) to be B~,r(Lp(t)) for any r > 0:.

."'" "'_'_""""'''''''10''-''

(c) For p = q = 2, B.y(L 2 (!1)) is the Sobolev space HC>(L 2 (rI. (d) For 0" < 1,1 < P < 00, and q = 00, B~(n) is Lip(a, Lp(O)) == {f E L,,(J)/II/(x + h) - l(x)II", < kho, k > Ooo",'ant}.

'1'
(e) IIfllv,j'(L2(f) is equivalent to the norm
dj,k

~ ~ 20kldj,klQ

,where

are wavelet coefficients of f.

(l)

(f) 1I/IIBg(L
1 0: - = -

is eqUivalent to the norm

where

+-. 2 2
Linear and Nonlinear Image Compression

11.6.2

We discuss here wavelet based image compression of observed pixel values. In digitized image, the pixel values (observations) are samples which depend on the measuring device OLill intensity field F(x) for x in / = [0,1] x [0,1]. In the simplest case, the pixel samples are modelled by averages of the intensity function F over small squares. In this case, Olle may choose a wavelet, say the Haar w;I.\'('le!s on the square. Wp assume that 2 2m pixel values Pi are indexed by J :::: (j\,h),O < JI,h < 2m in the usual arrangement of rows and columns, and that each measurement is the average value of F on the subsquare covered by that pixel. To fix notation, we note that the j-th pixel covers the square 1,,"1 with sidelength 2-"1 and lower left corner at the point j /2"1. We denote the characteristic function of I by X =-- Xl and the L2 (f) normalized characteristic function of 1,,"1 by X"m :::: 2.... XI" .... :::: 2"'X(2mx - j). One call write each pixel ,wue as P, :::: 22m

f
,
j

X(2"'x - j)F(x)llx

:::: 2"" (X"m, F).

The normal practice in wavelet-based image processing is to use the observed pixel values P, to construct the function

1m :::: LP,X(2"'x - j) :::: L(Xj,,,,, F)X"""

which we call the observed image.

...... "'_'_""""'''''''10''-''

Thus, if the wavelet expansion of the intensity field F is

then the wavelet expansion of image

I is
J ,k'1Jj,k'

1m ==

L Ld
O$k< ... j

The important point is that 1m is the L2(1) projection of F onto span{x"',i} == span{\'Jj,d09<m,n' [I,'[oroover, if F is in any function space whose norm is determined by a sequence norm of Haar wavelet coefficients, then ~o is 1m, and the sequence space norm uf 1m is less than the sequence space norm of F. In practice, we use smoother wavelets than Haar wavelet. Linear Compression. Let 'l/! be a wawlet and let F be an observed image mentioned above. Let IN == Ldj,k\'JJ,k' that is, we include in

k<N

the approximation all coefficients dj,k with frequency less than 2 N , N < m (IN is project.ion onto V;n). IN is ('.aIled t,he wavelet. approximation of F. We have

IIF ~ fNliLn

L L Id",I'
k>N
j
22a~
k~N

<

L L 2"N Id",I' :S TaN L L 22a~ )d ,kI 2 , ,

j j

== 2o:NllFllifQ(L~(l))
Therefore,

(By Remark lUU(e.

(11.6.4)

Nonlinear Compression: For nonlinear compresSIOn algorithm, we take

B . ,to<l',_""""",,"l1>"-I

.... ,

Thus, we consider all large coefficients without consideration offrequency 1 but k < m If we assume that F E B;(Lq(f)), - == 0/2+ 1/2, then N, the q

number of coefficients greater than >., satisfies

N>.q < ~)dj,klq == 11F1I:h~(Lq(l)r j.k

So

(11.6.5) and

II!> - 1.. 111,(1) <

L
Id,.kl<>'

Idj"I'
>.z-qldj,kl"
(11.6.6)

<

L
Id;.kl<>'

as 2 -q =20/1 +0. If N is nom-.,ero, then (11.6.5) implies that

(11.6.7)
By (lL6.6) and (11.6.7), we get

III>. -

ImIiL2(1)

< N-QI2IIFIIB~(I".(1))"

([1.6.8)

It may be remarked that the above analysis can be applied to any compression scheme that satisfies

with

It may also be lIoted that a given image, say

1 in one of the nonlinear smoothness spaces B;(Lq(J)), - = 0:/2+1/2 than in q the Sobolev spaces HU(Lz(l)). that is. if a given image I is in H"'(L 2(J)), 1 then it is also in Bg (L,,(f)), - == {J /2 + 1/2, for {J > 0:.

I, will have greater smoothness

.n",

'"'_'_""""'''''''10''-''

11.7 Problems
Problem 11.7.1 Let p.. denote the orthogonal projection of 2 (R) onto a scaling subspace l~., then show that

Ilf -

R" II 11'"

< KT"i N - m , ,

where [( is a positive constant, / E H m (R), for Daubechies wavelet of order N.

Problem 11.7.2 Let '.~, he an r-regular MRA and let Then prove that for all n > 0

E B;( L p ( f)).

III Problem 11.7.3

p"fIlB~(Lp(l)

S Cllflln~(L,,(l2-tlO.

Show that / E L 1' (R) if and only if

Problem 11.7.4 Write down the orthogonal wavelet decomposition of a timp series, XI., for t = 1" .. , n and apply it, 1.0 analyLR. a real-world data.

Problem 11.7.5 value problem - (au')'

Apply the wavelet method for solving thp boundary

+ 13'U + ~IU'

= /

u (0) = c, u (I) = d, where

E H- 1 (0, 1) .

Problem 11.7.6 Apply the wavelet method for solving the Hcgularh:ed Burgers Equation defined with It > 0 by
8u 82u 8t (x,t) -u(x,t) 8x (x,t) = Jl (x,t) 8X2
u(x,O) = uo(x)

au
at

for t

>

and 0< x < 1,

&u

(O,t)

0, u(I,I)

1.

.n",

"'_'_""""'''''''10''-''

Problem 11.7.7 Show that for bounded varialion,

E BV(R), the space of functions or

1<P

< 00, c is a positive constant.

Show that thl" Besov space
B~(L2(R))

Pf"Oblem 11. 7.8 space.

is a Hilbert

Pf"Oblem 11.7.9

Prow' t.hat

B~ (R))

if and only if

1:
Problem 11.7.10

1/(01'(1 + IW'Od{ < 00.

Let lx) be a wavelet such that

are constant:.. Show that

L
kez

dj,k'l/!j,k

<

2}!P

L
keZ

dj,ktP},k

Problem 11.7.11

Verify Remark 11.4.1. Let {w,,} be a frame for H with representation T.

Problem 11.7.12 Show that

where R t denote the pseudo-mverse of correlation R.

.,,,,, "'_'_""""'''''''10''-''

Problem 11.7.13 Problem 11.7.14 Problem 11.7.15

Verify Remark 11.4.2. Verify Remark 11.5.2. Let !p(x) be defined by

o
!p(x) =

x<O
0

:S x < J

Ox> 1.
Draw the graph of the wavelet. obtained by taking the convolution of the Haar wavelet with !p(x).
Problem 11.7.16

{w n (., .)
L,(R).

Prove that {wn(',-k)}I:EZ, 0 <n < 2J , where denotes a family of wi:welet packets, is an orthonormal basis of

..." . . '_'_""""'''''''10''--'

References
[1] A Aldroubi, [vI Unser. Wavelets in Medicine and Biology. Boca Raton: eRe Press, 1996 [2] eM Brislaw. Finger-prints go digital, Notices of The AMS, 42(1995), pp. 1278-1283. \Vebsite http://wwww.c3.lanl.gov/brislawn. [3] JS Byrnes, JL Byrnes, KA Hargreaves, KD Berry. eds. Wavelet and their Applications, NATO ASI Series, Dordrecht, Boston, London: Kluwer Academic Publishers, 1994. [4j A Charnbolle, RA DeVore, NY Lee, B Luder. Nonlinear wavelet image processing: Variational problems, compression and noise removal through wavelet shrinkage, IEEE Trans Image ProcefSing, 7:319-335, 1998.

[5J A Cohen Wavelet methods in numerical analysis. In: PC Ciarlet, JL Lions, eds. Hand Book of Numerical Anal"l\is. Vol. VIT. AmRt.erd:un: Elsevier Science, 2002, pp 417 710.

[6J W Dahmen. Wavelets and multiscale methods for operator equations,

Ada Numelica, 6:909-996, 1997, [7] W Dahmen. Wavelet methoeds for PDEs-Some recent developments, Journal of Computational and Applied Mathematics 128:133-185, 2001. [8] I Daubechies. Ten Lectures on Wavelets. Philadelphia: SIAl\L 1992. [9j RA DeVore, B Jawerth, BU Lucier. Image compression through wavelet transform coding, IEEE Transactions on lnformation Theory 38:719-746,1992.

...... "'_'_""""'''''''10''-''

(1OJ RA DeVore. Nonlinear approximation, Acta Numerica 51-150, 1998. [11] DL Donho. Orthogonal ridgelets and linear singularities, SlAM J Math Anal 31:1062-1099, 2000. [12J F Eli, P Kumar. eds Wavelets in Geophysics. New York: Academir Press, 1994. [13: R Gencay, F Seluk. An Introduction to Wavelets and Other Filtering l\'Iethuds ill Finance and Econumic:,;. London: Academic Press, 2001. [11] R Glowlnski, W Lawton, M Havachol, E Tenebaum. Wavelet Solutions of Linear and Nonlinear Elliptic, Parabolic and Hyperbolic Problems in One Space Dimension. [n: R Glowinski, A Lidmewski, OOs. Proceedings of the 9th Intel11ational Conference on Computer Methods in Applied Sciences and Engineering. Philadelphia: SIAM. 1990, PP 55-120. [15) M Kobyashi. (OOs.) Wavelets and their Applications-Case Studies, Philadelphia: SIAM, 1998 [161 J Kovacevic, I Daubechies. (oos), Special Issue un Wavelets, Proc. IEEE, 84:507-614, 1996. [17J S Mallat. A Wavelet Tour of Signal Processing, 2nd ed. London: Aca demic Press, 1999. [18J P Manchanda, K Ahmad, AH Siddiqi. Current Trends in Industrial and Applied Mathematics. New Delhi: Anamaya Publisher, 2002. [19J P Manchanda, AH Siddiqi. Role of functional analaytk methods in imaging science during the 2 Lst century. In: P Manchanda, K Ahmad, AH Siddiqi, eds. Current Trends in Applied Mathematics. Nf'w Delhi: Anarnaya Publisher, 2002, pp 1-28. [20J Y Meyer. Wavelets Algorithms and Applications. Philadelphia: SIAM, J993. [2J] M Meyer. Wavelets \lJbralion and Scalings, CRM, Monograph Series, Vol. 9, Providence: AMS, 1998.

0.

.... ,

,..-1"_""""'''''''10''-1

[22J Y Meyer. Lecture Notes, School on Mathematical Problems in Image Processing, 4-22, September 2000, Abdus Salam International Centre
for Theoretical Physics, SMR 1237/3, Trieste, Italy, 2000

[23J H Neunzert, AH Siddiqi Topics

Industrial Mathematics: Case Studies and Mathematical Methods. Dordrecht, Boston, London: Kluwer AC1'ldemic Publishers, 2000.
1I1

[24J DB Perdv;ll, AT Waldpn. Wavelet l\'fcthods for Time Series Analysis.

Cambridge: Cambrdige Series in Statistical and Probabilistic Series, Cambridge University Press, 2000.

[25J HL Resnikoff, RO Wells Jr. Wavelet Analysis, the Scalable Structure

of Information. New York, Berlin. Heidelberg: Springer. J998.

[26J AH Siddiqi, MK Ahmad, A Mukheimer. Certain recent developments

in fractal image compression. In: Brokate and AH Siddiqi, eds. Functional Analysis with Applications. London: Pitman Research Notes Series, Vol. 377, 1998, pp 177-191.
[27J AH Siddiqi, M Kocvara. eds_ Trends in Industrial <1m] Applied Mathematics. Dordrecht, Boston, London: Kluwer Academic Publishers, 2002.

[28J A Teolis. Computational Signal Processing with Wavelets. Doston:

Birkhiinser. 1998.

[29J JC Vall Den Berg. ed. Wavelets in Physics Cambridge: Cambridge

Ulliveristy PrefS, 1999, [301 DF Walnut. An Intmrlud,ioll to Wavelet Analysis. Boston. Basel: Birkhiiuser, 2000. [311 MV Wlckerhauser. Applied Wavelet Analysis from Theory to Software. Wellesley: AK Peters, 1994.

.n",

"'_'_""""'''''''10''-''

Appendix A

Set Theoretic Concepts

Definition A.l L Let X and l' be twO noncmpty selS. A function or ma1Jping or transformation or correspondence or operator, which we denote by j, is a rule assigning to each element x in X a single fully determined element y in 1'. The y which corresponds ill this way to a given x is usually written as f(x) or Ix, and is called the image of x under the rule I, or the vaLue uf f at the element x. The set X is called the domain of f and the set of all f(x)'t:/x E X is called the runge of f and it is a subset. of Y. The symbol f: X -) l' will mean that f is a function whose domain is .t, and whose rallge is contained in V 2 Let f : X ~ Y and 9 : V -+ Z. Then 9/, which is called the product of these mappings, is defined as (gf) (x) = aU (x) and 9/: X --t Z. 3. Let f: X -4 Y and Ac X. Then f(A) = {j(x)/x E A} is called the image of A under j. 4. Let / : X -+ Y. / is called into if the range of / is not equal to Y. / is called onto if the range of / equals Y. / is called one-one if two different elements always have different images. 5. Let /: X -+ Y be both one-one and Ol1to. fhen v.'C define its inverse mapping /-1 : Y -+ X as follows: for each yin Y we find a unique element x in X such that / (x) = y; we then define x to be /-1 (y). 6. let /: X -> Y and B be a subset ofY. Then /-I(B) = {xl f(x) E B} /-1 (B) is called the inverse image of B under j. 7. A function / is called an extension of a function 9 jf / (x) = 9 (x) 'r/ x belonging to K the domain of !J and [( c dom /
Note: (a) The term function in (1) is generally used in real and complex analysis. If the range of a function consists of real numbers, it is called a real funct.ion; and if the range consists of complex numbers, it is called a

...... '.'_'_""""'''''''10''-''

complex function. (b) The term operator of {1) is generally used if X and Yare normed spaces, (c) Th~ term transformation;s generally used inlincar algebra. (d) The term mapping is preferred in cases when Y is not necessarily a set of numbers, (e) The term operator IS replaced by functional if X is a normed space and Y is a set of real or complex numbers or some other field,

Theorem A.I [f f : X ----) Y, A l ,A2 ,'" ,An BI , B 2 , ' .. 1 B'I ar'e sublwts of Y, then
1. I() ~

a11l s1lbset.~

of X find

2. I(X)
.Y.

Y
C

A, <;; A, '* I (Ad

I (A,)

U(YAi)

~Y/(A,)

I(~A,) C~/(A,)

6. I~' () ~

7. I~' (Y) ~ X

8. BI C B2 :::}

i-I (Bd ~ i-I (B 2 )

i)

9.

I~'

(y B,) ~ Y/~' (B

10
11.

I~' (~B,) ~ 0/~'

(B,)
denote the complements of

i-I (B')::: (I I (B))', B' and (I-l (B))' B and i-I (B,), reslJeCtivdy.

Definition A.2

Let X be a nonempty set, then

1 A partition of X is a disjoint class {Xi} of nonempty subsets of X whose union is the full ~et X itself. The .Y, 's arc called par'lition sets.

...... "'_'_""""'''''''10''-''

2. A relation""'" (other symbols used in the literature for relations are S, C, C, and ,,) on (or in) the set X is a nonempty A subset of X xX. \Ve say that x is related by . . . , to y and we write x . . . ., y if (x,y) E A.

3. A relation"" on X is called an cquwalence 1'eLation if it has the following properties: (a) x"" x for every x E X (reflexive property), (b) if x......., y, then y "" x (symmetric proppIi.y), and (c) if x......, y and y"""" z, then x "" z (transitive property). 4. A partial order relation on X is a relation which is symbolized by < and satisfies the following properties: (a) x <x for every x (reflexivity), (b) if x <y and y < x then x = y (anti-sYllmletry), and (c) if x <y and y < z then x <z (transitivity).
X is called a par'tially ()nien:d set Two elements of the partially ordered set X are called compamhle if either x < y or y < x.

Note: The word 'partial' in (4) emphasizes that there may be pairs of elements in X which are not comparable.

Definition A.3

I. A partially ordered set in which any two elements are comparable is called a totally onlenxL set or a linoody on:le1w set or a chain or a compLetely ordered set. 2. An element x in a partially ordered set X is called ma.nmaL if y ~ x implies that y = x; i.e., if no element other than x itself is greater than or equal to x.
Definitton A.4 Let A be a nonempty subset of a partially ordered set X. An element x in X is called a lower hound of A if x < a for each a E A. A lower bound of A is called the !J7"OOft;st Lower' bound (!Jlb) of A if it is greater than or equal to every lower bound of A. An element y in X is called an 'l11J1JCr bound of A if a < y for eyery a E A. An upper hound of A is called the least upper hound (hlh) of A If it is less than or equal to every

.n",

"'_'_""""'''''''10''-''

upper bound of A. IThis means that if M is the set of all uppcr bounds of A, then an element m of M is lub of A provided m::::: b for all b E M. J

Remu'f'k A.1

I. The set R of real numbers is a partially ordered set, wher!" x < y means that y - x is nonnegativc. It is ",,'ell known that if a nonempty subset A of R has a lower bound, then it has a greatest lower bound which is usually called its infim11m and written as inf A. Similarly, if a noncmpty subset A of R has an upper bound, then it has a least upper bound and is usually called the 81lpn~m1lm and written sup A. 2. If A is a finite subset of R, then the supremum is often caJled ma.umum and infimum is oft!"11 called minimum 3. If A is a nonempty subset of R which has no upper bound and hence no least upper bound in R, then we express tlus by writing sup A = +00 and if A is an empty subset of R, we write sup A = -00. Similarly, if A has no lower bound and hence no greatest lower bound, we write inf A = -00, and A is empty, then we say that inf A = +00. Let A be any subset of the extended real-number system (the real number system with the symbols +00 and -00 adjoined). Then sup A and inf A exist. Thus, if we consider the extended real number system, no restriction on A is required for the existence of its supremum and infimum. 4. If A and B are two subsets of real numbers, theu sup (A

+ B)::::: supA +supB.

G. Let C be the set of all real functions defined on a Ilonempty set X, and let f : : : 9 mean that f (x) ::::: 9 (x) for evcry x E X. Then C is a partially ordered set and sup If (x) + 9 (x)1 < sup If (x)1 + sup 19 (x)l

Theorem A.2 (Zorn's Lemma). If X is a lJartiaLly onle1"d set in which every chain 01' totally onlenxl set lias an 111Jper bmmd. then X contains at Least one ma.rimal element Note. Zorn's lemma is equivalent to the axiom of choice and therefore it can itself be regarded as an axiom. It is an exceedingly powerful tool for the pruofs of several impurtaut results of functional analysis.

B.

..... ,

, to<l'._..

....,..",,"l1>"-I

A partially ordered set L in which each pair of elements has a greatest lower bound and a least upper bound is called a laltice. If x, y ELand L is a lattice, then x v y and x A y represent the least upper bound and the greatest lower bound of x and y, respectively. The set of all subsets of a set X is often denoted by 2.\" .

Definition A. 5

B.

.... ,

, to<l',_"""",,,,,"l1>"-I

Appendix B

Topological Concepts
Definition B.1
L A family ~. of subsets of a set X is called a topolOflY in X if the following conditions are satisfied: (a) Null set and X belong to (b) The
lIllIon

3' .
~

of allY collection of sets in

is again a set in

~.

(e) The intersection of a finite collection of sets of J is a set in J. (X, J) is called a topological space. However, for the sake of simplicity we write only X for a topological space, bearing in mind that a topology J is defined on X. The sets in J are called

the open sets of (X. 3).

2. An open set A containing an element a E X is called a 'ICfghbor'iIOOd of the element a. An open set B containing a subset P of X is called a neighbor'hood of the set P. 3. A point p is called a Limit point or a point of accumulation of a subset B of X if every neighhorhond of p r:ont.ains at lea,<;t one point of B other than p. 4. The interior of a subset of X is the union of its open subsets. A point of interior of a set is called its interior' point. 5. A subset of a topological space (X, J) is closed if its complement is open 6. The intersectIon of all closed sets containing a subset A of a topological space (X, j) is called the clos1l1"e of A and is usually denoted by

.n",

"'_'_""""'''''''10''-''

A. The set of points in A, which are not interior points of A, is called the boundary of A.
7. A collectiun ~ of upen subsets of a topologIcal space (X, 3) is called a base for the topology 3 if every dement of 3 can bf' written as a union of elements of '+I.
8. A collection '+I", of open subsets of (X, 3) is called a basis at the 1JOint x E X if, for any open set 0 cont.aining x, there exists a set A in ~'" such that x E A c O.

9. A nonempty collection '+I of open subsets of a topological space (X, 3) is called a subbase if the colledioll of all finite intersections of elements of '+I is a base for 3 .

10. If (X,;n is a topological space and Y ~ X, then the topology 3 y = {AlA = B n Y, BE 3} is called the natmul rdative topoloJJY of Y generated by 6
11. Let

61,32 be t\\'o topologies on a set X. 31 is said to be wwke1' than 32 (or 3z is said to be stronge,' than 3d if 31 c 32; i.e., if every open set of 31 is an open set in 32. Two topologies 31 and 32 are said to be equivalent if 31 = 32; Le., if they have the same open sets. The
stmnger topology contains more open sets.

12. A sequence {a,,} in a topological space X is said to be convergent to Q E X if every neighborhood of a contains all but a finite number of the points an'

Definition 8.2

I. Suppose (X, 31) and

in

(Y,32) are two topological spaces. and f : X ---t Y is a function. f is called continuous if f-I(A) E 31 , for every A

32.

2. If f is a continuous one-ta-one map of X onto Y such that the inverse function f- I is also continuous, then f is called a homeommphism or topologically isommphic. X and Yare called homeomorphic if there exists a homeomorphism betweeu X ilud V.

Theorem B.I Let '+I be a coUection of subsek! of an aF'bitmry set X, a'id 3 the ooUection of af'bitmry unions of elements of '+I. Then 3 is a tOlXJlogy for X if and only if

...... "'_'_""""'''''''10''--'

I, Fm' every pau' A, B E ':J3 and x E An B, the1'e is aCE ':J3 such that xECcAnB.
2. X
~

U
BE~

B.

Ways of Generating a Topology

Let X be any set. Then from the definition. it is clear that there are always two topologi~ on X: one consisting of X and , and the other consisting of all subsets of X; the first is known as an indiscrete t01Jology and the second as a discrete topology. Methods of generating topologies between these two extremes are described as follows: I. Let S be a collection of subsets of an arbitrary set X. The weakest topology containing S is the topology formed by taking all unions of finite intersections of elements of S, together with and X, 2. Let X be any set and let Y a topological space with thp topology 3 (Y) and {fa/a E A} a collection of functions, each defined on X with range in Y. The weak topology generated by {fa/a E A} is the we:l.kest topology on X under which each of t.hA function fa is continuous. This requires that 1;;1 (0) be all open subset of X for each a E A,O E .W). Le,S= {J;' (O)/a E A,O E .(Y)). Thenas;n (1), we can generate a topology. This topology with S as a subbase is then the weak topology generated by {ja} .

Definition B. 3
1. A topolugical space X is called a lfausdmjJ SlJUCfo if, for distinct points x, y EX, there are neighborhoods N x of x and Ny of y such that N x n N" = '

2, A topological space X is called n;gular' if for each closed set A, and each x i. A, there exist disjoint neighborhoods of x and A. 3. A topological space is called nonnal if, for en.ch pair of disjoint closed sets A, B, there exist disjoint neil;hborhoods U, V of A and B, respectively. Note: Here, the topological space satisfies the condition that sets consisting of single points are closed. Sets A and B are called disjoint if A n B = .

...... "'_'_""""'''''''10''-''

Definition B.4
1. A covermg of a set A in a topological space X IS a collectIOn of open sets whose union contains A The space X is said to be compact if every covering of X contains a finite subset which is also a covering of X. 2. A topological space X is said to be locally oomlXict if e\~ry point has a neighborhood whose closure is compact. 3. A topological space X is called oonnected ifit canuot be represented as a union of two disjoint nonempty open sets. X is called disoonnected if it is not connected. 4. A family of sets is said to have a finite intenu:ction IJ,ope,'ty if every finite subfamily hm; a nonvoid intersection.
lei X i ,X2 ,'" ,X" be topological spaces, with '.pi a base for the topology of X;. Their topoLogirol product XI X X 2 x x X" is defined as the set of all n -tuples (Xi,X2,'" ,x.. ) with Xi E Xi, taking as a base for the topology all products Vi x [-'2 x ... x U" of V, in '.p,.

Definition B.5

Theorem B.2
1. (TtJchonofJ theon;m) The t01JOIogirol 1woouct of oompacl spaces i,~ compact.

2. Every compact s'llbS1Jace of a fJausd01iJ tOPOlOglcaL s1wce is dosed. 3. A topological "'lwce is COfltlXlct if and only if eVf;ry family of closed sets with U~e finite inte1"Section property has a nonvoid intersection.

Jf f is a continuous function IJefined on a topologicaL s1wce X into a t01mlogical spaa Y aud A i.., a com1mct .~ul).'wt of X, Uwn f (A) is compact and if Y = R, then f atlaff~ its supn,m'llm and infimum on A.

5. Eve11/ clOlwd s,/lb..,et of a com,Xict tOlJOLogtrol space ts comlXict. 6. A continuous one-to-one function from a oompact 100Joiogiroi space on to a fJU'ilSa01iJ topologicaL space i.., a homoom01phism.

0 . ,>0<1"_""""'''''''10''-1

......

Appendix C

Elements of Metric Spaces

Definition C.l
L Let X be a set, and d a real function on X x X, with the properties

(a) d(x,y)

> O't/ x,

y E X, d(x,y) ~ 0 if and only ifx= y.

(b) d (x, y) = d (y, x) V X, Y E X (symmetry property).

(e) d(x,y) < d(x,z) +d(z,y) (the triangle inequality). Then d is called a mdric, or a metric function on or over or in X (X, d) is called a metric SfJuce.

2. Sr (x) = {yfd(x,y) ccnter x in X.

< '"}

is called all open sphere with radius rand

3. Let A and B be subsets of a metric space (X. d). Then

(a) d(A,B) =suPaEAinfbEB d(o,b).

(b) The diameter of A, which sup dCa, b).

""I,EA

v.'e

denote by 0 (A), is the number

(e) S(A,f) = {x/d(A,x) < f} is called the {-ncighborhood of A.

(d) A is bounded jf /j (A)

< 00.

4. A sequence in a metric space is a function whose domain is the set of pOfiitive integerfi and range is a fiubset of the lIlf!tric sparR.
5, A sequence {A,,} of subsets of a metnc space is called a decrem>mg sequence if A I ::J A..1 ;2 A 3 ::J, ... "

.n",

'"'_'_""""'''''''10''-''

Theorem C.l
1. The collection 01 open s!)hen;s in a metric space (X,d) lo,..,.",s a base lor the me17'ic topology, 2. EfJery melT"ic space
i.~

nonnal and hence a JIaust10r/J topological space.

Definition C.2
L. A scquclice {an} in a metric space P:,d) is said to be cunvergent to an element a E X if lim d(a,,,a) = O. {a,,} is railed a Cauchy
'1---+00

sequence if lim d (x"" X'l) = O.

m~=

n~=

2. A metric space is called complete if every Cauch)' sequence in convergent to an element of thi:; space.

It

is

3. Let (X,d.) and (Y,(!z) be metric spaces, and I a mapping of X into l'"'. I is called continuous at a point Xo in X if either of the following equh'alent conditions is satisfied: (a) For every to > 0, there exists a 0 implies d 2 (J (x), I (xo)) < l".

> 0 such that

d l (x, xo)

<0

(b) For each open sphere 5. (J (xo)) centered on I (xo) , there exists an open sphere 56 (xo) centered on Xo such that 1(5, (xo)) ~ 5. (f (xo)). f is called continuous ifit is continuous at each point of its domain. 4. In (3), it can be seen that the choice of 0 depends not only on f but also on the point xo, The concept of continuity, in which for each f > 0, a 0 > 0 can be found which works ulliformly over the entire metric spacc X, is called unilorm coutifmity. Theorem C.2
1_ If a converyent sequence in a metric space has infinitely many tlistinct points, then its limit is a limit point of Ule set 01 points of the sequence. 2. Let X and l'" be metric spaces awl I a 1nal)ping of X into Y. 77len I is continuous il ant1 O1dy il {x n } in X converyes to x E X implies that I (xu) in l' C01l1JC7yes to I (x) E l'". 3. A lKJint lJ is in the cl08ure 01 a set A in a metric space il and only il then; 1S a sequence {tJ'I} of lJOmts 01 A convCJying to p.

..... , Q.,,>0<1"_""""'''''''IO''-I

4. Let ( ,., d1 ) and (Y, dz) be metric spaces and f a mapping of . into y. Then f is continuous if and only if f~l (0) is open in X whenever

o is open in Y

5. Principle of e~tension by continuity: Let X be a metric space and F' a complet metric space. If f : A Y zs unifonnly continuous on a dense subset A of .. '", then f has un'ique extension 9 which is a uniformly continuous mapping of X into Y 6. Every continuous mapping defi1ted on a compact rnet7'ic space X into a 1net7'ic space Y is unifo7'1nly continuo'U.s.
7. Cantor-'s inter'section theorem: Let X' be a corltplete '1netric SpU(;e, und {Fn } a dec'roosing sequence of'lwnempty closed subsets of}{ such that

6 (Fn ) -+ O. Then F =

n=l

n Fn contains exactly one point.

00

8. Ba'tre's category the07'em,: Every complete metnc space 'ts of the second category (If a complete 1netr'zc spa,ce is the 'lin-ion of a seq'lienCe of its subsets, the closur'e of at least one set in the sequence must have a nonempty inter'i07'.)

Definition C.3
Let B be a subs t of (.:\:, d) . A subset A of .LY. is call dan -net for the set B if for each x E B, there exists ayE A such that d (x, y) < t. 2. A subs t B of a metric spac (X, d) is called totally bounded if for any > 0, th r exis s a finite -net for B.
Example C.l

1. In the metric space R 2 , the subset A=: {(m, n) 1m, n is an e-net for > V2/2 .

== 0, 1, 2, .. }

2. In 2, A. = {x E z/d (x, 0) = I}; i.e., th pointb of th surfaoe of the unit sphere in l!2 is bound d but not totally bounded.
ote:

Every totally bounded set is bounded.

ore,

II

C.3

1. Every com,pact subset of a metNc space 'ts totally bounded and closed.

2. A metric s!)(Jce is compact if and only if it complete.

IS

totally bouwletl anti

3. Every com!)(Jct metf'ic spare is scpamble.

Definition C.4
1. A metric space in which the triangle inequalit,Y is replaced b,Y a

stronger inequality d (x, y) :5 max {d (x, z) ,d (z, y)} is called an ultm mctric space or a nonaT"chimedean metric space.
2. Ld A be any nonempt,Y set of real or complex funcl.ion& defined on

an arbitrary nonempty set X. Then the functions in A are said to be uniformly bounded if there exists a k such that If (x)1 < k for every x in X and every / in A.

3. Let X be a compact metric space with metric d and A a set of continuous real or complex functions defined on X. Then A is said to be equicontinuous if, for each t > 0, a 0 > 0 can be found such that I/(x) - / (x')1 < t: whenever d(:l:,x') < oV / E A.
4. Let (X, dl ) and (Y, dz ) be metric spaces. Then a mapping / of X into Y is called an isomehy if d 1 (x, y) = (~(J (x), f (y)) foraJl x, y E X If, in addition, f is onto, then X and 1"" are said to be isometric.

5. A subset A of a metric space X is called eveqjwhe,-e (lense or (lense in it if A ::::: X; Le., every point of X is either a point or a limit point of A. This means that, given any point x of X, there exists a sequence of points in A that converges to x.
6. Let (X, (1) be an arbitrary metric space. A complete metric space (Y,dd is called a cumpletion of (X,d) if

(a) (X,d) is isometric to a subspace (W,dd of (Y,dd, and (b) the closure of lV, IV, is all of Y; that is, W = Y (IV is everywhere dense in V).

7. Let X be a metric space. A real-valued function f : X -t R is said to have compact SUPP07t if / (x) = 0 outside a compact subset of X The closure of the set {xl/ (x) :F O} is called the SU!Jp07t of f. It is denoted by supp f.

Theorem C.4 (Ascoli's Theorem). If X is a cnm!)(Jct metric space and C (X, R) (lenotes the set 0/ all nelli-valued continuous functions (lefined

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

on X, tlien a closed subspace of C (X, H) is compact if and only if it is uuif01'fnly boundC(1 and CQuiconliuuous.

Note: The theorem is also valid if R is replaced b:y the set of complex numbers C.

Theorem C.5 (Completion Theorem). Eve,y metrlc space lias a completion and all its completions are isometric.

Example C.2
1. The sets H, C, R'"!, c, m, p, L p, BF [a, b], AC la, b], co, Lip 0, which are defined in Appendix D, are metric spaces with respect to the metric induced by the norm (see Section 1.2) defined on the respective space&.

2. Let s bp the set of all infinite sequences of real numbers. Then s is a metric space with respect to the metric d, defined in the following manner:
d(x,y) ==

~ I

2; 1 + 10.

lai-Pil -.Bd'

where x= (01,02,'" ,0.,.) and ({JI,I32.

,.B,,,,.,)

belong tos.

3. Let K be the set of aJl nonempty compact subsets A, B of Hn and d(x,A) == inf {d(x,a)ja E A} Then K is a metric space with metric. dl , where 1 d l (A, B) == -2 [sup d (a, B)
(lEA

+ supd (b, A)]

bEn

A melri.wale space is a topological space X with the property that there exists at least one metric on the set X whose class of generated open sets is precisely the given topology.

Q.

......

, toOl'._..

....,..",,"l1>"-I

Appendix D

Notations and Definitions of Concrete Spaces

1. R, Z, N, Q and C will denote, respectively, the set of real numbers,

integers, positive integers, rational numbers and complex numbers unless indicated otherwise.

2. The coordlllate plane is defined to be the set of all ordered pairs (x, y) of real numbers. The notation for the coordinate plane is R x R or

n.2 which reflects the idea that

two repliOlS of the real line R.

it is the result of multiplying together

3. If z is a complex number, and if it has the standard form x + iy (:I; and yare real numbers), then we can identify z with the ordered pair (x,y), and thus with an element of H 2 When the coordinate plane R 2 is thought to be consisting of complex numbers and is enriched by the algebraic structure indicated below, it is called the c01n!Jlex plone am.! denoted b:y C. Let 21 = 0 + tb aJld 22 = C + 'fll belong to C. Theil (a)

= Z2 if and only if a = c (real parts of ZI and 2Z are equal) and b = d (imaginary parts of 21 and Z2 are equal).
21

(b) ZI2z=(ac)+i(bd). (c) 2i2Z= (ac-bd)+i(bd). (d) ZI = oc+bd +i bc - od . 2Z cZ +d2 cZ+((l (e) The conjugate of a complex number 2 = a + ib is denoted by z or z and is (IUal to a - ib. The real part of z is given by
Rc z = -2-' The imaginary part of z = -2-'

z+z

z-z

Re 2 = {z/Imaginary part of z = U} = {zJz = z}

Q.

......

, toOl',_"""",,,,,"l1>"-I

The properties of the conjugate of a complex number are listed below:

Z.

+2:2

= Zl +Z2

ZIZ2=ZI+Z2

and

z=o

iff2=O.

(f) The absolute value of 2 or mod of 2 is denoted by 121 = (a 2 + b2) 1/2, where 2 = a + ib. C is called the space of complex numbers. 4. Let 11 be a fixed natural number. Then R!' will denote the set of all ordered n-tuples x = (XI ,X2. ,x., 1 of real numbers. Note. For a detailed discussion of spaces R 2 C and fl", one may see Simmons [10, pp. 22,23,52-54 and 85-871. 5. A sequence r)f real numbers is a real-valued function whose domain is the set of natural numbers. A sequence {x,,} of real numbers is called bou7itlC(l if there exists a real number k > 0 such that Ixlll < k V n. The set of all bounded sequences of real numbers is usually denoted by m and it is called the SfJace of botmdC(l 1'001 sCfjucnccs. Every bounded sequence of real numbers always contains a convergent subsequence. 6. A sequence {x,,} of real numbers is said to be convergent to x E R if for every > 0 there exists a pasitive integer N such that Ix" - xl < V n > N; Le., it is convergent if Ix" - xl -t 0 as n -t 00. The set of all convergent sequences in R. is denoted by c and called the space of real convergent sequences. CO denotes the subset of c for which x = O. COO denotes the space of all real sequences x = {x,,} for which the series

L
.,=1

x"

< 00,

l.e., the series

L
I

x n is convergent.

Let :/:11 be a sequence. A real number M is called a superior bound of {x,,} if XII < M V n. A real number L is calk.od an inferior bound of {XII} if L < XII V 11. The limit superior {x,,} which we denote by lim XII or lim sup x" is defined as the greatest lower bound of
II ---+ 00 II ..... 00

the set of superior bounds of {X.,}. The limit inferior of (XII} which we Jenote b) lim XII or lim inf XII is defined as the least upper
~

n---+oo

buund of the set of inferior buunds of {x,,} . If x" is cUllVergent, then

Q . ,toOl',_""""",,"l1>"-I

.... ,

n ---+ 00

lim

Xu =

lim inf

n ---+ 00

Xn =

lim sup
"---+00

Xn-

The unit impulse signal 0" is defined by

0" = 1 ifn=O =Oifnf-O.

',f' . 1=) 0;1 is called the Kmnecker (lelIa. if if-j,

Note.

(a) For a bounded sequence lim X" and lim x" always exist. (b) lim
"---+00

Xn =

sup { lim

11.---+00

X"k / {X"k}

subsequence of {:I: n } } and

"---+00

lilll X" = iuf { lim x"j {X"k} liubsequence of {xn} }.

11.---+00

7. The set of all sequences X = that

{XI, X2, - -

in the case p = 2, is caJled the mfinile-(hmensio1101 Euclidean space. 8. For a,b E R and a < b,[a,b] = {x E R/a< x <b} (a,b) = {x E Ria < x < b} and (a,b] = {x E RIa < x <b} are called. respectively, the closed, open, semi-closed and semiopen-interval of R. R is a metric space with the metric d(x,y) = Ix - yl and therefore a dosed and bounded subset of R can be defined. As a special case, a real-valued function f (x) on [a, bl is called bounded if there exists a real num~ ber k such that If (x)J < k V x E [a, b] _ It is said to be continuous at Xo E [a, b] if, for l > 0, there exists 0 < 0 such that II (x) - I (:1:0)1 < l whenever Ix - xol < o. It is called uniformly continuous on [a, b] if the choice of 0 does not depend on the point Xo in the definition of continuity. 'Ve call define a continuous function in a similar fashion on an arbitrar,Y subset of R. C [u, bl denotes the :,et uf all continuous real functions defined on [a, b]. It is called the space of continuous functions on [a, b]. Similarly, C (T), where T C R, is called the space of continuous functions on T. If X is all arbitral',Y topological space, then C(X) is called the space of continuous real functions on the topological space X. 9. p(x) = ao + alx + ... + I1"X" for x E [a, b] . where 00, a1,' .. ,a" are reaJ numbers, is called a polYllomial of degree n. It is a continuous

,,=1

L jx"I P < 00,

,X"," . } of reaJ numbers such

by

for 1 ::; p

< 00, is denoted

ep _ Sometimes e2 ,

real function. P [a, b] denotes the set of all polynomials over [a, b], P" [a, b] denotes the set of all polynomials of degree less than or equal to n over [a, b]. Cn [0, To] denotes the set of all functions on [0, r.] of the foml

I (x)

= b1 COS X

+ ~ cos2x + ... + btl cosnx.

10 Let I(x) be a real-valued function defined on [a, b]. The limits

f~ (x) ~ lim fly) - fix) y-->'" y x ' ,<0

and
'() f + x = I'1m f(y)-f(x)
v......",

",

v-x

x and is denoted by I' or II or i or ~~. If the first derivative exists at ever,Y point of [a, h] , then it is said to be differentiable over [a, b].
If the first derivative of I is also differentiable, then I is called twice differentiable. III general, the nth derivative uf the function I is the derivative of the (n - l)th derivative. I is called n times tlifferenliable over [a, b] if nth derivative exists at every point of [a, b]. c(n) [a, b] denotes the set of all functions which have continuous derivatives up to and including the n-th order over [a, b]. [The derh'atives of all orders 011 [a, b] , are called infinitely tlijJercntiable.] Coo [a, b] denotes the class of all infinitely differentiable functions over [a, b]. 11 Let I(x) be a real-\'alued function defined on [a,bj. Let P; a = Xo

are called left and right derivatives, respectively. The function I is said to be dijJef'enliable if the right and left derivatives at x exist and their values are equal. This value is called the first derivative of I at

XI <X2

< ... <x"

= b be a partition of [a,b] , V(x) = sup L: I! (Xi)

"

<

- I (xi-d I is a

variation of I (x) over [a.b].

01 bountletl vaf'iolion on

la, b] if V(x) < 00. BV[a, b] denotes the space

I(x)

"

'''00

is called a junction

" of all functions of bounded variation over [a, b].

12, A real function I(x) defined on [a, b] is called absolutely continuous on [a, b] if, for t > 0, there exists a /j > 0 such that for any collection

{(ai. bi )} ~ of disjoint open subinten'als of [a, b] , L:

i"o 1

II (bi)

-I (ai)1 <

...... "'_'_""""'''''''10''-''

holds whenever

L
i=l

(bi -

Oi)

< o. ACfo,bl denotes the class of all

absoluteJy continuous functions on [a, bl 13. A real function f (x) defined on [a, bl is said to satisfy a Holdcr cmuli lion of exponent a over [a, b] or to be Holder continuous or Lipschitz class if
sup
"',/lEla,b]

If(x) - f(y)1 <

Ix - yl

(>

00.

The class of Holder continuous functions or Lipschitz class on [a, hI is denoted by C n [a, bl or Lipa [a, bl
14. We writ\':
f(x) :::: O(g(x)) if there exists a constant J{

> 0 such that If(x)1 <J<

19(x)1

f(x)

~ 0(9(X))

if

:i:~

-t O.

These relationships are valid when x -t 00, for x -t -00 or x -t Xo, where Xo is some fixed number. If bn > O. 11:::: O. 1. 2.... and ~: -t 0 as 11 a tl
::::

-t 00,

then we wnte a" :::: o(b,,). If :: is bounded, then we wnte

O(b.,).

Remark D.l
1.

a. Every function of the Lipschitzian class over [a, bl belongs to AC[a,bj. b. AC [0. hi c C[a, h]. c. AC[a,bj C BV[o,h].

2. All continuous differentiable functlOIlS over [a, b] are absolutely continuously over [a, bl.

Theorem D.1 (Generalize<! Heine-Borel Theorem). closed and bountleil subset of Rrl is comlJact.

Ev"1}

Theorem D.2
derivative

it

Let f(x,t) be continuous and have a continuous

in a tlomain of the Xl-plane which includes the rectangle

...". "'_'_""""'''''''10''-''

a < x < b, t l < t < t2. {1l a(ltlition, let Q (t) anti {3 (t) be (lefined and hove continuous (le'lvotives for" tl < t < t2. 17len for" t l < t < t2,

-d
t

d jt1(t)
ott)

I (x, t) dx

I [jJ (t) ,tl ~' (t) - flo (t) , tl 0' (t)

+
14. Inequalities Theorem D.3

a,,, 81 ,,(x,t)dx.
vi

0(1)

Let a and b be any real or complex numbers. Then

10 HI < lal Ibl 1 + la HI - 1+ lal + 1 + Ibl

Theorem D.4
(0) HoMer's inequality fOT" sequences: If 1) 1 1 - + - = 1, then p q

>

1 and q is (lefined by

Llx;y;! <
;=1

"

fOT" any COm1)lex nttmbers :/:1 X2, ... ,Xn , VI, Y2, ... ,y". (b) HoMer's mequality for integmls: Let f (t) E L p anti g(t) E L q. Then

f" J, I/(t)9 (t)1 dt s J, I/(t)I' <It

(f"

)'/' (f" 19(t)I'dt)'/' J.

+ 1I q =
1

where pond (1 are ,-datal by the rdoti()fl lip

Theorem D.5
(a) Minkowski's inequality for" seq1leftces: If p
~

1, theft

fOT" auy roUlli[ex uumbers

XI

,X2,' ..

,X'l' YI, Y2,

... ,y....

Q.

..... ,

, to<l',_"""",,,,,"l1>"-I

(b) Minkowski's inel]1wlity for> integrals: Let belong to L p Then

f (t)

E L p anti g (t)

(Ie'

I/(t) + g(t)I' <It) 'I' <

(Ie'

I/(t)I' <It) 'I'

+
15 Lebesgue Integration

(Ie'

19(t)I' dt)

'I' 10'- P2 L

(a) A a-algebra i.p on an abstract set 11 is a collection of subsets of 11 which contains the null set and is closed under countable set operations. A measurable space is a couple (0, i.p) where 0 is an ahstract. set and i.p a a-algebra of subspts of n. A subset A of 0 IS measurable if A E i.p . A mea:mre I-t on a measurable space (0, i.p) is a non-negative set function defined on i.p with the properties: J-l () = 0, Jl

(Q\

E i) =

~ It (Ei ) , where E; are

disjoint sets in ',J3. The tliple (0, ',J3,J-l) is caBed a meastwe space. If X is a normed space, (X, i.p) is a measurable space if ',J3 is the smallest a-algebra conl,aining all open subsets of X. ',J3 is called the Bor>el algebra and sets in ',J3 are called Borel sets. (b) Let be a set of infinitely many points, and i.p the class of all subsets of 11. Then (11, i.p) is a measurable space and a measure on (fl, ',J3) is defined by (E) = the number uf points in E if E is finite and Jl (E) = 00 otherwise. Consider the length function ). on the following class of subsets of R

'J,

~ = { E/ E =
Then

UC i , C; are disjoint half open intervals (u;. b

=1

i ]} .

i=1

The smallest a-algebra containing ~ is the algebra of Borel sets, 'lJ of R. ). may be extended to the measurable space (H, ',J3) as follows

A(A) =

inf U f.,:)A "

2:> (1n) ,

n "'_._""""'''''''10''-''
.n",

where 101 is a countable collection of intervals covering A. So (H, 1,J3, A) is a measure space. [n fact, A is a measure on a larger a-algebra , called the class of Lebes[J1le measurable sets ..c. So (R,..c,>.) is a measure space and A is called the Lebesgue measun;. Not all subsets of R are Lebesgue measurable but most reasonable sets arc, e.g., all sets which are countahle, are unions or intersections of open sets. (c) Let E i be subsets of 0, then the charucte,istic function of E~ is defined as

XE, (w)

~ {~

ifw E E; ifw E;.

A function f : 0 ---t H U {oo} is called a Stm,Jle fuuctton if there are nonzero constants Ci and disjoint measurable sets E i with p(E;) < 00 such that

(w) =

" I:
i=1

COXE,

(w).

The integral of a simple function

is defined as

!. fd~
E:

= tC",(E;
=1

n E)

for any E E '.P. Lf u is a nonnegative measurable function on (0, 'lJ, p) . Then u (tv) = lim Un (w) V W E 0, for some sequence
~Un}
"--->00

of monotonically increasing nonnegative simple functions.

We define

tion f on (0, l,J3,p) is integrable on E E '.P if JE: If I dp < 00. [f in particular we consider the Lebesgue measure space (R,.c,A), then we get a Lebc..~gtle integral, and in such a case we write

i r udp = E

n--->=

lim /, undpV E E 'lJ. A measurable funcE:

ia

r f(t)dt or 1

fdA.

L2 [a,b] is defined as

la,bJ

L 2 (a, b) =

{f: /[lfI'dt<oo}

[a, b]---t R Lebesgue measurable

."", "'_'_""""'''''''10''-''

Forp>l,
Lp(a, b) =

U Lebesgue measurable on

la, b] into R

/[If I'
1. If thc scquence

dt

< 00 }

Theorem D.6 (Lebesgue's Duminated Convergence Theorem).

Ud E L, (a,b)

has thc prope,ty that lim fk is finitc

,~~

a.e., on (a,b] awl if Ihl < h f01" $Orne nonnegative function h E L1(a.b] awl for all k ~ L thol lim hE L 1 (a. b) awl
k ......oo

Ja

{t> Jilll /kdx = lim {b /kdx.

k ...... oo

k-+oo

J"

2. 1f thc scqucnce {g4'} E LI (a, b) has thc 7WOPCJty that

9k convnycs

4'=1

a.c. on (a.b) and if L 1 (a,b) awl aUn

>

gk ~ h f01' somc n01l1lcgative function h E k=1 I, thcn L gk E L 1 (a,b) awl k=1

~

" E

Theorem D.7
1. (Beppo-Levi's The01'em) 1f the sequcnce {gkl E L 1 (a,b) has the pmpcrty that

I: 10
thcn k=1

r"

19,1 dx < 00,

k=1 "

9k convc,ycs a.c. on (a,b) to an mtegmblc functwn and

1,'(~9') dx~ ~ [9'dX.

B.
..... ,
, to<l'._..

....,..",,"l1>"-I

2. (Fatou's Lemma). If {fn} is a sequence of n01l1legative moosnruble functions and fn(x) --t f (x) everywlien; on a set E, then

fdx

< lim

fn dx . ,fn(x), ...

S. (Monotone Converyence Theorem). Iffl(x),h(x), ..

is a sequence of non-negative functions such that

and hill fn(x) = f(x) on a set E, tlien

n~~

lim /, f.,(x)dx = /, f(x)dx .

.. --->00

E:

E:

Theorem 0.8 (Fubini's Theorem). (a) If f(x,y) is integmble in the n;ctangle Q[a ::; X ::; 11, c ::; Y ::; dJ. then for all x E [a,ol except a sp-I. of meaS1we zem, the function f(x, y) is integmble witli J'C.~'Pect to y in [c, dJ. and for all y E Ic, dJ exec7)t a set of measure space f(x, y) is integmble with respect to x in [a, oj, the following equality holds

II
Q

f(x,y)dxdy

~ [<Ix

f(x,y)dy =

jd

dy 1'f(x,y)dX.

(b) If f is Lebc.~!J1le measumble in R2 , say a conti1I1wus function ex-

i: I:
(i)

cept at jinite number of pomts such that

i:i:

f(x,y)dx dy

and

f(x. y)dy dx exist and one of them is absolutely cofwe1yent. then

the two are equal. Eqttations of the type:

~(x) =

1'J((X,t)f(t)dt

(ii) f(x) = 1']{IX,Of(Odt+,,(x)

are called, ITS]ICCU"ely, Fre(UlOlm eqttations of the jif'St kiwi awl the secowl kind.

B.

.... ,

, to<l'._..

....,..",,"l1>"-I

Surface int.egral.

Suppose f(x, y, z) is a /tinction of three variables, contimlOus at all points on a smjace E. Suppose r is the gmph of z = S(x,y) for> (x,y) in a set D in the x, y-plane. it is assumed that r is smooth and that D is bouw[el!. The smjacc integral of / over a surjace r dwoted by [ / df, and
by
lS

defined

lfdf'~
when,

II f(x,y,z)d[,~ II
, D

f(x,y,S(x,y))dS,

dS~

1+

Dx Dy (8S)' + (ay)' dx.dy.

/ for which

L 2 (r) l[enotes the space

0/ /tmctions

[1/1 2 df

exists.

Appendix E

Vector Spaces
Vector Spaces
A vector space or a linear space X over a field J( consists of a set X ,a mapping (x,y) -t x+y of X x X into X, and a mapping (a,x) --t ax of [( x X into X, :mch that
I.

x+y=y+x

2. x+(y+z)=(x+y)+z

3. :3 0 E X with x

+0 =

x for all x E X
T+X I

4. for each x E X3 x' E X with

=0

5. (a+ IJ)x = ox

+ (3x

6. o:(x+y) = ux+o:y
7. ("'~) x ~ ~(ax)

8. Ix = x.
A subset Y C X IS called a vector subspace or simply a subspace of X if it is itself a vector space with the same operations as for X. A subset Y c X is subspace of X jf and only if for every Xl, X2 E Y and 0:, /3 E I<, O:XI + /3X2 E K. A subspace of a vector space X different from {O} and X is called a p''OfJer SlI0SJWCC. If ]{ = R, a vector "pace X over R is called the rent vector space. If ]( = C, a \'ector space over C is called the complex vector space. Let S be any subset of X. Then the subspace generated by S or spanned by S, which we denote by IS], is the intersection of all subspaces of X containing S. It can be shown that rSJ is the set of all linear combinations

.n",

"'_'_""""'''''''10''-''

" L
;=1

a;x; of finite sets in S. Let S = {Xt,X2,'" ,X.,} be a finite non-empty

subset uf a vector space X. Then S is called li1lcarty dependent if there exist

scalars a;. i = 1.2. . ,n sll(~h that

" L
;=1

Uj.xi

= 0 dops not imply that all ai's

are zero. If S is not linearly dependent.

" Le"L a,x; =

;=1

0 implies that all

Qi,i = 1,2,," ,71, are zero, then it is said to be lmeady tnllependcnt An arbitrary nonempty subset S is called lincm-[y independent if its every finite nonempty subset is linearly i1ldC7JCndcnt. It can be seen that a subset S of a vector space X is linearly indc,Jc7Ulc71t if and only if each vector in [S] is uniquely expressible as a lineaJ." combination of the vectors in S. A subset S of a vector space X of linearly independent elements is called a Hamel l)(l..~is or algebmic basis or basis in X if [5] ;:::: X. A vector space X may have many Hamel bases but all have the same cardinal number. This cardinal number is called the dimemion of X. X is called finite-dimemional if its dimension is 0 or a positive intcger and i7lfi7lite-dimemional otherwise. R" is a vector space of dimension 71 while C [a, b] is a vector space of infinite dimensions. Let l' be a subspace of a vector space X. We say that x, y E X are in relation denoted by x ,...., y, if x - y E 1'. This is an equivalence relation. This equivalence relation induces equivalence classes of X which are called cosets. If the coset of an element x in X is defined by x + l' = {x + yjy E Y}, then the distinct cosets form a partition of X. If addition and scalar multiplication are defined by

(x+Y)+(,+Y)

lX+')+Y) o:(x+ y) = QX+ Y,

then these cosets constitute a vector space denoted by X/1' and are called the quotient or factor space of X with respect to 1'. The origin in X/1' is the coset 0 + l' = Y, and the negative inverse of x + l' is (-x) + 1'. A mapping T defined on a vector space X into a vector space Y is called linear if

T(x+y) ~T(x)+T(y) T(o:x) ;:::: o:T(x) where

0:

is a scalar.

The set of all linear mappings detJned on X into Jl is called thc algebraic dual of X. Let M and N be subspaces of a vector SpacR X. We say that X is the direct SUJIl of At and N and write X = M (fI N if every z E X can be written

B.

......

, to<l',_"""",,,,,"l1>"-I

uniquely in the fonn z = x + Y with x E M and YEN. The mapping P, defined on X into itself by the relation P(z) = x, is called an algebraic pmjcctio71 or 1Jmjectio71 on M along N. A linear mapping P of X into itself is a projection if and only if
p'=P [PIP(x) = P(x)VxE X]

A \'ector space X over the field f( (= R or C) eqlllpped with a topology IS called a topological vector space if the mappings (a) X x X --t X : (x, y) --t x + y, and (b) [( x X --t X: (a,x) ---t ox is continuous.
If M and N are subspaces of a vector space X, then AI $ N is a subspace of X. C,p, P,.., [a,b], B\f[a,b], Lz(a,b) and AC[a,bj are examples of a vector space, where operations of addition and scalar multiplication are defined in a manner indicated below. In the space of real-valued continuous functions defined on [a, 0], C[a, bJ, the operations of addition and scalar multiplications are defined as follows:

Ii)

If + ")(x)

~ fIx)

+ "(x) V x E [",bl
V x E [a,b],
0

(ii) (of)(x) = o/(x)

real scalar.

The operations of addition and scalar multiplication in Lz(a,b), which is essentially the space of continuous functions on [a, oj having possible finite number of discontinuities, are defined in a similar way. These operations are defined similarly in P,..,[a,o], BV[a, oj and AC[a, b]. In f p and other spaces of sequences, the operations of addition and scalar multiplicatioll are defined as follows: Ll!t

x = (Xl,XZ,X3, ... ,x,,, ... ), y = (YI,Y2,Y3, ... ,Yn,) E 2

X

+Y

(Xl

+ YI, X2 + Y2, X3 + Y3, .

, X,..,

Y,.." )

ox=

(OX1,OX2,OX3,""OX,,, ... ),

is a scalar.

.n",

"'_'_""""'''''''10''-''

Appendix F

Fourier Analysis
Let f (t) be a periodic function with period T and Lebesgue int.egrable continuous functions having at most finite discontinuities over (- T /2, T {2). Then the Fourier series of f (t) is the trigonometric series

(F.I)
where

Ale = T Ao

21'1'/2 !(t)cosTdt,k = 1,2,3,'" 2r.kt

-'1'/2
'1'/2

(F.2) (F.3)
(FA)

T' {
L'I'/2

f(t)dt

Hk = T

21'1'/2 /(t)sin 211'kt dt,k=1,2,3, ... ,

-'1'/2

and

lye

write it as
(F.o)

Here,

lye

take
1V~'=T,k=O,1,2,3,

...

(F.6)

Ver}' often, we choose T = 21T. AI; and B~, are called cosine Fourier coefficient and sine Fourier coefficient, respectively. The set of ttiplex (A k , B k , w,,) where Ak,Bk,Wk are given by Equations (F.2), (F.3) and (F.6), respectively, is called the Fourier- series ft-eqtte7lcy content.

.n",

"'_'_""""'''''''10''-''

The complex form of the Fourier series of f(x) is

L Cn e21ti"t/T,
-=
where

== Co == Au
11

An+iB"
2

.n

>0

C ,,==
10'1

A" -iBn
n

2,n>U

==

T,n==-2,-1,O,I,2.

Let

T == 2TL, und 5" (J) (x) ==

1 "

" L
k=-n

Cke1b;

== "2Ao+ L(A~,coskx+Bksinkx)
k=1

be the n-th partial sum of the Fourier series of

5" (f)(x) = I
= -

f.

Then

T<

1/,'" f (t)
0

T<

1'"
0

fix -

t)

D n It) dt

(F.7)
D"(x - t)dt,

where I D n (x) == 2 is the "Dirichlpt kernel", and

an (f) =

+ ~ cos kx ==
~'=I

sin

(n + J-) x
25m:;
"

.;;

(F.B)

50 (f)

+ 5,

(f)

71+1

+ ... + 5" (f) = ~

TL10

r'" f (x

. t)]{" (t) dt,

(F.9)

where
Vo (x)

f{"

(x) ==

+ VI

lx) + ... n+1

+ On (x)
(F.IO)

is called tllf' "Fejer kernel".

Theorem F.l (Bessel's Inequality).

L
k=-o<

2 IC., 1 $ Ilfll~2(o,211") ,

0,.
1 , ') 2" Ao + " ( ' + Bn $ II f II' L An L2(O,211") .
=1

This also

mean.~

that {Ad and {Bd are elements of 2.

Theorem F.2 (Riesz-Fisher Theorem). Let {Cd E 2, Then lhe,"e e:l.lsts f E L 2 (-1T, 1T) s11m that {Cd is the k-tJl Fomie,' coefficient of f. HI,'thermo,-e,

L
k=-oo

lCd' ~ II!II~,(-.,.),

For {A k } , {BA-} belonging to 2, Ule,'C exists f E L 2 (0, 21T) s11m that AA" BA' are, nospectively, k-lh cosine and sine F01trie,' coefficients of f. ftwthen,IO'''e,

Theorem F.3

Let f E L z (-1T.1T), then

.~---+oo

lim

II! - S" (1)11,_2 (-u) = 0, ,

Theorem F.4

Let f E C[O,2r.] such thut

00,

/ /)

'. w(l t) =",'".<1t <

w (I, t) =

"'p I! (x + t) I.

! (x)1 '

Then the Pom'ter senes of f conve'yes 11fl1formly to f; that ts,

..---+00

lim

II! - S" (t)IIL~(_u) = O . '

Q,.,~,_ .._",,"l1>"-'

......

1fw (f,7}) = 0 (7t), then the condition 01 the the07'Cm holds.

Theorem F.5

111 is a f1tnction of b01tnded variation. then

S,,(f)-t I(xt-)~f(x )
whe1'C

flS

n-t

00,

I (x+)

h .....O+ h .....O+

lim Ilx,h)

I (x-)
exists ilt every x,a < x < b.

~ lim

Ilx-h)

Theorem F.6

Let 1 E L1(R), then tllP ser;,ps

il

L
k=-oo

l(x+21Tk) conve1yes

lllmost eve1ywhe1'C to coefficients

Ck

lunction .\(x) E L 1 (O.21T). Moreover'. the Founer"

Ck

of .\(x) are given by

I = -J(k).

2.

The Fourier Transform

Definition F.l (~OUriP.f Transform Then the function 1 defined by
Ilw) = I
in

L1(H)).

Let

E L,IR)

= 1= e-'w> I Ix) dx 211" _=

U (x)J
is used as the

is called the F01lrier" Translon/l of f. Very of~n, F notation for the Founer transform instead of I. It can be verified that

Theorem F.7

Let 1,9 E L) (R) and a,{3 E C. Then

Fiol

+ ~9)

~ aF(f)

+ Wig)

Theorem F.B The F011nef' tmnsform of an integmble function is a continlW11S function.

Theorem F.9 Jf f],h,'" ,In,eL l (RJ llndUf" -JilL, --t 0 then --+ j 11mformly on R.

llSn--+ 00,

1.,

Theorem F.ID (Riemann-Lebsegue Theorem).

Iwl--;.oo

IJ J E L] (R), then

lim

Ij(w)I~O

Th~orcm

F.11

Let fELl (R). Then (tmnsilltwn)

1. F{eiC.xJ(x)} =l(w-o:)

2. F (f (x - u)} ~ f(w)

,-'w"
(t>
0

(,hi!ting)
(scaling)

3. :F U (ax)} = l(~),

4. F(f(x)) ~F(f( xl)

(oonjugate)

Theorem F.12 1f f is a contin1lO11s piecewise dijJeJ'Cntiable function, f,/,EL1(R),llnd lim f(x)=O,thtm

Ixl--;.oo

F(f'}

~iwF(f).

Corollary F.I If J is a contimw11S function, n-time piecewise dijJe,'Cntiable, and f, /',. ,f(") E L1 (R), llnd
Ixl--;.oo

lim f(k) (x) =Ofmk=O, ,n-I,

then

'fi._.. . ~. . . . ~ __,

......

Definition F.2 Let f,9 E L] (R) then the COll"olution of denoted by I * 9 and is defmed by
FIfg)(x) ~
Me

and 9 is

[00
_<Xl

/(X-U)9(u)d".

V21T

Theorem F.13

Fo" 1,9 E L j (R) , F (f

*g) =

F (f) F (g) holds.

Theorem F.14 Let f b a contintlOus lunction on R vanishing outside a bounded intenJal. Then f E ~ (R) and

Definition F.3 (F01wier' Transform in L2(R). Let I E L2 (R) and {<p.,} be a sequence of continuous functions \",ith compact support convergent to I in L 2 (R); that is. III - lpnlJ[,,(R) --t O. The Fourier transform of I is defined by

= lim
,~oo

;p,,,

where the limit is with respect to the norm in L 2 (R) "

Theorem F.15

II IE L 2 (R), tlien

(i) (/,g)L2 = (!,Y)L2

(ii)

(PaJ'seval's IOf"1n1tla) (Planchen;llormula),

1I/1IT-2 =

II/11L 2

I ~I

In physical problems, the quantity II/11 L2 is a measure of energy while

[,2

represents the power spectrum of f.

Theorem F.16
1. LelIEL2(R), Then

~ I(w) = lim

,,~oo

v 211" -n

I . tn= [ " e-'WXI(x)dx.

...". '"'_'_""""'''''''10''-''

2. 111,9 E L 2 (R), then

l:

/(X)9(X)<lx

l:

1(x)9(x)dr.

Theorem F.17 (Inversion of the Fourier Thansform in 1... 2 (R)). Let I E L 2 (R). Then
/ (x) ~ lim
"->lXl

'"'
21T

11",'w,/
-fl

(w) dw,

where the convetyence is with n,speet to the norm in L'l (R) .

Corollary F.2
1,

JI IE L 1 (R) n

.). (R). then the equlliity

I (x) = -Iholds almost eVC1ywhen: in R.

.j2ir

1=

e1W ;>:[(w)tlw

-<XJ

2. F(F(J(x))) = f(-x) almost eve,ywhere m R

Theorem F.18 (Plancherel's Theorem). there exists [ E Lz (R) S1lch that:

1. I/ / E L, (RJ n L, (R). then /- (w)
~

For eve,y

E L'l (R),

I v21l"

= 1= ,-'w,/ (xl <Ix.

-<XJ

2. I (w) 3.

m= I V21l"

1"
L"

e-1W~

I (x) tlx

-n

/ (x) -

v21T

[" "w'itw) dw

5. The map

---t

I- ,.. .

aft ' . . o11let''Y 01 L'l (R) onto L'l (R) .

...... "'_'_""""'''''''10''-''

Theorem F.19

The Fourier tmTlSform is a unitary opemtm' on Lz(R).

Example F.2
1. If f(x) = (1 then
X z)

e-z~/'l = second derivative of a Gaussian function,

2. If the Shannon fUlIction is defined by

_sin 21TX - sin 1TX , f (xl nx then

J(w) ~ {

if

,J'iiiO'

1T

< Iwl < 21T,

otherwise .

",,,. '.'_'_""""'''''''10''-''

References
(11
ET Copson. Metric Spaces. Cambridge: Cambndge University Press, 1968 [21 [. Debnath, P Mikusinski. Hilbert Space Methods with Applications. San Diego. London. Boston: Academic Press. 1999. [31 RA DeVore, G Lorentz. Constructing Approximation. New York: Springer, 1993. [4] SA Douglas. Introduction to Mathematical Analysis. Reading, Massachusetts: Addison-Wesley Higher Mathematics, 1996.
[51 PR Halma:;. [\'[easure Theory, New Delhi: I'l'fad"fillan East-West Press Pvt Ltd, 1962.

[61 \V Kaplan. Advanced Calculus. 3rd ed. Reading: Addison Wesley, Inc., 1956. [7] JL Kelley. General Tbpology, Princeton: Van Nostrand, 1955. (81 WF Pfeffer. Integrals and Measure, New York, Basel: r-,'[arcel Dekker 1977. [9] RL Royden. Real Analysis. Ne\v York: MacMillan, 1988.

[101 GF Simmons. Introduction to Topology and 110dern Analysis. New York: McCraw-Hili, 1963.

",,,. "'_'_""""'''''''10''-''

Symbols and Abbreviations

( " -) ( )

1111
=} =}

3
~

d(', .)

-->

CBS

ViP NCP ONB

MRA

0"
a.e.

x
I

T'
6[X, Y!
x.1y

A.LB

Ai
ffi

a(, .) H' (!1) H"'(!1) DT(x)t ifF(x) DC T'(x)

of(x)

inlier product duality norm of a ,'ector implies equivalent exists for all Kronecker delta factorial distance between points or sets convergence in norm weak convergence Cauchy-Schwartz-Bunayakowski inequality variational inequality problem nonliear complementarity problem orthonormal basis multiresoluLion analysis unit impulse signal almost everywhere (except a set of measure zero) dual or adjoint space of a 1l00wed space adjoint of an upeator T identity operator space of all bounded linear operators on a uormed space X into another normed space Y orthogonality between elements x and y orthogonality between sets orthogonal complement direct sum bilinear fOITIl Sobole\' space of order J Sobole\' space of order m Gateau derivathe of operator T at x in the direction Frechet derh'<iti\'e at :.r subdifferentiul of functional F

",,,. "'_'_""""'''''''10''-''

V(U)

space of test fun ions

VfVg

=.

81

09

'"

"i=l

oX't OXi

Fouri r transform of