Matrix Algebra

As A Tool
Ali S. Hadi

Cornell University

An Alexander Kugushev Book

SUB GOttlngen
204 973 309

98A30508

Duxbury Press
An Imprint of Wadsworth Publishing Company
)P An International Thomson Publishing Company
Belmont Albany Bonn Boston Cincinnati? Detroit London Madrid Melbourne
Mexico City New York Paris San Francisco Singapore Tokyo Toronto Washington

Contents
Preface
1 . Introduction
1.1. Why Matrix Algebra?
-. . :
1.2. Some Definitions and Notation
1.2.1. What Is a Matrix?
1.2.2. Scalar Algebra and Matrix Algebra
1.2.3. Matrix Equality
1.2.4. Matrix Transpose
1.3. Special Types of Matrices
1.3.1. Square Matrices
1.3.2. Symmetric Matrices
1.3.3. Skew-Symmetric Matrices
1.3.4. Triangular Matrices
1.3.5. Diagonal Matrices
1.3.6. Identity Matrices
1.3.7. Unit Vectors
1.3.8. Matrices of Ones
1.3.9. Null Matrices
1.3.10. Constant Matrices
1.3.11. Partitioned Matrices
Exercises
-

....

1
1
2
2
4
5
5
6
6
6
7
7
8
8
8
8
9
9
9
10

2 . Some Matrix Calculations

2.1. Matrix Addition and Subtraction
2.2. Element-Wise Product
2.3. Matrix Inner Product
2.3.1. Definitions and Examples
2.3.2. Special Types of Matrix Products
2.3.3. Some Properties of Matrix Products
* 2.4. Kronecker Product
Exercises

13
13
14
15
15
18
21
23
26

3 . Linear Dependence and Independence

3.1. Introduction
3.2. Linear Dependence of Two Vectors
3.3. Linear Dependence of a Set of Vectors
3.4. Rank of a Matrix
3.5. Elementary Row Operations
3.6. Elementary Matrices
3.7. Elementary Column Operations
3.8. Permutation Matrices
Appendix: Reduction to Row-Echelon Form
v
Exercises

31
31
32
33
37
38
39
40
40
41
46
IX

CONTENTS

4 . Vector Geometry
4.1. Introduction
4.2. Algebraic and Graphic Views of Vectors
4.3. Geometric Views of Vectors
4.3.1. Length or Magnitude of a Vector
4.3.2. Angle Between Two Vectors
4.4. Vector Geometry and Linear Dependence
4.5. Geometry of Vector Algebra
4.5.1. Geometry of Vector Addition
4.5.2. Geometry of Vector Subtraction
4.5.3. Geometry of Multiplying a Vector by a Scalar
4.5.4. Geometry of Multiplying a Vector by a Matrix
4.6. Vector Space
Exercises

49
49
49
49
51
52
53
55
55
56
58
61
62
67

5 . Three Matrix Reductions

5.1. Trace,
5.2. Determinant
5.3. Vector Norms
Exercises

71
71
73
76
79

6 . Matrix Inversion
6.1. Element-Wise Division
6.2. The Regular Inverse
6.2.1. Definitions
6.2.2. Existence of Matrix Inverse
6.2.3. An Algorithm for Computing Matrix Inverse
6.2.4. Properties of Matrix Inverse
* 6.3. Generalized Inverse
6.3.1. Definitions . . . '.
6.3.2. Existence and Uniqueness of G-Inverse
6.3.3._An Algorithm for Computing a G-Inverse
6.3.4. Properties of G-Inverse
6.3.5. Particular Types of G-Inverse
6.3.6. Full-Rank Factorization
Appendix: Computing Matrix Inverse
Exercises

81
81
82
82
83
84
84
87
88
88
90
91
91
92
93
96

7 . Linear Transformation
7.1. Introduction
7.2. Definition
* 7.3. Orthogonal Rotation
* 7.4. Oblique Rotation
* 7.5. Orthogonal Projection
7.6. Singular and Nonsingular Transformations
Exercises
*

101
101
101
106
108
108
114
115

CONTENTS

xi

8 . Some Applications
8.1. Simultaneous Linear Equations
8.1.1. Definitions
8.1.2. Graphical Solution
8.1.3. Numerical Solution
8.1.4. The Case of Square Coefficient Matrix
8.1.5. Homogeneous Systems
8.2. Some Descriptive Statistics
8.2.1. The Mean Vector
8.2.2. The Variance-Covariance Matrix
8.2.3. The Correlation Matrix
* 8.3. Regression Analysis
8.3.1. Introduction
8.3.2. Least Squares Estimates
8.3.3. An Illustrative Example
8.3.4. The Fitted Values and Residual Vectors
* 8.4. Updating Computations
8.4.1. Adding Columns to a Data Matrix
8.4.2. Adding Rows to a Data Matrix
Appendix: Solving Systems of Linear Equations
Exercises

117
117
117
118
120
122
123
123
124
124
129
130
130
132
133
134
136
136
138
140
142

9 . Eigenvalues and Eigenvectors

9.1. Definition and Derivation
9.2. Some Properties
* 9.3. Singular Value Decomposition
Exercises

147
147
151
156
157

1 0 . Ellipsoids and Distances

10.1. Ellipsoids and Spheres
10.2. Distances
- 10.2.1. The Euclidean Distance
10.2.2. The Elliptical Distance
* 10.3. Smallest Ellipsoid Containing a Scatter of Points
Exercises

161
161
169
170
172
177
181

1 1 . Additional Applications
* 11.1. MatrixNorms
11.1.1. The Frobenius Norm
11.1.2. The Matrix P-Norm
* 11.2. Collinearity Diagnostics
11.3. Classification of Quadratic Forms
* 11.4. Markov Chains
* 11.5. Principal Component Analysis
* 11.6. Factor Analysis
Exercises

185
185
185
186
187
191
194
197
199
201

Bibliography

205

Index

207