4

Elementary Matrix Theory

We will be using matrices:

1.

For describing change of basis formulas.

2.

For describing how to compute the result of any given linear operation acting on a vector
(e.g., finding the components with respect to some basis of the force acting on an object
having some velocity.)

These are two completely different things. Do not confuse them even though the same computational apparatus (i.e., matrices) is used for both. For example, if you confuse rotating a vector
with using a basis constructed by rotating the original basis, you are likely to discover that your
computations have everything spinning backwards.
Throughout this set of notes, K , L , M and N are positive integers.

4.1

Basics

Our Basic Notation

A matrix A of size M N is simply
things,

a11
a
21

a
A =
31
..
.
a M1

a rectangular array with M rows and N columns of

a12 a13
a22 a23
a32 a33
..
..
.
.
a M2 a M3

..
.

a1N
a2N

a3N

..
.

aM N

As indicated, I will try to use bold face, upper case letters to denote matrices. We will use two
notations for the (i, j)th entry of A (i.e., the thing in the i th row and j th column):
(i, j)th entry of A = ai j = [A]i j

Until further notice, assume the thing in each entry is a scalar (i.e., a real or complex number).
Later, well use such things as functions, operators, and even other vectors and matrices.
9/22/2013

Chapter & Page: 42

Elementary Matrix Theory

The matrix A is a row matrix if and only if it consists of just one row, and B is a column
matrix if and only if it consists of just one column. In such cases we will normally simplify the
indices in the obvious way,



A = a1 a2 a3 a N

and

b1
b
2

b
B =
3
..
.
bN

Basic Algebra
Presumably, you are already acquainted with matrix equality, addition and multiplication. So all
well do here is express those concepts using our []i j notation:
Matrix Equality Two matrices A and B are equal (and we write A = B ) if and only if both
of the following hold:
(a)

A is the same size as B .

(b)

Letting M N be the size of A and B ,

[A] jk = [B] jk

for

j = 1... M

and

k = 1... N

Matrix Addition and Scalar Multiplication Assuming A and B are both MN matrices,
and and are scalars, then A + B is the M N matrix with entries
[A + B] jk = [A] jk + [B] jk

for

j = 1... M

and k = 1 . . . N

Matrix Multiplication Assuming A is a L M matrix and B is a M N matrix, their

product AB is the L N matrix with entries
[AB] jk = j th row of A times k th column of B

b1k
b
2k



b
= a j1 a j2 a j3 a j M
3k
..
.
b Mk

= a j1 b1k + a j2 b2k + a j3 b3k + + a j M b Mk

=

M
X

a jm bmk

M
X

[A] jm [B]mk

m=1

m=1

Basics

Chapter & Page: 43

If A is a row matrix, then so is AB and the above formula reduces to

[AB]k =

M
X

[A]m [B]mk

M
X

[A] jm [B]m

m=1

If, instead, B is a column matrix, then so is AB and the above formula reduces to
[AB] j =

m=1

Do remember that matrix multiplication is not commutative in general, but is associative.

That is, except for a few special cases,
AB 6= BA ,
but, as long as the sizes are appropriate, we can assume
A(BC) = (AB)C

Also, you cannot assume that if the product AB is a matrix of just zeros, then either A or B
must all be zeros.

?Exercise 4.1

a: Give an example where AB 6= BA .

b: Give an example of two nonzero matrices A and B for which the product AB is a matrix
of just zeros. ( A and B can have some zeros, just make sure that at least one entry in each
is nonzero.)

Inner Products of Matrices

Using the above definitions for addition and scalar multiplication, the set of all M N matrices
form a vector space of dimension M N . The default inner product for this vector space is the
natural extension of the inner product for C N ,
hA | Bi =

M X
N
X

a jk b jk =

j=1 k=1

a1
a
2

a
A =
3
..
.

and

hA | Bi =

X
j

version: 9/22/2013



B = b1 b2 b3 b N

b1
b
2

b
B =
3
..
.

bM

aM

then

[A] jk [B] jk

j=1 k=1

Note that, if A and B are both row matrices



A = a1 a2 a3 a N
and

or are both column matrices

M X
N
X

a j b j =

X
j

[A] j [B] j

Chapter & Page: 44

Elementary Matrix Theory

Conjugates, Transposes and Adjoints

In the following, assume A is a M N matrix and B is a N L matrix.

Complex Conjugates
The complex conjugate of A , denoted by A is simply the matrix obtained taking the complex
conjugate of each entry,
 


A mn = [A]mn
.
!Example 4.1:

If A =

"

2 + 3i
6i

5
4 + 4i

7 2i
8

then A =

"

2 3i
6i

5
4 4i

7 + 2i
8

We will say A is real if and only if all the entries of A are real, and we will say A is
imaginary if and only if all the entries of A are imaginary.
The following are easily verified, if not obvious:
1.

(AB) = A B .

2.

(A ) = A .

3.

A is real

4.

A is imaginary

A = A .
A = A .

Transposes
The transpose of the M N matrix A , denoted in these notes1 by AT , is the N M matrix
whose rows are the columns of A (or, equivalently, whose columns are the rows of A ),
 T
A mn = [A]nm .
!Example 4.2:

If A =

Also,
If

"

2 + 3i
6i

5
4 + 4i

7 2i
8

1 + 2i

|ai = 3i
5

1 but Arfken, Weber & Harris use e

A

then

6i

4 + 4i
8

2 + 3i

T
then A = 5
7 2i

|aiT = 1 + 2i

3i

Basics

Chapter & Page: 45

If you just think about what transposing a transpose means, it should be pretty obvious
that
AT

T

= A .

If you think a little bit about the role of rows and columns in matrix multiplication, then you may
not be surprised by the fact that
(AB)T = BT AT .
This is easily proven using our [] jk notation:
M
X


T
[A]km [B]m j
(AB) jk = [AB]k j =
m=1

M
X
 T  T
A mk B jm

m=1

M
X
 T  T


B jm A mk = BT AT jk

m=1

showing that

(AB)T

jk



= BT AT jk

for every

( j, k)

Hence, (AB)T = BT AT as claimed above.

Two more terms you may recall:
A is symmetric

AT = A

A is antisymmetric

AT = A

[A] jk = [A]k j

And

[A] jk = [A]k j

Note that, to be symmetric or antisymmetric, the matrix must have the same number of rows as
it has columns (i.e., it must be square).
You may recall from your undergraduate linear algebra days that there is a nice theory
concerning the eigenvalues and eigenvectors of real symmetric matrices. Well extend that
theory later.

Adjoints
Combining the transpose with complex conjugation yields the adjoint.2
The adjoint of the MN matrix A , denoted by either adj(A) or A , is the N M matrix
A = AT
Using the [] jk notation,

jk

= A

= [A]k j

T

2 The adjoint discussed here is sometimes called the operator adjoint. You may also find reference to the classical

adjoint, which is something totally different, and is of no interest to us at all.

version: 9/22/2013

Chapter & Page: 46

Elementary Matrix Theory

!Example 4.3:

If A =

Also,
If

"

2 + 3i
6i

5
4 + 4i

7 2i
8

1 + 2i

|ai = 3i
5

then

6i

4 4i
8

2 3i

then A = 5
7 + 2i

|ai = 1 2i

3i

From what weve already discussed regarding complex conjugates and transposes, we immediately get the following facts:


1. A = A .
2.

(AB) = B A .

3.

A = AT

4.

A = AT

A is real .
A is imaginary .

Any matrix that satisfies A = A is said to be either self adjoint or Hermitian, depending
on the mood of the speaker. Such matrices are the complex analogs of symmetric real matrices
and will be of great interest later.
Along the same lines, any matrix that satisfies A = A is said to be antiHermitian. I
suppose you could also call them anti-self adjoint (self anti-adjoint?) though that is not
commonly done.

4.2

Extending the Bra Ket Notation

Using an Orthonormal Basis

Assume we have an N -dimensional vector space V with orthonormal basis
B = { e1 , e2 , . . . , e N }

Remember, for any vector v in this space,

|vi = |viB = column matrix of components of v w.r.t. B

v1
N
v2
X

= .
where v =
vjej .
..
j=1

vN

We now define hv| to be the adjoint of |vi ,



hv| = hv|B = |vi = v1 v2 v N

Extending the Bra Ket Notation

Chapter & Page: 47

Observe that
A |vi
Also note that

hv| |wi = v1 v2

= hv| A

w1
N
X
 w2

vN . =
v j w j
..
j=1

wN

which, since we are assuming an orthonormal basis, reduces to

hv| |wi = h v | w i
| {z }
| {z }

vector
inner
product

matrix
product

The h| is the bra in the (Dirac) bra-ket notation.

Using an Arbitrary Basis

Now let

B = { b1 , b2 , . . . , b N }

be any basis for our vector space V , and let

D = { d1 , d2 , . . . , d N }

be the corresponding reciprocal basis. Recall from section 2.6 that

hv | wi =

N
X

vk wk

k=1

where

v1
v2

.. = |viD
.

and

v N

In terms of matrix operations, the above is

hv | wi =

N
X
k=1


vk wk = v1 v2

w1
w2

.. = |wiB
.

wN

w1


w2
v N . = |viD |wiB
..
wN

Because of this, the appropriate general definition of bra v is

hv| = hv|B = |viD

where D is the reciprocal basis corresponding to B . That way, we still have

hv| |wi = h v | w i
| {z }
| {z }
matrix
product

vector
inner
product

This material is optional. It requires the reciprocal basis from section 2.6.

version: 9/22/2013

Chapter & Page: 48

4.3

Elementary Matrix Theory

Square Matrices

For the most part, the only matrices well have much to do with (other than row or column
matrices) are square matrices.

The Zero, Identity and Inverse Matrices

A square matrix is any matrix having the same number of rows as columns. Two important
N N matrices are
The zero matrix, denoted by O or O N , which is simply the N N matrix whose entries are
all zero,
[O] jk = 0
for all ( j, k) .
The identity matrix, denoted by I or I N or 1 (Afkin, Weber & Harriss notation), which is
the matrix whose entries are all zero except for 1s on the major diagonal,
[I] jk = jk

Recall that, if A is any N N matrix, then

A + O = A
!Example 4.4:

AO = OA = O

AI = IA = A .

and

The 33 zero and identity matrices are

0 0 0
1 0 0

O = 0 0 0
and
I = 0 1 0
0 0 0
0 0 1

The inverse of a (square) matrix A is the matrix, denoted by A1 , such that

AA1 = A1 A = I

Not all square matrices have inverses. If A has an inverse, it is said to be invertible or nonsingular.
If A does not have an inverse, we call it noninvertible or singular.
It may later be worth noting that, while the definition of the inverse requires that
AA1 = I

and

A1 A = I

it can easily be shown that, if B is any square matrix such that

AB = I

or

BA = I

then
AB = I

and

BA = I

and thus, B = A1 .
Here are few other things you should recall about inverses (assume A is an invertible NN
matrix):

Determinants

Chapter & Page: 49

1.

AB = C

B = A1 C .

2.

BA = C

1
A1
= A.

B = CA1 .

3.
4.

If B is also an invertible N N matrix, then

(AB)1 = B1 A1

?Exercise 4.2:

4.4

Verify anything above that is not immediately obvious to you.

Determinants

Definition, Formulas and Cramers Rule

Recall that each N N matrix has an associated scalar called the determinant of that matrix.
We will denote the determinant of

a11 a12 a1N

a

21 a22 a2N
A =
..
..
..
..

.
.
.
.
aN 1 aN 2 aN N

by

det(A)

or

a11 a12 a1N

a

21 a22 a2N

det .
..
..
..

.
.
.
.
.
aN 1 aN 2 aN N



 a11 a12 a1N 



a
a

a
 21
22
2N 
 .
..
.. 
..
 .
.
 .
.
. 


a N 1 a N 2 a N N 

or

as seems convenient.
The determinant naturally arises when solving a system of N linear equations for N
unknowns
a11 x1 + a12 x2 + + a1N x N = b1
a21 x1 + a22 x2 + + a2N x N = b2
..
.

(4.1a)

a N 1 x1 + a N 2 x2 + + a N N x N = b N
where the a jk s and bk s are known values and the x j s are the unknowns. Do observe that we
can write this system more concisely in matrix form,
Ax = b
version: 9/22/2013

(4.1b)

Chapter & Page: 410

Elementary Matrix Theory

where A is the N N matrix of coefficients, and x and b are the column matrices


x1
b1
x
b
2
2

x =
and
b =
..
.. .
.
.
xN
bN

In solving this, you get (trust me)

det(A) xk = det(Bk )

for k = 1, 2, . . . N

(4.2)

where Bk is the matrix obtained by replacing the k th column in A with b . We can then find
each xk by dividing through by det(A) , provided det(A) 6= 0 . (Equation (4.2) is Cramers
rule for solving system (4.1a) or (4.1b), written the way it should be but almost never is
written.)
?Exercise 4.3:
the above?

What can you say about the possible values for the xk s if det(A) = 0 in

In theory, the definition of and formula for the determinant of any square matrix can be based
on equation (4.2) as a solution to system (4.1a) or (4.1b). (Clever choices for the b vectors may
help.) Instead, we will follow standard practice: Ill give you the formulas and tell you to trust
me. (And if you remember how to compute determinants as I rather expect just skip ahead
to Properties and Applications of Determinants.)
For N = 1 and N = 2 , we have the well-known formulas


a

a
 11 12 
det[a11 ] = a11
and

 = a11 a22 a12 a21 .
a21 a22 
For N > 2 , the formula for the determinant of an

a11 a12
a
21 a22
A =
..
..
.
.
aN 1 aN 2

N N matrix

a1N
a2N

.
..
..
.

aN N

can be given in terms of the determinants of (N 1) (N 1) submatrices via either the

expansion by minors of the first row
det A =

N
X
(1)k1 a1k det(A1k )
k=1

or the expansion by minors of the first column

det A =

N
X


(1) j1 a j1 det A j1
j=1

where A jk is the (N 1)(N 1) matrix obtained from A by deleting the j th row and k th
column.

Determinants

Chapter & Page: 411

Either of the above formulas gives you the determinant of A . If you are careful with your
signs, you can find the determinant via an expansion using any row or column. If you are
interested3, here is a most general formula for the determinant for the above NN matrix A :
det(A) =

N X
N
X

i 1 =1 i 2 =1

N
X

(i 1 , i 2 , . . . , i N ) ai1 1 ai2 2 ai N N

i N =1

where is a function of all N -tuples of the integers from 1 to N satisfying all the following:
1.

If any two of the entries in (i 1 , i 2 , . . . , i N ) are the same, then

(i 1 , i 2 , . . . , i N ) = 0

2.

If i 1 = 1 , i 2 = 2 , i 3 = 3 , etc., then
(i 1 , i 2 , i 3 . . . , i N ) = (1, 2, 3, . . . , N ) = 1

3.

Suppose ( j1 , j2 , . . . , j N ) is obtained from (i 1 , i 2 , . . . , i N ) by simply switching just two

adjacent entries in (i 1 , i 2 , . . . , i N ) . In other words, for some integer K with 1 K <
N,

if k 6= K or k 6= K + 1

ik
jk =

Then

i K +1

i
K

if k = K

if k = K + 1

( j1 , j2 , j3 . . . , j N ) = (i 1 , i 2 , i 3 . . . , i N )

Properties and Applications of Determinants

Let me remind you of many of the properties and a few of the applications of determinants.
In many ways, you can consider Cramers rule (equation (4.2)) for solving system (4.1a) or,
equivalently, matrix equation (4.1b) as a basic defining equation for the determinant. Cramers
rule tells us that system (4.1a) can be solved for every choice of the b j s if and only if det(A) 6= 0 .
But the ability to solve that system is equivalent to the ability to solve the matrix equation
Ax = b
for every choice of b , and since this matrix equation is solvable for every choice of b if and
only if A is invertible, we must have that:
An N N matrix A is invertible

det(A) 6= 0

This is doubtlessly our favorite test for determining if A1 exists. It will also probably be the
most important fact about determinants that we will use.
Another property that can be derived from Cramers rule is that, if A and B are both NN
matrices, then
det(AB) = det(A) det(B) .
(4.3)
3 If you are not interested, skip to Properties and Applications of Determinants below.

version: 9/22/2013

Chapter & Page: 412

Elementary Matrix Theory

(Just consider using Cramers rule to solve (AB)x = b , rewritten as A(Bx) = b .)

Now it is real easy to use the formulas (or even Cramers rule) to verify that

1
0

0
det(I) = det

..
.
0

0
1
0
..
.
0

0
0
1
..
.
0

0
0

0 = 1

. . ..
. .
1

From this and the observation regarding determinants of products you can easily do the next
exercise.
?Exercise 4.4:

Assume A is invertible, and show that

det A1

1
det(A)

A few other properties of the determinant are listed here. Assume A is an N N matrix
and is some scalar.
1.
2.
3.
4.

det(A)


det A


det AT


det A

= N det(A) .
= det(A) .
= det(A) .
= det(A) .

?Exercise 4.5: Convince yourself of the validity of each of the above statements. (For the
first, det(A) = N det(A) , you might first show that det(I) = N , and then consider
rewriting A as IA and taking the determinant of that.)
Also remind yourself of the computational facts regarding determinants listed on page 87
of Arfken, Weber & Harris.
Finally, let me comment on the importance of Cramers rule: It can be viewed as being
very important, theoretically, for the way it relates determinants to to solving of linear systems.
However, as a practical tool for solving these systems, it is quite unimportant. The number of
computations required to find all the determinants is horrendous, especially if the system is large.
Other methods, such as the Gauss elimination/row reduction learned in your undergraduate linear
algebra course are much faster and easier to use.