Mathematics

Part 1: Review of Matrices & Elements of Linear Algebra

Part 2: Multivariable Calculus
Review of Matrix Algebra
What is a Matrix?
A matrix is a rectangular array of numbers.
A =

1 2 0
9 1 2
1 3 4

; M =

1 2 5 7
4 1 0 8

; v =

1
3
2
6

Order of a matrix
A matrix has order n m if it has n rows and m columns.
Vectors
Denition
A (1 m) matrix is called as a row vector
v =

1 3 2 6

A (m1) matrix is called as a column vector
v =

1
3
2
6

By default, when we talk about a vector we mean a column vector.

Matrices
The elements of a matrix A are often written a
ij
where
i is the row number of the element
j is the column number of the element
Example
A =

1 2 0
9 1 2
1 3 4

a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33

Matrices
A general mn matrix will often be written
A =

a
11
a
12
a
1n
a
21
a
22
a
2n
.
.
.
.
.
.
.
.
.
.
.
.
a
m1
a
m2
a
mn

Notations
Square or round brackets. Dierent notations... same meaning.
A =

1 2 0
9 1 2
1 3 4

A =

1 2 0
9 1 2
1 3 4

Recall
the following notation has a dierent meaning (determinant):
det(A) =

1 2 0
9 1 2
1 3 4

Matrices
An obvious fact: Two matrices are said to be equal if they have
the same order and all their elements are equal.
A =

1 2 0
9 1 2
1 3 4

; B =

1 2 0
9 1 2
1 5 4

; C =

1 2 0 1
9 1 2 3
1 3 4 6

A = B and A = C.
Types of Matrices
An upper triangular matrix has the form

a
11
a
12
a
1n
0 a
22
a
2n
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a
nn

An lower triangular matrix has the form

b
11
0 0
b
21
b
22
0
.
.
.
.
.
.
.
.
.
.
.
.
b
n1
b
n2
b
nn

An diagonal matrix has the form

d
11
0 0
0 d
22
0
.
.
.
.
.
.
.
.
.
.
.
.
0 0 d
nn

All these matrices are square.

Matrix addition and subtraction
Two matrices may be added or subtracted if they have the same
order.

2 1
1 0
1 3

1 0
2 1
3 1

3 1
3 1
2 4

1 3 6 1
1 1 2 1

1 1 1 1
1 1 1 1

0 2 5 0
2 0 1 0

2 1
1 0
1 3

1 3 6 1
1 1 2 1

not dened
Matrix addition and subtraction
Matrices are commutative and associative under addition:
A + B = B + A; A + (B + C) = (A + B) + C
A zero matrix is any matrix with all elements equal to zero,and is
usually written 0:

0 0
0 0
0 0

0 0 0
0 0 0
0 0 0

A + 0 = 0 + A = A
Matrix Multiplication
It is straightforward to multiply matrices by a number k:
k

1 2 0
9 1 2
1 3 4

k 2k 0
9k k 2k
k 3k 4k

Multiplying two matrices together is more complicated.

1 2 0
0 1 2
1 3 4

1 0 1
1 1 3
2 0 2

=?
Matrix Multiplication
We dene the product of a row vector and a column vector as
being the sum of the products of their components. e.g:

1 2 0

1
1
2

= 1 1 +2 1 + 0 2 = 1
To multiply two matrices we multiply the rows of the rst one by
the columns of the second one.
Matrix Multiplication

1 2 0
0 1 2
1 3 4

1 0 1
1 1 3
2 0 2

Matrix Multiplication

1 2 0
0 1 2
1 3 4

1 0 1
1 1 3
2 0 2

1 2

Matrix Multiplication

1 2 0
0 1 2
1 3 4

1 0 1
1 1 3
2 0 2

1 2 5

Matrix Multiplication

1 2 0
0 1 2
1 3 4

1 0 1
1 1 3
2 0 2

1 2 5
5

and so forth. . .
Matrix Multiplication

1 2 0
0 1 2
1 3 4

1 0 1
1 1 3
2 0 2

1 2 5
5 1 7
12 3 18

Rule for matrix multiplication:

The number of columns of the rst matrix
must equal the number of rows of the
second.
Matrix Multiplication
When you multiply 2 matrices the order matters
Matrix multiplication is associative
A(BC) = (AB)C
but is non-commutative, i.e., in general
AB = BA
Matrix Multiplication
Because in general AB = BA, we must always specify on which
side we are doing the matrix multiplication:
X =

2 0
1 1

A =

1 0
3 1

Multiplying X on the left by A:

AX =

1 0
3 1

2 0
1 1

2 0
5 1

Multiplying X on the right by A:

XA =

2 0
1 1

1 0
3 1

2 0
4 1

Matrix Multiplication
Matrix multiplication has some other strange properties:
AB = 0 does not necessarily mean that A = 0 or B = 0

1 1
0 0

2 0
2 0

0 0
0 0

AD = AC does not necessarily mean that D = C

Matrix Multiplication
The product of two matrices is a matrix whose elements can be
written in a compact form as:
[AB]
ij
=
N

k=1
a
ik
b
kj
where N is the number of columns of A and the number of rows
of B.
Identity Matrix
The identity matrix I is the matrix with the property
AI = IA = A
I =

1 0
0 1

for 2 2 matrices
I =

1 0 0
0 1 0
0 0 1

for 3 3 matrices
Inverse of a Matrix
The inverse of a square matrix A is written A
1
and has the
property
A
1
A = AA
1
= I
A matrix only has one inverse
This means that if it exists, the inverse of a matrix is unique.
Inverse of a Matrix
Inverse of a 2 2 matrix.
if A =

a b
c d

, A
1
=
1
ad bc

d b
c a

Remarks:
ad bc is the determinant of the matrix.
The inverse is dened only if det(A) = ad bc = 0.
If det(A) = 0, A is said to be a singular matrix.
For larger square matrices there is a more general way to
compute inverses. Well see that shortly.
The sum of the diagonal elements of A is called the trace of
A: tr(A) = a + d.
Transpose of a Matrix
The transpose of a matrix is obtained by interchanging the rows
and the columns:
If A =

1 3 6 1
1 1 2 1

, A
T
=

1 1
3 1
6 2
1 1

If A and B have the same order

(A + B)
T
= A
T
+ B
T
(A
T
)
T
= A
(AB)
T
= B
T
A
T
Matrix Algebra
Using the rules for addition, subtraction, multiplication and
inverses, as well as the special matrices I and 0, we can re-arrange
matrix equations.
Example
Given
A =

1 0
2 1

; B =

1 2
2 1

; C =

0 0
0 1

Rearange the equation

A + 2BX = C
to nd X.
Matrix Algebra
Formally we have
A + 2BX = C
2BX = C A
BX =
1
2
(C A)
X =
1
2
B
1
(C A) (multiply on the left)
B =

1 2
2 1

B
1
=
1
1 4

1 2
2 1

=
1
5

1 2
2 1

C A =

1 0
2 0

X =
1
10

1 2
2 1

1 0
2 0

=
1
10

3 0
4 0

Matrix Algebra
Example
Given
B =

1 1
1 1

; C =

4 2
0 1

Rearange the equation

X + XB = C
to nd X.
Matrix Algebra
X + XB = C
X(I + B) = C
X = C(I + B)
1
(not (I + B)
1
C)
I + B =

2 1
1 2

(I + B)
1
=
1
3

2 1
1 2

X =
1
3

4 2
0 1

2 1
1 2

=
1
3

6 0
1 2

Linear Systems
Linear system of equations
A system of equations is a set of mathematical statements
containing several unknown quantities.
If unknowns can be found so that all the statements are
satised then we say that the system has a solution.
There are three possibilities when trying to nd the solution:
1
There is too little information to nd the unknowns.
2
There is too much information to nd the unknowns.
3
There is exactly the right amount of information.
Linear system of equations
Example
From the two statements
1
There are twice as many sheep as people in Australia.
2
There are 20 million more sheep than people in Australia.
we can construct the system of equations

s = 2p
s = p + 20
p = 20, s = 40 (in millions)
Unique solution
Linear system of equations
Example
From the two statements
1
There are 20 million more sheep than people in Australia
2
There is the same number of sheep as people in Australia
we can construct the system of equations

s = p + 20
s = p
No solution The two statements are inconsistent
Linear system of equations Graphical interpretation
Linear system of equations
An equation for several unknowns x
1
, x
2
. . . x
n
is linear if it can be
put in the form
a
1
x
1
+ a
2
x
2
+ + a
n
x
n
= b
where the coecients a
1
. . . a
n
and b are constants.
A set of linear equations is known as a linear system.
Linear system of equations
A linear system of m equations in n unknowns is a set of equations
of the form

a
11
x
1
+ a
12
x
2
+ + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ + a
2n
x
n
= b
2
.
.
.
a
m1
x
1
+ a
m2
x
2
+ + a
mn
x
n
= b
m
A solution is a set of numbers (x
1
, x
2
, . . . , x
n
) that satises
all m equations.
This set of numbers can be gathered into a vector.
Linear system of equations
The augmented matrix associated with the system is

a
11
a
12
a
1n
b
1
a
21
a
22
a
2n
b
2
.
.
.
.
.
.
.
.
.
.
.
.
a
m1
a
m2
a
mn
b
m

A system with zeros below the main diagonal is said to be in

row-echelon form.
A system in row-echelon form can be solved easily by
back-substitution.
Linear system of equations
Example

1 2 1 11
0 3 1 25
0 0 2 70

x
1
+ 2x
2
+ x
3
= 11
3x
2
+ x
3
= 25
2x
3
= 70
From the third row we get: 2x
3
= 70 x
3
= 35
From row 2 we get: 3x
2
+ x
3
= 25 x
2
=
10
3
Finally we get: x
1
+ 2x
2
+ x
3
= 11
x
1
+
20
3
+ 35 = 11 x
1
=
92
3
Row-echelon form
A matrix is in row echelon form if
All rows that contain only zeros are grouped at the bottom of
the matrix.
In any two consecutive non-zero rows, the leftmost non-zero
entry in the lower row occurs farther to the right than the
leftmost nonzero entry in the upper row.
We call a row (column) nonzero if it contains at least one nonzero
entry.
Row-echelon form
A matrix in row-echelon form has a staircase pattern like the
following matrix:

0
0 0 0 0
0 0 0 0 0 0 0

where the greek letters correspond to non-zero values and the stars
may or may not be zero.
For the augmented matrix in row echelon form, each leftmost
nonzero entry corresponds to a basic variable.
For instance, for the matrix above the basic variables are x
1
,
x
2
and x
5
.
Row-echelon form
The row-echelon form can give us some important information
about a system. If a system is consistent (i.e., if it has a solution)
if all variables are basic variables the solution is unique.
if there is at least one non-basic variable there are innitely
many solutions.
Row-echelon form
The number of solutions (unique, none, innitely many) can be
found by looking at the augmented matrix in row-echelon form:

a
11
a
12
a
1n
b
1
a
21
a
22
a
2n
b
2
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a
mn
b
m

unique solution if a
11
, a
22
, . . . , a
mn
= 0

a
11
a
12
a
1n
b
1
a
21
a
22
a
2n
b
2
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 b
m

no solution if b
m
= 0

a
11
a
12
a
1n
b
1
a
21
a
22
a
2n
b
2
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 0

innitely many solutions

The number of non-zero rows in row-echelon form is the rank of the
matrix.
Gauss elimination method for solving a linear system of
equations
Any system of equations is left unchanged by:
1
Multiplying a row by a non-zero scalar.
2
Adding or subtracting two rows.
3
Swapping any two rows.
These are known as row operations.
In Gauss Elimination, we use row-operations to change the
system to row-echelon form.
We can then solve the system by back-substitution.
Gauss elimination
Use Gauss elimination to solve the system
x
1
+ x
2
+ 2x
3
= 2
3x
1
x
2
+ x
3
= 6
x
1
+ 3x
2
+ 4x
3
= 4
The augmented matrix is

1 1 2 2
3 1 1 6
1 3 4 4

We need to use row operations to put the augmented matrix in

row-echelon form.
Gauss elimination

1 1 2 2
3 1 1 6
1 3 4 4

Example of a linear system: Balancing a chemical reaction

Consider the following chemical reaction:
C
2
H
6
+ O
2
CO
2
+ H
2
O
Balancing the reaction means nding positive integers n
1
, n
2
, n
3
,
and n
4
such that
n
1
C
2
H
6
+ n
2
O
2
n
3
CO
2
+ n
4
H
2
O
has the same number of atoms of each element on each side of the
equation.
We can cast the problem as a linear system.
Example of a linear system: Balancing a chemical reaction
n
1
C
2
H
6
+ n
2
O
2
n
3
CO
2
+ n
4
H
2
O
Inverse of a Matrix
We saw how to nd the inverse of a 2 2 matrix.
If we can nd the inverse of a n n matrix we could solve a
general system
AX = b X = A
1
b
Inverse of a 3 3 Matrix. Method 1: row operations
1
Write the LHS of the system in matrix form.

1 2 2
1 2 1
3 2 1

2
Augment the matrix with the identity matrix.

1 2 2 1 0 0
1 2 1 0 1 0
3 2 1 0 0 1

3
Use row operations to transform the left half into the identity.
The right half will then be the inverse.
Inverse of a 3 3 Matrix. Method 1: row operations
Example
Solve

x
1
2x
2
+ 2x
3
= 1
x
1
2x
2
+ x
3
= 1
3x
1
2x
2
+ x
3
= 1
by computing the inverse of the matrix associated with the linear
system.
Inverse of a 3 3 Matrix. Method 1: row operations
The augmented matrix is

1 2 2 1 0 0
1 2 1 0 1 0
3 2 1 0 0 1

R
1
R
1
2R
3
R
3
R
1
R
2

5 2 0 1 0 2
1 2 1 0 1 0
0 0 1 1 1 0

R
2
R
2
R
3

5 2 0 1 0 2
1 2 0 1 2 0
0 0 1 1 1 0

R
1
R
1
+ R
2
R
2
5R
2
+ R
1

4 0 0 0 2 2
0 8 0 4 10 2
0 0 1 1 1 0

R
1
R
1
/4
R
2
R
2
/8

1 0 0 0 1/2 1/2
0 1 0 1/2 5/4 1/4
0 0 1 1 1 0

Inverse of a 3 3 Matrix. Method 1: row operations

Therefore the inverse is

0 1/2 1/2
1/2 5/4 1/4
1 1 0

and the solution of the system is

x
1
x
2
x
3

0 1/2 1/2
1/2 5/4 1/4
1 1 0

1
1
1

0
1/2
0

Cramers rule for solving a linear system

Given the system

a
11
a
12
a
21
a
22

x
1
x
2

b
1
b
2

the solution can be expressed in terms of determinants:

x
1
=

b
1
a
12
b
2
a
22

a
11
a
12
a
21
a
22

; x
2
=

a
11
b
1
a
21
b
2

a
11
a
12
a
21
a
22

Of course this requires that the determinant of the system is

non-zero.
Cramers rule for solving a linear system
Cramers rule works for higher order systems:

a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33

x
1
x
2
x
3

b
1
b
2
b
3

x
1
=

b
1
a
12
a
13
b
2
a
22
a
23
b
3
a
32
a
33

a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33

; x
2
=

a
11
b
1
a
13
a
21
b
2
a
23
a
31
b
3
a
33

a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33

; x
3
=

a
11
a
12
b
1
a
21
a
22
b
2
a
31
a
32
b
3

a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33

But how do we compute a general determinant?

Determinant
There are two (main) methods for calculating the determinant of a
n n matrix:
1
Cofactor expansion.
2
Row operations.
Determinant Cofactor expansion
Steps to expand using cofactors with respect to a given row:
1
Multiply each coecients of this row by (1)
i+j
where i is its
row number and j its column number.
2
Multiply each coecients of this row with the determinant of
the minor matrix i.e., the matrix obtained by deleting the
coecients row and column.

a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33

= a
11

a
22
a
23
a
32
a
33

a
12

a
21
a
23
a
31
a
33

+ a
13

a
21
a
22
a
31
a
32

The cofactors are the signed determinants of the small matrices.

You can perform the cofactor expansion with respect to any row or
column.
Determinant Cofactor expansion
Calculate the determinant of

1 2 2
1 2 1
3 2 1

Inverse of a 3 3 Matrix. Method 2: Cofactors

1
Calculate the determinant of the matrix det(A).
2
If det(A) = 0 the inverse is given by
A
1
=
1
det(A)
C
T
where C is the matrix of the cofactors of A.
Determinant Row operations
Interchanging of two rows multiplies the determinant by -1.
Adding a multiple of one row to another leaves the
determinant unchanged.
Multiplying an entire row by a number k multiplies the
determinant by k.
These rules also hold for column operations.
Determinant Useful tips
Things to look for before trying to calculate a determinant
If any row or column contains only zeros then the determinant
is zero.
If any two rows or columns are identical then the determinant
is zero.
If any two rows or columns are multiples of each other then
the determinant is zero.
Determinant Special case
To nd the determinant of a matrix in row-echelon form, multiply
down the diagonal

1 2 3
0 5 2
0 0 1

= 1 5 1 = 5
General properties of determinant
det(A + B) = det(A)+det(B).
det(AB) = det(A)det(B).
det(A)det(A
1
)=1.
A has an inverse if and only if det(A) = 0.
Steps to evaluate determinants using row reduction:
1
Reduce the determinant to row-echelon form (keeping track of
row-swaps and multiplication by numbers).
2
Multiply the coecients along the diagonal.
Linear Systems Summary
We have seen 3 methods to solve a linear system
1
Gauss elimination.
2
Using the inverse matrix.
3
Cramers rule.