Linear systems graphically

Given

the linear system:

The solution is the intersection of both lines

Singular matrices
Singular matrices do not have a unique
solution
Parallel planes indicate that the system has no
solution
Overlapping planes result in infinite solutions

Matrices with multiple solutions or no solutions

Statistically these are rare (if you generate random
matrices)

no solution

infinite solutions

Ill-conditioned systems
Planes that almost overlap are almost singular
are ill conditioned
Small change in one or more coefficients results
in large changes in the solution
This results in large changes due to small errors
noise, roundof

ill conditioned

Ill-conditioned systems

Determinants
Notation:

Useful properties:

, where is an matrix
A linear system has a unique solution if and only if

Is a linear system with small ill-conditioned?

Not necessarily. but is completely solvable.

Calculating determinants
Determinant of a 2x2 matrix:

Determinant of a 3x3 matrix:

Calculating determinants
The determinant of an upper triangular matrix is
the product of the diagonal elements:

Implies that we can calculate the determinant

using GE
Additional rule:
Swapping two rows introduces a multiple of

Calculate using Gaussian elimination.

LU Decomposition
Generalized
form of Gaussian Elimination

Given the matrix:

Multiplying by the identity matrix yields :

The first step of GE is equivalent to:

Demonstrate that this can be reversed

using positive factors.

LU Decomposition
(continued)
The first step of GE is equivalent to:

Step 2:

LU Decomposition:

LU Decomposition
(continued)

factors
used
for GE
Lower
matrix

Upper
matrix

Perform LU decomposition on and verify.

LU Decomposition
(examples)
Perform LU decomposition on the following
matrices:

LU with Partial Pivoting

LU decomposition of a matrix gives:

Applying partial pivoting adds a permutation

matrix :

where is a sparse matrix of 1s:

Features of LU
Decomposition
Provides

a way to evaluate multiple right-handside vectors.

We can formulate the linear system
by using LU factorization:
This can be broken down into two triangular
systems:

Use forward substitution to calculate

Solving for multiple values

of
Solve the linear system for two values of :
and
From the previous slides, we know

and

Solve for using LU

factorization
Solve
the following systems for

based on your previous decomposition:

Matrix Inversion
For some applications, it may be necessary to
compute a matrix inverse
Computer graphics, image processing, multiple-input
multiple-output (MIMO) communication
Determining of a linear system is ill-conditioned

LU factorization can be used to compute inverse

matrices

Computing the Matrix

Inverse
Compute the matrix inverse column-by-column
using unit vectors
For the linear system :
if , the solution will be the first column of
use a corresponding unit vector to solve for each
column

Compute for and verify for

Matrix Inversion (example)

Compute
the inverse matrices and verify on any vector:

based on your previous decomposition:

Matrix and vector norms

A norm is a value that provides some measure
of the size or length of a vector or matrix
The norm of a vector or matrix is expressed as:
or
Often a subscript is used to specify the type of
norm.
The Euclidean norm is common:

Compute the Euclidean norm for

Types of matrix norms

Euclidean norm:

1-norm:

sum the magnitude of all values in each column, taking the largest
value as the norm

uniform matrix norm:

sum the magnitude of all values in each row, taking the largest
value as the norm

Matrix Norms (example)

Compute
, , and for

Compute , , and and note the magnitudes for


-Norm
is known as the spectral norm
Specifically:
Largest singular value of :
where is the largest eigenvalue of
(a.k.a. the spectral radius of )

Properties of matrix norms

if

where is the identity matrix

where is a vector

System conditioning
Ill-conditioned matrices are highly sensitive
Small changes in input create large changes in output
Small changes include noise and roundof!

Ill-conditioned systems are difficult to solve

computationally
Best you can really do is be aware of them

ill conditioned

Matrix condition number

How much the output value changes with
respect to the input can be quantified by the
matrix condition number:

The larger the condition number, the more illconditioned the system

Condition number derived

Assume
the linear system

where there is some perturbation in

this causes a perturbation in

Show that large is bounded by small :

Conditioning and loss of

precision
If then we expect to lose digits of precision
If we know the coefficients of to -digit precision,
and , then the result is accurate to
Specifically referring to the -norm but remember
that so a -digit precision loss is a conservative
estimate

Matrix condition number

(cont.)
Compute for and

Approximately 3 digits of precision are lost

Condition number
(examples)
Compute
for and calculate the precision loss.

1. LU decomposition
2. Compute the inverse
a. Column 1: , ; ,
b. Column 2: , ; ,
c. Assemble the inverse:

3. Compute the condition number

a.
b.
c.

approximately 3 digit precision loss

Condition number
(examples)
Compute

the condition number of the 3x3

Hilbert matrix: