Solving of linear Equations

using SVD
nSolving a linear equation
nGauss elimination and SVD
nHowTo
nSome tricks for SVD
J ochen Schieck
MPI Munich
Problems in Linear Equations
n generally in solving Ax=b two error
contributions possible:
1. intrinsic errors in A and b
2. numerical errors due to rounding
Mathematical description and handling
available
How does the final result (and its error) depend on
the input value and ?
General Error Analysis
Matrix Norm necessary
e.g.
with
Eigenvalues of matrix
Example for Condition of Matrix
Exact solution: (2,-2)
T
0.8642 g 0.86419999
0.1440 g 0.14400001
Change input
values:
approximate solution: (0.9911,-0.4870)
T
NOT ACCEPTABLE!
cond(A)=1.513x10
8
Gauss Elimination
n Error due to rounding:
with eps = machine accuracy (double: ~10
-16
)
However, this is the worst case
scenario!
n=m=30 g10
13
Singular Value Decomposition
n here for (nxn) case, valid also for (nxm)
n Solution of linear equations numerically
difficult for matrices with bad condition:
regular matrices in numeric approximation
can be singular
SVD helps finding and dealing with the
sigular values
How does SVD work
Definition of singular value decomposition:
with U and V orthogonal matrices

,
_

...
2
1
singular values

i
are eigenvalues of A
T
A
Solution
Solution of the Equation:
numeric behaviour of SVD can be determined:
SVD after Golub and Reisch
1
1
1
1
]
1

1
1
1
1
]
1

* * * 0
* * * 0
* * * 0
* * * *
' A PA
x x x x
x x x x
x x x x
x x x x
A
1
1
1
1
]
1

1
1
1
1
]
1

* * * 0
* * * 0
* * * 0
0 0 *
' ' '
* * * 0
* * * 0
* * * 0
* * * *
'
x
A Q A A
Householder Matrix P,Q=1-ww
T
(numerically OK)
P,Q unitary
SVD after Golub and Reisch
1
1
1
1
]
1

n
n
q
e q
e q
e q
J
0 0 0
0 0
* 0
0 0
3
3 2
2 1
0
after a couple of iterations:
2 1 1 0
... ...

n n
Q AQ P P J
with:
Q and P Householder matirces
A and J
0
have the same singular values
J
0
Bidiagonal form:
SVD after Golub and Reisch
apply Givens reflections to bidiagonal Matrix:
use iterative procedure to transform
bidiagonal J
0
matrix to diagonal form
n 1, n 23 12 0 12 23 n 1, n
_
0
...T T T J S ...S S J

Givens Reflection:
~cubic convergence expected
Arithmetic Expenses
n Gauss (normal) solution
n mn
2
+1/6n
3
n SVD Golub-Reinsch
n 2mn
2
+4n
3
m=n :
SVD is 9 times more expensive!
How to deal with Singularities
n singularities are determined with the SVD
n 1/
i
is used for solution of linear equation
n relate 1/
i
to machine accuracy and
resolution
n usage of values
i
< corrupts complet
result!
careful handling necessary!
n simple approach:
n negelect all valuesin matrix with
i
<
Smooth cutoff
n Regularisation of the
singularities
n Replace the singular
values with a
function:
( )
2
2 2
2
2
t s
s
s
1
+

=0.05
1/s
2
Example
Number-of-
events response
matrix
d
i
~ b
i
/b
i
d
i
< 1 g
statistically insignificant
Problem for alignment:
determination of
Conclusion
n Numeric Solution of a regular linear
equation can be distorted by singular
behaviour
n SVD returns singular values
n Singular values can be handeld with
smooth cut off
n Mathemaical well described procedure