Multigrid Tutorial

APPM 7400
Topics in Applied Math

Section 003
Multigrid Methods
Steve McCormick
ECOT 233 492-0662/442-0724
stevem@colorado.edu
http://amath.colorado.edu/faculty/stevem
Office hours: M@11, W@9, others TBA
U-Boulder 1 of 312
APPM 7400-003
Syllabus
• Coursework
– No exams
– Homework exercises & computing assignments
– Project & presentation--joint is OK
– Can tailor some or all work & later lectures to your interests
• Philosophy
– Big on fundamentals, principles, & simple ideas
– Think about the scientific method in general
– Understand by concrete examples & experience
– Know the whole picture
– Ask questions & interject comments
– Make sure I explain what you need
• Text
– A Multigrid Tutorial, 2nd edition, 2nd printing
– Supplemental material as needed
U-Boulder 2 of 312
Sources
MGNet http://www.mgnet.org/
Newsletter
Software repository
Copper Mountain Conference
March 30--April 4, 2003
We hope to support your attendance.
Course web site:

http://amath.colorado.edu/courses/7400/2003Spr/003/
Tech support: John <jruge@colorado.edu>

Homework: Ollie <Oliver.Roehrle@colorado.edu>
U-Boulder 3 of 312
Homework exercises
Due 1 week after the relevant chapter is covered.
1: 3,4 6: 1, 2, 6
2: 1-3, 8, 10, 13, 16 7: 1, 2, 10, 12, 15, 21
3: 2-6 8: 2, 6, 8
4: 6, 7, 11 9: 1, 3
5: 1-3, 12, 13 10: 1, 5, 8, 9
U-Boulder 4 of 312
Computing assignments
Due 2 weeks after the relevant chapter is covered.
Start assignments very early: computers are merciless!!!
Use whatever computing language/platform you like.
2: 20 4: 13, 15 6: 6 by computer!!!
7: Convert your 2-D code developed for Chapter 4

(standard coarsening & point relaxation) to solve (7.14).
Verify the code using ε = 1. Test its performance using
ε = 1/2,1/5, 1/10, 2, 5, 10. Interpret your results.
U-Boulder 5 of 312
Project & presentation
Due at the end of the semester.
• I am VERY flexible as to what constitutes a project.

• Think about what has captured your attention.
• See me, but only after you’ve thought a lot about it.
• Computing &/or theory &/or application &/or exploration.
• Presentation = in-class short talk &/or hand-in.
• In-class talk: Prepare! Speak to your peers, not me.
• This is meant to be instructive & FUN!
• Check course web site for a bit more detail.
U-Boulder 6 of 312
My thoughts on what’s good science
in order of importance to me
• Attitude
– You can do it! Be positive.
– But is it really right? Be critical.
– Think, think, think! Don’t just hope or guess.
– Control your emotions! Expect ups & downs in order to control them.
• Method
– Start simply. Reduce issue to the simplest possible case.
– Think with tiny steps, but keep the big picture in mind.
– Study concrete examples.
– Look for analogies. Can A be done in any way like how B was done?
• Creativity
– What do you really want? What end are you really aiming for?
– What do you really need? What you’re trying may be sufficient to do
what you want, but would an easier-to-prove weaker result do instead?
• Intelligence
– OK, so it doesn’t hurt to try to be “smart” too.
U-Boulder 7 of 312
A Multigrid Tutorial
2nd Edition, 2nd Printing
By
William L. Briggs
CU-Denver
Van Emden Henson
LLNL ← THANKS!
Steve McCormick
CU-Boulder
U-Boulder 8 of 312
Outline
by chapter
1. Model Problems 6. Nonlinear Problems

2. Basic Iterative Methods Full approximation scheme
Convergence tests 7. Selected Applications
Neumann boundaries
Analysis
Anisotropic problems
3. Elements of Multigrid
Variable meshes
Relaxation
Variable coefficients
Coarsening 8. Algebraic Multigrid (AMG)
4. Implementation Matrix coarsening
Complexity 9. Multilevel Adaptive Methods
Diagnostics FAC
5. Some Theory 10. Finite Elements
Spectral vs. algebraic Variational methodology
U-Boulder 9 of 312
Suggested reading
CHECK MY MG LIBRARY & MGNET REPOSITORY
• A. Brandt, “Multi-level Adaptive Solutions to Boundary Value Problems,”

Math Comp., 31, 1977, pp 333-390.
• A. Brandt, “Multigrid techniques: 1984 guide with applications to
computational fluid dynamics,” GMD, 1984.
• W. Hackbusch, “Multi-Grid Methods & Applications,” Springer, 1985.
• W. Hackbusch & U. Trottenberg, “Multigrid Methods”, Springer-Verlag,
1982.
• S. McCormick, ed., “Multigrid Methods,” SIAM Frontiers in Applied
Math. III, 1987.
• U. Trottenberg, C. Oosterlee, & A. Schüller, “Multigrid,” Academic
Press, 2000.
• P. Wesseling, “An Introduction to Multigrid Methods,” Wylie, 1992.
U-Boulder 10 of 312
Multilevel methods have been
developed for...
• PDEs, CFD, porous media, elasticity, electromagnetics.
• Purely algebraic problems, with no physical grid; for
example, network & geodetic survey problems.
• Image reconstruction & tomography.
• Optimization (e.g., the traveling salesman & long
transportation problems).
• Statistical mechanics, Ising spin models.
• Quantum chromo dynamics.
• Quadrature & generalized FFTs.
• Integral equations.
U-Boulder 11 of 312
Everyone uses multilevel methods
• Multigrid, multilevel, multiscale, multiphysics, …
Use local “governing rules” on the finest level to
resolve the state of the system at these detailed
scales, but--recognizing that these “rules” have
broader implications that are hard to determine
there--use coarser levels to resolve larger scales.
Continual feedback is essential because improving
one scale impacts other scales.
• Common uses
Sight, art, politics, thinking (scientific research),
cooking, team sports, …
U-Boulder 12 of 312
1. Model problems
• 1-D boundary value problem:
− u ″( x ) + σ u( x ) = f ( x ) 0 < x < 1, σ≥0

u( 0) = u( 1 ) = 0
• Grid:
1
h =
N
, xi = ih , i = 0, 1 , . . . N
x =0 x =1
x0 x1 x2 xi xN
• Let & for .
v i ≈ u( x i ) f i ≈ f ( xi ) i = 0, 1, . . . N
This discretizes the variables, but what about the equations?

U-Boulder 13 of 312
Approximate u’’(x) via Taylor series
• Approximate 2nd derivative using Taylor series:

0 h2 h3
0
u ( x i + 1 ) = u( x i ) + h u ′ ( x i ) + u″ ( x i ) + u′′′ ( x i ) + O( h 4 )
2! 3!
h2 h3
u( x i − 1 ) = u( x i ) − h u′ ( x i ) + u″ ( x i ) − u′′′ ( x i ) + O( h 4 )
2! 3!
• Summing & solving:
u( x i + 1 ) − 2 u( x i ) + u( x i − 1 )
u″ ( x i ) = + O( h 2)
h2
U-Boulder 14 of 312
Approximate equation via
finite differences
• Approximate the BVP
− u ″( x ) + σ u( x ) = f ( x ) 0 < x < 1, σ≥0

u( 0) = u( 1 ) = 0
by a finite difference scheme:
− vi − 1 + 2 vi − vi + 1
+ σ vi = f i i = 1, 2, . . . N − 1
2
h
v0 = vN = 0
U-Boulder 15 of 312
Discrete model problem
T&
Letting v = ( v 1, v 2, . . . , v N − 1 )
T
f = ( f 1, f 2, . . . , f N − 1 )
we obtain the matrix equation Av = f, where A
is (N-1) x (N-1), symmetric, positive definite, &
 2 + σh2 −1   f1 
   v1 
 − 1 2 + σh 2   v2   f 
−1  2 
   
1  2
− 1 2 + σh − 1   v3   f 
A= , v=  , f = 3 
2 
h      
 2   N −2 
v  f N −2
 − 1 2 + σ h − 1  v   
   N − 1  f
 − 1 2 + σh2   N −1
U-Boulder 16 of 312
Stencil notation
A = [-1 2 -1]
dropping h -2 & σ for convenience

-1
-1
U-Boulder 17 of 312
Basic solution methods
• Direct
– Gaussian elimination
– Factorization
– Fast Poisson solvers (FFT-based, reduction-based, …)
• Iterative
– Richardson, Jacobi, Gauss-Seidel, …
– Steepest Descent, Conjugate Gradients, …
– Incomplete Factorization, ...
• Notes:
– This simple 1-D problem can be solved efficiently in many ways.
Pretend it can’t & that it’s very hard, because it shares many
characteristics with some very hard problems. If we keep things
as simple as possible by studying this model, we’ve got a chance
to really understand what’s going on.
– But, to keep our feet on the ground, let’s go to 2-D anyway…
U-Boulder 18 of 312
2-D model problem
• Consider the problem
− ux x − uyy + σ u = f ( x , y ) , 0 < x < 1, 0<y<1

u = 0 , x = 0, x = 1, y = 0, y = 1; σ≥0
• Consider the grid
z
1 1
hx = , hy = ,
M N
( x i , yj ) = ( i hx , j hy )
y
0 ≤ i ≤ M
0≤ j ≤ N
U-Boulder x 19 of 312
Discretizing the 2-D problem
• Let v i j ≈ u( x i , yj ) & f i j ≈ f ( xi , yj ) . Again, using 2nd-
order finite differences to approximate u x x & uy y
we arrive at the approximate equation for the
unknown , for i =1,2, … M-1 & j =1,2, …, N-1:
u( x i , yj )
− v i − 1, j + 2v i j − v i + 1, j − v i , j − 1 + 2v i j − v i , j + 1
+ + σ vi j = f i j
hx2 hy2
v i j = 0 , the
• Ordering i = 0unknowns
, i = M , j (&
= 0,also
j =M
the vector f )
lexicographically by y-lines:
T
v = (v1,1 , v1,2 ,..., v1,N −1 , v2 ,1 , v2 , 2 ,..., v2 ,N−1 ,..., vM−1,1 ,vM−1,2 ,..., vM−1,N−1 )
U-Boulder 20 of 312
Resulting linear system
• We obtain a block-tridiagonal system Av = f :
 A1 − Iy  v1  f1 
 −I A − I    
 y 2 y  2 
v f
 2 
 − Iy A3 − Iy   v3   f 
   =  3 
    
 − Iy AN − 2 − Iy     
   vN − 1  f 
 Iy is a diagonal
where − Imatrix
y A N − 1 with on
 Nthe
−1 
diagonal & 1
hy2
 2 + 2 +σ −
1 
 hx2 hy2 hx2 
 1 2 2 1 
 − 2 + 2 +σ − 
 hx2 hx hy hx2 
Ai =  1 2 2 1 
− 2 + +σ −
 hx hx2 hy2 hx2 
 
 
1 2 2
 − 2 2 + 2 +σ

U-Boulder  hx hx hy  21 of 312
Stencils
preferred for grid issues
Stencils are much better for showing the grid picture:

again dropping h -2 & σ
0 −1
0  • -1• •
 h 2y 
 −12 2 2
2 + 2 +σ
−1 
• • •
-1 4 -1
 hx h x hy hx2

−1
0
 h 2y
0
 • -1• •
Stencils show local relationships--grid point interactions.
U-Boulder 22 of 312
Outline
Chapters 1-5: Chapters 6-10:
• √ Model Problems • Nonlinear Problems
• Basic Iterative Methods – Full approximation scheme
– Convergence tests • Selected Applications
– Analysis – Neumann boundaries
– Anisotropic problems
• Elements of Multigrid
– Variable meshes
– Relaxation
– Variable coefficients
– Coarsening
• Algebraic Multigrid (AMG)
• Implementation
– Matrix coarsening
– Complexity • Multilevel Adaptive Methods
– Diagnostics – FAC
• Some Theory • Finite Elements
– Spectral vs. algebraic – Variational methodology
U-Boulder 23 of 312
2. Basic iterative methods
• Consider Au = f where A is NxN & let v be an

approximation to u. (Generic A uses N not N-1.)
• Two important measures:

– The Error: with norms
e = u −v
N
|| e ||∞ = max | ei | & || e ||2 = ∑ e
i =1 i
2
– The Residual: with

What does
r = f − Av ||r|| measure???
|| r ||∞ & || r ||2 Why have both

r & e ???
U-Boulder 24 of 312
Residual correction
• Since e = u - v, we can write Au = f as A(v + e) = f
which means that Ae = f - Au ≡ r. r = f - Av

= Au - Av
• Residual Equation: = A(u - v)
= Ae
Ae = r
• Residual Correction:
What does this
do for us???
u = v +e
U-Boulder 25 of 312
Relaxation
• Consider the 1-D model problem
− ui − 1 +2 ui − ui + 1 = h 2 f i 1 ≤ i ≤ N −1 u0 = uN = 0
• Jacobi (simultaneous displacement): Solve the i th

equation for holding all other variables fixed:
vi
( ) 1 ( o ld)
vi n e w = ( v i − 1 + v i( +ol1d) + h 2 f i ) 1 ≤ i ≤ N −1
2
U-Boulder 26 of 312
Jacobi in matrix form
• Let A = D - L - U, where D is diagonal & L & U are
the strictly lower & upper triangular parts of A.
• Then becomes
Au = f
(D −L −U) u = f
D u = ( L + U) u + f
• Let u = D − 1.( L + U) u + D − 1 f
– RJR is = − 1the error propagation =orDiteration
-1(D-A) matrix.
I D-1A
J
called
D ( L + U) J = -
• Then the iteration is
v ( n ew) = R J v ( o ld ) + D − 1 f
U-Boulder 27 of 312
Error propagation matrix & the error
• From the derivation,
u = D − 1 ( L + U) u + D − 1 f
u = RJ u + D − 1 f
• the iteration is
v ( n ew) = R v ( o ld ) + D − 1 f
• subtracting, J
• or
u − v ( n ew) = R J u + D − 1 f − ( R J v ( o ld) + D − 1 f )
• hence
u − v ( n ew) = R J u − R J v ( o ld)
Error propagation! e ( n ew) = R J e ( o ld) J = I-D-1A

U-Boulder 28 of 312
A picture
RJ = D-1 (L + U) = [ 12 0 1
2 ]
so Jacobi is an error averaging process:
ei(new) ← (ei-1(old) + ei+1(old))/2

U-Boulder 29 of 312
But…

U-Boulder 30 of 312
Another matrix look at Jacobi
v (new) ← D-1 (L + U) v (old) + D-1 f (L + U = D-A)
= (I - D-1 A) v (old) + D-1 f
v (new) = v (old) - D-1 (Av (old) - f) = v (old) - D-1 r
• Exact: u =u - D-1 (Au - f)

• Subtracting: e (new) = e (old) - D-1 Ae (old)
• General form: u = u - B (Au - f) with B ~ A-1
• Damped Jacobi: u = u - ωD-1 (Au - f) with 0<ω<2
• Gauss-Seidel: u = u - (D - L)-1 (Au - f)
• Exact: u = u - A-1 (Au - f) = A-1f
U-Boulder 31 of 312
Weighted Jacobi • • •
safer (0<ω<1) changes; politics in the ‘hood • o •
• • •
• Consider the iteration
( ) ( ) ω ( o ld )
vi n e w ← ( 1 − ω) v i o l d + ( v i − 1 + v i( +o l1d) + h 2f i )
2
• Letting A = D-L-U, the matrix form is

v ( n ew) = ( 1 − ω) I + ω D − 1( L + U) v ( o ld) + ω h 2D − 1f
= R ω v ( old) + ω h 2D − 1f
• Note that
R ω = [ ( 1 − ω) I +ω R J ]
( a ppro x )
• It is easy to see that if e ≡ u ( ex a c t ) − u , then
e ( n ew) = R ω e ( o ld)
U-Boulder 32 of 312
Gauss-Seidel (1-D)
• Solve equation i for ui & update immediately.
• Equivalently: set each component of r to zero in turn.
• Component form: for i = 1, 2, . . . N − 1, set
1
vi ← ( vi − 1 + vi + 1 + h 2 f i )
• Matrix form: 2
A = (D −L −U)
( D − L) u = U u + f
• Let u = ( D − L) − 1 U u + ( D − L) − 1 f
• ThenRiterate:
= ( D − L ) − 1U
G
v ( n ew) ← R G v ( old) + ( D − L ) − 1f
• Error propagation:
e ( n ew) ← R G e ( o ld)
U-Boulder 33 of 312
Red-black Gauss-Seidel
• Update the EVEN points:
1
v 2i ← ( v 2i − 1 + v 2i + 1 + h 2 f 2i )
2
• Update the ODD points:
1
v 2i + 1 ← ( v 2i + v 2i + 2 + h 2 f 2i + 1 )
2
x0 xN
• 2-D:
U-Boulder 34 of 312
Numerical experiments
• Solve A u = 0 , − ui − 1 + 2 u i − u i + 1 = 0
• Use Fourier modes as initial iterates, with N = 64:
 ikπ  1 ≤ i ≤ N − 1, 1 ≤ k ≤ N −1
vk = ( v i ) k = sin  
 N  component mode
sin(kπx), x =i/N k =1
1
0.8
0.6
k =3
0.4
0.2
0
-0.2
-0.4
k =6
-0.6
-0.8
-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
U-Boulder 35 of 312
Convergence factors differ for
different error components
Error, ||e||∞ , in weighted (ω=2/3) Jacobi on Au = 0
for 100 iterations using initial guesses v1, v3, & v6
1
k =1
0.9
0.8
0.7
0.6
0.5
0.4
k =3
0.3
0.2
k =6
0.1
0
0 20 40 60 80 100 120
U-Boulder 36 of 312
Stalling convergence
relaxation shoots itself in the foot
• Weighted (ω=2/3) Jacobi on 1-D problem.

• Initial guess:
1  j π  6 jπ   32 j π  
v 0 =  s in   + s in  N  + s in  N  
3  N      
• Error plotted against iteration number:
|| e ||∞
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 10 20 30 40 50 60 70 80 90 100
U-Boulder 37 of 312
Analysis of stationary linear iteration
• Let v(new) = Rv(old) + g . The exact solution is
unchanged by the iteration: u = Ru + g .
• Subtracting: e(new) = Re(old) .
• Let e(0) be the initial error & e(i) be the error

after the i th iteration. After n iterations, we have
e(n) = Rne(0) .
a b c 4
 
I can deal with 24, but d e f ???
 
g h i
a b c 4
 
What if d e f
e = 24 e ???
 
g h i
U-Boulder 38 of 312
Quick review of
eigenvectors & eigenvalues
• The number λ is an eigenvalue of a matrix B & w ≠ 0
its associated eigenvector if Bw = λw.
• The eigenvalues & eigenvectors are characteristics
of a given matrix.
• Eigenvectors are linearly independent, & if there is a
complete set of N distinct eigenvectors for an NxN
matrix, then they form a basis: for any v, there exist
unique scalars vk such that
N
• Propagation: v= Σ k
vk
k=1
w Why is an
eigenvector
N useful???
n
Σ
n
B v= λ vk wk
U-Boulder k=1 39 of 312
“Fundamental Theorem of Iteration”
• R is convergent (Rn → 0 as n → ∞) iff
.
ρ(R) = max1≤k≤N λk < 1
Thus, e(n) = Rne(0) → 0 for any initial vector v (0)
iff ρ(R)<1.
• ρ(R)<1 assures convergence of the iteration for R.

• ρ(R) is the spectral convergence factor.
– But ρ is doesn’t tell you much by itself--it’s generally valid
only asymptotically. It’s useful for the symmetric case in
particular, so we’ll accept it for now, but first a little
background & then a warning…
U-Boulder 40 of 312
Rayleigh quotient vs. spectral radius
assume A is symmetric (wk orthogonal) & nonnegative definite (λ ≥ 0)
• RQ(v) ≤ ρ(A): v = Σvkwk
RQ(v) < Av, v > < A Σ vk wk , Σ vk wk >

= =
< v,v > < Σ vk wk , Σ vk wk >
< Σ λk vk wk , Σ vk wk > Σ λ
2
k vk ≤ λ1N = ρ(A)
≥
= =
< Σ vk wk , Σ vk wk > Σ vk2
• supv ≠ 0 RQ(v) = ρ(A):
< AwN ,wN > < λN wN ,wN >

RQ(wN ) = = = λN = ρ(A)
< wN ,wN > < wN , wN >
U-Boulder 41 of 312
Euclidean norm vs. spectral radius
use RQ
• ||R||2 = ρ1/2(RTR):
||R||22 = supe≠0 ||Re||22 /||e||22
= supe≠0 <Re,Re >/<e,e>
= supe≠0 <RTRe,e >/<e,e> = ρ(RTR)
note: ||Re||2 ≤ ||R||2·||e||2
• ||A||2 = ρ1/2(A2) = ρ(A) for symmetric A!!!
U-Boulder 42 of 312
ρ(R) vs. ||R||2
ρ(R) = sup |λ(R)|. ||R||2=supe≠0 ||Re||2 /||e||2= ρ1/2(RTR).
ρ(R) is norm independent. ||R||2 depends on ||· ||2.
ρ(R) is asymptotic: ||R||2 bounds ||e(n + 1)||2 /||e(n)||2:

could get large ||R||2 but ρ(R)<1 worst case! Probably pessimistic
⇒ “convergence”. Next slide. initially, but sharp sooner or later.
ρ(R) = inf ||R||

taken over all matrix norms.
ρ(R) = ||R||2
for symmetric R.
U-Boulder 43 of 312
Example: R = ( ), K large
ρ(R) = 0 but ||R||2 = K !
(1) (0 )
 0 e 2 Re
=   ⇒ ( 0) = (0 )
( 0)
e 2
=K
 1 e 2 e 2
Thus, 1 iteration with e(0) shows dramatic L2 divergence!

But R2 = 0, so e(2) = Re(1) = R2e(0) = 0!
Thus, 2 iterations with e(0) show complete convergence!
On one hand, this is special (λ=0, large K, 2x2), so this
behavior would be more subtle & persistent in general.
On the other, this behavior would vanish for symmetric R.
U-Boulder 44 of 312
Convergence factor & rate
• How many iterations are needed to reduce the initial error
by 10-d ?
e (n ) n n −d
(0 ) ≤ R ≤ R ~ 10
e
• So, we have
d
n~
− log10 ρ( R)
• Convergence factor = ||R|| or ρ(R).
• Convergence rate = or -log10(ρ(R)).

digits
− log10 R
iteration
U-Boulder 45 of 312
Convergence analysis: Weighted Jacobi
= I − ωD − 1 A
 2 −1 
 −1 2 −1 
ω 
Rω = I −  −1 2 −1  1-D
2 ⋅⋅ ⋅⋅ ⋅⋅ 
 ⋅ ⋅ ⋅
 −1 2 
For our 1-D model, the eigenvectors of weighted

Jacobi Rω & the eigenvectors of A are the same! Why???
The eigenvalues are related as well.

ω
λ ( Rω ) = 1 − λ ( A)
U-Boulder 2 46 of 312
Eigenpairs of A
The eigenvectors of A are Fourier modes!
 kπ   j kπ  [-1 2 -1]
λk ( A ) = 4 sin 
2
 , wk , j = sin   λN-1 ≅ 4
 2N   N  λ1 ≅ πh2
1 1
0.8 0.8
0.6 0.6
0.4 0.4
N = 64 0.2
0
-0.2
0.2
0
-0.2
-0.4 -0.4
-0.6 -0.6
-0.8 -0.8
-1 -1
0 10 20 30 40 50 60 0 10 20 30 40 50 60
k = 1 k = 2
1 1 1
0.8 0.8 0.8
0.6 0.6 0.6
0.4 0.4 0.4
0.2 0.2 0.2
0 0 0
-0.2 -0.2 -0.2
-0.4 -0.4 -0.4
-0.6 -0.6 -0.6
-0.8 -0.8 -0.8
-1 -1 -1
0 10 20 30 40 50 60 0 10 20 30 40 50 60 0 10 20 30 40 50 60
k = 4 k = 8 k = 16
U-Boulder 47 of 312
Eigenvectors of Rω = eigenvectors of A
 kπ 
λk ( R ω ) = 1 − 2 ω sin2  2N 
 
• Expand the initial error in terms of the eigenvectors:

N−1
e ( 0)
= ∑ ck w k
k =1
• After n iterations:
N −1
Re n (0 )
= ∑ ck λ w k
n
k
k=1
• The kth error mode is reduced by λk (Rω) each iteration.
U-Boulder 48 of 312
Relaxation suppresses eigenmodes unevenly
• Look carefully at λk ( R ω ) = 1 − 2 ω sin2  k π  .

 2N 
1
0.8
Note that if 0 < ω ≤ 1,
0.6 ω = 1/3
then | λ ( R ω ) | < 1 for
k
.
0.4 k = 1, 2, . . . , N − 1
0.2 ω = 1/ 2 For 0 < ω ≤ 1,
λ 0
N
π 
-0.2
2 λ1 = 1 − 2 ω sin 2
 2N 
-0.4
ω=1
 
ω = 2/ 3
-0.6  πh 
= 1 − 2 ω s in 
2

-0.8
 2 
-1
= 1 − O ( h2) ≈ 1
0 k N
U-Boulder 49 of 312
Low frequencies are “undamped”
Notice that no value of ω will efficiently damp out
long waves or low frequencies.
1
What value of ω gives
0.8
the best damping of
0.6 ω = 1/3
short waves or high
0.4
frequencies N/2≤k≤N-1?
λ
0.2
ω = 1/ 2
axis 0
-0.2
N Choose such that
-0.4
2
ω
ω=1
-0.6
ω = 2/ 3 λN ( R ω ) = − λN ( R ω )
-0.8 2
2
-1 ⇒ ω=
0 N 3
U-Boulder
k axis 50 of 312
Smoothing factor
• The smoothing factor is the largest magnitude of

the iteration matrix eigenvalues corresponding to
the oscillatory Fourier modes:
smoothing factor = max |λk(R)| for N/2≤k≤N-1.
• Why only the upper spectrum? “MG” spectral radius?
• For Rω with ω=2/3, the smoothing factor is 1/3:
|λN/2|=|λN|=1/3 & |λk|<1/3 for N/2<k<N.
• But |λk| ≈ 1 - ωk2π2h2 for long waves (k << N/2).
U-Boulder 51 of 312
Convergence of Jacobi on Au = 0
100 100
Unweighted Jacobi Weighted Jacobi
90 90
80 80
70 70
60 60
50 50
40 40
30 30
20 20
10 10
0
0 0 10 20 30 40 50 60
0 10 20 30 40 50 60
Wavenumber, k Wavenumber, k
• Jacobi on Au = 0 with N = 64. Number of iterations
needed to reduce initial error ||e||∞ by 0.01.
• Initial guess :
 jkπ 
vkj = sin  
 N 
U-Boulder 52 of 312
Weighted Jacobi = smoother (error)
 2j π  1  16j π  1  32j π 
• Initial error: v k j = sin 
N  + 2 sin  N  + 2 sin  N 
     
2
-1
Many relaxation schemes
-2
are smoothers:
0 0 .5 1 oscillatory error modes
are quickly eliminated,
• Error after 35 iteration sweeps: but
smooth modes are slowly
2
damped.
1
-1
-2
0 0 .5 1
U-Boulder 53 of 312
Similar analysis for other smoothers
• Gauss-Seidel relaxation applied to the 3-point
difference matrix A (1-D model problem):
−1
RG = ( D − L ) U
• A little algebra & trigonometry shows that
kπ  kπ jkπ 
λk (RG ) = cos2  wk , j = cos j   sin 
 N  N  N 
1
1
0.8
0.8
0.6 π xj )
sin(3π
0.4
0.2
0.6
0
j
0.4 λk -0.2
-0.4
λ3 sin(3π
πxj )
-0.6
0.2
-0.8
-1
0
0 10 20 30 40 50 60
k 0 10 20 30
j 40 50 60
U-Boulder
What’s wk look like for large k ? 54 of 312
Gauss-Seidel eigenvectors
1 0.1
0.8 0.08
0.06
0.6
k=1,n=64 0.04
0.4
0.02
k=30,n=64
0.2 0
0 −0.02
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
0.2 1
0.1 k=60,n=64
0.5
0
−0.1
0
−0.2 k=48,n=64
−0.3
−0.5
−0.4
−0.5 −1
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
U-Boulder 55 of 312
Gauss-Seidel convergence
Au = 0
Eigenvectors of RG are not the same as those of A.
Gauss-Seidel mixes the modes of A.
100 Gauss-Seidel on Au = 0, with

90
80
N = 64. Number of iterations
70 needed to reduce initial error
60
50
||e||∞ by 0.01.
40
30 Initial guess (modes of A):

20
10
0
 jkπ 
vkj = sin  
0 10 20 30 40 50 60
 N 
U-Boulder 56 of 312
Outline
√ • Selected Applications
– Convergence tests
– Variable meshes
– Relaxation
– Coarsening
• Implementation
U-Boulder 57 of 312
3. Elements of multigrid
1st observation toward multigrid
• Many relaxation schemes have the smoothing
property: oscillatory error modes are quickly
eliminated, while smooth modes are often very
slow to disappear.
• We’ll turn this adversity around: the idea is to
use coarse grids to take advantage of smoothing.
x0 xN
Ωh
2h
x0 xN Ω
2
How?
U-Boulder 58 of 312
Reason #1 for coarse grids:
Nested iteration
• Coarse grids can be used to compute an improved
initial guess for the fine-grid relaxation. This is
advantageous because:
– Relaxation on the coarse-grid is much cheaper: half as

many points in 1-D, one-fourth in 2-D, one-eighth in 3-D,…
– Relaxation on the coarse grid has a marginally faster

(|λ1(R)| ≈ 1 - ωπ2h2 ):
instead of
1 − O( 4h 2 ) 1 − O ( h 2)
U-Boulder 59 of 312
Idea! Nested iteration
•
•
•
• Relax on Au = f on Ω4h to obtain initial guess v2h.
• Relax on Au = f on Ω2h to obtain initial guess vh.
• Relax on Au = f on Ωh to obtain … final solution???
• What is A2hu2h = f2h? Analogous to Ahuh = fh for now.

• How do we migrate between grids? Hang on…
• What if the error still has large smooth components
when we get to the fine grid Ωh ? Stay tuned for 2nd
observation toward multigrid…
U-Boulder 60 of 312
Reason #2 for coarse grids:
Smooth error becomes more oscillatory
• A smooth function:
1
On the coarse grid, the
smooth error appears to
0 . 5 be relatively higher in
0
frequency: in this example
it is the 4-mode out of
- 0 . 5 a possible 15 on the fine
grid, ~1/4 the way up the
- 1
0 0 . 5 1 spectrum. On the coarse
grid, it is the 4-mode out
of a possible 7, ~1/2 the
can be represented by linear way up the spectrum.
interpolation from a coarser grid:
Relaxation on 2h is
(cheaper &) faster
on this mode!!!
U-Boulder 61 of 312
For k=1,2,…N/2, the kth mode is
preserved on the coarse grid.
 2j k π   j kπ 
1
0.8
wk , 2j = sin 
h
 = sin   = wk2,hj 0.6
 N   N /2 
0.4
0.2
0
-0.2
Also, note that

-0.4
-0.6
-0.8
-1
h →0
wN/2 0 2 4 6 8 10 12
on the coarse grid. k=4 mode, N=12 grid
1
0.8
0.6
0.4
What happens 0.2

0
-0.2
to the modes -0.4
-0.6
between N/2 & N ?

-0.8
-1
0 1 2 3 4 5 6
k=4 mode, N=6 grid

U-Boulder 62 of 312
For k > N/2, wkh is disguised on
the coarse grid: aliasing!!!
• For k > N/2, the kth mode on 1
the fine grid is aliased & 0.8

0.6
appears as the (N - k)th

0.4
0.2
0
mode on the coarse grid: -0.2

-0.4
-0.6
-0.8
-1
 ( 2j ) πk  0 2 4 6 8 10 12
( wkh ) = s in 
N 
k=9 mode, N=12 grid
2j 
 2πj ( N − k ) 
1
0.8
= − sin   0.6
 N 
0.4
0.2
0
-0.2
 π ( N − k) j 
-0.4
-0.6
= − sin   -0.8
 N / 2 
-1
0 1 2 3 4 5 6
k=3 mode, N=12 grid
= − ( wN
2h )
−k j
U-Boulder 63 of 312
1-D interpolation (prolongation)
to migrate from coarse to fine grids
• Mapping from the coarse grid to the fine grid:
I 2hh : Ω 2h → Ω h
• Let , be defined on h, . Then
2h
vh v 2h Ω Ω
where I 2hh v 2h = v h
v 2hi = v i2h
v 2hi + 1
1 2h
= ( v i + v i2h+ 1 )
2
} for 0≤i ≤
N
2
− 1.
U-Boulder 64 of 312
1-D interpolation (prolongation)
• Values at points on the coarse grid map unchanged
to the fine grid.
• Values at fine-grid points NOT on the coarse grid
are the averages of their coarse-grid neighbors.
U-Boulder 65 of 312
1-D prolongation operator P
h
• P = I 2h is a linear operator: ℜN/2-1 ℜN-1 .
 v 1h 
• N = 8:  1/2   
 1   v 2h 
   
 1/2 1/2   v 2h 
1  v 3h 
   2h   h 
 1   2  v =  v4 
 1/2 1/2   v 2h   h 
  3  3x 1  v5 
 1   h 
   v6 
 1/2  7x 3
 vh 
• has full rank, so η(P ) = {0} .  7  7x 1
I 2hh
When is I 2hh v2h = 0?
U-Boulder 66 of 312
“Give To” stencil for P
] 1/2 1 1/2 [
o x o x o x o
1/2 1 1/2
U-Boulder 67 of 312
How well could v 2h approximate u?
• Imagine that a coarse-grid approximation v2h has

been found. How well could it approximate the
exact solution u ?
• If u is smooth, a coarse-grid interpolant v2h might

do very well.
U-Boulder 68 of 312
How well could v 2h approximate u?
• Imagine that a coarse-grid approximation v2h has

been found. How well could it approximate the
exact solution u ?
• If u is oscillatory, a coarse-grid interpolant v2h

cannot work well.
U-Boulder 69 of 312
Moral
• If what we want to compute is smooth, a
coarse-grid interpolant could do very well.
• If what we want to compute is oscillatory, a
coarse-grid interpolant cannot do very well.
• What if u is not smooth? Can we make it so?
• Can we make something smooth?
• How about e ? Can we smooth it? How would we

get e & use it to get u ? Ae = r & u ← v + e !
• Thus, use nested iteration on residual equation
to approximate the error after smoothing!!!
• Just because the coarse grid can approximate e well
doesn’t mean we know how to do it! But we will soon!
U-Boulder 70 of 312
2nd observation toward multigrid
• The residual equation: Let v be an approximation to
the solution of Au = f, where the residual r = f -Av.
Then the error e = u - v satisfies Ae = r.
• After relaxing on Au = f on the fine grid, e will be

smooth, so the coarse grid can approximate e well.
This will be cheaper & e should be more oscillatory
there, so relaxation will be more effective.
• Therefore, we go to a coarse grid & relax on the

residual equation Ae = r.
What’s a good initial guess on grid 2h? e = 0 !
How do we get back to grid 2h? Stay tuned…

U-Boulder 71 of 312
A way to coarsen Ae = r
• Assume we’ve relaxed so much that e is smooth.
• Ansatz: e = Pv2h for some coarse-grid v2h.
• How do we characterize e so we can hope to compute it?
Ae = r ⇒ A P v2 h = r
7x7 7x3 3x1 = 7x1
• Too many equations now & too few unknowns!

• Multiply both sides by some 3x7 matrix?
PT
P T A P v2h = P Tr
A2h
)
3x3
)
3x7 7x7 7x3 3x1 = 3x1
U-Boulder 72 of 312
Idea! Coarse-grid correction
2-grid
h to get an approximation
• Relax on Au = f on Ω .
• Compute r = f − A v h .
• Relax on Ae = r on 2h to obtain an approximation to
Ω
the error, .
e 2h
• Correct the approximation v h ← v h + e2h.
I 2hh
This is the essence of multigrid.
• Clearly, we need a mapping for Ωh → Ω2h.
U-Boulder 73 of 312
1-D restriction by injection
• Mapping from the fine grid to the coarse grid:
I h2h : Ω h → Ω 2h
• Let , be defined on ,
h 2h
. Then
vh v 2h Ω Ω
2h h
where I
. h v = v 2h
v i2h = v 2hi
Ωh
Ω 2h
U-Boulder 74 of 312
1-D restriction by full weighting
• Let v h , v 2h be defined on Ω h, Ω 2h. Then

I h2h v h = v 2h
where
1 h .
v i = ( v 2i − 1 + 2 v 2hi + v 2hi + 1 )
2h
4
Ωh
Ω 2h
U-Boulder 75 of 312
1-D restriction (full-weighting)
2h
• R = I h is a linear operator: ℜN-1 ℜN/2-1 .
 v 1h 
• N = 8:  
 v 2h 
 
1/4 1/2 1/4  v 3h   v 12h 
  h   2h 
 1/4 1/2 1/4   v4  =  v2 
 1/4 1/2 1/4   v h   2h 
 5   v3 
 h 
 v6 
 vh 
• has rank , so dim(η(R))  7  .
2h N N
Ih ∼ ∼
2 2
Look at the columns of R associated with grid 2h.
U-Boulder 76 of 312
Prolongation & restriction are
often nicely related
• For the 1-D examples, linear interpolation & full
weighting are
1 
2 
 
1 1  1 2 1
1   2h 1 
I 2hh =  2  Ih =  1 2 1 
2   4
 1 1   1 2 1
 2
 
 1 
• So they’re related by the variational condition
for c in ℜ.
T
I 2hh = c ( I h2h )
U-Boulder 77 of 312
2-D prolongation
1 1 1
4 2 4
1 2h 1 1
v 2i + 1, 2j = ( v i j + v ih+ 1, j )
h
1
2 2 2
1 2h
v 2hi , 2j + 1 = ( v i j + v ih, j + 1 ) 1 1 1
2 4 2 4
1 2h
v 2i + 1, 2j + 1 = ( v i j + v ih+ 1, j + v ih, j + 1 + v ih+ 1, j + 1 )
h
4
We denote the operator by

1 1 1
using a “give to” stencil ] [.
4 2 4
Centered over a C-point ,
1 1
it shows what fraction of 1
2 2
the C-point’s value 1 1 1
contributes to a neighboring 4 2 4
F-point .
U-Boulder 78 of 312
“Get From” interpolation
1/4 1/4
[1/2 1/2]
1/4 1/4
Centered over a coarse grid point .
1/4 1/4
1/2 1/2 1/4 1/4
U-Boulder 79 of 312
2-D restriction (full weighting)
1 1 1
16 8 16
1 1 1
8 4 8
1 1 1
16 8 16
We denote the operator by

1 1 1
using a “get from” stencil [ ].
16 8 16
Centered over a C-point , it
1 1 1
shows what fraction of the 8 4 8
value of the neighboring 1 1 1
F-point contributes to the 16 8 16
value at the C-point.
U-Boulder 80 of 312
Now we put all these ideas together
• Nested Iteration
– effective on smooth solution (components).
• Relaxation
– effective on oscillatory error (components).
• Residual Equation
– characterizes the error.
– enables nested iteration for smooth error (components)!!!
• Prolongation (variables) & Restriction (equations)

– provides pathways between coarse & fine grids.
U-Boulder 81 of 312
2-grid coarse-grid correction
v h ← CG ( v h , f h ) , α1 , α2
1) Relax α times on A h u h = f h on Ω h with arbitrary

1
initial guess h.
v
2) Compute h h h.
r =f −A v
h
3) Compute .
r 2h = I 2h r h
4) “Solve” h on .
2h 2h 2h
=
5) Correct fine-grid solution
A e r 2h Ω .
6) Relax times on v hon I 2hh initial
← v h +with e 2h
guess .
α2 A h uh = f h Ωh
vh
U-Boulder 82 of 312
2-grid coarse-grid correction
Correct
Relax on Ah v h = f h
Compute r h = f h− Ahv h vh ← vh + eh
Restrict Interpolate
r 2h = I h2h r h eh = I2hh e2h
2h 2h
Solve A e = r2h
e 2h = ( A 2h) − 1 r 2h
U-Boulder 83 of 312
What is A2h?
• For this scheme to work, we must have 2h, a coarse-

A
grid operator. For the moment, we will simply assume
that is “the coarse-grid version ” of the fine-grid
2h
A
operator .
Ah
• Later we’ll return to the question of constructing .
2h
A
U-Boulder 84 of 312
How do we “solve” the coarse-
grid residual equation? Recursion!
α1 h h α2 = 0
vh ← G (v , f ) v h ← v h + eh
h
f
2h 2h
← Ih ( f
h h
−A v )
h
e h ← I 2h v 2h
v 2h ← Gα1( 0 , f
2h 2h
) v 2h ← v 2h + e 2h
4h 4h 2h 2h
f ← I2h ( f − A v 2h ) 2h
e 2h ← I 4h v 4h
v 4h ← Gα1( 0
4h
,f
4h
) v 4h ← v 4h + e 4h
8h 8h 4h 4h
4h 4h
f ← I4h ( f −A v ) e 4h ← I 8h v 8h
v 8h ← Gα1( 0 , f
8h 8h
) v 8h ← v 8h + e 8h
1 H
vH = eH = ( A ) − f
H
U-Boulder 85 of 312
V-cycle (recursive form)
v h ← M V h ( v h , f h ) , α 1, α2
1) Relax α1 times on A h u h = f h with initial v h given.
2) If h is the coarsest grid, go to 4;

Ω
else:
f 2h ← I2h
h ( f h
− A h h
v )
v 2h ← 0
v 2h ← M V 2h ( v 2h , f 2h )
3) Correct: .
v h ← v h + I 2hh v 2h
4) Relax times on with initial guess .
α2 A h uh = f h vh
U-Boulder 86 of 312
Outline
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
U-Boulder 87 of 312
4. Implementation
Storage cost: vh & fh on each level.
In 1-D, a coarse grid has about half as many
points as the fine grid.
In 2-D, a coarse grid has about one-fourth

as many points as the fine grid.
In d-dimensions, a coarse grid has about 2-d

as many points as the fine grid.
−d − 2d − 3d − Md 2N d
d
Total storage cost: 2N ( 1 + 2 +2 +2 + …+2 ) <
1 − 2− d
less than 2, 4/3, & 8/7 the cost of storage on the fine
grid for 1-D, 2-D, & 3-D problems, respectively.
U-Boulder 88 of 312
Computational cost
• Let one Work Unit (WU) be the cost of one
relaxation sweep on the fine grid.
• Ignore the cost of restriction & interpolation
(typically about 20% of the total cost).
• Consider a V-cycle with 1 pre-Coarse-Grid (α1 = 1)
correction sweep & 1 post-Coarse-Grid (α2 = 1)
correction sweep.
• Cost of a V-cycle (in WUs):
2− d 2 − 2d 2 − 3d 2− Md
2
2 ( 1+ + + + ... + ) <
1 − 2− d
• Cost is about 4, 8/3, & 16/7 WUs per
V-cycle in 1, 2, & 3 dimensions.
U-Boulder 89 of 312
Convergence analysis
• First, a heuristic argument:
– The convergence factor for the oscillatory error modes
(smoothing factor) is small & bounded uniformly in h.
smoothing factor = max |λk(R)| for N/2≤k≤N-1.

– Multigrid focuses the relaxation process on attenuating the
oscillatory components on each level.
Relax 1st coarse grid Relax on fine grid
smooth oscillatory
k=1 k=N/2 k=N
⇒ The overall multigrid convergence factor is

small & bounded uniformly in h !
Bounded uniformly in h ≠ independent of h.

U-Boulder 90 of 312
Convergence analysis
a little more precisely...
• Continuous problem: Au = f, u i = u(x i)
• Discrete problem: Ah uh = fh, vh ≈ uh
• Global/discretization error: Ei = u(x i) - uih

||E || ≤ Kh p
(p = 2 for model problem & proper norm)
• Algebraic error: eih = uih - vih
• For tolerance ε, assume the aim is to find vh so that the

total error ||e || = ||u(h) - vh || ≤ ε , where u(h) = (u (x i)).
• Then this objective is assured as follows:
||e || ≤ ||u (h) - u h || + ||u h - v h || = ||E|| + ||e h|| ≤ ε .
U-Boulder 91 of 312
We can satisfy the convergence
objective by imposing two conditions
1) ||E|| ≤ ε/2. Achieve this condition by choosing an

appropriately small grid spacing h*:
K(h*)p = ε/2
2) ||e h|| ≤ ε/2. Achieve this condition by iterating until
||E|| ≤ ε/2 = K(h*)p on grid h; then we’ve
converged to the level of discretization error.
→ Once discretization error & algebraic error

are in balance, then it would be better to
go to grid h/2 than to iterate more!
U-Boulder 92 of 312
Convergence to the level
of discretization error
• Use an MV scheme with convergence factor γ < 1
bounded uniformly in h (fixed α1 & α2).
• Assume a d-dimensional problem on an NxNx…xN

grid with h = N -1.
• Initial error is ||e h || = ||u h - 0 || = ||u h || = O(1) .
• Must reduce this to ||e h || = O(h p) = O(N -p) .
• We can determine θ, the number of V-cycles needed

to do this, if we can bound its convergence factor γ.
U-Boulder 93 of 312
Work to converge to the level
of discretization error
• Using θ V-cycles with convergence factor γ gives an
overall convergence factor of γ θ.
• We must therefore have γθ ∼ O( N − p),
or .
θ ∼ O( l og N )
• Since 1 V-cycle costs O(1) WUs & 1 WUs is O(N d ),
then converging to the level of discretization error
using the MV method costs
d
O( N direct
• This compares to fast )
l og Nmethods (fast Poisson
solvers). But we’ll see that MV can do even better.
U-Boulder 94 of 312
Numerical example
• Consider the 2-D model problem (with σ = 0):
− ux x − uy y = 2 ( 1 − 6x 2) y 2 ( 1 − y 2) + ( 1 − 6y 2) x 2 ( 1 − x 2)
in the unit square, with u = 0 Dirichlet boundary.
• The solution to this problem is
u( x , y ) = ( x 2 − x 4 ) ( y 4 − y 2)
• We examine the effectiveness of MV cycling to

solve this problem on NxN grids [(N-1) x (N-1)
interior points] for N = 16, 32, 64, 128.
U-Boulder 95 of 312
Numerical results
MV cycling
Shown are the results of

16 V(2,1)-cycles. We
display, after each cycle,
residual norms, total
error norms, & ratios of
these norms to their
values after the previous
cycle.
N = 16, 32, 64, 128.
||r h||h = h ||r h||2

scaled residual error
||e ||h = h ||u (h) - v h||2
scaled discrete total error
U-Boulder 96 of 312
A warning about bounds
• Bounds like ||en + 1|| ≤ γ ||en || & ||u (h) - u h|| = O(h)
are only just that--bounds!
• If you see behavior that suggests that these
bounds are sharp (e.g., halving h halves the
discretization error), then great. If you don’t see
this behavior, don’t assume things are wrong.
• Think about this:

O(h2 ) = O(h) but generally O(h) ≠ O(h2 ) !!!
(Any process that is O(h2) is also O(h),
but the converse isn’t necessarily true.)
U-Boulder 97 of 312
Reconsideration
You want to approximate uh.
A good iteration is the V-cycle.
What’s a good way to start it?
Can you do better than vh ← 0?
Start on the coarse grid!
Use nested iteration for the V-cycle!
U-Boulder 98 of 312
Look again at nested iteration
• Idea: It’s cheaper to solve a problem (fewer iterations)
if the initial guess is good.
• How to get a good initial guess:

– “Solve” the problem on the coarse grid first.
– Interpolate the coarse solution to the fine grid.
• Now, let’s use the V-cycle as the solver on each grid

level! This defines the Full Multigrid (FMG) cycle.
U-Boulder 99 of 312
Full multigrid (FMG)
v h ← FM G ( f h )
• Initialize f h , f 2h , f 4h , . . . , f H
• Solve on coarsest grid −1 H

v H = ( A H) f
• Interpolate initial guess

• Perform V-cycle v 2h ← I 42hh v 4h
• Interpolate initial guess v 2h ← MV 2h ( v 2h , f 2h )
• Perform V-cycle v h ← I 2hh v 2h

v h ← MV h ( v h , f h )
U-Boulder 100 of 312

FMG-cycle
• Restriction
• Interpolation
• High-Order Interpolation?

FMG-cycle (recursive form)
v h ← FM G ( f h ) , η
1) Initialize f h , f 2h , f 4h , . . . , f H
2) If h is the coarsest grid, then solve exactly;

Ω
else:
v 2h ← FM G ( f 2h )
3) Set initial guess: .

v h ← I 2hh v 2h
4) Perform η times.
v h ← M V( v h , f h )

FMG cycle cost
One2 V-cycle is performed per level, at a cost of

1 − 2 − d WUs per grid (where the WU is for the
size of the finest grid involved). − jd
The size of the WU for coarse-grid j is 2 times
the size of the WU for the fine grid (grid 0).
Hence, the cost of the FMG(1,1) cycle in WUs is
less than
 2  −d − 2d 2
  ( 1 + 2 + 2 + . . . ) =
 1− 2−d  −d 2
( 1−2 )
d = 1: 8 WUs; d = 2: 32/9 WUs; d = 3: 128/49 WUs.

Has discretization error
been reached by FMG?
• If discretization error is achieved, then ||eh|| = O(h2)

& the error norms at the “solution” points in the cycle
should form a Cauchy sequence:
||eh|| ≈ 0.25 ||e2h||
Let’s see this algebraically…

Interpolation stability
how interpolation affects error
• Property: || P e 2h || ≤ β|| e 2h ||
• Reasoning:
|| P e 2h || ≤ ||P|| || e 2h ||
= ||PT P||1/2 || e 2h ||
⇒ ||PT P||1/2 ≤ √2

Approximation property
how the discrete solutions approximate each other
Property: ||u h - P u 2h || ≤ αKh 2

Reasoning:
|(u (h) - P u (2h))i| = |u(xi) - (u(xi - 1 ) + u(xi + 1))/2| ≤ |u”(ξ)|h 2/2
⇒ ||u (h) - P u (2h) || ≤ cKh 2
⇒ ||u h - P u 2h ||
≤ ||u h - u (h) || + ||u (h) - P u (2h) || + || P u (2h) - P u 2h ||
≤ Kh 2 + cKh 2 + || P ||·||u (2h) - u 2h ||
≤ Kh 2 + cKh 2 + βK(2h)2
≤ (1 + c + 4 β)Kh 2

FMG accuracy
||e 2h || ≤ K(2h)2
Assume:
||e 2h || ≤ K(2h)2 induction hypothesis
||u h - P u 2h || ≤ αKh 2 approximation property (α≈5)
|| P w 2h || ≤ β|| w 2h || interpolation stability (β≈1)
Triangle inequality:
||e h || = ||u h - P v 2h ||
≤ ||u h - P u 2h || + || P (u 2h - v 2h)||
≤ αKh 2 + β K(2h)2
= (α + 4β)Kh 2
⇒ ||e h || ≤ “9”Kh 2
So we need only reduce ||eh|| by “0.1”!!!

Numerical example
• Consider again the 2-D model problem (with σ = 0):
− ux x − uy y = 2 ( 1 − 6x 2) y 2 ( 1 − y 2) + ( 1 − 6y 2) x 2 ( 1 − x 2)
inside the unit square, with u = 0 on the boundary.
• We examine the effectiveness of FMG cycling to

solve this problem on NxN grids [(N-1) x (N-1)
interior] for N = 16, 32, 64, 128.

FMG results
3 FMG cycles & comparison with MV cycle results
1.03e-04
2.58e-05
6.44e-06
1.61e-06
||e ||h = h ||u (h) - v h||2

scaled discrete total error
Diagnostic tools
for debugging the code, the method, the problem
• Finding mistakes in codes, algorithms, concepts, & the

problem itself challenges our scientific abilities.
• This challenge is especially tough for multigrid:
– Interactions between multilevel processes can be very subtle.
– It’s often not easy to know how well multigrid should perform.
• Achi Brandt:
– “The amount of computational work should be proportional to
the amount of real physical changes in the computed solution.”
– “Stalling numerical processes must be wrong.”
• The “computational culture” is best learned by lots of
experience & interaction, but some discussion helps.

Tool # 1: Be methodical
• Modularize your code.

• Test the algebraic solver first.
• Test the discretization next.
• Test the FMG solver last.
• Beware of boundaries, scales, & concepts.
• Ask whether the problem itself is well posed.

Tool # 2: Start simply
• Start from something that already works if you can.
• Introduce complexities slowly & methodically, testing
thoroughly along the way.
• Start with a very coarse fine grid (no oxymoron intended).
• Start with two levels if you can, using a direct solver or lots
of cycles on coarse grids if nothing else.

Tool # 3: Expose trouble
Start simply, but don’t let niceties mask trouble:
• Set reaction/Helmholtz terms to zero.
• Take infinite or very big time steps.
• Don’t take 1-D too seriously, not even 2-D.

Tool # 4: Test fixed point property
Relaxation shouldn’t alter the exact solution of
the linear system (up to machine precision).
• Create a right side: f = Au* with u* given.
• Make sure u* satisfies the right boundary conditions.
• Test relaxation starting with u* : Is r = 0, is it zero
after relaxation, does u* change?
• Test coarse-grid correction starting with u* : Is the
correction zero?

Tool # 5: Test on Au = 0
• The exact solution u* = 0 is known!

• Residual norm ||Au|| & error norm ||u|| are computable.
• Errors ||Au|| & ||u|| should eventually decrease steadily
with a rate that might be predicted by mode analysis.
• Multigrid can converge so fast that early stalling suggests
trouble when it’s just that all machine-representable
numbers in a nonzero u* have already been computed!
Computing r = f - Au & updating u shouldn’t have trouble
with machine precision if you have u* = 0 & thus f = 0.

Tool # 6: Zero out residual
• Using a normal test, try multiplying the residual by 0
before you go to the coarse grid.
• Check to see that the coarse-grid corrections are 0.
• Compare this test with a relaxation-only test--the
results should be identical.

Tool # 7: Print out residual norms
• Use the discrete L 2 norm:
||r||h = (hd Σ ri2)1/2 = hd/2 ||r ||2

• Output ||r||h after each pre- & post-relaxation sweep.
• These norms should decline to zero steadily for each h.
• The norm after post-relaxation should be consistently
smaller than after pre-relaxation--by the predictable
convergence factor at least.

A warning about residuals
• Residuals could mask large smooth errors:
– If ||e||=O(h2), then ||r||=O(h2) for smooth e, but
||r||=O(1) for oscillatory e. (λN /λ1 = O(h-2)!!!)
– This means that if we are controlling the error by
monitoring ||r||, if we don’t know the nature of e,
& if we don’t want to risk large e by requiring only
||r||=O(1), then we may have to work very hard to
make ||r||=O(h2). (We could in effect be trying to
make ||e||=O(h4) !!!)
• Multigrid tends to balance the errors, so ||r||=O(h2)
tends to mean ||e||=O(h2).

Tool # 8: Graph the error
• Run a test on a problem with known solution (Au = 0).
• Plot algebraic error before & after fine-grid relaxation.
• Is the error oscillatory after coarse-grid correction?
• Is the error much smoother after fine-grid relaxation?
• Are there any strange characteristics near boundaries,

interfaces, or other special phenomena?

Tool # 9: Test two-level cycling
• Replace the coarse-grid V-cycle recursive call with a direct

solver if possible, or iterate many times with some method
known to “work” (test ||r || to be sure it’s very small), or use
many recursive V-cycle calls.
• This can be used to test performance between two coarser

levels, especially if residual norm behavior identifies trouble
on a particular level.

Tool # 10: Beware of boundaries
• Boundaries usually require special treatment of the
stencils, intergrid transfers, & sometimes relaxation.
• Special treatment often means special trouble, typically
exposed in later cycles as it begins to infect the interior.
• Replace the boundary by periodic or Dirichlet conditions.
• Relax more at the boundary, perhaps using direct solvers.
• Make sure your coarse-grid approximation at the boundary
is guided by good discretization at the fine-grid boundary.

Tool # 11: Test for symmetry
• If your problem is symmetric or includes a symmetric

case, test for it.
• Check symmetry of the fine-grid & coarse-grid matrices:

are aij & aji relatively equal (to machine precision).
• Be especially watchful for asymmetries near boundaries.

Tool # 12: Check for compatibility
• This is a bit ahead of schedule, but consider the problem
-u ” = f with u ’(0) = u ’(1) = 0.
• It’s singular: If u = 1, then -u ” = 0 & u ’(0) = u ’(1) = 0.
• It’s is solvable iff f ∈ Range(∂xx) = η⊥(∂xx ) = {1}⊥ or f ⊥ 1.
• First fix the grid h right side: f h ← f h - (<f h, 1>/<1,1>)1.
• Do this on coarse grids too: f 2h ← f 2h - (<f 2h, 1>/<1,1>)1.
• Uniqueness is also a worry: u h ← u h - (<u h, 1>/<1,1>)1.

Tool # 13: Test for linearity
• Again, this is a bit ahead of schedule, but if you’re writing

a nonlinear FAS code, it should agree with the linear code
when you test it on a linear problem. Try it.
• Move gradually to the target nonlinear test problem by

putting a parameter in front of the nonlinear term, then
running tests as the parameter changes slowly from 0 to 1.

Tool # 14: Use a known PDE solution
• Set up the source term (f = Au* in Ω) & data (g = u* on Γ).
• Do multigrid results compare qualitatively with sampled u* ?
• Monitor ||u* - u h||h .
• Test a case with no discretization error (let u* = 2nd degree
polynomial). The algebraic error should tend steadily to 0.
• Test a case with discretization error (let u* have a nonzero
3rd derivative). The algebraic error should decrease rapidly
at first, then stall at discretization error level. Check error
behavior as you decrease h. Does it behave like O(h2) (h
halved ⇒ error halved) or whatever it should behave like?

Tool # 15: Test FMG accuracy
• Make sure first that the algebraic solver converges as

predicted, with uniformly bounded convergence factors.
• Test the discretization using Tool # 14.
• Compare FMG total error to discretization error for

various h. You might need to tune the FMG process here
(play with the number of cycles & relaxation sweeps).

Computing assignments
• Document:
norms/weights, V(ν1, ν2), errors, labels (table, graph)
• Use various scenarios:
Ax = 0, Ax = f, varying N & νi & ω, Jacobi/Gauss-Seidel
• Thoroughly test:
don’t stop until you get what you expect.
compare with known solution, text, others.
study discretization & algebraic errors.
report on “asymptotic” factors.
• Be kind to the reader:
code = zzz… tables = + tables ok = ++
& graphs
by hand
↓
tables & graphs & discussion (clear, ↓ = +++
concise)
• Discuss, discuss, discuss:
what do you see & think? what did you learn?

Outline
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
– Complexity
√ • Multilevel Adaptive Methods

5. Some theory
What is A2h?
• Recall the 2-grid coarse-grid correction scheme:
– 1) Relax on Ah u h = f h on Ωh to get v h.
– 2) Compute f2h = Ih2h (f h - Ah v h) .
– 4) Solve A2h u 2h = f 2h on Ω2h .
– 5) Correct fine-grid solution vh ← vh + I2h u2h.
h
• Assume that eh ∈ Range(I2h ). Then the residual
equation can be written h
rh = Aheh = Ah I2h u2h for some u2h ∈ Ω2h.
h 2h
This characterizes u , but with too many equations.
• How does Ah act on Range(I2h )?
h
h
How does A act on R a n ge ( I 2hh ) ?
u 2h
I 2hh u 2h
h h 2h
A I 2h u
• Therefore, the odd rows of Ah I2hh are zero (for 1-D

only) & r2i + 1 = 0. We therefore keep the even rows of
h the residual equations on Ω2h. These rows
Ah I2h for
can be picked out by applying restriction!
I h2h A h I 2hh u 2h = I h2h r h

Building A2h: The Galerkin condition
• The residual equation on the coarse grid is
I h2h A h I 2hh u 2h = I h2h r h
• We thus identify the coarse-grid operator as
2h 2h h h
A = I h A I 2h A2h = RAP
• How do we know RAP is symmetric?

If PT = R & RT = P, then
(RAP)T = PTATRT = RAP.
Computing the i th row of A 2h.
• Compute A 2h e 2i h where e 2i h = ( 0, 0, . . . , 0, 1, 0, . . . , 0) T
↑ Why is this the i th row of A2h ?
i −1 i i +1
2h
0 1 0 ei
1 1 h 2h
0 2 1 2 0 I 2h e i
1 1 1 h h 2h
− 0 0 − A I 2h e i
2 2
2h h 2h 2
1 1 1 2h h h 2h
− − I h A I 2h e i
2 2
4h 2h 4h 2
The i th row of A 2h looks a
lot like the i th row of A h !
• The i th row of 2h is 1
A −1 2 −1 ,
( 2h) 2
which is the 2h version of h.
Ω A
• Note that IF relaxation on h leaves only error
in the range of interpolation,Ωthen solving
A 2h u 2h = f 2h
determines the error exactly!
• This is generally not feasible, but this logic
leads to a very plausible representation for .
A 2h
Variational properties of coarsening
• The definition for A 2h that resulted from the
foregoing line of reasoning is useful for both
theoretical & practical reasons. Together with
the commonly used relationship between
restriction & prolongation, we have the
variational properties:
A 2h = I h2h A h I 2hh Galerkin Condition
h 2h T
I 2h = c ( I h ) for c in ℜ.
Properties of restriction
in a little more detail…
2h h 2h or
• Full Weighting: Ih :Ω → Ω
2h ℜN-1 ℜN/2-1
Ih :
• N = 8:
1 2 1
2h 1
Ih = 1 2 1
4
1 2 1 3× 7
• has rank & null space η( ) with dim .

2h N N
Ih 2
−1 I 2h
h 2
Spectral properties of restriction
• How does I 2h act on the eigenvectors of h?

h A
• Consider  j kπ  , 1 ≤ k ≤ N - 1, 0 ≤ j ≤ N.
h = s in
wk, j  N 
 
kπ  2h
( I h2h wkh ) = co s 2 2N  wk , j
j  
for 1 ≤ k ≤ N/2. ≡ c k wk2,hj

Spectral properties (cont’d)
• i.e., I 2h [kth mode on h] = c [ kth mode on 2h ]
h Ω k Ω
• • •
• h : N = 8, k = 2
• • Ω
• • •
I h2h
• : N = 4, k = 2
• • • Ω 2h
•
N N
• Let k ′ = N − k for 1 ≤ k < , so that < k′ < N .
2 2
kπ  2h
( I h2h wkh′ ) = − sin 2
 2N  wk , j
j  
≡ s k wk2,hj

h] = -sk [ kth mode on Ω 2h]
I h2h [(N-k) mode on
i.e., th
Ω
•
• •
h : N = 8, k = 6
• • Ω
• •
•
2h
Ih
• : N = 4, k = 2
•
• 2h
• Ω
•
• Summarizing:
I h2h wkh = c k wk2h
}{
N
1<k <
2
I h2h wkh′ = − s k wk2h k′ = N − k
I h2h wN
h
/2 = 0
• Complementary modes:
W k = span { wkh , wkh′ }
I h2h W k → {wk2h}∞

Null space of restriction
• Observe that η(Ih2h) = span(Ahêih), where i is odd &

êih is the i th unit vector.
• Let ηi = Ahêih.

• While the ηi looks oscillatory, 0 − 1 contains

0 0it 0generally 2 −1 0 0
all Fourier modes of :
h
A
N −1
ηi = ∑ a k wk ak ≠ 0
• All the Fourierkmodes
=1 of are needed to
represent the null space of restriction!
h
A

Properties of interpolation
• Interpolation: I h : Ω 2h → Ω h or
2h
h : ℜN/2-1 ℜN-1
I 2h
• N = 8: 1
2
1 1
1
I 2hh = 2
2
1 1
2
• has full rank & null space
1 {0}.
I 2hh
Spectral properties of interpolation
• How does I h act on the eigenvectors of 2h ?

2h A
• Consider  j kπ  , 1 ≤ k ≤ N/2-1,
( wk ) = sin 
2h

j  N / 2  0 ≤ j ≤ N/2.
• A bit of work shows that the modes of are

NOT “preserved” by A 2h
, but that the space is
“preserved”: I 2hh Wk
kπ  h  kπ  h
I 2hh wk2h = co s 2
 2N  wk − sin  2N
2
 wk ′
   
= c k wkh − s k wkh′
Spectral properties of interpolation
I2hh wk2h = ck wkh − sk wkh′
• Interpolation of smooth Ω 2h modes excites

oscillatory modes on h.
Ω
• Note that if N , then
k «
2
  2   h  k2  h
k
I 2hh wk2h =  1 − O   wk + O  2  wk ′
2
 N   N 
• is 2nd-order
≈w interpolation.
h
k
h
I 2h
Range of interpolation
• The range of h is the span of the columns of h .

I 2h I 2h
• Let be the i th column of h .
ξi I 2h
ξi :
N −1
h
bk wkh ,
ξi = ∑ bk ≠ 0
=
k 1
• All the Fourier modes of are needed to represent
Range( ).
Ah
h
I 2h
Use all the facts to analyze the
coarse-grid correction scheme
relax only before correction
1) Relax once on Ωh : vh ← R vh + B f h
2) Compute & restrict residual f 2h ← I h2h ( f h − A h v h )
3) Solve residual equation
2h − 1 2h
v =(A )
2h f
4) Correct fine-grid solution .
v h ← v h + I h v 2h
• The entire process appears as 2h
v hexactα vh h 2h − 1 2h h h
• The ← R solution
I satisfies
+ 2h ( A ) Ih ( f − A ( R v h + B f h ))
v h = R uh + I2hh ( A 2h ) − 1 Ih2h ( f h − A h ( R uh + B f h ))
CG error propagation
• Subtracting the previous two expressions, we get
h 2h − 1 2h h
eh ← I − I2h ( A ) Ih A R e h
e h ← CG e h
• How does CG act on the modes of ? Assume e h
consists of the modes & Ah
for
& . wkh h
wk′ 1 ≤ k ≤ N /2
k′ = N − k
• We know how
act on & .α h 2h 2h − 1
R , A , I , (A )h , Ih 2h
wkh h
wk′

CG error propagation
• For now, assume no relaxation. Then
W k = spa n { wkh , wkh′ }
is invariant under CG:
CG wkh = s k wkh + s k wkh′

CG wkh′ = c k wkh + c k wkh′
where
 kπ   kπ 
c k = co s 2   s k = sin2  
 2N   2N 

CG error propagation for k <<N
• Consider the case k << N (extremely smooth &

oscillatory modes):
 k2   k2 
wk → O  + O
2  k 2  k′
w w
N  N 
  k2     k2  
wk ′ →  1 − O  +  1 − O
2  k 2   k′
w w
 N   N 
• Hence, CG eliminates the smooth modes but does
not damp the oscillatory modes of the error!

CG with relaxation
• Next, include one relaxation sweep. Assume that

the relaxation R preserves the modes of A h
(although this is often unnecessary). Let
λk
denote the eigenvalue of associated with .
For k < < N/2: R wk
wk → λk sk wk + λk sk wk ′ Small!
wk ′ → λk ′ ck wk + λk ′ ck wk ′ Small!

Crucial observation
• Between relaxation & coarse-grid correction,

both smooth & oscillatory components of the
error are effectively damped.
• This is the “spectral” picture of how multigrid

works. We examine now another viewpoint,
the “algebraic” picture of multigrid.

Recall the variational properties
• All the analysis that follows assumes that the

variational properties hold:
2h 2h h h
A = I h A I 2h Galerkin Condition
h 2h T
I 2h = c ( Ih ) for c in ℜ.

Algebraic interpretation of CG
h& 2h .
• Consider the subspaces that make up Ω Ω
From now on, ‘R( )’ refers
to the Range of a linear
h
Ω R ( I 2hh ) N ( I h2h ) operator & ‘N( )’ to its
Null Space.
h From the Fundamental Theorem

I 2h of Linear Algebra:
I h2h
2h 2h T
N( Ih ) ⊥ R( ( Ih ) )
or
Ω 2h R ( I h2h )
N ( I h2h ) ⊥ R ( I 2hh )
But we really care about N (Ih2h Ah ) !?!?

Subspace decomposition of Ω h
• If uh ∈ N (Ih2h Ah ), then for any u 2h we have
0 = 〈 Ih2h A hu h, u 2h 〉 = 〈 Ahu h, I2hhu 2h 〉 ,
so
R ( I h ) ⊥ h N ( I 2h A h ) ,
2h h
A
where means .
“energy”
x ⊥any
• Moreover, h y can be written
h
A x , y as= 0 inner product,
A
h h
where e& . e h = s h + t
h
• Hence, s h ∈ R ( I 2h ) t h ∈ N ( I h2h A h )
.
Ω h = R ( I 2hh ) ⊕ N ( I h2h A h )

Characteristics of the subspaces
• Since s h = I 2hh q2h for some q 2h ∈ Ω 2h, we
associate s h with the smooth components of e h .
But, s h generally has all Fourier modes in it.
Recall the basis vectors for h :
I 2h
• Similarly, we associate with oscillatory

components of , although t h h generally has all
Fourier modes ine hit as well. Recall
t that is
spanned by , so is spanned 2h by
the unit vectors h ( I h odd
Nfor ) i,
2h h
which “look”ηoscillatory.
i = A ei N ( Ih A )
h T
e i = ( 0, 0, . . . , 0, 1, 0, . . . , 0)

Algebraic analysis CG
• Recall that (without relaxation)
h 2h − 1 2h h
CG = I − I 2h ( A ) Ih A
• First note that if h , then .
This follows since s ∈ R ( I 2h )
h G sh = 0
for Csome
& therefore s h = I 2hh q 2h q 2h ∈ Ω 2h
2h − 1 2h h
CGsh = I h
− I2h ( A ) Ih A h 2h
I2h q =0
• It follows that N(CG)=R( ), that is, 2htheby null
A Galerkin property
space of CG is the range ofh interpolation.
I2h

More algebraic analysis of CG
• Next, note that if t h ∈ N ( I h2h A h ) , then
h h 2h − 1 2h h h
CG t = I − I2h ( A ) Ih A t
0
h h
⇒ CG t = t
• Thus, CG is the identity on .
2h h
N ( Ih A )

How does the algebraic picture
fit with the spectral view?
h in two ways:
• We may view Ω
h = Low frequency modes

Ω 1 ≤ k ≤ N/2
High frequency modes
N/2 < k < N
that is,
h
Ω = L ⊕ H
or
h h 2h h
Ω = R ( I 2h ) ⊕ N( Ih A )
Actually, each view is just part
of the picture
• The operations we have examined work on different
spaces!
• While 2h h is mostly oscillatory, it isn’t ,

N ( I h A ) is mostly smooth, it isn’t H
& while .
h
R ( I 2h ) L
• Relaxation eliminates error from .
H
• Coarse-grid correction eliminates error from .
R ( I 2hh )

How it actually works (cartoon)
H N ( I h2h A h )
eh
th
R ( I 2hh )
sh

Relaxation eliminates H, but
increases the error in R ( I 2hh ).
H N ( I h2h A h )
h
t L
eh
R ( I 2hh )
sh

h
CG eliminates error in R ( I 2h ),
but increases error in H
H N ( I h2h A h )
h
t L
eh
R ( I 2hh )
sh

Outline
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
– Complexity
√

Some reflection
• What do you want out of this course?
• Do you seem to be getting there?
• Can we get there more directly?
• Start thinking about your project!
• Think about doing it in two parts.

6. Nonlinear problems
• How should we approach the nonlinear system

A ( u) = f
& can we use MG to solve it?
• A fundamental relation we’ve relied on is the linear

residual equation:
We A u −rely
can’t A v on=this
f −now ⇒ Aa enonlinear
A v since = r A(u)
generally means
A ( u ) − A ( v ) ≠ A ( e)
The nonlinear residual equation
• We still base our development around the residual

equation, now the nonlinear residual equation:
A ( u) = f
A ( u) − A ( v ) = f − A ( v )
A ( u) − A ( v ) = r
• How can we use this equation as the basis for a
solution method?

Let’s consider Newton’s method
• Best known & most important nonlinear problem solver!

• We wish to solve F(x) = 0 .
• Suppose for the moment that F is scalar F : ℜ→ ℜ.
• Expand F in a Taylor series about x :
F( x + s) = F( x) + sF ′( x) + s 2 F ″ ( ξ)
• Dropping higher order terms (hot), if x + s is a solution,
= F(arrive
• We0thus x) + sat
F ′Newton’s
( x) ∴ s = − F( x ) / F ′ ( x )
method:
F( x )
x← x −
F ′( x)
Newton’s method for systems
• The system A(u) = f in vector form is
 a1 (u1 ,u2 ,...,u N )   f1 

   
 a2 (u1 ,u2 ,...,u N )  =  f2 
 M  M
   
a N (u1 ,u2 ,...,u N )  f N 
• Expanding A(v + e) in a Taylor series about v :
A ( v + e) = A ( v ) + J( v ) e + hot
 ∂a 
J(u) =  i 
U-Boulder
 ∂u j  168 of 312
Newton for systems (cont’d)
• J(v) is the Jacobian
 ∂a1 ∂a1 ∂a1 
 ∂u L
∂u2 ∂u N 
 1 
 ∂a2 ∂a2
L
∂a2 
J(v) =  ∂u1 ∂u2 ∂u N 
 M M O M 
 ∂a N ∂a N ∂a N 
 L 
 ∂ ∂
• If u = v + e is a solution,
u u ∂ 
2 f = A(v)N + J(v)e + hot, so
u
1
u=v
e ≈ [J(v)]-1(f - A(v))
• This leads to the iteration
v ← v + [J(v)]-1(f - A(v))
Newton’s via the residual equation
• The nonlinear residual equation is
A ( v + e) − A ( v ) = r
• Expanding A(v + e) in a two-term Taylor series about v
& ignoring hot:
A ( v ) +J( v ) e − A ( v ) = r
• Newton’s method is J( v)
thus: e =r
r = f − A( v)
−1
v ← v + J( v) r
How does multigrid fit in?
• One obvious method is to use multigrid to solve

J(v)e = r at each iteration step. This method is
called Newton-MG & can be very effective.
• However, we would like to use multigrid ideas to

treat the nonlinearity directly.
• Hence, we need to specialize the multigrid

components (relaxation, grid transfers,
coarsening) for the nonlinear case.

What is nonlinear relaxation?
• Several of the common relaxation schemes have
nonlinear counterparts. For A(u) = f, we describe
nonlinear Gauss-Seidel:
For each j = 1, 2, …, N :
Set the jth component of the residual to 0
& solve for vj . That is, solve (A(v))j = fj .
• Equivalently,
For each j = 1, 2, …, N :
Find s ∈ ℜ such that ,
where εj is the canonical jth unit basis vector.
( A ( v + s εj ) ) j = f j

How is nonlinear Gauss-Seidel done?
• Each ( A ( v ) ) j = f j is a nonlinear scalar equation for
vj . We can use the scalar Newton’s method to solve!
• Example: ( ) may be
− uso″ (that
discretized x ) + u( x ) e u x =isf given
, by
( A( v) ) j = f j
− v j − 1 + 2v j − v j + 1
vj
2
1 ≤ j ≤ N −1
+ vj e = fj
h iteration for v is given by
• Newton j
− vj − 1 + 2 vj − vj + 1 vj
2
+ vj e − fj
h
vj ← vj −
2 vj
2
+ e ( 1 + vj)
U-Boulder
h 173 of 312
How do we do coarsening for
nonlinear multigrid?
• Recall the nonlinear residual equation
A ( v + e) − A ( v ) = r
• In multigrid, we obtain an approximate solution vh
on the fine grid, then solve the residual equation
on the coarse grid.
• The residual equation on appears as

Ω 2h
A 2h ( v 2h + e 2h ) − A 2h ( v 2h ) = r 2h

Look at the coarse residual equation
• We must evaluate the quantities on Ω 2h in
2h 2h 2h 2h
A (v
2h +e ) −A 2h (v ) =r
• Given vh, a fine-grid approximation, we restrict
the residual to the coarse grid:
r 2h = I h2h ( f h − A h ( v h ) )
• For v2h we restrict vh by
v 2h = I h2h v h
Thus,
A 2h ( Ih2h v h + e2h ) = A 2h ( Ih2h v h ) + Ih2h ( f h − A h( v h) )

We’ve obtained a coarse-grid
equation of the form A2h(u2h) = f2h
• Consider the coarse-grid residual equation:
u 2h f 2h
coarse-grid unknown all quantities are known
• We solve for &
obtain A 2h ( u2h) = f 2h u 2h = I h2h v h + e 2h
e 2the
• We then apply u 2h − I h2h v h
h = correction:
h 2h
vh = vh + I 2h e
Full approximation scheme (FAS)
2-grid form
• Perform nonlinear relaxation on A h( u h) = f h to
obtain an approximation v h .
• Restrict the approximation & its residual
v 2h = I 2h v h r 2h = I 2h ( f h − A ( v h ) )
• Solve the coarse-grid
h equation h
• Extract 2Ah2happroximation
( u 2h) = A 2h( vto2hh) error
+ r 2h
• Interpolate & correct
e 2h = u 2h − v 2h
v h = v h + I 2hh e 2h
A few observations about FAS
• If A is a linear operator, then FAS reduces directly to

the linear two-grid correction scheme.
• An exact solution to the fine-grid problem is a fixed

point of the FAS iteration.

A few more observations about FAS
• The FAS coarse-grid equation can be written as
2h 2h
A ( u 2h) =f + τ2hh
where 2h h is the so-called tau
τh =term
correction A(Ih &v ) − Iish A(v
2h h
2h
2h
the )original 2h source term,
f
provided you choose it that way: .
2h 2h h
f =I h f
• In general, since , the solution to the FAS
coarse-grid equationτ2hh is
≠ not
0 the same asuthe 2h solution
to the original coarse-grid problem
A 2is
• The tau correction ( u ) = f 2halter the coarse-
h as 2ah way to
grid equation to enhance its approximation properties.

Still more observations about FAS
• FAS may be viewed as an inner & outer iteration:

the outer iteration is the coarse-grid correction,
the inner iteration the relaxation method.
• A true multilevel FAS process is recursive, using

FAS to solve the nonlinear Ω2h problem using Ω4h.
Hence, FAS is generally employed in a V- or W-
cycling scheme.

Even more observations about FAS
• For linear problems, we use FMG to obtain a good
initial guess on the fine grid. Convergence of
nonlinear iterations depends critically on having a
good initial guess.
• When FMG is used for nonlinear problems, the
interpolant is generally accurate enough to
I 2hh of
be in the basin u 22hhattraction of the fine-grid
solver.
• Thus, whether FAS, Newton, or Newton-multigrid
is used on each level, one FMG cycle should provide
a solution accurate to the level of discretization,
unless the nonlinearity is extremely strong.

Intergrid transfers for FAS
• Generally speaking, the standard operators (linear
interpolation, full weighting) work effectively in
FAS schemes.
• For strongly nonlinear problems, higher-order

interpolation (e.g., cubic interpolation) may be
beneficial.
• For an FMG scheme, where is the

interpolation of a coarse-grid hsolution
2h to become a
I 2h u interpolation
fine-grid initial guess, higher-order
is often advised.

What is A2h(u2h) in FAS?
As in the linear case, there are two basic possibilities:
1. A2h(u2h) is determined by discretizing the nonlinear operator

A(u) in the same fashion as was employed to obtain Ah(uh) ,
except that the coarser mesh spacing is used.
2. A2h(u2h) is determined from the Galerkin condition
where the action2h

of the2hGalerkin
2hproduct
h h can2hbe captured in an
A ( u
implementable formula. ) = hI A ( I2h u )
The first method is usually easier & more common.

Nonlinear problems: An example
• Consider
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
on the unit square [0,1] x [0,1] with homogeneous
Dirichlet boundary conditions & a regular h = 1/128
Cartesian grid.
• Suppose the exact solution is
u( x, y) = ( x 2 − x 3 ) sin ( 3 π y)

Discretization of nonlinear example
• The operator can be written (sloppily) as
−1
1   h u i,h j
− 1 4 − 1  ui, j + γ ui, j e
h = f i, j
2
h  −1 
h
A ( u h)
• Relaxation is given by
( A h( u h) ) − f i, j
i, j
uih, j ← uih, j −
4 uhi, j
2
+ γ ( 1 + ui , j ) e
h
h

FAS & Newton’s method on
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
N = 128
• FAS V(2,1)-cycles until ||r|| < 10-10
γ
1 10 100 1000
convergence factor 0.135 0.124 0.098 0.072
number of FAS cycles 12 11 11 10
• Newton’s Method with exact inner solves until ||r|| < 10-10
γ
1 10 100 1000
convergence factor 4.00E-05 7.00E-05 3.00E-04 2.00E-04
number of Newton iterations 3 3 3 4

Newton, Newton-MG, & FAS on
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
N = 128
• Newton uses exact solves, Newton-MG is inexact Newton
with a fixed number of inner V(2,1)-cycles for the Jacobian
problem, overall stopping criterion ||r|| < 10-10.
Outer Inner
Method iterations iterations Megaflops
Newton 3 1660.6
Newton-MG 3 20 56.4
Newton-MG 4 10 38.5
Newton-MG 5 5 25.1
Newton-MG 10 2 22.3
Newton-MG 19 1 24.6
FAS 11 27.1
Comparing FMG-FAS & FMG-Newton
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
• We will do one FMG cycle using a single FAS V(2,1)

-cycle as the “solver” at each new level. We then
follow that with sufficiently many FAS V(2,1)-cycles
as is necessary to obtain ||r|| < 10-10.
• Next, we will do one FMG cycle using a Newton-

multigrid step at each new level (with a single linear
V(2,1)-cycle as the Jacobian “solver.”) We then
follow that with sufficiently many Newton-multigrid
steps as is necessary to obtain ||r|| < 10-10.

Comparing FMG-FAS & FMG-Newton
u( x, y)
− ∆u( x, y) + γ u( x, y) e = f ( x, y)
N = 128
Cycle || r h || || e h || Mflops || r h || || e h || Mflops Cycle
FMG-FAS 1.10E-02 2.00E-05 3.1 1.06E-02 2.50E-05 2.4 FMG-Newton
FAS V 6.80E-04 2.40E-05 5.4 6.70E-04 2.49E-05 4.1 Newton-MG
5.50E-09 2.49E-05 17.7 Newton-MG
1.80E-09 2.49E-05 19.4 Newton-MG
5.60E-10 2.49E-05 21.1 Newton-MG
1.80E-10 2.49E-05 22.8 Newton-MG
5.70E-11 2.49E-05 24.5 Newton-MG

Remembering coarse-grid correction
Au = f
• Relax (Jacobi) to smooth error e = u - v :
v ← v - D-1(Av - f) ↑
I2hh
• Form the residual equation Ae = r :

Ae = A(u - v) = f - Av = r
• Use premise that smooth error ⇒ e = I2he2h :
h
h
AI2he = r
2
h T
h fewer unknowns
• Use transpose I = (I2h) to reduce equations:
2
h 2h
Ih2hAI2he = I2hr
h
h h fewer equations
A2h
Motivating FAS for nonlinear A
A(v + e) = f → A2h(v2h + e2h) = f2h
2h 2h h 2h 2h h 2h h h
A ( Ih v + e 2h ) = A ( Ih v ) + Ih ( f − A ( v h) )
• Analogous to asking how to discretize a PDE of this form:

A(v + e) = f
with v known.
• Example:
γuu’ - u’’ = f → γ (v + e) (v + e)’ - (v + e)’’ = f
• Expand to get an equation in e of form g + ae + be’ + …:
γvv’ - v’’ + γ (v’e + ve’ + ee’ ) - e’’ = f
A(v)
Differential residual equation
γ (v’e + ve’ + ee’ ) - e’’ = f - A(v) ≡ r(v)
• Right side (analogous to injection):

r(v) → rjh = r(xj)
• Coefficients:
v → vjh = v(xj), v’ → (v’)jh = (v(xj+1) - v(xj-1))/(2h)
• Unknowns:
e → e j h, e’ → (e’)jh = (ej+1 - ej-1)/(2h),
e’’ → (e’’)jh = (ej+1 - 2ej h+ ej-1)/
h 2
h
h h h

Leading to FAS…
γuu’-u’’ = f → A(v+e) = f → γ (v’e+ve’+ee’ )-e’’ = f-A(v)
• Fine-grid residual equation (at h point 2j):

h h h h h h h h h
v 2 j +1 −v
h e2 j +1 − e2 j −1
2 j −1 h h e2 j +1 − e2 j −1 −e2 j +1 + 2e2 j − e2 j −1 h
γ e + γv2 j 2j + γe2 j + 2 = r2 j
2h 2h 2h h
• Coarse-grid residual equation (at 2h point j):
h h 2h 2h 2h 2h 2h 2h 2h
v −v
2j+2 2j−2 2h h e
j +1 −e
j −1 2h e j +1 −e j −1 −e
j +1 + 2e − e
j j −1 2h h
γ e + γv
j 2j + γe
j + 2 = (I r ) j
h
4h 4h 4h (2h)
• FAS
2h 2h h 2h 2h h2h 2h h h
h
A ( Ih v + e ) − A ( Ih v ) = Ih ( f − A (v ) )

Example: Newton-MG vs. FAS
• PDE (er, ODE):
− u ″ ( x ) + u( x ) e u( x ) = f ,
• Discretization:
− v j − 1 + 2v j − v j + 1
vj
+ vj e = fj
2
h

One Newton-MG step
• Step 0. Given v, form the grid h linear correction equation:
−e j −1 + 2e j − e j +1 −v j −1 + 2v j − v j +1
( )
vj vj
2 + + j
(1 v )e e j = rj ≡ f j − 2 + j
v e
h
Initialize the Newton correction approximation: e = 0. h
• Step 1: Relax on the grid h linear equation.
• Step 2: Solve the grid 2h error correction equation:
2h 2h 2h
−e j −1 + 2e j − e j +1 −e2 j −1 + 2e2 j − e2 j +1
• Step
+ + 2j
(1 v )e
v 2j 2h
e j = 2j −
r (
3: Correct the grid h Newton correction:
(2h)
2
h
2 + + 2j
(1 v ) e
v2j
e2 j )
• Step 4: Stop if you’vee “solved” hthe2 hlinear equation well
← + 2h e
e I
enough for Newton correction e & set v ← v + e. Else,
leave v alone & return to Step 1.

One FAS step
• Step 0. Given v, form the grid h nonlinear equation:
− vj − 1 + 2vj − vj + 1 vj
2 + vj e = fj
h
• Step 1: Relax on the grid h nonlinear equation to improve v.
• Step 2: Solve the grid 2h FAS correction equation:
2h 2h 2h
−e j −1 + 2e j − e j +1 h 2h v2h j + e 2h h
h v2 j 2h h
+ ( + ) − = ( h r )j
j
2 v2j e j e v2j e I
(2h) the grid h approximation v :
• Step 3: Correct
• Step 4: Stop if you’vev “solved”

← + 2h
v I the
h
e
2 hnonlinear problem well
enough for the solution u. Else, return to Step 1.

Outline
• √ Model Problems •√ Nonlinear Problems
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
– Complexity
√

7. Selected applications
7a. Neumann boundary conditions
• Consider the 1-D problem

− u ″ ( x) = f ( x) 0<x <1
u ′ (this
• We discretize 0) =onu ′the
( 1) interval
=0 [0,1] with 1
h=
grid spacing & nodes , j=1,2, …, N+1 . N +1
xj = j h
• We extend the interval with two ghost points
0 x 1
-1 0 1 2 3 … j ... N-1 N N+1 N+2

Central differences at boundary
• We use differences as before, but now also for
the derivative in the Neumann condition:
-1 0 1 j-1 j j+1 N N+1 N+2
u1 − u − 1 − uj − 1 + 2 uj − uj + 1 uN + 2 − uN
u′ ( 0) ≈ u″ ( x ) ≈ u′ ( 1) ≈
• This yields
2h the system
j
h2 2h
− uj − 1 + 2 uj − uj + 1
= fj 0 ≤ j ≤ N +1
h2
u1 − u − 1 uN + 2 − uN
=0 =0
2h 2h

Eliminating the ghost points
• Use the boundary conditions to eliminate u − 1 , uN + 2

u1 − u − 1 uN + 2 − uN
u − 1 = u1 =0 uN + 2 = uN
=0 2h
2h
• Eliminating the ghost points in the j = 0 & j = N + 1
equations gives the (N+2)x(N+2) system of equations:
− uj − 1 + 2 uj − uj + 1
= fj 0 ≤ j ≤ N +1
h2
2u0 − 2u1 − 2uN + 2uN + 1

= f0 = f N +1
2 2
h h
Write the system in matrix form
h h h , where
• We can write A u =f
 2 −2 
 −1 2 −1 
 
h 1 −1 2 −1 
A =
2 ⋅⋅ ⋅⋅
⋅ ⋅

h  
 −1 2 −1
 
 − 21 2 
• Note that is (N+2)x(N+2) & nonsymmetric, &
the system A h
involves unknowns & at the
boundaries. uh uh
0 N +1

We must consider compatibility
• The problem − u ″ ( x ) = f ( x ), for 0 < x < 1 & with

u ′ ( 0) = u ′ ( 1) = 0 is not well-posed!
• If u(x) is a solution, so is u(x) + constant.
• We cannot be certain a solution exists. If one does,
it must satisfy
1 1 1
− ⌠ u″( x) dx = ⌠ f ( x) dx − [ u′( 1) − u′( 0) ] = ⌠ f ( x) dx
⌡ ⌡ ⌡
0 0 0
1
0 = ⌠ f ( x) dx
⌡
• This integral compatibility condition is necessary!
0
If f(x) doesn’t satisfy it, there is no solution!

The well-posed system
• The compatibility condition is necessary for a
solution to exist. In general, it is also sufficient:
∂2 is a well-behaved operator on the space of
−
∂x 2
functions u(x) that have zero mean.
• Thus we may conclude that if f(x) satisfies the
compatibility condition, this problem is well-posed:
− u ″ ( x) = f ( x) 0<x <1
u ′ ( 0) = u ′ ( 1) = 0
1
⌠ u( x ) dx = 0
• The last says:
⌡ of all possible solutions u(x) + constant,
we choose0 the one with zero mean.

The discrete problem is not well posed
• Since all row sums of A h are zero, then 1h ∈ N S ( A h )
• Putting A h into row-echelon form shows that
h hence h h
dim(N S( A ) ) = 1 N S( A ) = spa n ( 1 )
• By the Fundamental Theorem of Linear Algebra,
has a solution if & only if Ah
f h ⊥ N S ( ( A h) T )
• It is easy to show that
h T T
• Thus, N S ( ( A ) ) = c (
has a solution if & only if 1/ 2, 1, 1, . . . , 1, 1/ 2)
h h h
A
• That is, u = f
h
f ⊥ c ( 1/2, 1, 1, . . . , 1, 1/2) T
1 h N 1 h
f 0 + ∑ f j + f N +1 = 0
h
2 j =1
2
We have two issues to consider
• Solvability. A solution exists iff f h ⊥ N S ( ( A h) T )
• Uniqueness. If h solves h h h, then so does u h + c1h

u A u =f
• Note that if h , then
h T h h T
A = ( A )
& solvability & uniqueness can be handled together) )
NS ( A ) = NS ( ( A
• This is easily done. Multiply 1st & last equations by 1/2,

giving
 1 −1 
 −1 2 −1 
 
h 1 −1 2 −1 
A =  ⋅⋅ ⋅⋅ 
h2 ⋅ ⋅ 
 −1 2 −1
 
 − 1 1 
The new system is symmetric
h h
• We have the symmetric system A uh =f :
 f h/2 
 
u0h  0 
 1 − 1    h 
 −1 2 −1  
u1h  f1 
    
1 − 1 2 − 1  
u2h  f2 h
 ⋅⋅ ⋅⋅    =  
2 ⋅ ⋅
h   M   M 
 − 1 2 − 1   uN h   h 
    fN 
 − 1 1   uh   
 N +1 f h
/2 
• Solvability is guaranteed by ensuring that
 N + 1 is
orthogonal to the constant vector : h
f
h
1
h N +1 h
f , 1h = ∑ fj =0
j =0

The well-posed discrete system
• The (N+3)x(N+2) system is:
− uj − 1 + 2 uj − uj + 1
= fj 0 ≤ j ≤ N +1
2
h
u0 − u1 f 0
=
2 2
h
− uN + uN + 1 f N + 1
=
h 2 2
N +1
∑ uih = 0 (choose the zero mean solution)
i =0
or, more simply
h h
A uh =f
u h, 1h =0
Multigrid for the Neumann problem
• We must have the interval endpoints on all grids
x 0h x 1h h
xN /2
h
xN
x 02h 2h
xN 2h
xN
/4 /2
• Relaxation is performed at all points, including endpoints:
h
v jh − 1 + + v jh+ 1 + h 2 f j
2 h 2 h
1 ← v N + h on
v 0 ←add
h v 1 + ah global
h h h
• We f0 Gram-Schmidt
v jh ←
2step after
v N +relaxation f N +1
each level to enforce the zero-mean condition
v h, 1h
vh ← vh − 1h
1h, 1h
Interpolation must include the
endpoints
• We use linear interpolation:
 1 
 1/2 1/2 
 
 1 
 1/2 1/2 
 
I 2hh =  1 
 1/2 1/2 
 
 1 
 1/2 1/2 
 
 1 
Restriction also treats the
endpoints
1 h T
• For restriction, we use I 2h = ( I 2h) , yielding the
h 2
values
2h 1 h 1 h
f 0 = f0 + f1
2 4
2h 1 h 1 h 1 h
f j = f 2j − 1 + f 2j + f 2j + 1
4 2 4
2h 1 h 1 h
f N +1 = f N + f N +1
4 2

The coarse-grid operator
• We compute the coarse-grid operator using the

Galerkin condition 2h 2h h h
A = I h A I 2h
j2h = 0 1
ε20h 1 0  1 −1 
 −1 2 −1 
1  
I 2hhε20h 1 0 2h 1  −1 2 −1 
2 A =
2 ⋅⋅ ⋅⋅
⋅ ⋅

4h  
2h h 2h 1 1 
A h I 2hε0 0 − −1 2 −1
2  
2h 2h 2  − 1 1 
2h h h 2h 1 1
I h A I 2hε0 −
4h 2 4h 2
Coarse-grid solvability
h h
• Assuming f satisfies f , 1h = 0 , the solvability
condition, we can show that theoretically the coarse-
grid problem A 2h u 2h = I 2h ( f h − A hv h ) is also solvable.
h
• To be certain numerical round-off does not perturb

solvability, we incorporate a Gram-Schmidt-like step
each time a new right-hand side is generated for
2h
the coarse grid: f
2h
f , 12h
2h 2h
f ← f − 12h
12h , 12h

Neumann problem: An example
• Consider the problem − u″ ( x ) = 2 x − 1 ,
0 <x <1 u′ ( 0 ) = u′ ( 1 ) = 0
which has x2 x3 as a solution for any c
u( xthe
(c = -1/12 gives ) =zero− mean
+ csolution).
2 3
grid size average number

|| r h || || e h ||
N conv. factor of cycles
31 6.30E-11 0.079 9.70E-05 9
63 1.90E-11 0.089 2.40E-05 10
V(2,1)
127 2.60E-11 0.093 5.90E-06 10 cycles
255 3.70E-11 0.096 1.50E-06 10
511 5.70E-11 0.100 3.70E-07 10
1027 8.60E-11 0.104 9.20E-08 10
2047 2.10E-11 0.112 2.30E-08 10
4095 5.20E-11 0.122 5.70E-09 10
7b. Anisotropic problems
1
• All problems considered thus far have had −
2
as
h
the off-diagonal entries.
• We consider two situations when the matrix has

two different constants on the off-diagonals.
These situations arise when
– the (2-d) differential equation has constant but
different coefficients for the derivatives in the
coordinate directions
– the discretization has constant but different mesh
spacing in the different coordinate directions

We consider two types of anisotropy
• Different coefficients on the derivatives
− ux x − α uyy = f
discretized on a uniform grid with spacing h .
• Different mesh spacings:
1
hx = h =
N
hy hx
hy =
α
U-Boulder hx 215 of 312
Both problems lead to the same
stencil
− u j − 1, k +2 u j , k − u j + 1, k − uj , k − 1 + 2 uj , k − uj , k + 1
+α
2
h h2
−α
h 1  
A = − 1 2 + 2α − 1 
2
h  −α 
− u j − 1, k +2 u j , k − u j + 1, k − uj , k − 1 + 2 uj , k − uj , k + 1
+
2 2
h  h 
 
 α 
Why standard multigrid fails
−α
h 1  
A =  − 1 2 + 2α − 1 
h2 −α 
• Note that Ah has weak connections in the y-direction.
MG convergence factors degrade as α gets small, with
poor performance already at α = 0.1 .
• Consider the limiting case α ⇒ 0: 0
h 1  
A ⇒  − 1 2 +22α − 1 
2
• Collection of disconnected 1-D problems!h  0 
• Point relaxation will smooth oscillatory errors in the
x-direction (strong connections), but with no
connections in the y-direction, the errors in that
direction will generally be random; point relaxation
provides no smoothing in the y-direction.

We analyze weighted Jacobi
• The eigenvalues of the weighted Jacobi iteration
matrix for this problem are
2ω   iπ   lπ  
λi, l = 1 − 
2
sin   + α sin 
2

1+α   2N   2N 
1 1
1 0.9 0.9
0.8 0.8 0.8
0.7 0.7
0.6
0.6 0.6
0.4
0.5 0.5
0.2
0.4 0.4
0 15
0 0.3 0.3
10 0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
5
10
15
5
l i l
i
0 l
k

Two strategies for anisotropy
• Semicoarsening: Because we expect MG-like

convergence for the 1-D problems along lines of
constant y, we could coarsen the grid in the
x-direction, but not in the y-direction.
• Line relaxation: Because the the equations are

strongly coupled in the x-direction, we could solve
simultaneously for entire lines of unknowns in the
x-direction (along lines of constant y).

Semicoarsening with point relaxation
−α
• Point relaxation on Ah 1 
smooths in the
=  − 1 2 + 2α − 1 
h2 −α 
x-direction. Coarsen by removing every other y-line.
h
Ω Ω 2h
• We do not coarsen along the remaining y-lines.
• Semicoarsening is not as “fast” as full coarsening. The

number of points on is about half the number of
points on , instead of the usual one-fourth.
Ω 2h
Ωh
Interpolation with semicoarsening
• We interpolate in the 1-D way along each line of

constant y.
• The coarse-grid correction equations are
v 2h j , k = v 2h j , k + v j2,hk
v j2,hk + v j2h+ 1, k
v 2h j + 1, k = v 2h j + 1, k +
2

Line relaxation with full coarsening
• The other approach to this problem is to do full

coarsening, but to relax entire lines (constant y)
of variables simultaneously.
• Write Ah in block form as
 D − cI 
 − cI D − cI 
 
Ah =  − cI D − cI 
 ⋅⋅ ⋅⋅ 
 ⋅ ⋅ − cI 
where  − cI D 
&
 2 + 2α − 1 
α 1  − 1 2 + 2α − 1 
c = D =  ⋅⋅ 
h2 h2 ⋅ 
 − 1 2 + 2α 
Line relaxation
• One sweep of line relaxation consists of solving a
tridiagonal system for each line of constant y.
• The kth such system has the form , where h

is the kth subvector of D v k = gk
with entries
h h v&k
h
the kth right-hand side vsubvector is ( v kh ) j = v jh, k
h α
( gk ) j = f j , k +
h ( v jh, k − 1 + v jh, k + 1 )
2
h
• Because D is tridiagonal, the kth system can be solved
very efficiently.

Why line relaxation works
• The eigenvalues of the weighted block Jacobi
iteration matrix are
2ω  2  i π  + α s in2  l π  
λi , l = 1 −  s i n  2N   2N  
 iπ      
2 s in2   + α
 2N 
0.8 0.8
0.8
0.6 0.6 0.6

0.4
0.4
0.2 0.4
0
0.2
0.2
-0.2
0
15 0
15
10
10 -0.2
5 -0.2
5
0 2 4 6 8 10 12 14 16
0 0
l
0 2 4 6 8 10 12 14 16
i i l

Semicoarsening & line relaxation
• We might not know the direction of weak coupling
or it might vary.
• Suppose we want a method that can handle either
−α or −1
1   h 1  
A1 h = − 1 2 + 2α − 1  A2 = − α 2 + 2α − α 
2 2
h 
h  −α  −1 
• We could use semicoarsening in the x-direction to

handle & line relaxation in the y-direction to
take care hof .
A1
A 2h

Semicoarsening & line relaxation
• The original grid. • Original grid • Coarsening is

viewed as a done by deleting
stack of every other
“pencils.” Line pencil.
relaxation is
used to solve
problem along
each “pencil”.
An anisotropic example
• Consider − ux x − α uyy = f with u = 0 on the boundaries
of the unit square, & stencil given by
−α
1  
Ah = − 1 2 + 2α − 1 
2
h  −α 
• Suppose so the exact
solution isfgiven
( x, y) by
= 2 ( y − y 2) + 2 α ( x − x 2)
u( x, y) = ( y − y 2) ( x − x 2)
• Observe that if α is small, the x-direction dominates,
while if α is large, the y-direction dominates

What is smooth error?
• Consider α = 0.001 & suppose point Gauss-Seidel is
applied to a random initial guess. The error after
50 sweeps appears as:
0.12
0.1
0.08
0.06
0.1 0.04
0.02
0.08
00
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.06
Error along line of constant x
0.04
0.02
0.12
0 0.1
1 0.08
1
0.8 0.06
0.5 0.6 0.04
0.4
y 0 0
0.2
x
0.02
00 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Error along line of constant y

We experiment with 3 methods
• Standard V(2,1)-cycling, with point Gauss-Seidel
relaxation, full coarsening, & linear interpolation
• Semicoarsening in the x-direction. Coarse & fine

grids have the same number of points in the y-
direction. 1-D full weighting & linear interpolation
are used in the x-direction, with no y-coupling in
the intergrid transfers
• Semicoarsening in the x-direction combined with

line relaxation in the y-direction. 1-D full weighting
& interpolation.

With semicoarsening, the
operator must change
• To account for unequal mesh spacing, the residual
& relaxation operators must use a modified stencil
 α 
−
 hy2 
 
 
 1  2 2α  1 
A = −  2 + −
2 2 2
 h x  hx hy  hx 
 
 1 
−
 2 
• Note that as grids become coarser,
h y  grows
while remains constant.
hx
hy
How do the 3 methods work for
various values of α ?
N = 16
• Asymptotic convergence factors:

α
scheme 1000 100 10 1 0.1 0.01 0.001 1E-04
V(2,1)-cycles 0.95 0.94 0.58 0.13 0.58 0.90 0.95 0.95
semiC in x 0.94 0.99 0.98 0.93 0.71 0.28 0.07 0.07
semiC / line relax 0.04 0.08 0.08 0.08 0.07 0.07 0.08 0.08
y-direction strong x-direction strong

• Note: semicoarsening in x works well for α < .001
but degrades noticeably even at α = .1.

A semicoarsening subtlety
• Suppose α is small, so that semicoarsening in x is
used. As we progress to coarser grids, hx− 2 gets
small but − 2 remains constant.
hy
• If, on some coarse grid, − 2 becomes comparable

to hx effectively becomes
, then the problem
α hy− 2 in the y-direction. Continued
recoupled
semicoarsening can produce artificial anisotropy,
strong in the y-direction.
• When this occurs, it is best to stop semicoarsening &

use full coarsening on any further coarse grids.

7c. Variable meshes
• Non-uniform grids are commonly used for domain or
data irregularities or emerging solution features
• Consider how we might approach the 1-D problem
− u″( x) = f ( x) 0<x <1

posed onu(
the = u( 1 ) = grid:
0) following 0
x =0
x0 xj − 1 xj xj + 1 xN

We need some notation for the
mesh spacing
• Let N be a positive integer. We define the
spacing interval between x & x :
j j +1
h j + 1/2 ≡ x j + 1 − x j j = 0, 1, ... , N − 1
h j + 1/2
x0 xj − 1 xj xj + 1 xN

Second divided differences
’’ = (uj+1/2
uj-1/2 ’ - uj-1/2
’ )/hj
uj'-1/2 = (uj - uj-1)/hj-1/2 Uj'+1/2 = (uj+1 - uj)/hj+1/2
uj-1 ↓ uj ↓
uj+1
| |
hj-1/2 hj+1/2
hj = (hj-1/2 + hj+1/2)/2

We define the discrete
differential operator
• Using second-order finite differences (& wading
through a mess of algebra!), we obtain the
discrete representation
− αhj u jh− 1 + ( αhj + βhj ) u jh − βhj u jh+ 1 = f jh 1 ≤ j ≤ N −1
u0h = uN
h = 0
where
h 2 h 2
=hαj βj = h
j − 1/ 2 ( hj − 1/ 2 + hj + 1/2 ) j + 1/2 ( hj − 1/2 + hj + 1/2 )
• Multiply by (hj-1/2 + hj+1/2)/2 yields an SPD matrix.

We modify standard multigrid to
accommodate variable spacing
• We choose every second fine-grid point as a
coarse-grid point
x 0h x 2h j h
xN
2h
xN
x 02h x 2h
j /2
• In [x0 , x2 ], linear means
h h
v (x) = v0 + (v1 - v0 ) (x - x0 )/(x2 - x0 )

2h 2h 2h h h h
• Now just plug in x = x1 to get…
h

Interpolation formula
• If v h = I 2hh v 2h , then for 1 ≤ j ≤ N/2 − 1 we have
h2 j + 3/2 v j2h + h2 j + 1/2 v j2h+ 1

v 2h j = v j2h v 2h j + 1 =
h 2 j + 1/2 + h 2 j + 3/2

We use the variational properties
to derive restriction & A2h
2h 2h h h 2h 1 h T
A = I h A I 2h Ih = ( I 2h )
2
• This produces a stencil on Ω 2hthat is similar, but
not identical, to the fine-grid stencil. If the
resulting system is scaled by ( h + h ) , then
the Galerkin product is the same j −as + 1/2
1/2 thej fine-grid
stencil.
• For 2-d problems this approach can be generalized
readily to logically rectangular grids. However, for
irregular grids that are not logically rectangular,
AMG is a better choice.

7d. Variable coefficients
• A common difficulty is variable coefficients,
exemplified in 1-D by
− ( a( x) u′( x) ) ′ = f ( x) 0<x <1
u( 0 ) = u( 1 ) = 0
where a(x) is a positive function on [0,1]
• We seek to develop a conservative, or self-adjoint,
method for discretizing this problem.
• Assume we have available to us the values of a(x)
at the midpoints of the grid
a j + 1/2 ≡ a( x j + 1/2 )
x 0h x jh− 1 x jh h
xN
x jh + 1
We discretize using central differences
• We can use second-order differences to

approximate the derivatives. To use a grid spacing
of h, we evaluate a(x)u’(x) at points midway
between the gridpoints:
( au′) |x j + 1/2 − ( a u′) |x j − 1/2

( a( x) u′( x) ) ′ ≈
= + O( h 2 )
xj
h
Points used to evaluate (au’ )’ at xj
x 0h x jh − 1 h
x jh x jh + 1 xN
We discretize using central differences
(cont’d)
• To evaluate ( au′) |x j + 1/2 , we must sample a(x) at the

point x & use second order differences:
j + 1/2
uj + 1 − uj uj − uj − 1
( a u′ ) |x j + 1/2 ≈ a j + 1/2 ( a u′ ) |x j − 1/2 ≈ a j − 1/2
h h
where
a j + 1/2 ≡ a( x j + 1/2 )
Points used to
evaluate u’ at Points used to evaluate (au’ )’ at xj
x j + 1/2
x 0h x jh − 1 x jh h
xN
x jh + 1
The basic stencil
• We combine the differences for u’ & for (au’ )’ to

obtain the operator
uj + 1 − uj uj − uj − 1
aj + 1/2
h − aj − 1/2 h
− ( a( x j ) u′ ( x j ) ) ′ ( x j ) ≈ −
h
& the problem becomes, for

1 ≤ j ≤ N −1
1
2( − aj − 1/2 uj − 1 + ( a j − 1/2 + a j + 1/2 ) uj − a j + 1/2 uj + 1 ) = f j
h
u0 = uN = 0

Coarsening the variable
coefficient problem
• A reasonable approach is to use a standard multigrid
algorithm with linear interpolation, full weighting, &
the stencil
1
A 2h = 2
( − a 2h
j − 1/2 a j2h− 1/2 + a j2h+ 1/2 − a j2h+ 1/2 )
( 2h)
where a 2h j + 1/2 + a 2h j + 3/2
a j2h+ 1/2 =
2
• The same stencil is obtained by the Galerkin relation

Variable mesh vs. variable coefficients
after scaling by ηj = (hj-1/2 + hj+1/2)/2 & h
• Variable mesh
1 h 1 1 h 1 h h
− uj −1 + ( + )uj + uj +1 = ηj f j
hj −1/ 2 h j −1/ 2 hj +1/ 2 h j +1/ 2
• Variable coefficients
1 h
( − aj − 1/2 uj − 1 + ( a j − 1/2 + aj + 1/2 ) uj − a j + 1/2 uj + 1 ) = h f j
h
• Correspondence
1 aj −1/ 2 1 aj +1/ 2
⇔ ⇔
hj −1/ 2 h h j +1/ 2 h

Standard multigrid degrades if
a(x) is highly variable
• So MG for variable coefficients is equivalent to MG (with

simple averaging) for Poisson’s equation on a variable mesh.
-(au’ )’ = f
-u’’ = f
• But simple averaging won’t accurately represent smooth
components if is close to but far from .
h
x 2j +1 x 2h j x 2h j + 2
x 2h j h
x 2j +1 x 2h j + 2
x j2h x j2h+ 1
One remedy is to apply
operator interpolation
• Assume that relaxation does not change smooth
error, so the residual is approximately zero.
Applying at x h yields
2j + 1
− a 2h j + 1/2 e2h j + ( a 2h j + 1/2 + a 2h j + 3/2 ) e2h j + 1 − a 2h j + 3/2 e2h j + 2
2
=0
h
• Solving for :
h
e2j +1
a 2h j + 1/2 ej2h + a 2h j + 3/2 ej2h+ 1

+1 =
h
e2j
+ 1/2 + a 2j + 3/2
h
a 2j h

Thus, operator induced interpolation is
v 2h j = v j2h
a 2h j + 1/2 v j2h + a 2h j + 3/2 v j2h+ 1

+1 =
h
v 2j h
a 2j + 1/2 + a h
2j + 3/2
• As usual, the restriction & coarse-grid operators

are defined by the Galerkin relations
2h 2h h h 2h h T
A = I h A I 2h I h = c( I 2h )
A variable coefficient example
• We use V(2,1) cycle, full weighting, linear interpolation.
• We use a(x) = 1 + ρ sin(kπx) & a(x) = 1 + ρ rand(kπx)
N = 128 a(x) = 1 + ρ sin(kπx)

1 + ρ rand(kπx)
ρ k=3 k=25 k=50 k=100 k=200 k=400

0 0.085 0.085 0.085 0.085 0.085 0.085 0.085
0.25 0.084 0.098 0.098 0.094 0.093 0.083 0.083
0.5 0.093 0.185 0.194 0.196 0.195 0.187 0.173
0.75 0.119 0.374 0.387 0.391 0.39 0.388 0.394
0.85 0.142 0.497 0.511 0.514 0.514 0.526 0.472
0.95 0.191 0.681 0.69 0.694 0.699 0.745 0.672

Outline
√ – Neumann boundaries
– Analysis
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
– Complexity
√

8. Algebraic multigrid (AMG)
unstructured grids, variable coefficients,…
Automatically defines coarse “grid”
AMG has two distinct phases:

— setup phase: define MG components
— solution phase: perform MG cycles
AMG approach is opposite of geometric MG

— fix relaxation (point Gauss-Seidel)
— choose coarse “grids” & prolongation, P, so that
error not reduced by relaxation is in range(P)
(in contrast, geometric MG fixes coarse grids,

then defines suitable operators & smoothers)
— use Galerkin principle so that coarse-grid

correction eliminates error in range(P)

AMG has two phases
• Setup Phase
– Select coarse “grids,” Ω m + 1, m = 1, 2, . . .
m
+ 1 , m = 1, 2, . . .
– Define interpolation, Im
– Define restriction & coarse-grid operators

m+1 m T m+1 m +1 m m
Im = ( Im +1 ) A = I m A Im + 1
Solve Phase
Standard MG processes: V-cycle, W-cycle, FMG, FAS, …
Only the selection of coarse grids does not parallelize

well using existing techniques!
AMG fundamental concept:
Smooth error = “small” residuals
• Consider the iterative method error recurrence
ek + 1 = ( I − Q − 1 A ) ek
• Error that is slow to converge satisfies
−1 −1
( I− Q A ) e ≈ e ⇒ Q Ae≈ 0
⇒ r ≈ 0
• More precisely, it can be shown that smooth error
satisfies
r −1 « e A (1)
D

AMG uses strong connection to
determine MG components
• It is easy to show from (1) that smooth error
satisfies
A e , e « De , e (2)
• Define i is strongly connected to j by

− aij ≥ θ max { − aik } , 0 <θ ≤1
k ≠i
• For M-matrices, we have from (2)
1  − a i j   ei − e j  2
2 ∑  2 a ii   ei  « 1
i ≠j
implying that smooth error varies slowly in the

direction of strong connections
Some useful set definitions
• The set of strong connections of a variable u i ,
that is, the variables upon whose values the value
of ui depends, is defined as
 
S i =  j : − a i j > θ max − aij 
 j ≠i 
• The set of points strongly connected to a variable ui
is denoted S iT = { j : j ∈ S i } .
• The set of coarse-grid variables is denoted C.
• The set of fine-grid variables is denoted F.
• The set of interpolatory coarse-grid variables
used to interpolate the value of the fine-grid
variable ui is denoted C i .
Choosing the coarse grid
• Two Criteria
– (C1) For each i ∈ F, each point j ∈ S should either be in C

i
or should be strongly connected to at least one point in C .
i
– (C2) C should be a maximal subset with the property that no

two C-points are strongly connected to each other.
• Satisfying (C1) & (C2) is sometimes impossible. We use

(C2) as a guide while enforcing (C1).

Selecting the coarse-grid points
C-point selected
(point with
largest “value”)
Neighbors of
C-point
become F-
points
Next C-point
selected (after
updating “values”)
F-points
selected, etc.
Examples: Laplacian operator
5-pt FD, 9-pt FE (quads), & 9-pt FE (stretched quads)
9-pt FE
(stretched quads)
−1 −1 − 1 −1
   
 −1 4 −1  −1 8 −1 
 −1   −1 − 1 −1 
−1 −4 −1
5-pt FD   9-pt FE (quads)
 2 8 2 
 −1 −4 −1 
Prolongation is based on smooth
error, strong connections (from
M-matrices)
Smooth error is given by:
∑
C
F
F
ri = a i i ei + a i j ej ≈ 0
j ∈N i
i Prolongation :
C
C
ei , i∈C

( P e) = 
i  ∑ ωik ek , i∈F
F
k∈Ci

Prolongation is based on smooth
error, strong connections (from
M-matrices)
Sets:
Ci Strongly connected C-pts.
F
C
Dis Strongly connected F -pts.
F
Diw Weakly connected points.
i The definition of smooth error,

C
C a i i ei ≈ − ∑ a i j ej
j ≠i
F gives
a i i ei ≈ − ∑ aij ej − ∑s aij ej − ∑waij ej

j ∈Ci j ∈Di j ∈Di
Strong C Strong F Weak pts.
Finally, the prolongation
weights are defined
• In the smooth-error relation, use e j = ei for weak
connections. For the strong F-points use :
   
e j =  ∑ a j k ek  ⁄  ∑ a j k 
 k ∈Ci   k ∈Ci 
yielding the prolongation weights:

a ika k j
aij + ∑ s ∑ akm
j ∈Di m ∈C
i
wi j = −
aii + ∑ w a in
n ∈Di

AMG setup costs:
A bad rap
• Many geometric MG methods need to compute

prolongation & coarse-grid operators
• The only additional expense in the AMG setup phase is

the coarse-grid selection algorithm
• AMG setup phase is only 10-25% more

expensive than in geometric MG & may be
considerably less than that!

AMG performance:
Sometimes a success story
• AMG performs extremely well on the model problem
(Poisson’s equation, regular grid)- optimal convergence
factor (e.g., 0.14) & scalability w.r.t. problem size.
• AMG appears to be both scalable & efficient on

diffusion problems on unstructured grids (e.g., 0.1-0.3).
• AMG handles anisotropic diffusion coefficients on

irregular grids reasonably well.
• AMG handles anisotropic operators on structured &

unstructured grids relatively well (e.g., 0.35).
So, what could go wrong?
Strong F-F connections: weights depend on each other
• For point i the value ej is interpolated from k 1 , k 2 , &

is needed to make the interpolation weights for
approximating ei .
• For point j the value ei is interpolated from k 1, k 2 , &
is needed to make the interpolation weights for
approximating ej .
• It’s an implicit system!
k 1 ∈ Ci
k 1 ∈ Cj
k4 ∈ Cj
k 3 ∈ Ci i j
k 2 ∈ Ci
U-Boulder k 2 ∈ Cj 264 of 312
Is there a fix?
A Gauss-Seidel-like iterative approach to weight
definition is implemented. Usually, two passes
suffice. But does it work? Yes, frequently:
convergence factors
2D Laplacian, N = 10,000,
stretched quadrilaterals θ standard iterative
0.25 0.47 0.14
∆ x = 10 ∆ y 0.5 0.24 0.14
0.25 0.93 0.93

∆ x = 100 ∆ y 0.5 0.55 0.28
V(1,1) cycles/Gauss-Seidel (used throughout chapter 8).

How’s it perform (vol I)?
regular grids, plain, old, vanilla problems,
unit square, N = 64, Dirichlet boundaries
• The Laplace operator:
Convergence Time Setup
Stencil per cycle Complexity per Cycle Times
5-pt 0.054 2.21 0.29 1.63
5-pt skew 0.067 2.12 0.27 1.52
9-pt (-1,8) 0.078 1.30 0.26 1.83
9-pt (-1,-4,20) 0.109 1.30 0.26 1.83
• Anisotropic 5-Point Laplacian: − ε Ux x − Uyy

Epsilon 0.001 0.01 0.1 0.5 1 2 10 100 1000
Convergence/cycle 0.084 0.093 0.058 0.069 0.054 0.079 0.087 0.093 0.083

How’s it perform (vol II)?
structured meshes, rectangular domains
• 5-point Laplacian on regular rectangular grids

Convergence factor (y-axis) plotted against number of nodes (x-axis)
0 .1 6
0 .1 4 6
0 .1 4 0 .1 3 5
0 .1 4 2
0 .1 3 3
0 .1 2
0 .1
0 .0 8 0 .0 8 2
0 .0 6 0 .0 5 5
0 .0 4
0 .0 2
0
0 500000 1000000

How’s it perform (vol III)?
unstructured meshes, rectangular domains
• Laplacian on random unstructured grids (regular

triangulations, 15-20% nodes randomly collapsed into neighboring nodes)
Convergence factor (y-axis) plotted against number of nodes (x-axis)
0 .3
0 .2 5 0 .2 5 3
0 .2 2 3
0 .2
0 .1 7 5
0 .1 5
0 .1 1 1
0 .1 0 .0 9 7
0 .0 5
0
0 20000 40000 60000

How’s it perform (vol IV)?
isotropic diffusion, structured/unstructured grids
∇ • ( d ( x , y ) ∇ u ) on structured, unstructured grids
0 . 4
0 . 3 5
6 a
0 . 3 6 b
0 . 2 5 7 a
0 . 2 7 b
0 . 1 5 8 a
8 b
0 . 1
9 a
0 . 0 5
9 b
0
N=16642 N=66049 N=13755 N=54518

Structured Structured Unstruct. Unstruct.
Problems used: “a” means parameter c = 10, “b” means c = 1000
1. 0 0. 125 ≤ max { x − 0. 5 , y − 0. 5 } ≤ 0. 25

6: d ( x , y ) = 1. 0 + c x − y 8: d( x, y) = 
c otherwise
1. 0 x ≤ 0. 5 1. 0 0. 125 ≤ ( x − 0. 5) 2 + ( y − 0. 5) 2 ≤ 0. 25
7: d ( x , y ) =  9: d( x, y) = 
 c x > 0. 5 c otherwise
How’s it perform (vol V)?
Laplacian operator, unstructured grids
Convergence factor
0.3
0.2754
0.25
0.2198 0.2237
0.2
0.1888
0.1715
0.15
0.1 0.1002
0.05
0
0 50000 100000 150000 200000 250000 300000 350000
Gridpoints

AMG for systems
• How can we do AMG on systems?
 A 11 A 12   u   f 
A A  v  = g
 21 22     
• Naïve approach: “Block” AMG (block Gauss-Seidel,
using scalar AMG to “solve” at each cycle)
−1
u ← ( A 11 ) ( f − A 12 v )
v ← ( A 22 ) − 1 ( g − A 21 u )
Great Idea! Except that it often doesn’t work!

Block AMG doesn’t account for strong inter-variable coupling.

AMG for systems: A solution
• To solve the system problem, allow interaction between
the unknowns at all levels:
k
k
 11 12 
A A k
 ( Ik + 1) 0 
k u
A = Ikk + 1 = 
k 
& k 
k ( Ik +1
 A 21 A 22   0 + 1) v 
• This is called the “unknown” approach.

• 2-D biharmonic, Dirichlet boundaries, unit square,
uniform quadrilateral mesh:
Mesh spacing 0.125 0.0625 0.03135 0.015625

Convergence factor 0.22 0.35 0.42 0.44

αAMG)
Adaptive AMG (α
to broaden applicability
adaptive interpolation
to determine sense of smoothness
+
adaptive C-point choice
to determine strength of connection

Adaptive interpolation
Standard AMG collapses stencils by assuming smooth

error is locally constant (Poisson “sense of smoothness”):
   
e j =  ∑ a j k ek  ⁄  ∑ a j k 
*** smooth ≈ 1 ***  k ∈Ci   k ∈Ci 
ej = ei
a i i ei ≈ − ∑ aij ej − ∑s aij ej − ∑waij ej
j ∈Ci j ∈Di j ∈Di
Strong C Strong F Weak pts.

Pre-relaxation
• What if we found a smooth error x that’s far from 1 ?
• If we found that xj = 1.6xi for j∈Diw, say, then we could
just make the replacement ej → 1.6ei .
• If, say, xj = 0.2xk for j∈Dis & all k∈Ci, then we could set
ej → (Σ ajk 0.2ek )/( Σ ajk ).
j∈Ci j∈Ci
• How do we obtain a representative smooth error x ?
Relax on Ax = 0 !!!
Adaptive C-point choice
• AMG uses the ansatz that residuals are 0 at F-points.
• What if we had rf = 0 exactly?
 A ff A fc e f   0  e f   −A−1
ff A fc

   =   ⇒ A ff e f + A fcec = 0 ⇒  = ec
 Acf Acc  ec   rc   ec   I 
• Choosing −A−1  (ideal interpolation) makes MG exact:

A
P =  ff fc 
 I 
MGe = [CG]Pec = [I − P[P AP] P A]Pec T −1 T
−1
= Pec − P[P AP] [P AP]ec = 0.
T T

Compatible relaxation
• How do we make the residuals zero at F-points?
Relax on Affxf = -Afcxc !!!

• This is the same as relaxing on the F-points while
maintaining 0 at the C-points.
• If Aff is well-conditioned, then this converges quickly.
• Quick “CR” convergence means that the F-point residuals
will be small after relaxation & that there is thus a P
(the ideal one) that gives good MG convergence.
• Is there a good local P ? Maybe, since CR is local & fast.

Outline
– Analysis
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
– Complexity √ – Matrix coarsening
√

9. Multilevel adaptive methods
fast adaptive composite grid method (FAC)
two-spike problem

Model 1-D problem
- u”(x) = f(x), 0<x<1

u(0) = u(1) = 0
0 1
h-2 (- uhi-1 + 2uhi - uhi-1 ) = fhi ,1 ≤ i ≤ 7,

uh0 = uh8 = 0, h = 1/8

Local refinement
suppose f(x) has a spike at x = 3/4
but is smooth elsewhere
y= f(x)
O
fine grid ( = interface)
coarse grid

Strategy
• Recognize that there’s little value of having the fine grid in

the smooth region, [0, 1/2].
• Start with standard MG

– eliminate relaxation
– eliminate intergrid transfers & residual calculations
• Interpret this process via the composite grid

(= 2h-points in [0, 1/2] + h-points in [1/2, 1]).
• We’ll try absurdly hard to eliminate work @ x = 1/2,

but we have in mind multi-dimensions & smaller patches.

Eliminate relaxation
notation
vh = (vih )
• Initialize vh = 0 & f12h (f1h + 2f2h + f3h )/4 .

• Relax on vh on the local fine grid.
• Compute rh = fh - Ah vh & transfer to 2h:
f22h (r3h + 2r4h + r5h )/4 & f32h (r5h + 2r6h + r7h )/4 .
• Compute an approximation, v2h, to the solution of the 2h
residual equation, A2h u2h = f2h .
• Update the residual at x12h for later cycles:
f12h f12h - (- v02h + 2 v12h - v22h )/(2h)2.
• Correct: vh vh + Ih2hv2h.

Eliminate more
f12h involves
rh at x3h & x4h!
Store vh only @ x3h - x7h & save v2h w12h !
• Initialize vh = 0, w12h = 0, & f12h (f1h + 2f2h + f3h )/4 .

• Compute rh = fh - Ah vh (not at x1h or x2h) & transfer to 2h:
f22h (r3h + 2r4h + r5h )/4 & f32h (r5h + 2r6h + r7h )/4 .
f12h f12h - (- v02h + 2 v12h - v22h )/(2h)2.
• Accumulate the 2h approximation: w12h w12h + v12h.
• Correct: vh vh + Ih2hv2h (not @ x1h or x2h).

Eliminate the rest
r3h doesn’t change on h.
Compute change in r3h.
2h residual @ x1h = g12h - (- w12h + 2 w22h - w32h )/(2h)2 !
• Initialize vh = 0, w12h = 0, & f12h (f1h + 2f2h + f3h )/4 .

• Compute the right sides for 2h:
f22h g12h - (- w12h + 2 w22h - w32h )/(2h)2 &
f32h (r5h + 2r6h + r7h )/4 .
f12h f12h - (- v02h + 2 v12h - v22h )/(2h)2.
• Accumulate the 2h approximation: w12h w12h + v12h.
• Correct: vh vh + Ih2hv2h (not @ x1h or x2h or x3h).

Interpretation
composite grid
(- v0c + 2 v1c - v2c )/(2h)2 = f1c ≡ (f1h + 2 f2h + f3h )/4
(- v1c + 2 v2c - v3c )/(2h)2 = f2c ≡ (f3h + 2 f4h)/4
(- vci-1 + 2 vic - vci+1 )/h = fic ≡ fhi+2

Issues
• Adaptivity
• Error estimates
• Norms (proper scaling)
• Multiple dimensions
– Slave points
– More complicated stencils
• Data structures
• Parallel algorithms (AFAC)
• Time-space

Two-spike example (Laplacian)
• Global grid h & one two-patch refined level h/2.

• V(1,0)-cycles, Gauss-Seidel
• Asymptotic convergence of the solver
• Scaled L2 discretization error estimate

Outline
– Analysis
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
• Some Theory •√ Finite Elements
√

10. Finite elements
FE, a variational methodology
FD methodology
FE methodology
method
method method

Discretization
What do you see?
•
• • ••
• •
• •
Localize
•
What choice
do you have?

Continuous piecewise linear functions
•
• • ••
• •
• •

Representation
a basis of “hat” functions
h
ε(i)
is piecewise linear &
• 0 at all nodes except

h
ε(i)
node i where it’s 1
••• •• •••
a general piecewise linear
function can be represented by
uh = Σi uihε(hi)

How do we use piecewise linears?
How do you take

2nd derivatives of this ???
•
• • ••
• •
• •
Weak form!
−u"= f
⇒ − u"v = fv ∀v
⇒ ∫ (−u")vdx = ∫ fvdx ∀v
⇒ ∫ u'v' dx − u'v | = ∫ fvdx
1
0 ∀v
⇒ ∫ u'v' dx = ∫ fvdx ∀v ∋ v(0) = v(1) = 0
⇒ (u',v') = ( f ,v ) ∀v ∋ v(0) = v(1) = 0
↓ ↓
h
Σjuj ε(j)
h h
ε(i)
FE basics
• Model problem Ω ⊂ ℜ2
Lu ≡ - uxx - uyy = f in Ω • Duality: Solving Lu = f is
u = 0 on ∂Ω equivalent to minimizing
the functional
• Sobolev spaces
F(u) = “(Lu, u)”/2 - (f, u)
L2(Ω) = {u : ∫Ωu2dΩ < ∞}
( ) = {u : u, ux , uy ε L2(Ω),
H01(Ω) ( ), u|∂Ω = 0} • Short story:
(u, v) = ∫Ωu v dΩ 1st derivative test

∇F(u) = Lu - f = 0
• L is self-adjoint positive definite: 2nd derivative test
(Lu, v) = (∇u, ∇v) = ∫Ω(uxvx+uyvy)dΩ = (u, Lv) F”(u) = L > 0
(Lu, u) > 0 if u ≠ 0
 ux 
∇u =  
U-Boulder
 uy  297 of 312
Long story
Using symmetry & linearity of L & bilinearity of the inner product:
F(u + v) = (L(u + v), u + v)/2 - (f, u + v)
= [(Lu, u) + 2(Lu, v) + (Lv, v)]/2
- (f, u) - (f, v)
= F(u) + (Lu, v) - (f, v) + (Lv, v)/2
= F(u) + (Lu - f, v) + (Lv, v)/2
The last term is positive for v ≠ 0. Thus,

F(u + v) ≥ F(u) ⇔ (Lu, v) = (f, v) ∀ v ε H01(Ω)
⇔ Lu = f

FE constructs
• Assume that Ω is the unit square.
• Consider an nxn grid of square “cells”.
• Continuous piecewise bilinear elements:
Hh ⊂ H01(Ω).
• Each uh in Hh is determined by its node
values. This is how we’ll represent them!
• Within each square:
uh = axy + bx + cy + d
uh is linear in a coordinate direction
• If uh on one side of an element matches uh
on the other at the nodes, then it matches
on the common edge. So uh is continuous.

Continuity
•
•

FE discretization
• Minimize F(u) = (Luh, uh)/2 - (f, uh)

over uh ε Hh.
• Equilavent to solving
(Luh, vh) = (f, vh) ∀ vh ε Hh.
• Basis: ε(ij) is the element of Hh that
h
equals 1 @ node ij & 0 elsewhere.
• Expansion: uh(x,y) = Σij uij ε(ij) (x,y).
• Problem: What is Luh ≡ - huxxh - uyy ???
h h

Weak form
• The Gauss Divergence Theorem & homogeneous
boundary conditions yield
(Lu, v) = (- uxx - uyy, v) = (-∇•∇u, v) = (∇u, ∇v)
• Note:
(∇u, ∇v) = ∫Ω(uxvx + uyvy)dΩ.
• So the problem becomes

(∇uh, ∇vh) = (f, vh) ∀ vh ε Hh
or
∫Ω(uxvx + uyvy)dΩ = ∫ΩfvhdΩ ∀ vh ε Hh.
h h h h

Towards the matrix equation
for nodal values
(∇uh, ∇vh) = (f, vh) ∀ vh ε H h
• Using uh(x,y) = Σij uijh ε(hij) (x,y) & choosing vh = ε(hkl) :

Σij uijh (∇ε(hij), ∇ε(hkl)) = (f, ε(hkl)) ∀ k, l
• Matrix terms (∇ε(ij), ∇ε(kl)) are 0 when
h h
|i - k| or |j - l| > 1.
• We compute
(∇ε(ij), ∇ε(ij)) = 8/3 & (∇ε(ij), ∇ε(i±1j ±1)) = -1/3.
• Assumeh f is bilinear:
h h h
(f, ε(kl)) = ∫Ωf ε(kl)dΩ ≈ h2f(xkl).
h h
actually,
Bh fh
uh = (uijh) & fh = (h2f(xkl)) (mass matrix)
The matrix equation
Ahuh = fh
where
uh = (uij) & fh = (h2f(xkl))
h
& the matrix is given by the stencil
−1 −1 −1
1 
Aij = −1 8 −1.
h
stiffness matrix
3 
−1 −1 −1
Some matrix properties
−1 −1 −1
1 
Aij = −1 8 −1
h
3 
−1 −1 −1
• Symmetric! ij “reaches” to i±1j±1 as i±1j±1 to ij.
• Singular? Ah 1 = 0 ?! Not next to boundaries!
@ west boundary: −1 −1

1 
Aij =  8 −1
h
3 
• Positive definite! −1 −1
Diagonally dominant (strictly so @ boundaries).

Abstract FE relaxation
focus on functions, not nodal vectors
• Relaxation involves “local” changes: uh ← uh - s εh for some

scalar s & εh = ε(hij).
But how do we pick s ???
• Use FE principle of minimizing F(uh - sεh) over s :
F(uh - sεh) = (L(uh - sεh), uh - sεh)/2 - (f, uh - sεh)
= F(uh) - s (Luh - f, εh) + (s2/2) (Lεh, εh).
• So just pick s by taking the derivative of this quadratic
polynomial & setting it to 0: s = (Luh - f, εh)/(Lεh, εh).
This is just Gauss-Seidel!

Abstract FE coarsening
again with focus on functions
• Coarsening involves a “global” change: uh ← uh + w2h for

some coarse-grid function w2h.
But how do we pick w2h ???
• Use FE principle of minimizing F(uh + w2h) over w2h:

F(uh + w2h) = (L(uh + w2h), uh + w2h)/2 - (f, uh + w2h)
= F(uh) + (Luh - f, w2h) + (Lw2h, w2h)/2.
• So just pick w2h by taking the gradient of this quadratic
functional & setting it to 0. This is tricky because you need
to write the gradient as a function in the subspace H2h.
We go instead from abstract functions to nodal vectors…

h
Concrete FE coarsening: I2h
now with focus on nodal vectors
• Adding nodal representations of vh & v2h (ε = ε(ij)) :

v2h(x,y) = Σij v2hij ε2h (x,y) (sum over 2h indices
2h nodal values ↔ even h indices)
= Σij vhij εh (x,y) ???

h nodal values
• We should be able to do this because v2h ε H2h ⊂ Hh.

• Row 2j:
vh2i 2j = v2hij
vh2i+1 2j = (v2hij + v2hi+1 j)/2
vh2i 2j+1 = (v2hij + v2hij+1)/2
vh2i+1 2j+1 = (v2hij + v2hi+1 j +
v2hij+1 + v2hi+1 j+1)/4
Concrete FE coarsening (cont’d)
Ahuh = fh
• Solving this matrix equation is equivalent to minimizing
Fh(vh) ≡ (Ah vh, vh)/2 - (fh, vh) (parens here mean Euclidean norm)
over vh ε Hh. So how do we now correct vh ???
• We minimize Fh(vh + I2h v2h) over v2h ε H2h :
h
Fh(vh + I2h v2h)
h
= (Ah (vh + I2h v2h), vh + I2h v2h)/2 - (fh, vh + I2h v2h)
h h h
= Fh(vh) + (A2h v2h, v2h)/2 - (f2h, v2h)
variational conditions
h T
f2h = I2h(fh- Ah vh)
A2h = I2hhTAhI2hh
≡ Fh(vh) + F2h (v2h)
Outline
– Analysis
– Variable meshes
√ – Relaxation
– Coarsening
• Implementation
• Some Theory •√ Finite Elements
√ √

Multigrid rules!
We conclude with a few observations:
o We have barely scratched the surface of the myriad ways

that multigrid has been, & can be, employed.
o With diligence & care, multigrid can be made to handle many

types of complications in a robust, efficient manner.
o Further extensions to multigrid methodology are being sought

by many people working on many different problems.

Multigrid/multilevel/multiscale
an important methodology
• Multigrid has proved successful on a wide variety of

problems, especially elliptic PDEs, but has also found
application in parabolic & hyperbolic PDEs, integral
equations, evolution problems, geodesic problems, …
• Multigrid can be optimal, often O(N ).
• Multigrid can be robust in a practical sense.
• Multigrid is of great interest because it is one of the

very few scalable algorithms, & it can be parallelized
readily & efficiently!

Multigrid Tutorial

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Multigrid Tutorial

Încărcat de

Drepturi de autor:

Formate disponibile

APPM 7400

Topics in Applied Math

Course web site:

Tech support: John <jruge@colorado.edu>

Due 1 week after the relevant chapter is covered.

2: 1-3, 8, 10, 13, 16 7: 1, 2, 10, 12, 15, 21

5: 1-3, 12, 13 10: 1, 5, 8, 9

7: Convert your 2-D code developed for Chapter 4

• I am VERY flexible as to what constitutes a project.

1. Model Problems 6. Nonlinear Problems

• A. Brandt, “Multi-level Adaptive Solutions to Boundary Value Problems,”

− u ″( x ) + σ u( x ) = f ( x ) 0 < x < 1, σ≥0

This discretizes the variables, but what about the equations?

• Approximate 2nd derivative using Taylor series:

• Summing & solving:

− u ″( x ) + σ u( x ) = f ( x ) 0 < x < 1, σ≥0

− ux x − uyy + σ u = f ( x , y ) , 0 < x < 1, 0<y<1

Stencils are much better for showing the grid picture:

• Consider Au = f where A is NxN & let v be an

• Two important measures:

– The Residual: with

|| r ||∞ & || r ||2 Why have both

• Since e = u - v, we can write Au = f as A(v + e) = f

which means that Ae = f - Au ≡ r. r = f - Av

• Consider the 1-D model problem

• Jacobi (simultaneous displacement): Solve the i th

Error propagation! e ( n ew) = R J e ( o ld) J = I-D-1A

• Exact: u =u - D-1 (Au - f)

• Letting A = D-L-U, the matrix form is

• Weighted (ω=2/3) Jacobi on 1-D problem.

• Let e(0) be the initial error & e(i) be the error

• ρ(R)<1 assures convergence of the iteration for R.

• RQ(v) ≤ ρ(A): v = Σvkwk

RQ(v) < Av, v > < A Σ vk wk , Σ vk wk >

< AwN ,wN > < λN wN ,wN >

• ||A||2 = ρ1/2(A2) = ρ(A) for symmetric A!!!

ρ(R) is norm independent. ||R||2 depends on ||· ||2.

ρ(R) is asymptotic: ||R||2 bounds ||e(n + 1)||2 /||e(n)||2:

ρ(R) = inf ||R||

Thus, 1 iteration with e(0) shows dramatic L2 divergence!

• Convergence rate = or -log10(ρ(R)).

For our 1-D model, the eigenvectors of weighted

The eigenvalues are related as well.

• Expand the initial error in terms of the eigenvectors:

• The kth error mode is reduced by λk (Rω) each iteration.

• Look carefully at λk ( R ω ) = 1 − 2 ω sin2  k π  .

• The smoothing factor is the largest magnitude of

• Why only the upper spectrum? “MG” spectral radius?

• For Rω with ω=2/3, the smoothing factor is 1/3:

|λN/2|=|λN|=1/3 & |λk|<1/3 for N/2<k<N.

• But |λk| ≈ 1 - ωk2π2h2 for long waves (k << N/2).

100 Gauss-Seidel on Au = 0, with

30 Initial guess (modes of A):

– Relaxation on the coarse-grid is much cheaper: half as

– Relaxation on the coarse grid has a marginally faster

• What is A2hu2h = f2h? Analogous to Ahuh = fh for now.

Also, note that

on the coarse grid. k=4 mode, N=12 grid

What happens 0.2

between N/2 & N ?

k=4 mode, N=6 grid

the fine grid is aliased & 0.8

appears as the (N - k)th