MTH 309E LINEAR ALGEBRA

2010.04.21

Lecture 36:

• spectraldecomposition and
image compression
Recall:
� �
1) A square matrix Q = u1 u2 � � � u� is an orthogonal matrix
if {u1� u2� � � � � u�} is an orthonormal set of vectors, i.e.
�
0 if � �= �
u� · u� =
1 if � = �

2) If Q is an orthogonal matrix then Q −1 = Q T

3) A square matrix A is a symmetric matrix if AT = A.

4) If A is a symmetric matrix then A is orthogonally diagonaliz-

able; i.e. there is an orthogonal matrix Q and a diagonal matrix
D such that
A = QDQ −1 = QDQ T
Yet another view of matrix multiplication

Note. If M is an � × 1 matrix and N is an 1 × � matrix then MN

is an � × � matrix.

Fact. Let A be an � × � matrix with columns v1� � � � � v� and let

B be an � × � matrix with rows w1� � � � � w�:
 
w1
� �  
w 
A = v1 v2 � � � v� B =  .. 2 
 . 
w�

Then
AB = v1w1 + v2w2 + � � � v�w�
 Theorem. Let A be a symmetric matrix with orthogonal diago-
nalization
A = QDQ T
If  
λ1 · · · 0
� �  . . 
Q= u1 � � � u� and D =  .. . . . .. 
0 · · · λ�
then
A = λ1(u1uT1 ) + λ2(u2uT2 ) + · · · + λ�(u�uT� )

Note. This formula is called the spectral decomposition of the

matrix A.
Application: image compression

• 600 × 600 pixels

• color of each pixel is represented

by a number between 0 (= black)
and 1 (= white)

• this picture is described by

a (symmetric) matrix A consisting
of 600 × 600 = 360� 000 numbers

Application: image compression • each number is stored in 8 bytes

• image
• this picture is described by file size is
a (symmetric) matrix A 360� 000 × 8 = 2� 880� 000 bytes
consisting
of 600 × 600 = 360� 000≈ 2�9 MB
numbers

Idea:
1) To compress this image find a spectral decomposition of A

A = λ1(u1uT1 ) + · · · + λ600(u600uT600)

where |λ1| � |λ2| � � � � � |λ600|.

2) For � = 1� � � � � 600 define

B� = λ1(u1uT1 ) + · · · + λ� (u� uT� )

3) We have B� ≈ A.

4) Matrix B� can be stored using only 600 · � numbers.

 original picture k=1

360,000 numbers 601 numbers

2.9 MB 0.005 MB
(compression: 99.8%)

k=5 k=10

3,005 numbers 6,010 numbers

0.024 MB 0.048 MB
(compression: 99.1%) (compression: 98.3%)
 k=20 k=50

12,020 numbers 30,050 numbers

0.096 MB 0.240 MB
(compression: 96.7%) (compression: 91.7%)

k=100 k=150

60100 numbers 90150 numbers

0.481 MB 0.721 MB
(compression: 83.3%) (compression: 75.0%)
Note.

• A similar technique can be used to compress all (i.e. non-

square, non-symmetric) images.

• In such case the spectral decomposition of a symmetric ma-

trix can be replaced by the singular value decomposition
which works for any matrix (see Sec. 7.4. of the textbook).