EE547: Fall 2018

Note: A note on computing induced norms

Author: L.J. Ratliff

Suppose A is an m × n matrix and we have the induced norm

kAxkp
kAkp = sup
x6=0 kxkp

(’sup’ stands for supremum—aka, least upper bound). Since kxkp is a scalar, we have

kAxkp Ax
kAkp = sup = sup

x6=0 kxkp x6=0 kxkp p

Since x/kxkp has unit length, we can express the induced norm as follows:

kAkp = sup kAxkp

kxkp =1

That is, kAkp is the supremum of kAxkp on the unit ball kxkp = 1. Note that kAxkp is a continuous function
of x and the unit ball B1 (0) = {x| kxkp = 1} is closed and bounded (compact).

Fact. On a compact set, a continuous function always achieves its maximum and minimum values.

The above fact enables us to replace the ’sup’ with a ’max’. Indeed,
kAxkp
kAkp = max = max kAxkp
x6=0 kxkp kxkp =1

When computing the norm of an operator A, the definition is the starting point and the remainder of the
process is the following:
1. First, find a candidate for the norm, call it K for now, that satisfies

kAxkp ≤ Kkxkp , ∀ x

2. Then, find at least one non-zero x0 for which

kAx0 kp = Kkx0 kp

3. Finally, set kAkp = K.

We can do this because
kAxkp ≤ kAkp kxkp
which is clearly the case by inspecting the definition of kAkp .

Recall that
kxk∞ = max |xi |
i

0-1
0-2 Note: A note on computing induced norms

Claim.
 
n
X
kAk∞ = max  |aij | (max of row sum)
i
j=1

Proof. We prove this using the above steps. Let A ∈ Rm×n . Then, Ax ∈ Rm is a vector so that

Pn
k=1 a1k xk


kAxk∞ =
.
.
Pn .


a
k=1 mk k x
∞
 n 
X 
= max  aik xk 
 
i∈{1,...,m}  
k=1
n
X
≤ max |aik xk |
i∈{1,...,m}
k=1
n
!
X
≤ max |aik | max |xk |
i∈{1,...,m} k∈{1,...,n}
k=1
n
!
X
= max |aik | kxk∞
i∈{1,...,m}
k=1

so that we have completed step 1—that is,

n
X
K= max |aik |
i∈{1,...,m}
k=1

so that

kAxkp ≤ Kkxk∞ , ∀ x

Step 2 requires that we find non-zero x0 for which equality holds in the above inequality. Examination
reveals that we have equality if x0 is defined to have the components

 ai∗ k
|ai∗ k | , aik 6= 0
xk = 1≤k≤n
1, a i∗ k = 0

where i∗ above is
n
X
i∗ = arg max |aij |
i∈{1,...,m}
j=1

You can check this by starting with the definition for kAxk∞ and plugging in this choice of x0 to see that

kAx0 k = Kkx0 k
Note: A note on computing induced norms 0-3

P 
 n
maxi∈{1,...,m}  j=1 aij xj 

kAxk∞
kAk∞ = sup = sup
kxk6=0 kxk∞ kxk6=0 maxi∈{1,...,m} |xi |
 
 n 
X 
= sup max  aij xj 
kxk=1 i∈{1,...,m}  j=1 
n
X
≤ sup max |aij ||xj | (triangle inequality)
kxk=1 i∈{1,...,m} j=1
n
X
≤ sup max |aij | max |xj |
kxk=1 i∈{1,...,m} j=1 j

n
X
= max |aij |
i∈{1,...,m}
j=1

where equality holds when we let x be the vector with kxk∞ = 1 such that it extracts the max row sum,
i.e. for 1 ≤ j ≤ n, let  ai∗ j
|ai∗ j | , 6 0
ai∗ j =
xj = (0.1)
1 , ai∗ j = 0
where
n
X
i∗ = arg max |aij |
i∈{1,...,m}
j=1

Pn
Allowing for the vector x to have the above property, we have kAk∞ = maxi∈{1,...,m} j=1 |aij |