Vector norms

A vector norm is a scalar quantity that reflects the “size” of a

vector x.
The norm of a vector x is denoted as kxk.
There are many ways to define the size of a vector. If x ∈ Rn , the
three most popular are
n
X
one-norm : kxk1 = |xk |,
k=1
v
u n
uX
two-norm : kxk2 = t |xk |2 ,
k=1

∞-norm : kxk∞ = max |xk |

1≤k≤n

Math 365 Vector and Matrix Norms March 9, 2018 2/9

Vector norms

However a vector norm is defined, it must satisfy the following three

properties to be called a norm:
1 kxk ≥ 0 and kxk = 0 if and only if x = 0 (i.e. x contains all zeros
as its entries).
2 kαxk = |α|kxk, for any constant α.
3 kx + yk ≤ kxk + kyk, where y ∈ Rn . This is called the triangle
inequality.

Math 365 Vector and Matrix Norms March 9, 2018 3/9

Unit vectors

A vector x is called a unit vector if its norm is one, i.e. kxk = 1.

Unit vectors will be different depending on the norm applied.
Below are several unit vectors in the one, two, and ∞ norms for
x ∈ R2 .

U n i t v e c t or s w i t h r e sp e c t t o t h e on e n or m U n i t v e c t or s w i t h r e sp e c t t o t h e tw o n or m U n i t v e c t or s w i t h r e sp e c t t o t h e ∞ n or m
1 1 1

0.8 0.8 0.8

0.6 0.6 0.6

0.4 0.4 0.4

0.2 0.2 0.2

x2
0
x2

x2

0 0

−0.2 −0.2 −0.2

−0.4 −0.4 −0.4

−0.6 −0.6 −0.6

−0.8 −0.8 −0.8

−1 −1 −1
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
x1 x1 x1

(a) One-norm (b) Two-norm (c) ∞-norm

Math 365 Vector and Matrix Norms March 9, 2018 4/9

Matrix norms

A matrix norm is a scalar quantity that reflects the “size” of a

matrix A ∈ Rm×n .
The norm of A is denoted as kAk.
Any matrix norm must satisfy the following four properties:
1 kAk ≥ 0 and kAk = 0 if and only if A = 0 (i.e. A contains all zeros
as its entries).
2 kαAk = |α|kAk, for any constant α.
3 kA + Bk ≤ kAk + kBk, where B ∈ Rm×n .
4 kABk ≤ kAkkBk, where B ∈ Rn×p . This is called the
submultiplicative inequality.

Math 365 Vector and Matrix Norms March 9, 2018 5/9

Matrix norms

Each vector norm induces a matrix norm according to the following

definition:
kAxkp
kAkp = max = max kAxkp ,
kxkp 6=0 kxkp kxkp =1

where x ∈ Rn and p = 1, 2, . . ..
Induced norms describe how the matrix stretches unit vectors with
respect to that norm.

Math 365 Vector and Matrix Norms March 9, 2018 6/9

Induced matrix norms

Two popular and easy to define induced matrix norms are

m
X
One-norm : kAk1 = max |ajk |,
1≤k≤n
j=1
n
X
∞-norm : kAk∞ = max |ajk |.
1≤j≤m
k=1
The one-norm corresponds to the maximum of the one norm of
every column.
The ∞-norm corresponds to the maximum of the one norm of
every row.

The two-norm of A is defined as the largest eigenvalue of the matrix

AT A. This is computationally expensive to compute.

Math 365 Vector and Matrix Norms March 9, 2018 7/9

Non-induced matrix norms

The most popular matrix norm that is not an induced norm is the
Frobenius norm:
v
u m X n
uX
kAkF = t |ajk |2 .
j=1 k=1

Math 365 Vector and Matrix Norms March 9, 2018 8/9

Important results on matrix norms

The following are some useful inequalities involving matrix norms.

Here A ∈ Rm×n :
kAxk ≤ kAkkxk
1 √
√ kAk1 ≤ kAk2 ≤ nkAk1
m
1 √
√ kAk∞ ≤ kAk2 ≤ mkAk∞
n
p
kAk2 ≤ kAk1 kAk∞

Math 365 Vector and Matrix Norms March 9, 2018 9/9