A

Convex and Concave Functions

Convex Function. A function f (X) is said to be convex if for any pair of points
(1) (2)

x1
x1

x (2)

x (1)

2 2
X1 = .. and X 2 = ..
. .








(1)
(2)

xn xn

and all , 0 1,
f [X2 + (1 )X1 ] f (X2 ) + (1 )f (X1 ) (A.1)

that is, if the segment joining the two points lies entirely above or on the graph of
f (X). Figures A.1a and A.2a illustrate a convex function in one and two dimensions,
respectively. It can be seen that a convex function is always bending upward and
hence it is apparent that the local minimum of a convex function is also a global
minimum.

Concave Function. A function f (X) is called a concave function if for any two
points X1 and X2 , and for all 0 1,
f [X2 + (1 )X1 ] f (X2 ) + (1 )f (X1 ) (A.2)

that is, if the line segment joining the two points lies entirely below or on the graph
of f (X).
Figures A.1b and A.2b give a concave function in one and two dimensions, respec-
tively. It can be seen that a concave function bends downard and hence the local
maximum will also be its global maximum. It can be seen that the negative of a con-
vex function is a concave function, and vice versa. Also note that the sum of convex
functions is a convex function and the sum of the concave functions is a concave
function. A function f (X) is strictly convex or concave if the strict inequality holds
in Eqs. (A.1) or (A.2) for any X1 = X2 . A linear function will be both convex and
concave since it satisfies both inequalities (A.1) and (A.2). A function may be con-
vex within a region and concave elsewhere. An example of such a function is shown
in Fig. A.3.

Testing for Convexity or Concavity. In addition to the definition given, the following
equivalent relations can be used to identify a convex function.

Engineering Optimization: Theory and Practice, Fourth Edition Singiresu S. Rao 779
Copyright 2009 by John Wiley & Sons, Inc.
780 Convex and Concave Functions

Figure A.1 Functions of one variable: (a) convex function in one variable; (b) concave function
in one variable.

Figure A.2 Functions of two variables: (a) convex function in two variables; (b) concave
function in two variables.

Figure A.3 Function that is convex over certain region and concave over certain other region.
 Convex and Concave Functions 781

Theorem A.1 A function f (X) is convex if for any two points X1 and X2 , we have
f (X2 ) f (X1 ) + f T (X1 )(X2 X1 )

Proof : If f (X) is convex, we have by definition

f [X2 + (1 )X1 ] f (X2 ) + (1 )f (X1 )

that is,
f [X1 + (X2 X1 )] f (X1 ) + [f (X2 ) f (X1 )] (A.3)

This inequality can be rewritten as

f [X1 + (X2 X1 )] f (X1 )

f (X2 ) f (X1 ) (X2 X1 ) (A.4)
(X2 X1 )
By defining X = (X2 X1 ), the inequality (A.4) can be written as
f [X1 + X] f (X1 )
f (X2 ) f (X1 ) (X2 X1 ) (A.5)
X
By taking the limit as X 0, inequality (A.5) becomes
f (X2 ) f (X1 ) f T (X1 )(X2 X1 ) (A.6)

which can be seen to be the desired result. If f (X) is concave, the opposite type of
inequality holds true in (A.6).

Theorem A.2 A function f (X) is convex if the Hessian matrix H(X) = [ 2 f (X)/
xi xj ] is positive semidefinite.

Proof : From Taylors theorem we have

n
 f
f (X + h) = f (X ) + hi (X )
xi
i=1
n 
n
2 f 

1 
+ hi hj (A.7)
2! xi xj X=X +h
i=1 j =1

where 0 < < 1. By letting X = X1 , X + h = X2 and h = X2 X1 , Eq. (A.7) can

be rewritten as
f (X2 ) = f (X1 ) + f T (X1 )(X2 X1 ) + 12 (X2 X1 )T
H{X1 + (X2 X1 )}(X2 X1 ) (A.8)

It can be seen that inequality (A.6) is satisfied [and hence f (X) will be convex] if H(X)
is positive semidefinite. Further, if H(X) is positive definite, the function f (X) will be
strictly convex. It can also be proved that if f (X) is concave, the Hessian matrix is
negative semidefinite.
782 Convex and Concave Functions

The following theorem establishes a very important relation, namely, that any local
minimum is a global minimum for a convex function.

Theorem A.3 Any local minimum of a convex function f (X) is a global minimum.

Proof : Let us prove this theorem by contradiction. Suppose that there exist two different
local minima, say, X1 and X2 , for the function f (X). Let f (X2 ) < f (X1 ). Since f (X)
is convex, X1 and X2 have to satisfy the relation (A.6), that is,
f (X2 ) f (X1 ) f T (X1 )(X2 X1 ) (A.6)

or
f T (X1 )S 0 (A.9)

where S = (X2 X1 ) is a vector joining the points X1 to X2 . Equation (A.9) indicates

that the value of the function f (X) can be decreased further by moving in the direction
S = (X2 X1 ) from point X1 . This conclusion contradicts the original assumption that
X1 is a local minimum. Thus there cannot exist more than one minimum for a convex
function.

Example A.1 Determine whether the following functions are convex or concave.
(a) f (x) = ex
(b) f (x) = 8x 2
(c) f (x1 , x2 ) = 3x13 6x22
(d) f (x1 , x1 , x3 ) = 4x12 + 3x22 + 5x32 + 6x1 x2 + x1 x3 3x1 2x2 + 15

SOLUTION
(a) f (x) = ex : H (x) = d 2f/dx 2 = ex > 0 for all real values of x. Hence f (x) is
strictly convex.
(b) f (x) = 8x 2: H (x) = d 2f/dx 2 = 16 < 0 for all real values of x. Hence
f (x) is strictly concave.
(c) f = 2x13 6x22 :
 
2 f/x12 2 f/x1 x2 12x1 0
H(X) = 2 =
f/x1 x2 2 f/x22 0 12

Here 2f/x12 = 12x1 0 for x1 0 and 0 for x1 0, and

 
H(X) = 144x1 0 for x1 0 and 0 for x1 0

Hence H(X) will be negative semidefinite and f (X) is concave for x1 0.

 Convex and Concave Functions 783

(d) f = 4x12 + 3x22 + 5x32 + 6x1 x2 + x1 x3 3x1 2x2 + 15:

f/x12
2
2 f/x1 x2 2 f/x1 x3

f/x1 x2 2 f/x22
2
2 f/x2 x3

H(X) =
2 f/x1 x3 2 f/x2 x3 2 f/x32

8 6 1

= 6 6 0
1 0 10
Here the principal minors are given by
|8| = 8 > 0
 
8 6
6 6 = 12 > 0
 
 
8 6 1
 
6 6 0 = 114 > 0
 
1 0 10

and hence the matrix H(X) is positive definite for all real values of x1 , x2 , and
x3 . Therefore, f (X) is a strictly convex function.