Inequality constraints

Karush-Kuhn-Tucker Conditions
Inequality constraints
Constraint qualification
Equality constraints
Henrik Juel, DTU Management

min. f (x) s.t. gi (x) 0 i

Feasible solution xo
Binding constraints I = {i : gi (xo ) = 0}
Constraint qualification
KKT necessary optimality conditions:
If xo is a local minimum,
then there exist multipliers
ui 0 for i I
P
such that f (xo ) + iI ui gi (xo ) = 0

Karush-Kuhn-Tucker Conditions p.1/8

Inequality example

Karush-Kuhn-Tucker Conditions p.2/8

Geometric interpretation
f (x) points in the direction of steepest ascent
f (x) points in the direction of steepest descent

min. (x 3)2 + (y 2)2

s.t. x2 + y 2 5, x + 2y 4
and x, y 0

In two dimensions:
f (xo ) is perpendicular to a level curve of f
gi (xo ) is perpendicular to the level curve gi (x, y) = 0

The gradients are: f = (2x 6, 2y 4)

g1 = (2x, 2y), g2 = (1, 2)
g3 = (1, 0), g4 = (0, 1)

Constraint qualification example:

max. x s.t. y (1 x)3 , y 0

Consider the point (x, y) = (2, 1)

It is feasible with I = {1, 2}
(2, 2) + u1 (4, 2) + u2 (1, 2) = (0, 0)
u1 = 1/3, u2 = 2/3

Consider the global max: (x, y) = (1, 0) at a cusp

After reformulation, the gradients are: f = (1, 0)
g1 = (3(x 1)2 , 1) = (0, 1), g2 = (0, 1)

Class exercise: Solve max. ln(x + 1) + y

s.t. 2x + y 3, x, y 0

No u1 , u2 exist with
(1, 0) + u1 (0, 1) + u2 (0, 1) = (0, 0)

Karush-Kuhn-Tucker Conditions p.3/8

Karush-Kuhn-Tucker Conditions p.4/8

Constraint qualifications

Sufficient condition

KKT constraint qualification:

gi (xo ) for i I are linearly independent

min. f (x) s.t. gi (x) 0 i

Slater constraint qualification:

gi (x) for i I are convex functions
A nonboundary point exists: gi (x) < 0 for i I
Class exercise: Solve
min. x2 + y 2
s.t. x2 + y 2 5, x + 2y = 4
and x, y 0

If f and gi for i I are convex functions,

then a feasible KKT point is optimal
An equality constraint is equivalent to two inequality
constraints:
h(x) = 0
h(x) 0 and h(x) 0
The corresponding two nonnegative multipliers may
be combined to one free one:
u+ h (x) + u (h (x)) = vh (x)

Karush-Kuhn-Tucker Conditions p.6/8

Karush-Kuhn-Tucker Conditions p.5/8

Equality constraints also

Alternative formulation

min. f (x) s.t. gi (x) 0 i, hj (x) = 0 j

min. f (x) s.t. gi (x) 0 i, hj (x) = 0 j

As before, let xo be a feasible solution, define

I = {i : gi (xo ) = 0},
and assume that a constraint qualification holds.
Necessary optimality condition:
If xo is a local minimum,
then there exist multipliers
ui 0 for
P
Pi I ando vj j
o
o
such that f (x ) + iI ui gi (x ) + j vj hj (x ) = 0

This form of KKT condition is more common:

P
P
f (xo ) + i ui gi (xo ) + j vj hj (xo ) = 0
ui gi (xo ) = 0, ui 0 i
xo feasible
Or i vector form:
min. f (x) s.t. g(x) 0, h(x) = 0

Sufficient optimality condition:

If f and gi for i I are convex functions,
and hj j are affine (linear),
then a feasible KKT point is optimal
Karush-Kuhn-Tucker Conditions p.7/8

f (xo ) + ug (xo ) + vh (xo ) = 0

ug(xo ) = 0, u 0
xo feasible
Class exercise: Write the KKT conditions for
max. cx s.t. Ax b, x 0

Karush-Kuhn-Tucker Conditions p.8/8