MATH160

NUMERICAL
OPTIMIZATION
 At the end of the period, you should
be able to:
 Use Newton’s method to find the
maximum/minimum of a function.
 Use Quadratic interpolation to
find the maximum/minimum of a
function.

2
Root location and optimization are
related in the sense that both involve
guessing and searching for a point on a
function.
Root location involve searching for
zeros of a function while optimization
involves searching for either the
minimum or the maximum.
The optimum is the point where the
curve is flat, these corresponds to the
x value where the derivative f’(x) is
equal to zero.
Additionally, the second derivative
f”(x), indicates whether the optimum
is a minimum or a maximum:
if f”(x)<0 the point is a maximum;
if f”(x)>0 the point is a minimum.
Newton’s Method:
The Newton-Raphson method is an
open method that finds the root of
a function using
f xi 
xi 1  xi 
f xi 
Newton’s Method:
A similar open approach can be used
to find the optimum of f(x) by
defining a new function , g(x) = f’(x),
leading to the equation
f xi 
xi 1  xi 
f xi 
Example:
Use Newton’s method to
approximate the maximum of
2
f  x   2 sin x 
x
10
with initial guess of 2.5.
Solution:
x2
f  x   2 sin x 
10
The first and second derivatives are

f  x   2 cos x  f  x   2 sin x 

x 1

5 5
Solution:
Substitute the derivatives to the
equation x
2 cos x i 
x i 1  x i  5
1
 2 sin x i 
5

Substitute the initial guess

x1  0.99508  f  x   1.57859
Solution:
The process can be repeated, with
the following result
x 2  1.46901  f  x   1.77385
x 3  1.42764  f  x   1.77573
x4  1.42755  f  x   1.77573
Quadratic Interpolation
Quadratic interpolation takes
advantage of the fact that a second
order polynomial often provides a good
approximation to the shape of f(x)
near an optimum.
If we have three points that jointly
bracket an optimum, we can fit a
parabola to the points.
Then we can differentiate it, set the
result equal to zero, and solve for an
estimate of the optimal x.

Given the initial guesses x0 , x1 , and

x2 , the quadratic equation of the
parabola is defined by
 x  x1  x  x2   x  x0  x  x2   x  x0  x  x1 
f  x      f  x0      f  x1      f  x2 
 x0  x1  x0  x2   x1  x0  x1  x2   x2  x0  x2  x1 
It can be shown through some
algebraic manipulation that the result
is
f  x0 x12  x 2 2  f  x1 x 2 2  x0 2  f  x 2 x0 2  x12 
x3 
2 f  x0  x1  x 2   2 f  x1  x 2  x0   2 f  x 2  x0  x1 

where x3 is the value of x that

corresponds to the maximum value of
the quadratic fit to the guesses.
Example:
Use quadratic interpolation to
approximate the maximum of
2
f  x   2 sin x 
x
10
with initial guesses of 0, 1, and 4
Solution:
The points are (0,0), (1,1.5829),
(4, -3.1136)

x3  1.5055,  f 1.5055  1.7691

Since the new value is to the right
of x1 and the new point is higher
than the intermediate point we
discard x0.
Solution:
The new points are (1,1.5829),
(1.5055, 1.7691), (4, -3.1136) so
x4  1.4903,  f 1.4903  1.7714
The process can be repeated, with
the following result
x5  1.4256,  f 1.4256  1.7757
x6  1.4266,  f 1.4266  1.7757

x7  1.4275,  f 1.4275  1.7757

Exercise:
Find the extrema of the function

f  x   5 sin2 x   2 x  4 x
2

using Newton’s method

using Quadratic Interpolation
method