Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function Module 8 Lecture Notes 1 Piecewise Linear

ar Approximation of a Nonlinear Function Introduction

In the previous lectures, we have learned how to solve a nonlinear problem using various methods. It is clear that unlike in linear programming, for nonlinear programming there exists no general algorithm due to the irregular behavior of nonlinear functions. One commonly used technique for solving nonlinear problems is to first represent the nonlinear function (both objective function and constraints) by a set of linear functions and then apply simplex method to solve this using some restrictions. In this lecture we will be discussing about a method to approximate a nonlinear function using linear functions. This method can be applied only to a single variable nonlinear function. For a nonlinear multivariable function consisting of n variables, this method is applicable only if the function is in separable form i.e., can be expressed as a sum of n single variable functions. In this lecture, only single variable nonlinear functions are discussed. Piecewise Linearization Any nonlinear single variable function f(x) can be approximated by a piecewise linear function. Geometrically, this can be shown as a curve being represented as a set of connected line segments as shown in the figure below.

f(x) f(t4)

f(t3)

f(t2) f(t1) t1 t2 t3 t4 x

D Nagesh Kumar, IISc, Bangalore

M8L1

Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function Method 1:

Consider an optimization function having only one nonlinear term f(x). Let the x-axis of the nonlinear function f(x) be divided by p breaking points t1, t2, t2, , tp and the corresponding function values be f(t1), f(t2),, f(tp). Suppose if x can take values in the interval

0 x X , then the breaking points can be shown as,

0 t1 < t 2 < ... < t p X .

Expressing x as a weighted average of these breaking points,

x = w1t1 + w2t 2 + ... + w p t p

i.e., x = wi t i
i =1

The function f(x) can be expressed as, f ( x ) = w1 f (t1 ) + w2 f (t 2 ) + ... + w p f (t p ) = wi f (ti )

i =1 p

where

w
i =1

=1

Finally the model can be expressed as Max or Min

f ( x ) = wi f (ti )
i =1

subject to the additional constraints

wt
i =1 p

i i

=x

w
i =1

=1

Linearly approximated model stated above can be solved using simplex method with some restrictions. The restricting condition specifies that there should not be more than two wi in the basis and also two wi can take positive values only if they are adjacent. This is because, if the actual value of x is between ti and ti+1, then x can be represented as a weighted

D Nagesh Kumar, IISc, Bangalore

M8L1

Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function

average of wi and wi+1 only i.e., x = wi t i + wi +1t i +1 . Therefore, the contributing weights wi and wi+1 only will be positive, rest all weights be zero. Similarly for an objective function consisting of n variables (n terms) represented as Max or Min f ( x ) = f1 ( x1 ) + f 2 (x2 ) + .... + f n ( xn ) subjected to m constraints
g 1 j ( x1 ) + g 2 j ( x 2 ) + .... + g nj ( x n ) b j for j = 1,2,..., m

The linear approximation of this problem is

Max or Min wki f k (t ki )
k =1 i =1 n p n p

subjected to wki g kj (t ki ) b j
k =1 i =1 p

for j = 1,2,..., m for k = 1,2,..., n

w
i =1

ki

=1

Method 2: Let x be expressed as a sum, instead of expressing as the weighted sum of the breaking points as in the previous method. Then, x = t1 + u1 + u 2 + .... + u p 1 = t1 + ui
i =1 p 1

where ui is the increment of the variable x in the interval (ti , ti +1 ) , i.e., the bound of ui is 0 u i t i +1 t i . The function f(x) can be expressed as

f ( x ) = f (t1 ) + i ui
i =1

p 1

where i represents the slope of the linear approximation between the points ti +1 and ti given by

D Nagesh Kumar, IISc, Bangalore

M8L1

Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function

i =

f (ti +1 ) f (ti ) . ti +1 ti

Finally the model can be expressed as Max or Min f ( x ) = f (t1 ) + i ui

i =1 p 1

subjected to the additional constraints

t1 + ui = x
i =1 p 1

0 ui ti +1 ti , i = 1,2,...., p 1

Example The example below illustrates the application of method 1. Consider the objective function Maximize f = x13 + x2 subject to
2 x12 + 2 x2 15 0 x1 4 x2 0

The problem is already in separable form (i.e., each term consists of only one variable). So we can split up the objective function and constraint into two parts g1 = g11 ( x1 ) + g12 ( x2 ) where f = f1 ( x1 ) + f 2 ( x2 )

g11 ( x1 ) = 2 x12 ; g12 ( x2 ) = 2 x2

Since f 2 ( x2 ) and g12 (x2 ) are in linear form, they are left as such and therefore x2 is treated as a linear variable. Consider five breaking points for x1 .

f1 ( x1 ) = x13 ; f 2 ( x2 ) = x2

D Nagesh Kumar, IISc, Bangalore

M8L1

Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function Table 1 i 1 2 3 4 5 t1i 0 1 2 3 4 f i (t1i ) 0 1 8 27 64 g1i (t1i ) 0 2 8 18 32

Thus, f1 ( x1 ) can be written as,

f1 ( x1 ) = w1i f1 (t1i )
i =1

= w11 0 + w12 1 + w13 8 + w14 27 + w15 64

and g11 (x1 ) can be written as,

g11 ( x1 ) = w1i g1i (t1i )
i =1

= w11 0 + w12 2 + w13 8 + w14 18 + w15 32

The linear approximation of the above problem thus becomes, Maximize subject to 2w12 + 8w13 + 18w14 + 32w15 + 2 x 2 + s1 = 15 w11 + w12 + w13 + w14 + w15 = 1 w1i 0 for i = 1,2,...,5 This can be solved using simplex method in a restricted basis condition. Table 2 Variables Iteration Basis f f = w12 + 8w13 + 27 w14 + 64w15 + x2

br
w15 -64 32 1

br c rs

w11
1 f 1 0 0 0 0 1

w12
-1 2 1

w13 -8 8 1

w14
-27 18 1

x2
-1 2 0

s1
0 1 0 0 --

s1 w11

15 1.87 1 1

D Nagesh Kumar, IISc, Bangalore

M8L1

Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function

From the table above, it is clear that w15 should be the entering variable. For that s1 should be the exiting variable. But according to restricted basis condition w15 and w11 cannot occur together in basis as they are not adjacent. Therefore, consider the next best entering variable w14 . This also is not possible, since s1 should be exited and w14 and w11 cannot occur together. Again, considering the next best variable w13 , it is clear that w11 should be the exiting variable. Table 3 Variables Iteration Basis f

br
w15 -56 24 1

br crs

w11
1 f 1 0 0 8 -8 1

w12
7 -6 1

w13 0 0 1

w14
-19 10 1

x2
1 2 0

s1
0 1 0 8 7 1 -3 15

s1
w13

For the table 2 above, the entering variable is w15 . Then the variable to be exited is s1 and this is not acceptable since w15 is not an adjacent point to w13 . Next variable w14 can be admitted by dropping s1 . Table 4 Variables Iteration Basis f

br
w15 -10.4 2.4 -1.4

br c rs

w11
f 1 1 0 0 -7.2 -0.8 1.8

w12
-4.4 -0.6 1.6

w13 0 0 1

w14
0 1 0

x2
4.8 0.2 -0.2

s1
1.9 0.1 -0.1 21.3 0.7 0.3 --

w14
w13

D Nagesh Kumar, IISc, Bangalore

M8L1

Optimization Methods: Advanced Topics in Optimization - Piecewise linear approximation of a nonlinear function Now, w15 cannot be admitted since w14 cannot be dropped. Similarly w11 and w12 cannot be

entered as w13 cannot be dropped. Thus, the process ends at this point and the last table gives the best solution. w13 = 0.3; w14 = 0.7 Now,

x1 = w1i t1i
i =1

= w13 2 + w14 3 = 2.7

and x2 = 0
The optimum value of f = 21.3 . This may be an approximate solution to the original nonlinear problem. However, the solution can be improved by taking finer breaking points.

D Nagesh Kumar, IISc, Bangalore

M8L1