Sunteți pe pagina 1din 7

Finding the minimum of the banana function using steepest descent

 fx( ,y) := (1x− ) 2 + ( y − x 2 ) 2 <= Expression defining the banana function dfdx(x , y) := d fx( ,y) → (−2) + 2x⋅ − ( 4y ⋅ − x 2 ) ⋅x d x <= Expressions defining the terms dfdy(x , y) := d fx( ,y) → 2⋅y − 2 ⋅ x 2 of the gradient d y S x (x , y) := −dfdx(x , y) S y (x , y) := −dfdy(x , y) <= The search direction at any (x,y) point is just the negative of the gradient - the direction of steepest descent x 0 := −3 y 0 := 1 ⎛ ⎞ 5 <= Initial guess obj 0 := ( f x , y 0 0 ) obj 0 = 32 <= location of minimum as found using the Mathcad ( Minimize f x , 0 , y 0 ) = 1 ⎝ ⎠ minimize function - this is the 'right answer' f1( ,1)0= f

Iteration 1 S
, y 0
) = 56
x()d
:=
x
d
S
(
, y
0 )
x
( x 0
0 +
x
x 0
S
, y 0
) = 8
<= Define x and y in terms of distance
in the search direction, S
y
( x 0
y()d
:=
y
d
S
(
, y
0 )
0 +
y
x 0
F(d) := fxd( ( )yd,
(
))
<= Define the objective function in terms of d so that the
multi-dimensional problem is now a one-dimensional problem
13
12
F(d)
11
10
0.01
0.012
0.014
0.016
0.018

dd := .01

d star := Minimize(F, dd)

d

<= Initial guess for the Mathcad optimizer

d star = 0.016

<= d* is the location of the minimum value of the objective function along the line defined by the current search direction :=
d
⋅ dfdx
( x
, y
0 )
x 1
x 0
star
0
x 1 =
−2.101
<= New estimate of x* and
y*
:=
d
dfdy x
(
, y
0 )
y 1 = 5.128
obj 1 :=
f x
(
1 , y
1 )
obj 1 = 10.126
y 1
y 0
star
0
Iteration 2
S
, ) =
0.204
x()d
:=
x
d
S
( x
, y
1 )
x
( x 1
y 1
1 +
x
1 <= Define x and y in terms of distance
in the search direction, S
S
, ) = −1.427
y()d
:= y
d
S
( x
, y
1 )
y
( x 1
y 1
1 +
y
1 F(d) := fxd( ( )yd,
(
))
15
F(d) 10
5

012345

d

dd := .01

<= Initial guess for the Mathcad optimizer

d star := Minimize(F, dd)

d star = 2.172 :=
x
d
⋅ dfdx
( x
, y
1 )
=
−1.658
x 2
1 −
star
1
x 2
<= New estimate of x* and
y*
:=
y
d
dfdy x
(
, y
1 )
y
2 = 2.028
obj 2 :=
f x
(
2 , y
2 )
obj 2 = 7.587
y 2
1 −
star
1
Iteration 3
S
, y 2
) = 10.103
x()d
:=
x
d
S
x (
, y
2 )
x
( x 2
2 +
x 2
<= Define x and y in terms of distance
in the search direction, S
S
, y 2
) = 1.443
y()d
:=
y
d
S
y (
x
, y
2 )
y
( x 2
2 +
2
F(d) := fxd( ( )yd,
(
))
15
10
F(d)
5
0
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35

x 3

y 3

d

dd := .01

<= Initial guess for the Mathcad optimizer

d star := Minimize(F, dd)

:=

:=

x

y

2

2

d

d

star

star

(

(

dfdx x

dfdy x

2

2

,

,

y

y

2

2

)

)

d star = 0.055

x 3 =

1.101

y 3 = 2.108

<= New estimate of x* and y*

obj 3 :=

(

f x

3

, y

3

)

obj 3 = 5.216

Iteration 4

(

(

S y x 3

S x x 3

,

,

y 3

y 3

) = 0.256

) = −1.792

x()d

y()d

:=

x

:= y

3

3

+

+

d

d

S

S

x

y

(

(

x

x

3

3

)

, <= Define x and y in terms of distance in the search direction, S

, F(d) := fxd( ( )yd,

(

y

y

3

3

)

)) 6
5
F(d)
4
3
0
0.5
1
1.5
2
d

dd := .01

<= Initial guess for the Mathcad optimizer d star := Minimize(F,
dd)
d star = 0.965
:=
d
⋅ dfdx
( x
, y
3 )
=
−0.854
x 4
x star
3
3
x 4
<= New estimate of x* and
y*
:=
d
dfdy x
(
, y
3 )
y
4 = 0.379
obj 4 :=
f x
(
, y
4 )
obj 4 = 3.559
y 4
y star
3
3
4
Iteration 5
S
, ) =
4.903
x()d
:=
x
d
( x
, y
4 )
x
( x 4
y 4
4 +
S x
4
<= Define x and y in terms of distance in the
search direction, S
S
, ) = 0.7
y()d
:=
y
d
( x
, y
4 )
y
( x 4
y 4
4 +
S y
4
F(d) := fxd( ( )yd,
(
))
4
F(d) 2
0
0
0.1
0.2
0.3
0.4

x

y

5

5

d

dd := .01

<= Initial guess for the Mathcad optimizer

d star := Minimize(F, dd)

d star = 0.347

:=

:=

x

y

4

4

d

d

star

star

(

(

dfdx x

dfdy x

4

4

,

,

y

y

4

4

)

)

x 5 =

0.848

y 5 = 0.622

<= New estimate of x* and y*

obj 5 :=

(

f x

5

, y

5

)

obj 5 = 0.033

Iteration 6

S

S

x

y , ) =
−0.028
x()d
:= x
d
S
x (
, <= Define x and y in terms of distance in the
search direction, S
y
)
( x 5
y 5
5 +
x 5
5
, ) = 0.195
y()d
:=
y
d
S
y (
, F(d) := fxd( ( )yd,
y
)
(
))
( x 5
y 5
5 +
x 5
5
F(d)
0.03
0.025
0
0.1
0.2
0.3
0.4

d

dd := .01

<= Initial guess for the Mathcad optimizer

d star := Minimize(F, dd)

d star = 0.326

x

:=

6

:=

6

0.26

x

5

d

star

dfdx x

(

(

y dfdy x

y 5

d

star

Iteration 7

S

x

(

x

6

,

y

6

) =

x()d

:=

x

5

5

6

,

,

y

y

+

5

5

)

)

d

S

x

(

x

6

x 6 =

0.839

y 6 = 0.685

<= New estimate of x* and y*

obj 6 :=

(

f x

6

, y

6

)

obj 6 = 0.026

)

, <= Define x and y in terms of distance in the search direction, S

y

6

S

y , ) = 0.037
y()d
:=
y
+
d
S
( x
, F(d) := fxd( ( )yd,
y
)
(
))
( x 6
y 6
6
y
6
6
0.03
0.0283
0.0267
F(d)
0.025
0.0233
0.0217
0.02
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
0.26
0.28
0.3

d

dd := .01

<= Initial guess for the Mathcad optimizer d star := Minimize(F,
dd)
d star = 0.143
:=
x
d
star ⋅ dfdx
( x
, y
)
=
0.876
<= New estimate of x* and y*
x 7
6
6
6
x 7
:=
d
dfdy x
(
, y
)
y
= 0.691
obj 7 :=
f x
(
7 , y
7 )
obj 7 = 0.021
y 7
y 6
star
6
6
7
Iteration 8
S
, ) =
−0.022
x()d
:= x
d
S
x (
, <= Define x and y in terms of distance in the
search direction, S
y
)
x
( x 7
y 7
7 +
x 7
7
S
, ) = 0.154
y()d
:=
y
d
S
y (
, F(d) := fxd( ( )yd,
y
)
(
))
y
( x 7
y 7
7 +
x 7
7
0.022
0.02
F(d)
0.018
0.016
0
0.1
0.2
0.3
0.4

x 8

y 8

d

dd := .01

<= Initial guess for the Mathcad optimizer

d star := Minimize(F, dd)

d star = 0.322

:=

:=

x

y

7

7

d

d

star

star

(

(

dfdx x

dfdy x

7

7

,

,

y

y

7

7

)

)

x 8 = 0.869

y 8 = 0.74

<= New estimate of x* and y*

obj 8 :=

(

f x

8

, y

8

)

obj 8 = 0.017

Iteration 9

(

(

S y x 8

S x x 8

,

,

y 8

y 8

) = 0.21

) = 0.03

x()d

y()d

:=

:=

x

y

8

8

+

+

d

S

d S

x

y

(

(

x

x

8

8

, <= Define x and y in terms of distance in the search direction, S

,

(

)

y

y

8

8

)

F(d) := fxd( ( )yd,

)) 0.018
F(d)
0.016
0.014
0
0.05
0.1
0.15
0.2
0.25

d

dd := .01

<= Initial guess for the Mathcad optimizer

d star := Minimize(F,

dd)

d star = 0.137

x 9

y 9

:=

:=

x

y

8

8

d

d

star

star

(

(

dfdx x

dfdy x

8

8

,

,

y

y

8

8

Now, make a convergence plot

)

)

x 9 =

0.898

y 9 = 0.744

i := 09

<= New estimate of x* and y*

obj 9 :=

(

f x

9

, y

9

)

obj 9 = 0.014 100
10
obj i
1
0.1
0.01
Iteration Number

0123456789

i

Objective Function