Optimal control of an oscillator system

J.C.C. Henriques
2015/02/06

Optimal control

1.1

Problem statement

Let x(t ) the state of the system with input u(t ) that satisfies the state equation [1]
= F (x, u) ,
x

(1)

x (0) = x0

(2)

uU

(3)

t [0, T ]

(4)

with

with T fixed. The set U shows the constraints applied to u.

The goal is to determine u, defined in [0, T ] which maximizes the cost functional J defined as
J (u) = (xT ) +

L (u, x) dt

(5)

where
xT = x (T ) .

(6)

(xT ) is the cost associated with the terminal state xT . The Lagrangian L (u, x) is the function to
be maximized in [0, T ].

1.2

Pontriagyn maximum principle

Along optimal path for (x, u, (t )) it is verified the following necessary conditions for maximizing J
= F (x, u) ,
x
with
x (0) = x0
uU

t [0, T ]

(7)

= T F (x, u) + L (x, u)

(8)

and

subjected to the terminal condition

(T ) = (T ) .

(9)

The vector (t ) is designated by co-state and Eq. (8) by adjoint equation.

For each t , the Hamiltonian H defined by
H (x, u, ) = T F (x, u) + L (x, u)

(10)

is maximum for the optimal input u. If the maximum is for an u in the interior of U then
H
= 0.
ui

(11)

In the case that the optimum is in the boundary of U then Eq. (11) does not apply.

Oscillator system
Let as consider a simple oscillator system comprising a spring, k, a energy extraction damper,c,

and control damper, G, with damping as function of, u, see Fig. 1. The equation that describes the
motion of this system is
m z + c z + g u z + k z = f cos(t )

(12)

where the dot denotes time derivative, m is the block mass, f and are the modulus are angular
frequency of the external force. The initial conditions are z(0) = z0 and z(0) = v0 .
The extracted energy, e, can be computed using the instantaneous power of the damper c which
is given by
e = c z2 .

(13)

The initial condition is e(0) = 0.

The system of equations (12) and (13) can be written as a first-order system in vectorial form as
= F (x, u) ,
x

(14)

T
T
x = v z e = x1 x2 x3 ,

(15)

defining

Figure 1: Oscillating system with spring, k, energy extraction damper,c, and control damper, g .
and

F1
( f cos(t ) c x1 g u x1 k x2 ) /m

,
F (x, u) =
x1

F2 =
2
F3
c x1

(16)

with initial conditions

T
x0 = v0 z0 0 .

(17)

We want to determine u(t ) for 0 t T such that

J (u) = e(T ),

(18)

is maximum subjected to the constraint 0 u 1.

In the present case (xT ) = e(T ) and L (x, u) = 0, see Eq. (5). Using this, the adjoint equation
is given by
= T F (x, u) .
The gradient of F is given by

F =

F1
x1
F2
x
1
F3
x1

F1
x2
F2
x2
F3
x2

F1
x3
F2

x3
F3
x3

(19)

(c + g u) /m k/m 0

0
,

2c x1

(20)

resulting in

1
1 (c + g u) /m + 2 + 23 c x1

=
.
1 k/m
2

3
0
The final condition of is (T ) = (T ), giving

1 (T )

x1

(T ) =

x
2

3 (T )
x
3

x = xT

=
0 .
1

(21)

(22)

For each t , the Hamiltonian H

( f cos(t ) c x1 g u x1 k x2 ) /m

x1

H (x, u, ) = T F = 1 2 3

c x12

(23)

= 1 ( f cos(t ) c x1 g u x1 k x2 ) /m + 2 x1 + 3 c x12
is maximum for the optimal input u. The conditions to maximize H are
u(t ) =

1,

if (1 (t )x1 (t )) 0

0,

otherwise

(24)

The usual solution of this problem is:

1. Set u(t ) = 0.
2. Compute (14) using the (17).
3. Compute (21) backward (from T to 0) to impose the final condition (22).
4. Determine u(t ) using (24).
5. If not converged go to step 2.

Demonstration of the Pontriagyn maximum principle

Let us define the functional J as
J =J

T [
x F (x, u)] dt .

(25)

The vector is introduced so J is optimal and satisfies the system of ordinary differential equations (1) for all t [0, T ]. In other words,
T

T [
x F (x, u)] dt ,

(26)

is a J constraint that is zero for the optimal solution1 . The vector is called co-state. Introducing
Eq. (5) in Eq. (28) we get
J = (xT ) +

L (u, x) dt

= (xT ) +

T [
x F (x, u)] dt

0
T

L (u, x) + F (x, u) dt
T

0
1

Similar to a Lagrange multiplier.

(27)
T

dt .
x

Defining the Hamiltonian function

H (x, u, ) = T F (x, u) + L (x, u) ,

(28)

we get
J = (xT ) +

H (u, x, ) dt

dt .
T x

(29)

Let us assume that uopt is the function which maximizes then functional J then a small perturbation u results in a decrease of J . The system will follow another path x + x and the variation
of the objective function is negative


J = J (x + x, v) J x, uopt < 0,

(30)

where v = uopt + u. Using Eq. (29) we get

J = (xT + xT ) (xT ) +

[H (v, x + x, ) H (u, x, )] dt

T
x dt .

(31)

Integrating by parts the last term of Eq. (31)

Z

x dt = (T ) x (T ) (0) x (0)

T
x dt

(32)

Since the optimal control does not change x (0) we have x (0). As a result,
J = (xT + xT ) (xT ) T (T ) x (T )
ZT
Z
+
[H (v, x + x, ) H (u, x, )] dt +
0

T
x dt .

(33)

Performing a first order Taylor series expansion of the terms on x we get

(xT + xT ) (xT ) + (xT ) xT ,

(34)

H (v, x + x, ) H (v, x, ) + H (v, x, ) x

(35)

and

Replacing both expansions in Eq. (33)

Z

J = (xT ) (T ) x (T ) +

i
T

H (u, x, ) + x dt

(36)

[H (v, x, ) H (u, x, )] dt .

We can choose
T
= H (u, x, ) ,

(37)

satisfying
T (T ) = (xT ) ,

(38)

to assure that the first term of the right-hand-side of Eq. (36) is zero. This results in
J =

[H (v, x, ) H (u, x, )] dt .

(39)

The states x and co-states are computed for uopt and independently of v. If uopt is the optimum
then


H uopt , x, H (v, x, )

(40)

v U . This needs to be proved because we compute uopt by maximizing H (u, x, ).

Suppose that there is an time instant, t1 where a function w satisfies


H (w (t1 ) , x (t1 ) , (t1 )) > H uopt (t1 ) , x (t1 ) , (t1 )

(41)

Since H is a continuous function, there is an interval [t1 , t1 + ] where this inequality holds.
Let w = uopt except in this interval. Using this choice, the variation is
J =

t1 +


H (w (t ) , x (t ) , (t )) H uopt (t ) , x (t ) , (t ) dt > 0

t1

However, this contradicts the hypothesis that uopt is the optimal control.

References
[1] J. L. M. Lemos, Lectures on Optimal Control, Instituto Superior Tcnico, 2012.

(42)