NEW DEVELOPMENTS IN

PREDICTIVE CONTROL
FOR NONLINEAR SYSTEMS

M. J. Grimble, A. Ordys, A. Dutka, P. Majecki

University of Strathclyde
Glasgow Scotland, U.K
 Introduction
Model Predictive Control (MPC) is one of the
most popular advanced control techniques
The MPC algorithms are well established for
linear systems
Recent developments extended this methodology
to the Non-linear systems control
Techniques developed at the University of
Strathclyde are presented
Linear Quadratic Gaussian Predictive
Control
LQGPC
 Preview Control
The algorithm was first presented by Tomizuka M. and D.E. Whitney. The algorithm
uses the LQG approach to optimisation and a stochastic model for the reference signal
beyond the preview horizon.

x( t + 1 ) = Ax( t ) + Bu( t ) + Gw( t )

State space model:
y( t ) = Cx( t ) + v( t )

Performance index:
R
|
J ( t ) = S LMb y( t + j + 1 ) r( t + j + 1 )g b y( t + j + 1 ) r( t + j + 1 )g
M
T
|T j = 0 N
+ u( t + j )T u( t + j ) }

Reference generator: RN ( t + 1 ) = N RN ( t ) + N ( t + N )
 Preview Control

LM x( t + 1 ) OP = ( t + 1 ) = L A OP ( t ) + LM BOP u( t ) + LMG OP LM w( t ) OP
MN xR ,N ( t + 1 )PQ M
MN N PQ MN PQ MN N PQ MN ( t + N )PQ

A B G
C

LM y( t ) OP = LMC OP LM x( t ) OP + LM OPv( t )
MN RN ( t )PQ MN PQ MN xR ,N ( t )PQ MN PQ

A standard (infinite horizon) LQG approach can

now be used.
 Linear Quadratic Gaussian Predictive Control
LQGPC

The standard predictive control performance index:

b
J t = Y ( t ) R( t ) gT e bY ( t ) R( t )g + bU ( t )gT u bU ( t )g
x ( t +1 ) = Ax ( t ) + Bu( t ) + Gw ( t )
Starting with the state-space model:
y( t ) = Dx ( t ) + v ( t )

re-formulate it to be compatible with the above index:

state equation:
The control
x( t + 1 ) = A x( t ) + U ( t ) + W ( t ) vector U(t)
contains all
output equation: control
actions within
~ the horizon N
Y ( t ) = N A x( t ) + S N U ( t ) + S N W ( t ) + V ( t )
into the future
 Linear Quadratic Gaussian Predictive Control
LQGPC
Define the LQG-type performance index (finite or infinite)
as a sum of predictive performance indices:
R
J DPC = S
1 t +T U
J GPC ( l )V
R
J DPC = S lim
1 t+ U
J GPC ( l )V
T l=t
T + 1 W T l = t
T + 1 W
Substituting the criterion from the previous slide obtains:
R
Jt = S
1 T L O U
MbY ( l ) R( l )g e bY ( l ) R( l )g + bU ( l )g u bU ( l )gP V
T T
T l=t N
T + 1 QW
The solution can be obtained through Dynamic Programming
with two Riccati equations involved.
The reference generator is used in a similar way as in
Preview Control
Solution of Quadratic Gaussian
problem for nonlinear systems
and
Non-Linear Quadratic Gaussian
Predictive Control
 NLQG and NLQGPC

NLQG - an extension of SDRE method

SDRE method State Dependent Riccati
Equations
The NLQGPC algorithm: predictive
extension of SDRE
 System representation

The model:
xt +1 = f ( xt ) + g ( xt )ut + Gt
yt = h( xt )
Linear State Dependent form of the model:
xt +1 = A( xt ) xt + B( xt )ut + Gt
yt = C ( xt ) xt
Assumption on the parameterisation of the
model:
( A( xt ), B ( xt ) ) is controllable
x ,u
 System representation
Simplified notation:

At = A( xt ), Bt = B( xt ), Ct = C ( xt )

The Prediction of the trajectory:

ut , ut +1 ,..., ut + N 1 xt +1 , xt + 2 ,..., xt + N

Assumption on the trajectory after the

prediction horizon.
 The NLQG algorithm
Estimate (or measure) the state x(t)

Use previous feedback gain K(t-1) to calculate prediction of current

control u(t)

u (t ) = K (t 1) x(t )

Use current control prediction u(t) and the model re-calculated at time
instant t (with the state x(t)) to obtain future state prediction x(t+1).
 The NLQG algorithm
The state prediction x(t+1) together with the state feedback gain K(t-1)
from previous iteration of the algorithm is used for a calculation of the
future control prediction u(t+1).

u (t ) = K (t 1) x(t )

The model once again is re-calculated using future state prediction,

stored and sequence is repeated n times.
 The NLQG algorithm
Use the model prediction for time instant t+n and solve Algebraic
Riccati Equation.

The solution at time instant t+n is obtained: P(t + n, )

 The NLQG algorithm
Use as a boundary condition P(t + n) = P(t + n, ) for iterations of
the Riccati Difference Equation and use appropriate prediction of the
model throughout iterations of Riccati Equation.

Use P(k + 1) to calculate the feedback control gain and calculate the
current control.
 The NLQG algorithm
Use P(k + 1) to calculate the feedback control gain and calculate the
current control.

u (t ) = K (t ) x(t )

K (t ) = function( A(t ), B(t ), P(t + 1))

Calculated current control is used for the plant input signal manipulation.
 The NLQG cost function

The following cost function is minimised:

1 t +Th x (k )Qc (k ) x(k ) + u (k ) Rc u (k )

T T
J t = E lim
Th 2Th k =t +u (k ) M c x(k ) + x (k ) M c u ( k )
T T T

The cost function may be split in two parts:

infinite
J t = E lim
h
T 2
1
T h
Jt (
finite
+ Jt

)
 The NLQG cost function

First part is an infinite cost function:

t +Th xT (k )Qc (t + n p ) x(k ) + uT (k ) Rc u (k )

J tinfinite =
k = t + n + u T ( k ) M T x ( k ) + xT ( k ) M u ( k )
c c
and this part is minimised by Algebraic Riccati Equation.
Second part is a finite cost function

t + n 1 (k )Qc ( k ) x(k ) + uT ( k ) Rc u ( k )
T
x
J t finite =
k =t + u ( k ) M x ( k ) + x ( k ) M u ( k )
T T T
c c
And is minimised by Difference Riccati Equation with border condition
given by the solution of ARE.
 Remarks

To obtain accurate results it should be assumed that system will remain

time invariant after t+n time instant:

Therefore if real behaviour of the system is closer to the assumption

results are more accurate
Example
 Example

Plant model

1.7
( )
atan x p ,2 (t )
0 0.01
x p (t + 1) = x p ,2 (t ) x p (t ) + u (t ) + p (t )
0.3 0
0 1
yh (t ) = y (t ) = [1 0] x p (t )

Reference model

xr (t + 1) = [1] xr (t ) + [0] r (t )
rh (t ) = [1] xr (t )
 Results
output
1.6
S DRE
P ropos ed algorithm
1.4
Cons tant gain feedbac k

1.2

0.8

0.6

0.4

0.2

-0.2
0 10 20 30 40 50 60 70
 Results
c ontrol
2
S DRE
P ropos ed algorithm
1.5
Cons tant gain feedbac k

0.5

-0.5

-1

-1.5

-2

-2.5
0 10 20 30 40 50 60 70
 The NLQGPC control law derivation
Re-written state equation:
xt +1 = At xt + tU t , N + Gt
where
t = [ Bt , 01 , 02 ,..., 0 N ]

State-space model with prediction-output equation

xt +1 = At xt + tU t , N + Gt
Yt +1, N = t , N At xt + St , N U t , N + Gt , N t , N 1

Note: that state equation is identical to the state equation of

the controlled system.
 The NLQGPC control law derivation

Reference signal model:

X tR+1 = AR X tR + G RtR
Rt +1, N = C R X tR
Augmented system :
t +1 = t t + tU t , N + tA
t +1, N = t t + St , N U t , N + Gt , N t , N 1
with
xt At 0 A t t G 0
t = R , t = , = R , t = , =
R t ,
t 0
R
X t 0 A 0 G
t +1, N = Yt +1, N Rt +1, N , t = t , N At C R

 The NLQGPC control law derivation
The cost function:
1 t +Th N
{( yk +i rk +i ) } { }
N
J t = E lim T
iE ( yk +i rk + i ) + uk + i Ui uk + i
T

Th 2Th k =t i =1 i =0
or:
k T k T E k k + U k , N T Sk , N T E k k +
t +Th
1 + J
J t = E lim T T S
k k E k ,N k ,N k ,N k ,N E k ,N k ,N 0
U + U T
S T
S U +

Th 2Th k =t
+U T U
k,N U k,N

Introduce notation:
Qk = k T E k , M k = k T E Sk , N , Rk = S k , N T E S k , N + U
Final form:
1 t +Th Q M k k

J t = E lim k T
Uk,N T k
M T + J 0

Th 2Th k =t k Rk U k , N

 NLQGPC control law derivation - Algorithm 1

The control law minimising the cost function:

( ) ( )
1
Ut,N = Tt Pt , t + Rt Tt Pt , t + M tT t

where Pt , : solution of Algebraic Riccati

Equation

( )( ) (M )
1
Pt , = Qt + Tt Pt , t M t + Tt Pt , t Rt + Tt Pt , t t
T
+ t T Pt , t

This Algebraic Riccati Equation contains state

dependent matrices Qt , M t , Rt calculated at time t,
which contain the prediction of future system
behaviour
 NLQGPC control law derivation - Algorithm 2
A more accurate solution of the minimisation
problem:
t + N 1 Q M
kT U k ,N T +
k k k


1 k =t M
k Rk U k , N
T
J t = E lim + J0
Th 2Th t + N +Th Qk M k k
k T
U k,N T
T

k =t + N U k , N
M k R k

Cost function split in two parts

finite horizon
infinite horizon.
 NLQGPC control law derivation - Algorithm 2

Difference Riccati Equation :

( )( ) (M )
1
Pk = Qk + Tk Pk +1 k M k + Tk Pk +1 k Rk + Tk Pk +1 k k
T
+ k T Pk +1 k

boundary condition Pt + N = Pt + N ,
iterated backwards for k = t + N 1, t + N 1,..., t + 1

The control vector minimising cost

function: T
( ) ( )
1
U t , N = t Pt +1t + Rt
T T
t Pt +1t + M t t
 Example
The model is given by the following non-linear state space equations

( )
t1+1 = t1 0.3 sin t1 + t2 + gt1

0.3 ( ) + + g
3
t2+1 = t2 1
t t t
2

yt = t

Next the model is re-arranged into Linear State Dependent form:

1 ( )
0.3 sin 1
t
1


t +1 = t1 + 0 + g 0
t
t 1 t 0 g


( )
2
0 1 0.3 t2
yt = [1 0] t + [ 0]t ,
 Example

The system is controllable since

0 1

rank 2 = 2



( )
1 1 0.3 ( 2 )
t


 Example
The step response for two NLQGPC algorithms is compared with
SDRE with g=0.01 (noise level)
output
4

3.5

2.5

1.5

0.5 Alg. 2, Setpoint=1: J=2.5178Setpoint=3: J=23.6322

Alg. 1, Setpoint=1: J=2.5214 Setpoint=3: J=24.1597
SDRE, Setpoint=1: J=2.5799 Setpoint=3: J=27.2109
0
0 5 10 15 20 25
 Example
Now compare noise rejection (for two levels of process noise)

output
4

2
Alg. 2, g=0.01: J=23.5602
Alg. 1, g=0.01: J=23.9984
1 SDRE, g=0.01: J=26.9404

0
0 5 10 15 20 25

output
4

2
Alg. 2, g=0.1: J=22.4603
1 Alg. 1, g=0.1: J=22.852
SDRE, g=0.1: J=25.7435
0
0 5 10 15 20 25
Advantages & Disadvantages of NLQGPC

Advantages:
1. Controls based on solutions to the NLQGPC
have been shown to offer high performance.
2. Less computational burden than other non-linear
predictive control techniques.
Disadvantages:
1. Since NLQGPC utilizes the Riccati equation, it
is an unconstrained predictive control technique.
2. Like SDRE, NLQGPC doesnt guarantee closed-
loop global stability.
Dealing with constraints in NLQGPC
The input constraints can be approximated by means of
smooth limiting functions, and then included into the
dynamics of the plant in a state-dependent state-space form.

0 u u
 Improving stability via Satisficing

Satisficing is based on a point-wise

cost/benefit comparison of an action.
The benefits are given by the Selectability
function Ps(u,x), while the costs are given by
the Rejectability function Pr(u,x).
The satisficing set is those options for which
selectability exceeds rejectability: i.e.,
S ( x , b ) = {u : Ps ( u , x ) bPr ( u , x )}
CLF-Based Satisficing Technique

The selectability criteria is defined to be:

T
Ps ( u , x ) = V x ( f + gu )
The rejectability criteria is defined to be:
Pr (u , x) = l ( x) + x RxT

b=0, therefore, the satisficing set S:

S ( x, b) = {u : VxT ( f + gu ) 0}
Augmenting NLQGPC with Satisficing

By projecting the NLQGPC controller

point-wise onto the satisficing set, the good
properties of the NLQGPC approach are
combined with the analytical properties of
satisficing.
 Example: Control of F-8 aircraft

The non-linear
dynamical
model of the F-8
fighter aircraft:

x1 = 0.877 x1 + x3 0.088 x1 x3 + 0.47 x12 0.019 x 22

x12 x3 + 3.846 x13 0.215u + 0.28 x12 u + 0.47 x1u 2 + 0.63u 3 ,
x 2 = x3 ,
x 3 = 4.208 x1 0.396 x3 0.47 x12 3.564 x13
20.967u + 6.265 x12 u + 46 x1u 2 + 61.4u 3
u 0.05236 rad
 0.5

Blue linesUnconstrained NLQGPC.

0.4
Black lines Constrained NLQGPC.
0.3 Magenta lines Constrained NLQGPC
with guarantee of global asymptotic stability
Angle of Attack (rad)

0.2

0.1

-0.1

-0.2
0 2 4 6 8 10 12 14
Time
 Elevator Deflection
0.04

0.02

0
u (rad)

-0.02

u 0.05236 rad
-0.04

-0.06

-0.08
0 2 4 6 8 10 12 14
Time
 NLQGPC

High Deals with input

Performance Constraints

Low Guarantee of
Computational Robustness &
Burden Asymptotic
Stability
Non-Linear Generalized Minimum
Variance Control
 Contents

Introduction

Nonlinear GMV control problem and solution

Relationship to the Smith Predictor

Incorporating future information:

Feedforward and Tracking

Simulation example
 Introduction

1969: strm introduces Minimum Variance (MV) controller assuming linear

minimum phase plant. Successful applications in pulp and paper industry.
1970s: Clarke and Hastings-James modify the MV control law by adding a control
costing term. This is termed a Generalized Minimum Variance (GMV) control law
and is the basis for their later self-tuning controller.

The GMV control law has similar characteristics to LQG design in some cases and
is much simpler to implement
However, when the control weighting tends to zero the control law reverts to the
initial algorithm of strm, which is unstable for non-minimum phase processes.
 Introduction
Aim: introduce a GMV controller for nonlinear, multivariable,
possibly time-varying processes

The structure of the system is defined so that a simple solution is obtained.

When the system is linear the results revert to those for the linear GMV controller.
There is some loss of generality in assuming the reference and disturbance models
are represented by linear subsystems.
However, plant model can be in a very general nonlinear operator form, which
might involve state-space, transfer operators, neural networks or even nonlinear
function look-up tables.
 Nonlinear system description
0 = Pc e + Fcu

+ +

Error Control Disturbance

weighting Pc Fc weighting
Wd
model

Nonlinear
plant d
Reference Controller
r + e u m +
Wr C0 W
- +
y

Nonlinear plant model: ( Wu )( t ) = z -k ( Wk u ) ( t )

Linear disturbance model: Wd = A f 1Cd

Linear reference model: Wr = A f 1Er

 Plant model
Nonlinear plant model can be given in a very general form, e.g.:
state-space formulation
neural network / neuro-fuzzy model
look-up table u y
Fortran/C code
f (u , y ) = 0
It can include both linear and nonlinear components,
e.g. Hammerstein model:

Plant subsystems Wd

d
u m y
W1k W0k z k +
Control Output
Nonlinear Linear Delay

Just need to obtain the output to given input signal

 NGMV problem formulation
To minimize: variance of the generalized output 0(t):

J NGMV = E [ 02 (t )]

with 0 (t ) = Pc e ( t ) + ( Fc u ) ( t )

Pc = Pcn Pcd1 - linear error weighting

( Fc u ) ( t ) = z k ( Fc k u ) ( t ) - control weighting (possibly nonlinear)

Control weighting assumed invertible and potentially nonlinear to

compensate for plant nonlinearities in appropriate cases

The weighting selection is restricted by closed-loop stability

 NGMV problem solution
The approach also similar: k
PY
c f = F + z R
0 (t ) = Pc ( z k ( Wk u ) ( t ) + Y f (t )) + ( Fc u ) ( t ) Diophantine equation
= z k (Fck Pc Wk )u ( t ) + PcY f (t ) Y f Y f* = WdWd* + WrWr*
0 (t ) = F (t ) + (Fck Pc Wk )u ( t k ) + R (t k ) Spectral factorization

(t) white noise (sequence of

statistically independent
independent random variables)

Optimal control: u NGMV (t ) = (Fck Pc Wk ) 1 R (t )

stable causal nonlinear operator inverse

 Controller implementation
u NGMV (t ) = [(Fck FY f1Wk )1 RY f1e](t )
Disturbance
d
Controller
Reference
Plant
r e
+ 1 - 1 u Output
-
RY f + F ck W +
+ y
Wk
FYf1
linear blocks
Existence of a Stable Operator Inverse
Necessary condition for optimality:

operator ( PcWk Fck ) must have a stable inverse

For linear systems: the operator must be strictly minimum-phase.

To show this is satisfied for a very wide class of systems consider the case where
Fck is linear and negative so that Fck = -Fk . Then obtain:

( PcWk + Fk ) u = Fk ( Fk1PcWk + I ) u

return-difference operator for a feedback system with

A delay-free plant and controller

K c = Fk1Pc .
 PID-based initial design
Consider the delay-free plant Wk and assume a PID controller KPID exists to stabilize
the closed-loop system.
Then a starting point for the weighting choice that will ensure the operator
( PcWk + Fk ) is stably invertible is

Pc = K PID , Fk = 1

To demonstrate this selection reasonable consider scalar case and let controller

( k0 + k1 + k2 ) ( k0 + 2k2 ) z 1 + k2 z 2
K c = k0 +
k1
1 z 1
( 1
+ k2 1 z = ) 1 z 1

Assume the PID gains are positive numbers, with small derivative gain. Then
simple to confirm if Fk = 1 the Pcn term is minimum phase and has real
zeros.
 Relationship to the Smith Predictor
The optimal controller can be expressed in a similar form to that of a Smith
Predictor. This provides a new nonlinear version of the Smith Predictor.

Compensator d

Reference Plant
r _ u
+ ( Ap Pcd )1G0Yf 1 + 1
Fck
+ y
_ _ + W
+

Wk

F0Y f1

mk
( Ap Pcd )1G0Yf 1Dk

Dk

-
+
 Smith Predictor form of NGMV controller
The system may be redrawn and the compensator rearranged as shown below.
This structure is essential if Pc includes an integrator.

Compensator
Disturbance
Plant
Reference Output
u
0 - + y
+ Ap1G0Y f1 Fck1 Pcd1 W +
r - +

Wk

Pcn

Dk

p
_
+
 Comments
This last structure is intuitively reasonable. With no plant-model mismatch, the
control is not due to feedback but involves an open-loop stable compensator.
1
The nonlinear inner-loop has weightings Fck Pc acting like an inner-loop
controller. If weightings are chosen to be of usual form this will represent a filtered
PID controller.

Such a choice of weightings is only a starting point, since stability is easier to

achieve. However, control weighting can have additional lead term and high
frequency characteristics of optimal controller will then have more realistic roll off.

Stability: Under the given assumptions the resulting Smith system is stable. This
follows because the plant is stable, the inner-loop is stable and there are only
stable terms in the input block.
 Feedback + Feedforward + Tracking Control
future reference 0 (t ) = Pc e ( t ) + ( Fc u ) ( t )
information

Measurable Un-measurable
+ +
Error Control
Pc
weighting Fc weighting
r C2
Reference
Disturbance
Wr yf Wd Wd0
+ + + models
r C1
+
Nonlinear
Setpoint Controller plant d1 d0

+ + u m + +
Ww Hf C0 W y
w - e0 + + +
Scaling

Hf

Feedback gain/dynamics
 Feedback, Tracking and Feedforward Control
Signal Generation Control Modules

yf Total
Measured
disturbance (G P 1 1 i 1 1
0 0 d D f 1 z G1P1d Dd 1 )W 1
di Wd 3
u
f
disturbance

ur + +
d
Reference r (t + p ) G2 Prd1Er1
Nonlinear
plant
+ u Output
+ - +
Setpoint Hf w G0 P0 d D f 11
1
Fck1 W
- e + + m + y
Wk
F0Y f1

Feedback, Feedforward and Tracking Controller

Hf
 Simulated example
2-by-2 model given in the non-linear state space form:
x2 (t )
x1 (t + 1) = + u1 (t )
1 + x12 (t )
x12 (t )
x2 (t + 1) = 0.9 x2 (t )e + u2 (t )
y (t ) = x(t )

Both outputs are followed by a transport delay of k = 6 samples so the time-

delay matrix:
z 6 0
Dk =
0 z 6

Models:
1 1 0.1
1 z 1 0 1 z 1 0 1 0.5 z 1 0
Wr = Wd = z 6 Wd 0 =
0 1 0 1 0.1
0
1 z 1 1 z 1 1 0.5 z 1

reference measurable unmeasurable

disturbance disturbance
 Transient responses
Feedforward action

Output 1 setpoint
2 NGMV Control 1
NGMV+FF 1.5
NGMV+FF+TR
1.5
1
1

0.5
0.5

0 0
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200
Output 2 Control 2
2 1.5
NGMV
1.5 NGMV+FF
1 NGMV+FF+TR
1
0.5
0.5

0 0
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200

Future reference information incorporated

 Stochastic performance
Output 1 setpoint Control 1
3 NGMV 2
NGMV+FF
NGMV+FF+TR
2
1
1

0
0

-1 -1
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200
Output 2 Control 2 NGMV
3 1.5 NGMV+FF
2 NGMV+FF+TR
1
1 0.5
0 0
-1 -0.5
-2
0 20 40 60 80 100 120 140 160 180 200 -1
0 20 40 60 80 100 120 140 160 180 200

Controller Var[e] Var[u] Var[0]

NGMV FB 0.61 8.43 2.45
NGMV FB+FF 0.51 8.41 0.82
NGMV FB+FF+TR 0.48 8.37 0.27
 Concluding Remarks

State dependent models gives a useful structure for NL

predictive controllers
NGMV has much potential for development with multi
- step predictive control an obvious development.
Key to success in NL control is to show works and practical
on real processes - why NGMV seems great potential.