Chapter 10

Linear Quadratic Regulator Problem

Minimize the cost function J given by
J=

1
2 0

( x' Qx + u ' Ru )dt

R>0
Q0

positive definite (symmetric with positive eigenvalues)

positive semi definite (symmetric with nonnegative eigenvalues)

subject to

x& = Ax + Bu

( y = Cx )

LQR SOLUTION:
Find the positive-definite solution P of the ARE (Algebraic Ricatti Equation)
A' P + PA + Q PBR 1 B' P = 0
u = Kx
where
K = R 1 B ' P
The positive-definite solution of the ARE results in an asymptotically stable closed-loop system if:
1)
2)
3)

the system is controllable

R>0
Q = C q ' C q where (Cq,A) is observable

These conditions are necessary and sufficient

We can define another output z where

z = Cq x

controlled or regulated output

therefore

x' Qx = x' C q ' C q = z ' z

LQR design of double integrator

0 1
A=

0 0
assume

0
B=
1

1 0
Q=

0 0

C q = [1 0]

R =1

1
Q = [1 0] = C q ' C q
0
1

(A, B) is controllable
(Cq, A) is observable

note P is symmetric

ARE:

0 0 p1
1 0 p

p2 p1
+
p3 p2

p2 0 1 1 0 p1 p2 0 0 p1 p2
+

p3 0 0 0 0 p2 p3 0 1 p2 p3
1442443 1424
3
0
0

p2
p3

p1
p
2

p2
p3

14444244443
p22

p2 p3

p2 p3

p32

0 0
=

0 0
solving
0
p
1

0 0
+
p 2 0

p1 1 0 p 22
+

p 2 0 0 p 2 p3

1 p 22 = 0 p 22 = 1
p1 p 2 p3 = 0
p1 = p 2 p3
p1 p 2 p3 = 0

(1)
(2)
(3)

2 p 2 p32 = 0

(4)

p 2 p 3 0 0
=

p32 0 0

p2 = 1
p3 = 2
p1 = 2

2
P=
1

2
K = R 1 B' P = 1[0 1]
1

1
=1
2

The closed loop system matrix becomes

1
0
A BK =

1 2
Closed loop roots are:
2 + 2 + 1 = 0

2
(1 j )
2

damping ratio is 0.707

2

The loop transfer function is:

K ( s ) B
= K ( sI A) 1 B

2 s+

2
2

65 phase margin
infinite gain margin

(s)
R(s) +

B
-

1
s

L( s ) =
=
=
=

2 s+
s2
2
2

2
2

[1 + s]
2
2

s2
1+

s
1
2

s2
s
1 + 0.7071
s2
L( j ) >0.7071
1.4142

mag

z = 0.7071

j
0.7071
2

( j )

1.4142

L ( j c ) = 1 =

1.4142

c = 1.4142
c
Phase@ c = - 180 + tan -1

0.7071

144244
3

phase
-180

phase m arg in

1
o
Phase margin = tan 1 10..4142
7071 = tan ( 2) = 63.4

10 z

10

USING MATLAB TO GET EXACT RESULTS

Matlab
num = sqrt(2)*[ 1

sqrt(2)/2 ]

den = [ 1

0]

results:

margin(num,den)
gm=
pm=65.53

@ =1.554

Properties of LQR design

From the ARE we can derive the relation

1 + L ( j ) = 1 +
2

G q ( j )

(*)

is a scalar
where L( s ) = K ( s ) B

and

-loop gain
( s) ( sI A) 1

Q = Cq ' Cq
G q ( s ) = C q ( s ) B

From (*) we see

1 + L ( j ) 1

This implies that the Nyquist plot of the loop transfer function of an LQR design
always stays outside of a unit circle centered at (-1,0).

In SISO case, LQR design has > 60 phase margin, infinite gain margin and a gain reduction tolerance
of -6dB (i.e. the gain can be reduced by a factor of 12 before instability occurs).
Recall pole placement does not guarantee stability margins.
High-frequency roll-off rate
Closed loop transfer function T ( j ) = K ( jI A + BK ) 1 B
lim T ( j ) =

1
j

KB =

1
j

R 1 B' PB < 0

-20dB/dec roll off rate at high frequencies

- not good for noise suppression

Optimal Observers Kalman Filter

State estimation plant represented as

x& = Ax + Bu +
y = Cx + v

process noise

measurement noise

The optimal filter is given by

x& = Ax + Bu + L( y Cx )
5

where L = C ' R01

where is the positive definite solution of

A + A'+Q0 C ' R01C = 0

Qo and Ro are noise covariance matrices, which represent the intensity of the process and sensor noise
inputs.
Require Q0 0, R0 > 0 and system to be observable.
If we combine the Kalman-Bucy Filter (optimal estimator) with LQR design, we have LQG (Linear
Quadratic Gaussian). Lets do a LQG design for double integrator plant. We already have the LQR
design.
For Kalman filter, assume

1 0
Q0 =
and R0 = 1
0 0
a b
Solving Ricatti equation with =

b c
a 2 = 2b + 1

we find

ab = c
b2 = 1

3
=
1

3
and L = C ' R01 =
1
Transfer function of compensator is given by

H ( s ) = K ( L A + BK + LC ) 1 L
3.14( s + 0.3)
=
( s + 1.57 + j1.4)( s + 1.57 j1.4)
Comparison of LQR and LQG
-LQR has guaranteed stability margins
-LQG has no guaranteed stability margins
6

-high freq. roll off in LQG can be > 20 dB/dec exhibited by LQR greater noise filtering in
LQG
-LQG is not robust uncertainty in plant may cause system to go unstable
Loop Transfer Recovery (LTR)

> 60 phase margin

infinite gain margin

LQG

no guaranteed margins

LQR

The properties of LQR can be recovered asymptotically by using Qo and R0 as tuning parameters

LQR

loop gain, L( s ) = K ( s ) B
= K ( sI A) 1 B

LQG

sX = ( BK + A LC ) X + LY ( s )
X ( s )
= ( sI A + BK + LC ) 1 L
Y (s)

loop gain, LLQG ( s ) = K ( sI A + BK + LC ) 1 LC ( s ) B

If the following two conditions hold then LQR loop properties can be recovered if
1)
2)

G(s) is minimum phase

R0=1 and Q0=q2BB

Then it can be shown

lim LLQG ( s ) = L( s )
q

The variable y that is recovered may be different from the variable z that is to be controlled
where y = Cx

and

z = Cq x

Loop Shaping Steps

1)

Determine the controlled variable and set

Q=CC

and

Q=CqCq

2)

Get a desired loop gain in LQR design. Use R as tuning parameter.

3)

Select scalar q and solve the filter Ricatti equation

A + A'+ q 2 BB'C ' C = 0

L = C '

4)

Increase q until the resulting loop transfer function is close to the LQR design
Do not make q too high since
1)
large gains in L are required
2)
the undesirable -20dB/dec high freq. roll-off of LQR will be recovered

Example
Double integrator system
with

1 0
Q=

0 0

R=1

Gave 65 phase margin for LQR design

10

Robustness
1)
2)

Robust stability stable in the face of plant uncertainties

Robust performance performance met even in the face of plant uncertainties

Two important properties of feedback

1)
2)

sensitivity reduction
disturbance rejection

General feedback system

Y (s) =

G ( s) H ( s)
1
G ( s) H ( s)
R( s) +
D( s)
N ( s)
1 + G ( s) H ( s)
1 + G ( s) H ( s)
1 + G ( s) H ( s)

Tracking error e = r - y

E ( s) =

1
1
1
R( s)
D( s)
N (s)
1 + G ( s) H ( s)
1 + G ( s) H ( s)
1 + G ( s) H ( s)

Actuator output (i.e. plant input) is given by

U (s) =

H ( s)
[R( s) D( s) N ( s)]
1 + G ( s) H ( s)

note:

U ( s) = H ( s) E ( s)
E ( s ) = H 1 ( s )U ( s )

Define the following terms

J ( s ) = 1 + GH
1
S (s) =
1 + GH
GH
T (s) =
1 + GH
note:

return difference
sensitivity
complementary sensitivity

S ( s) + T ( s) = 1

Using these definitions

system output:

Y ( s ) = S ( s ) D( s ) + T ( s )[R( s ) N ( s )]

tracking error:

E ( s ) = S ( s )[R( s ) D( s ) N ( s )]
11

U ( s ) = H ( s ) S ( s )[R( s ) D( s ) N ( s )]

plant input:

From these expressions we see that we need

1)

Disturbance rejection: From Y(s) expression we see we require S small GH>>1

(since SD)

2)

Tracking: S small

3)

Noise suppression: From Y(s) we have T(s)N(s) require T small

4)

Actuator limits: From U(s) expression want H(s)S(s) bounded

Tracking and Disturbance rejection require small S

Noise suppression requires small T

Also

however
however

S+T=1
command inputs and disturbances are low frequency whereas measurement noise
is high frequency signal

keep S small in low frequency range and T small in high frequency range

H (s)S (s) =

H (s)
T ( s)
=
1 + G ( s) H ( s) G ( s)

making T small we reduce control energy

Loop Gain Properties

Low Frequency

Mid. Frequency

Performance (R)

High Gain

Smooth Transition

Disturbance Rejection (D)

Noise Suppression (N)

High Gain

High Frequency

(for good margins)

Low Gain

Uncertainty Modeling
Two categories --

1)
2)

structured uncertainty
unstructured uncertainty

We will deal with unstructured uncertainty

12

~
Additive uncertainty: actual model G ( s )

~
(s) +
G ( s) = G
{
model

Multiplicative uncertainty:

a (s)
1442
44
3
uncertainty or error

~
G ( s ) = [1 + m ( s )]G ( s )

input uncertainty

output uncertainty

Robust Stability
We say a compensator robustly stabilizes a system if the closed-loop system remains stable for the true
~
plant G ( s) .
Robustness results can be derived using the small gain theorem.
Small Gain Theorem

The closed-loop system will remain stable if

G ( s) H ( s) < 1

no since G ( s ) H ( s ) G ( s ) H ( s )
13

then closed-loop stability is guaranteed if

G ( s) H ( s) < 1

There is no possibility of encirclements of (-1,0) point by Nyquist plot.

Two equations that the small gain theorem can help us to answer
1)
Given that the uncertainty is stable and bounded, will the closed-loop system be stable
for the given uncertainty?
2)
For a given system, what is the smallest uncertainty that will destabilize the system?
To answer these questions we first do some block diagram manipulation

With multiplicative output uncertainty

where M ( s ) =

G(s) H (s)
1 + G ( s) H ( s)

Determine M(s), the transfer function seen by m

14

By small gain theorem, closed-loop system will be robustly stable if

m <

i.e.

m <

1
GH (1 + GH ) 1
1
T

T complementary sensitivity

If the uncertainty is bounded by so that

m <

then the closed-loop system will be stable if

T <

T < 1

or

This answers the first question

Second question: find the size of the smallest stable uncertainty that will destabilize the system
Because the uncertainty must be smaller that 1/T, it must be smaller that the minimum of 1/T. We
must find the maximum of T.
Define

M r = sup T ( j )

sup = supremum (least upper bound)

Then the smallest destabilizing uncertainty, we call this the multiplicative stability margin or MSM, is
given by

MSM =

1
Mr

For additive uncertainty

M ( s) =

H ( s)
1 + G(s) H (s)

closed-loop will be robustly stable if

a <

1
H (1 + GH )

or

a <

1
HS

15

if uncertainty is stable and bounded by

a <

then we guarantee closed-loop stability if

HS <

or

HS < 1

we can define additive stability margin (ASM) by

ASM =

1
sup H ( j ) S ( j )

Example

G ( s) =

5 s
,
( s + 5)( s 2 + 0.2 s + 1)

H (s) =

5( s + 0.1) s + 0.2
s
s+5

phase margin: 38
gain margin: 2.8 (9dB)
Find MSM and ASM:
MSM
Find peak of T (complementary sensitivity function)
peak = 1.52 MSM = 0.65

the system will be robustly stable against unmodelled multiplicative uncertainties

with transfer function magnitude < 0.65

16

Problem 10.9
a.)

~
G = (1 + m )G

~
G
m = 1
G

2( s + 1)
s ( s 2 + s + 1)
m =
1
1
s2
2

2( s + 1)
s2 + s +1

s 2 ( s 2 + s + 1) s 2 ( s 2 + s + 1)

s2 + s +1
s2 + s +1

b.)

M(s)

20( s + 1)
GH
20( s + 1)
20( s + 1)
s 2 ( s + 10)
=
= 2
= 3
20( s + 1)
(1 + GH )
s ( s + 10) + 20( s + 1) s + 10 s 2 + 20 s + 20
1+ 2
s ( s + 10)

c.)

SGT:

m M < 1

m <

1
=
M

1
GH
1 + GH

m < 1 + (GH ) 1

17

Additive uncertainty

~
G = G + a
~
a = G G
=

2( s + 1)
1
2
2
s ( s + s + 1) s

1 2 s + 2 s 2 s 1

s2
s2 + s + 1

s2 + s + 1
s 2 ( s 2 + s + 1)

M (s) =
SGT:

H
(1 + GH )

a M < 1
a <

1
M

a < H 1 + G

18

s + 10
1
+ 2
20 s + 20 s

a <

<

a <

s 2 ( s + 10) + 20 s + 20
s 2 (20 s + 20)

s 3 + 10 s 2 + 20 s + 20
s 2 (20 s + 20)

19

Basic Bode Magnitude Plots

G ( s) =

1 + s0

A o

-20dB/dec
o

G ( s ) = A(1 + 0 )
s

+20dB/dec

QA
G ( s) =

1 + Q1 ( s0 ) + ( s0 ) 2

}Q
A( o ) 2
-40dB/dec o

+40dB/dec

G(s) = A 1+ Q1 ( s0 ) + ( s0 ) 2

A( o ) 2

A
Q{

A
Q

If Q < 12 then roots are real. Factor the expression and use the resulting product of two first order
transfer functions to find magnitude response.

20

Example

G (s) =

(1 + s ) 1 + Q1 ( s ) + ( s ) 2
0

0 < 1

A
1+ so

A
A 1o

A o
o

Q
1
1+

1 s
Q 1

}Q

( 1 ) 2
o

()
s

Q 10

A 10

A 03 1

21

Example
The M structure of a system has been determined to be given by

A
1 + Q1 ( sa ) + ( sa ) 2

M =

AM (1 + s2 )
(1 + s1 )(1 + s3 )

where 1 << 2 << 3 and a = 2 3

Determine the conditions under which robust stability is assured.
Answer
By SGT we require M < 1 or

<

1
M

1
s
s
1 AM (1 + )(1 + )
=
(1 + s )
M
1

AM1 12

1
M 1

A
1
M

o 2

AM1

3
QA

From the above diagram we can see that we require

A < AM1

and

QA < AM1

2
1

or
A < AM1

2
Q1

22

23