Lecture 3

2.
153 Adaptive Control

Lecture 3
Simple Adaptive Systems: Control & Stability
Daniel Wiese
dpwiese@mit.edu
Wednesday 18-Feb-2015
(dpwiese@mit.edu) Wednesday 18-Feb-2015 1 / 41

Introduction
Last time:
Identification of multiple parameters in first order plant
Error model 1 and 3
Determining update law using Lyapunov functions
Adaptive control
Today:
Quick overview of error models 1 and 3
Finish adaptive control of a first-order plant
Using tuning gain in adaptive control
Stability

Identification of a Vector Parameter - Error Model 1
Error Model 1: e = θ̃> u
θ:
e parameter error
˙
Adaptive law: θe = −eu
Lyapunov function: e = 1 θe> θe
V (θ) 2
˙
V̇ = θe> θe
= −θe> eu
= −e2 ≤ 0
⇒ θ(t)
e is bounded for all t ≥ t0
Error Model 3:
Error Model 3: ė = −ae + θe> u
θ:
e parameter error
˙
Adaptive law: θe = −eu

Lyapunov function: e = 1 e2 + θe> θe
V (e, θ) 2
˙
V̇ = eė + θe> θe
˙
= −ae2 + eθe> u + θe> θe
˙
= −ae2 + θe> (eu + θ)
e
= −ae2 ≤ 0
⇒ e(t) and θ(t)
e are bounded for all t ≥ t0
Stability using Error Models
So far our stability approach has been

pick an “energy-like” function V
take its time derivative
choose a suitable update law that ensured V̇ ≤ 0
What does this actually tell us about what happens?

This we will talk more about today
Also, what happened to error model 2?

Similar to error model 3, but arises in situations where the entire
system state is accessible
We will see error model 4 later in the course
But first, we will revisit adaptive control of our first-order plant.

Model Reference Adaptive Control
This model-reference approach is represented by the following block

diagram
Goal: Choose u so that e(t) → 0 as t → ∞.
ap unknown
kp unknown, but with known sign
Certainty Equivalence Principle
Step 1: Algebraic Part: Find a solution to the problem when parameters

are known.
Step 2: Analytic Part: Replace the unknown parameters by their estimates.
Ensure stable and convergent behavior.
The use of the parameter estimates in place of the true parameters is
known as the certainty equivalence principle.

Certainty Equivalence Principle- Step 1
Step 1: Algebraic Part: Propose the control law

u(t) = θc xp + kc r
and choose θc , kc so that closed-loop transfer function matches the
reference model transfer function.
ẋp = ap xp + kp u
= ap xp + kp (θc xp + kc r)
= (ap + kp θc )xp + kp kc r
Now compare this to the reference model equation
ẋm = am xm + km r
Desired Parameters: θc = θ∗ and kc = k ∗ must satisfy the matching
condition
ap + kp θ∗ = am and kp k ∗ = km
Certainty Equivalence Principle- Step 1
Solve for the nominal or ideal parameters

am − ap km
θ∗ = and k∗ =
kp kp
This is represented with the following block diagram

Certainty Equivalence Principle - Step 2
Step 2: Analytic Part: Replace the unknown parameters by their estimates.

Ensure stable and convergent behavior. From Step 1, we have
am − ap km
u(t) = θ∗ xp + k ∗ r, θ∗ = , k∗ =
kp kp
Replace θ∗ and k ∗ by their estimates θ(t) and k(t) and determine stable
update laws.

Adaptive control input:
u(t) = θ(t)xp + k(t)r
Define the parameter errors as
e = θ(t) − θ∗
θ(t)
k(t) = k(t) − k ∗
e
e + θ∗
Plug the control law into the plant equation (note that : θ(t) = θ(t)
k(t) + k ∗ ):
and k(t) = e
ẋp = ap xp + kp u(t)

= ap xp + kp θ(t)xp + k(t)r
∗ ∗

= ap xp + kp θ(t)x
e p + θ xp + k(t)r + k r
e
= ap + kp θ∗ xp + kp θ(t)x ∗

p + kp k r + kp k(t)r
e e
= am xp + kp θ(t)x
e p + km r + kp k̃(t)r

Plant with control law substituted in, and after some algebra:
ẋp = am xp + kp θ(t)x
Reference Model:
ẋm = am xm + km r
Define the tracking error as
e = xp − xm
Error model 3:
ė = am e + kp θ(t)x
e p + kp k̃(t)r
>
= am e + k p e
θ (t)ω

Plant:
ẋp = am xp + kp θ(t)x
Reference Model:
ẋm = am xm + km r
Define the tracking error as
e = xp − xm
Error model 3:
e p + kp k̃(t)r
>
 
xp
θ (t)ω
= am e + k p e ω =  
r
 
θ(t)
e
θ
e =  
k̃(t)

This is where we left off last time.
u(t) = θ(t)xp + k(t)r

>
ė = am e + kpe
θ (t)ω
Let’s finish proving stability

Note one difference versus when we last saw error model 3: the
presence of kp in the error model!
Propose a slightly modified candidate Lyapunov function

1 2 >
V = e + |kp |θ θ
e e
2

>
error model: ė = am e + kpe
θ (t)ω

1 2 >
Lyapunov function: V = e + |kp |θ θ
e e
2
Take the time derivative of V
>˙
V̇ = eė + e
θ e
θ
>˙
= am e2 + kp eθe> ω + |kp |e
θ eθ
˙
= am e2 + θe> (kp eω + |kp |e
θ)
˙
Propose the update law: θe = −sign(kp )eω
V̇ = am e2 ≤ 0
⇒ e(t) and θ(t)
e are bounded for all t ≥ t0
Adaptive Gain γ
We have reduced the adaptive control problem to error model 3, and

proved stability using a Lyapunov function
Now that we have shown stability, what about performance?
To provide an extra degree of freedom to tune the closed-loop

performance, we add an additional term, an adaptive gain γ
We will look at performance quantitively later on - for now we will

just look at it qualitatively

Adaptive Gain γ
Same plant, reference model, and control las as before give the same error
model:
e p + kp k̃(t)r
Now introduce a gain γ in the update law
θ̇(t) = −γsign(kp )exp
k̇(t) = −γsign(kp )er
Choose the following candidate Lyapunov function and differentiate

1 2 |kp | e>e
V = e + θ θ
2 γ
>˙
V̇ = eė + e θ e
θ
|kp | e˙

>
2
= am e + θ kp eφ +
e θ
γ
= am e2 ≤ 0
Adaptive Gain Example
Simulation Parameters: am = −1, km = 1, ap = 1, kp = 2
γ =1 γ =10 γ =100
3 x 3 x 3 x
xpm xpm xpm
2.5 2.5 2.5
2 2 2
State
State
State
1.5 1.5 1.5
1 1 1
0.5 0.5 0.5
0 0 0
0 10 20 30 0 10 20 30 0 10 20 30
time [s] time [s] time [s]
1 1 1
θ θ θ
k k k
0 0 0
Parameter
Parameter
Parameter
−1 −1 −1
−2 −2 −2
−3 −3 −3
0 10 20 30 0 10 20 30 0 10 20 30
time [s] time [s] time [s]

Adaptive Gain Example
These simulations show (qualitatively) the effect that γ has on the

performance of the system
The rate at which the parameters is adjusted increases
The state follows the reference model more closely
This results in increased control rates and fast oscillations
There are ways to reduce the oscillations, which we will see later in the
course.
We will look at this from a quantitative view in coming lectures as well
For now, we will go through stability in a little more depth.

Equilibrium Points
Consider the following dynamical system
ẋ(t) = f (x(t), t)
(1)
x(t0 ) = x0
Definition: equilibrium point (pg 45) The state xeq is an equilibrium

point of (1) if it satisfies:
f (xeq , t) = 0 (2)
for all t ≥ t0 .
Definition: autonomous (pg 45) If the RHS of (1) does not depend
on t, the equation is called autonomous.

Equilibrium Points
Definition: isolated equilibrium (pg 45) If xeq is the only constant

solution in the neighborhood of xeq , it is called an isolated equilibrium.
Both linear and nonlinear systems can have multiple equilibrium points
Linear systems will have a single equilibrium point, or an infinity of
non-isolated equilibrium points e.g. mass

ẋ 0 1 x 0
= + 1 f
ẍ 0 0 ẋ m
Nonlinear systems can have an infinity of isolated equilibrium points

e.g. pendulum
Note: when we talk about stability, we talk about the stability of a
particular equilibrium point.

Stability Definitions
Definition 2.1: stable (pg 51) The equilibrium state xeq of (1) is said
to be stable if for every > 0 and t0 ≥ 0, there exists a δ(, t0 ) > 0 such
that kx0 k < δ implies that kx(t; x0 , t0 )k < ∀t ≥ t0 .
Definition 2.2: attractive (pg 51) The equilibrium state xeq of (1) is
said to be attractive if for some ρ > 0 and every η > 0 and t0 > 0, there
exists a number T (η, x0 , t0 ) such that kx0 k < ρ implies that
kx(t; x0 , t0 )k < η ∀t ≥ t0 + T .
Attractivity: all trajectories starting in a neighborhood of xeq

eventually approach xeq .
Attractivity and stability are independent concepts
Definition 2.3: asymptotically stable (pg 51) The equilibrium state
xeq of (1) is said to be asymptotically stable if it is both stable and
attractive.
Stability Definitions (Continued)
Definition 2.4: uniformly stable (pg 52) The equilibrium state xeq of
(1) is said to be uniformly stable if in Definition 2.1 δ is independent of
initial time.
Definition: globally stable (pg 52) If lim→∞ δ() = ∞ in the

definitions above, then the equilibrium stat is said to be globally stable.
See page 52 for more stability definitions.

Lyapunov’s Stability Methods
A motivating example: determine the stability of the origin for the

following scalar system
ẋ(t) = ax(t)
Can determine the stability of the origin by solving the differential
equation:
x(t) = eat
The origin is globally exponentially stable.
What about the Van der Pol oscillator?
ẍ = −x + (x2 − 1)ẋ
which has an equilibrium point at the origin.
Lyapunov’s methods allow us to determine the stability of an equilibrium

for such a system without solving the differential equation!
Lyapunov’s Indirect (1st ) Method
Also called Lyapunov’s linearization method

Concerned with local stability of an equilibrium point of a nonlinear
system
Idea: all physical systems are nonlinear, but a linearized
approximation is valid in a neighborhood of an equilibrium
Formally:
ẋ = f (x)
In general, the nonlinear system can be non-autonomous, and the

resulting linearized system will be time varying.
Define perturbations about an equilibrium point xeq as
x = xeq + δx
Differentiating
˙
ẋ = ẋeq + δx
Lyapunov’s Indirect (1st ) Method (Continued)
Recalling the definition of an equilibrium point, we obtain

˙ = f (x) = f (xeq + δx)
δx
Now perform a Taylor Series expansion of f (xeq + δx)

∂f
f (xeq + δx) = f (xeq ) + δx + ε(x)
∂x eq
since f (xeq ) = 0 we have

˙ = ∂f
δx δx + ε(x)
∂x eq
which we write, after neglecting the higher order terms, as

˙ = Aδx
δx
∂f
We evaluate A = ∂x as
 ∂f1 ∂f1   
∂x1 ... ∂x2 f1
∂f
=  ... ..  where f =  ... 
  
∂x . 
∂f n ∂fn fn
∂x1 ... ∂xn
This is called the Jacobian matrix of f .
Theorem: Lyapunov’s linearization method When a nonlinear system

f is linearized about an equilibrium point xeq with Jacobian matrix A:
If A is strictly stable, then xeq is asymptotically stable for f
If A is unstable, then xeq is unstable for f
If A is marginally stable, then we can draw no conclusion about
stability of xeq for f

Return to the Van der Pol oscillator:
Ẍ = −X + (X 2 − 1)Ẋ
write as
Ẋ1 X2
=
Ẋ2 −X1 + (X12 − 1)X2
where X1 = X2 = 0 is an equilibrium point. The nonlinear system is
represented as
Ẋ = f (X)
We want to linearize about the origin to get
∂f
ẋ = x = Ax
∂x eq

With
f1 = X2
f2 = −X1 + (X12 − 1)X2
We linearize by taking partial derivatives and obtain

ẋ1 0 1 x1
= 2

ẋ2 −1 + 2X1 X2 X1 − 1 eq x2
Evaluating the Jacobian at the equilibrium point we obtain

ẋ1 0 1 x1
=
ẋ2 −1 −1 x2
which is stable.
Using Lyapunov’s Indirect Method, we can conclude that the origin of the
nonlinear system Ẍ = −X + (X 2 − 1)Ẋ is stable.
Lyapunov’s Direct (2nd ) Method
Lyapunov’s first method does not work if the linearization is

marginally stable.
Need a new tool: Lyapunov’s second, or direct method
Again, we will have another tool which gives us information about the
stability of an equilibrium point for a nonlinear system, without
solving the differential equation
Lyapunov’s direct method is motivated by the observation that as

physical systems dissipate energy, they eventually settle to an
equilibrium point.
Idea: determine a scalar, positive definite energy-like function and

look at how this quantity changes with time.

Lyapunov’s Direct (2nd ) Method (Continued)
For the system

ẋ = f (x)
Let
(i) V (x) > 0, ∀x 6= 0, and V (0) = 0
T
(ii) V̇ (x) = ∂V
∂x f (x) < 0
(iii) V (x) → ∞ as kxk → ∞
Then x = 0 is asymptotically stable. If instead of (ii), we have

(ii-b) V̇ ≤ 0
Then x = 0 is stable.
Now that we have learned more about stability, we know what this
means
But what more can we say?
Asymptotic Convergence of e(t) to Zero
Stability is an important and necessary first step, but recall our

control goal of ensuring e(t) → 0 as t → ∞
In order to prove that this will in fact be the case, we require some
additional tools
We will soon introduce Barbalat’s Lemma and a corollary
To understand these new tools, we first introduce some notation
regarding the norms of signals

Norms
Given two real numbers, the notion of the “size” of these numbers is
apparent
However, given quantity such as a vector, matrix, or time-varying
signal, we may by interested in how “big” they are when compared to
another vector, matrix, or time-varying signal respectively
A norm is a non-negative measure of the magnitude of a given
quantity that satisfy three basic properties
We will not go into the details of norms in lecture, but will post a
short handout online

Signal Norms
Given a signal (either vector-valued or scalar), we wish to determine

some measure of its magnitude
Perhaps the signal is very large at one instance of time, and small
everywhere else, or maybe it is moderately large for all time
Whatever the case may be, we wish to have some way to quantify
these varying degrees of the “largeness” of a signal
Often signal norms are denoted using the capital letter Lp
For the vector valued signal x(t) the following norms are given

Signal Norms
Lp Norm The general definition of the Lp norm is the following, where

p ∈ N+ .
Z t p1
p
kx(t)kLp = kx(τ )k dτ
0
L1 Norm Z t
kx(t)kL1 = kx(τ )kdτ
0
L2 Norm sZ
t
kx(t)kL2 = kx(τ )k2 dτ
0
L∞ Norm
kx(t)kL∞ = sup kx(t)k
t
In this class we will mostly be using the L2 and L∞ norms.

Existence of Signals in Normed Spaces
In proving stability for adaptive systems, we are initially concerned

with showing boundedness of signals, such as the error e(t)
If this error signal is bounded for all time, we write this formally as
e ∈ L∞
where this implies the L∞ norm of e(t) exists and is finite

Similarly, if the L2 norm of a signal e(t) exists and is finite, we write
e ∈ L2
Using this notation, we will now present Barbalat’s Lemma

Barbalat’s Lemma
Lemma 2.12 (Barbalat’s) page 85

(i) If f : R+ → R is uniformly continuous for t ≥ 0
Rt
(ii) And if limt→∞ 0 |f (τ )|dτ exists and is finite
Then limt→∞ f (t) = 0
What does this say about the function f ?
Corollary If g ∈ L2 ∩ L∞ , and ġ is bounded, then limt→∞ g(t) = 0.
Notice:
Replace g with e in the corollary, and that is what we need to prove!
So we need to show e ∈ L2 ∩ L∞ for the adaptive systems we have
seen so far

Using Barbalat’s Lemma
Recall the error model 3 stability proof we did on slide 17

We left off with V̇ ≤ 0 ⇒ e ∈ L∞ and θe ∈ L∞
For all bounded inputs r, xm will be bounded, as it is the output of
our stable reference model
Since xm ∈ L∞ and e ∈ L∞ ⇒ xp ∈ L∞
Now to show e ∈ L2

Note that Z t
V̇ (τ )dτ = V (t) − V (0)
0
Since V is non increasing and positive definite, V (0) − V (t) ≤ V (0). This
gives Z t
− V̇ (τ )dτ ≤ V (0)
0
Substituting in our expression for V̇ = am e2 , remembering that am < 0

Z t
|am | e2 (τ )dτ ≤ V (0)
0
which is equivalent to
Z t
|am | ke(τ )k2 dτ ≤ V (0) < ∞
0

Z t
|am | ke(τ )k2 dτ ≤ V (0) < ∞
0
simplifies to sZ
t
ke(τ )k2 dτ < ∞
0
Recognize that this is just ke(t)kL2 < ∞ we write e ∈ L2
Finally, we need to show the boundedness of ė so we can apply Barbalat’s

lemma
>
ė = am e + kpe
θ (t)ω
Everything on the right side is bounded ⇒ ė is bounded
Thus, the conditions to apply Barbalat’s lemma (corollary) are met, so we

conclude limt→∞ e(t) = 0
Brief Summary
Today we:
Reviewed and finished stability for the first-order adaptive control
problem from last lecture
Introduced the adaptive gain γ and showed qualitatively some of its
effects on the performance of the system
Covered lots of different stability definitions
Went over Lyapunov’s first and second methods for proving stability
for nonlinear systems
Stated what signal norms were, and discussed the existence of signals
in normed spaces
Gave Barbalat’s Lemma and showed how to use it to show e → 0

Lecture 3

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Lecture 3

Încărcat de

Drepturi de autor:

Formate disponibile

2.

153 Adaptive Control

(dpwiese@mit.edu) Wednesday 18-Feb-2015 1 / 41

(dpwiese@mit.edu) Wednesday 18-Feb-2015 2 / 41

Error Model 3: ė = −ae + θe> u

So far our stability approach has been

What does this actually tell us about what happens?

Also, what happened to error model 2?

But first, we will revisit adaptive control of our first-order plant.

(dpwiese@mit.edu) Wednesday 18-Feb-2015 5 / 41

This model-reference approach is represented by the following block

Goal: Choose u so that e(t) → 0 as t → ∞.

Step 1: Algebraic Part: Find a solution to the problem when parameters

(dpwiese@mit.edu) Wednesday 18-Feb-2015 7 / 41

Step 1: Algebraic Part: Propose the control law

Solve for the nominal or ideal parameters

This is represented with the following block diagram

(dpwiese@mit.edu) Wednesday 18-Feb-2015 9 / 41

Step 2: Analytic Part: Replace the unknown parameters by their estimates.

(dpwiese@mit.edu) Wednesday 18-Feb-2015 10 / 41

(dpwiese@mit.edu) Wednesday 18-Feb-2015 11 / 41

(dpwiese@mit.edu) Wednesday 18-Feb-2015 12 / 41

(dpwiese@mit.edu) Wednesday 18-Feb-2015 13 / 41

This is where we left off last time.

u(t) = θ(t)xp + k(t)r

Let’s finish proving stability

(dpwiese@mit.edu) Wednesday 18-Feb-2015 14 / 41

We have reduced the adaptive control problem to error model 3, and

Now that we have shown stability, what about performance?

To provide an extra degree of freedom to tune the closed-loop

We will look at performance quantitively later on - for now we will

(dpwiese@mit.edu) Wednesday 18-Feb-2015 16 / 41

0.5 0.5 0.5

(dpwiese@mit.edu) Wednesday 18-Feb-2015 18 / 41

These simulations show (qualitatively) the effect that γ has on the

(dpwiese@mit.edu) Wednesday 18-Feb-2015 19 / 41

Consider the following dynamical system

Definition: equilibrium point (pg 45) The state xeq is an equilibrium

(dpwiese@mit.edu) Wednesday 18-Feb-2015 20 / 41

Definition: isolated equilibrium (pg 45) If xeq is the only constant

Nonlinear systems can have an infinity of isolated equilibrium points

(dpwiese@mit.edu) Wednesday 18-Feb-2015 21 / 41

Attractivity: all trajectories starting in a neighborhood of xeq

Definition: globally stable (pg 52) If lim→∞ δ() = ∞ in the

See page 52 for more stability definitions.

(dpwiese@mit.edu) Wednesday 18-Feb-2015 23 / 41

A motivating example: determine the stability of the origin for the

which has an equilibrium point at the origin.

Lyapunov’s methods allow us to determine the stability of an equilibrium

Also called Lyapunov’s linearization method

In general, the nonlinear system can be non-autonomous, and the

Recalling the definition of an equilibrium point, we obtain

Now perform a Taylor Series expansion of f (xeq + δx)

since f (xeq ) = 0 we have

which we write, after neglecting the higher order terms, as

This is called the Jacobian matrix of f .

Theorem: Lyapunov’s linearization method When a nonlinear system

(dpwiese@mit.edu) Wednesday 18-Feb-2015 27 / 41

Return to the Van der Pol oscillator:

(dpwiese@mit.edu) Wednesday 18-Feb-2015 28 / 41

We linearize by taking partial derivatives and obtain

Evaluating the Jacobian at the equilibrium point we obtain

Lyapunov’s first method does not work if the linearization is

Need a new tool: Lyapunov’s second, or direct method

Definition: globally stable (pg 52) If lim→∞ δ() = ∞ in the