Propagation of Uncertainty in

Dynamic Systems!
Robert Stengel!
Optimal Control and Estimation MAE 546 !
Princeton University, 2015"
!! Propagation of the mean and variance in linear, time-
varying discrete-time systems"
!! Markov processes and the transition function property"
!! White and colored noise inputs"
!! Sampled-data representation of continuous-time systems"
!! Propagation of the mean and variance in continuous-time
systems"

Copyright 2015 by Robert Stengel. All rights reserved. For educational use only.!
http://www.princeton.edu/~stengel/MAE546.html!
http://www.princeton.edu/~stengel/OptConEst.html! 1!

Uncertain Linear, Time-Varying

(LTV) Dynamic Model"
•! Discrete-time LTV model with known coefficients"

x k = !k "1x k "1 + # k "1u k "1 + $ k "1w k "1

•! Initial condition and disturbance inputs are not
known precisely"
•! All random variables are Gaussian, i.e., they are
fully described by means and covariances"

2!
 Disturbance Input of the
LTV Dynamic Model"
x k = !k "1x k "1 + # k "1u k "1 + $ k "1w k "1

Disturbance input (


process noise) is a white-noise
sequence"

E (wk ) = 0
( ) ( )
E w k w Tk = Q'k ; E w k w Tk !l = 0, l = any non-zero integer
or #% 1, j = k
! jk ! Kronecker delta function = $
( )
E w j w Tk = Q'k " jk &%
0, j " k

3!

Initial Condition and Control of

the LTV Dynamic Model"
Initial condition described by mean and covariance"

E ( x 0 ) = x0 ! mo ; E "( x 0 ! mo ) ( x 0 ! mo ) $ = P0
T
# %
dim ( x o ) = n ! 1

Control input is known precisely"

E [ u k ] = u k = u k ; E "( u k ! u k ) ( u k ! u k ) $ = U k = 0
T
# %
dim ( u k ) = m ! 1

Cross-covariances are zero"

E "#( x k ! x k ) w k T $% = M k = 0
E "#( x k ! x k ) u k T $% = 0
E "# w k u k T $% = 0 4!
 Probability Density Function of
the LTV Dynamic Model"
Initial probability density function depends
only on the mean and covariance of the
Gaussian distribution"

1 "
1
( x0 " m0 )T P0"1 ( x0 " m0 )
pr(x 0 ) = n /2 1/2 e 2

( 2! ) P0

5!

Expected Value of the State"

First moment of x"

E ( x k ) = E ( !k"1x k"1 + # k"1u k"1 + $ k"1w k"1 )

x k = !k"1x k"1 + # k"1u k"1 + $ k"1 ( 0 )
m k = !k"1m k"1 + # k"1u k"1

6!
 Expected Value of the
State Covariance"
Second central moment of x"
E "( x k ! m k ) ( x k ! m k ) $ ! Pk
T
# %

{
= E "# &k!1 ( x k!1 ! m k!1 ) + ' k!1 ( u k!1 ! u k!1 ) + ( k!1w k!1 $% "# &k!1 ( x k!1 ! m k!1 ) + ' k!1 ( u k!1 ! u k!1 ) + ( k!1w k!1 $%
T
}
With negligible cross-covariance"

Pk = !k"1E #( x k"1 " m k"1 ) ( x k"1 " m k"1 ) % !k"1T + ' k"1E ( w k"1w T k"1 ) ' k"1T
T
$ &
= !k"1Pk"1!k"1 + ' k"1Q'k"1 'Tk"1
T

! !k"1Pk"1!k"1T + Qk"1

Qk !1 ! " k !1Q'k !1 "Tk !1

7!

Conditional Probability
Density Function of the
State " 1 ( ) "
1
x k " mk
T
Pk"1 ( x k " m k )
pr(x k ) = 1/2 e
2

( 2! )n /2 Pk

m k = !k"1m k"1 + # k"1u k"1

Pk = !k"1Pk"1!k"1T + Qk"1

The density function is conditioned on the prior state"

pr(x k x k!1 ) =
1 1
(T
)
!1
! %& x k !( #k!1m k!1 +$
$ k!1u k!1 ) '( #k!1Pk!1#k!1T +Qk!1 %& x k !( #k!1m k!1 +$
$ k!1u k!1 ) '(
1/2 e
2

( 2" )n/2 #k!1Pk!1#k!1T + Qk!1

... and propagation is a Markov process"

pr(x k ) = pr(x k x k !1 )pr(x k !1 ) = pr(x k x k !1 ) if pr(x k !1 ) = 1 8!
 Gauss-Markov Sequence"
m k = !k "1m k "1 + # k "1u k "1
Pk = !k "1Pk "1!kT"1+ Qk "1
•! The mean and covariance are completely
specified by the prior probability distribution"
•! The random sequence"
–! has the transition function property"
–! is a Gauss-Markov sequence"

pr(x k ) = pr(x k x k!1 ) pr(x k!1 ) = pr(x k x k!1 ) pr(x k!1 x k!2 ) pr(x k!2 ) = !
# k &
= % " pr(x i x i!1 ) ( pr(x 0 )
$ i=1 '
9!

Probability Mass and

Density Functions"
The random sequence"
has the transition function property"
is a Gauss-Markov sequence"
Probability Mass Function!
Pr(!x k ) = Pr(!x k !x k"1 ) Pr(!x k"1 )
$ k '
= Pr(!x k !x k"1 ) Pr(!x k"1 !x k"2 ) Pr(!x k"2 ) = & # Pr(!x i !x i"1 ) ) Pr(!x 0 )
% i=1 (
Pr(!x k ) " pr(x k )!x
Pr(!x k !x k#1 )Pr(!x k#1 ) " pr(!x k !x k#1 )pr(x k#1 )!x
Probability Density Function!

lim Pr(!x k ) = lim Pr(!x k !x k#1 ) Pr(!x k#1 )

!x"0 !x"0

pr(x k ) = pr(x k x k#1 ) pr(x k#1 )

10!
 Sampled-Data Representation
of Continuous-Time Systems!

11!

Sampled-Data Representation of
Continuous-Time Systems"
Continuous-time LTV model with known coefficients"

x! (t) = F(t)x(t) + G(t)u(t) + L(t)w(t), x(t o ) given

t

x(t) = x(t o ) + " [ F(! )x(! ) + G(! )u(! ) + L(! )w(! )] d!

to

Incremental solution"
tk

x(t k ) = x(t k!1 ) + # [ Fx(" ) + Gu(" ) + Lw(" )] d"

t k!1
tk

= $ ( t k ,t k!1 ) x(t k!1 ) + # $ (t ," ) %&G (" ) u(" ) + L (" ) w(" )'( d"
k
t k!1 12!
 Descriptions of
Random Variables"
E ( x 0 ) = mo ; E "( x 0 ! mo ) ( x 0 ! mo ) $ = P0
T
# %

E !" w ( t ) #$ = 0
E !" w ( t ) w T (% ) #$ = Q'& ( t ' % )

{
E !" u ( t ) #$ = u ( t ); E !" u ( t ) % u ( t ) #$ !" u ( t ) % u ( t ) #$
T
}=0
13!

Mean Value and

Covariance Solutions"
Mean value propagation from tk–1 to tk"
tk

m k = !k m k "1 + $ ! (t ,# ) G (# ) u(# )d#

t k"1
k

Covariance propagation"
Double integration over time "(! and " )
T
, % tk ( % tk ( 0.
.
Pk = !k"1Pk"1!k"1T + E - ' $ ! ( t k , # ) L (# ) w (# ) d# * ' $ ! ( t k , + ) L (+ ) w (+ ) d+ * 1
./ '& tk"1 *) '& tk"1 *) .2
tk tk

= !k"1Pk"1! $ $ ! (t ,# ) L (# ) E %& w (# ) w (+ )() L (+ ) ! (t ,+ ) d# d+

T T T T
k"1 + k k
t k"1 t k"1
14!
 Covariance Propagation"

E #$ w (! ) w T (" ) %& = Q'C ' (! ( " )

tk tk

Pk = !k"1Pk"1! & & ! (t ,# ) L (# ) Q' $ (# " % ) L (% ) ! (t ,% ) d# d%

T T T
k"1 + k C k
t k"1 t k"1
tk

= !k"1Pk"1! & ! (t ,# ) L (# ) Q' LT (# ) ! T ( t k , # ) d#

T
k"1 + k C
t k"1

! !k"1Pk"1!k"1T + Qk"1

15!

Relationship Between Discrete- and

Continuous-Time Disturbance Models"
tk

Qk!1 = $ " ( t k , # ) L (# ) Q'C LT (# ) " T ( t k , # ) d#

t k!1

Qk!1 : Covariance matrix ( n " n )

Q'C : Spectral density matrix (s " s)
For small !t

Qk !1 " L ( t k !1 ) Q'C ( t k !1 ) LT ( t k !1 ) #t 16!

 Autocovariance Functions for
Continuous-and Discrete-Time
White Noise"
Dirac Delta ! Kronecker Delta !
Function! Function!

!ww ( "t ) = Q'C # ( "t ) !ww ( "t k ) = Qk #1$ ( "t k )

%' #, "t = 0
! ( "t ) = & $& 1,
'( 0, "t $ 0
j=k
! jk = ! ( "t k ) =%
&' 0, j#k
# "t1

* ! (t ) dt = lim
"t1 +0 * ! ( "t ) d ( "t ) = 1
)# ) "t1 17!

Colored Noise, Variance,

Spectral Density Matrices"
•! Spreading of the autocovariance
function is accompanied by"
&
–! Finite variance " ! xx (" )
–! Low-pass filtering of the power
! xx (" ) = ' # ($ ) e
xx
% j"$
d$
spectral density" %&

Autocovariance Function! Power Spectral Density Function!

18!
 Spectral Density Matrices"
•! Dynamic systems
subject all white-noise
inputs to low-pass
filtering"
•! Nyquist (or folding)
frequency"
1 "
! Nyquist = ! Sampling =
2 T
T = Sampling interval, sec

•! Signals above this

frequency fold and
corrupt lower-
frequency sampled
signals (aliasing)" Caveat: Covariance and Spectra
capture stochastic equilibrium 
effects but not transient effects"
19!

Frequency
Folding
(Aliasing)"

20!
Quantization, Zero-Order Hold,
and Inter-sample Ripple"
•!Digital control subject to zero-
order hold in D/A" Zero-order hold!

•!Sampled signal misses inter-

sample transient response"
•!Effective delay of sampled signal"

Analog-to-Digital Converter!
•!Continuous signal sampled
with finite precision in A/D"

21!

Next Time:!
Kalman Filter for
Discrete-Time Systems!

22!
 Supplemental
Material!

23!

Colored Noise Disturbance"

•! Disturbance may not be white
 noise"
–! Power may vary with frequency"
•! Mean"
•! Correlation with
E (wk ) = 0 adjacent signal"
dim ( w k ) = s ! 1
•! Covariance"
( )
E w k w Tk !1 = Vk
(
E wkw T
k )=W k

•! Linear model for disturbance propagation"

–! Driven by white noise sequence"

E ( !k ) = 0
w k = A k !1w k !1 + "k !1
( )
E ! j !Tk = Q!k " jk
24!
 Calculation of Disturbance
Process Parameters"
Autocovariance of scalar Markov process"

! xx (1) = E ( xi xi +1 ) = bE ( xi2 ) = b" x2

! xx ( k ) = b k " x2 = b k " u2

Autocovariance of vector Markov process"

( )
E w k w Tk !1 = E #$( A k !1w k !1 + "k !1 ) w Tk !1 %&
(
= A k !1E w k !1w Tk !1 )
= A k !1Wk !1
! Vk

Therefore, the state transition matrix of the noise model is"

A k !1 = Vk Wk!1!1
25!

Propagating the Disturbance"

Wk = A k !1Wk !1A Tk !1 + Q"k

•! System state vector is ! m $

augmented to include the x ' ! ## w & , dim ( x 'k ) = ( n + s ) ' 1
k
k
" &%
disturbance"
k

•! Augmented equation for the mean"

! mk $ ! 'k (1 ) k (1 $ ! m k (1 $ ! * $ ! 0 $
# &=# &# & + # k (1 & u k (1 + # & +k (1
#" w k &% #" 0 A k (1 &% #" w k (1 &% #" 0 &% #" I s &%

•! Augmented covariance equation"

T
! Pk 0 $ ! 'k (1 ) k (1 $ ! Pk (1 0 $ ! 'k (1 ) k (1 $ ! 0 $
# &=# &# &# & +# & Q ! 0 I s $%
#" 0 Wk & # 0 A k (1 & # 0 Wk (1 &% #" 0 A k (1 & # I s & *k(1 "
% " %" % " %
26!