Documente Academic
Documente Profesional
Documente Cultură
41
Transfer-Function Identification
4.1: Introduction to transfer functions
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
42
kD0
q "1
yk! D
uk!
D
uk!.
.1 " q "1/
.q " /
G.q/
1
2
Note the different summation starting indices, due to assumption that G.q/ has no
strictly feedthrough path (i.e., g0! D 0). If g0! 0, then both indices start at k D 0.
The second line of this equation does not really follow from the first, since q "1 is an
operator and not a variable. However, we can use a similar approach with transforms
to rigorously come up with the same result, if we permit rational-polynomial functions
of q. This is really an abuse of notation, but when everybody understands what youre
talking about, I suppose its okay.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
43
! This is much more compact than trying to find the entire unit-pulse
or frequency responses.
function;
transfer function.
fast or slow.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
44
Unit step:
gk! D 1k!.
q
G.q/ D
; jqj > 1.
q"1
Exponential (geometric):
"4"2 0 2 4 6 8
"4"2 0 2 4 6 8
"4"2 0 2 4 6 8
3. jaj D 1;
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
45
4. jaj < 1;
0.8
0.6
0.4
$ duration N 43
21
N D
2"
360
D
:
! rad
! deg
N D3
N D8
N D 10
N D 20
N D2
I.q/
Plot to right
shows
discrete-time
unit-pulse
responses
versus pole
locations.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
R.q/
46
Then,
9
a D 0:3567=T =
"=4
;
b D
T
T D sampling period:
$
"k
gk! D g.kT / D e
cos
1k!
4
# $
"k
1k!:
D 0:7k cos
4
!
"
"0:3567 k
s1 T
and e s2T .
EXAMPLE :
"
T
"j
s-plane
"
T
q-plane
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
47
Good
Damping #
Good
Frequency !n
Good
Settling time
Higher-order systems:
! Pole moving toward q D 1, system slows down.
! Zero moving toward q D 1, overshoot.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
48
w-plane
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
49
and
If true,
1. Take H.q/ H.w/ via the bilinear transform.
CHECK :
q D re j!T
2 q " 1
2 re j!T " 1
:
wD
D
T q C 1 qDrej!T
T re j!T C 1
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
410
the real part of w is positive when jrj > 1 (w in RHP). Therefore, the
bilinear transformation does exactly what we want.
When r D 1;
#
$
!T
2 2 sin.!T /
2
wDj
D j tan
.
T 2 C 2 cos.!T /
T
2
s-plane
!s
j
2
!s
j
4
!s
"j
4
!s
"j
2
q-plane
w-plane
RD1
RD1
2
j
T
"
2
T
PROCEDURE:
EXAMPLE :
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
411
Let G.q/ D
(1,2)
0:368q C 0:264
:
q 2 " 1:368q C 0:368
'
(
0:368 1C0:5w
C 0:264
1"0:5w
G.w/ D '
' 1C0:5w (
(
1C0:5w 2
" 1:368 1"0:5w C 0:368
1"0:5w
D
(3)
!
" ! !w
"
" j !2w " 1 j 12:14
C1
"
! !w
:
G.j!w / D
j!w j 0:924
C1
Magnitude (dB)
(4)
Bode Plots
40
20
0
20
40 1
10
10
10
10
10
Phase (deg)
180
90
0
90
180
270
1
10
10
10
10
10
412
Do all of A.q/, B.q/, C.q/, D.q/ exist? And, what are the values
for nk , na , nb , nc , and nd ?
ISSUES :
ek!
uk!
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
B.q/
A.q/
yk!
413
B.q; $/
and H.q; $/ D 1, and
A.q; $/
B.q; $/
uk! C ek!.
A.q; $/
i
h
The OE model is parametrized by $ D
a1 # # # ana b1 # # # bnb .
yk! D
The noise source is the difference (error) between the actual and
noise-free output.
Good to use when system dominated by white sensor noise.
Expect problems when the noise spectrum is shaped (colored noise,
process noise). Why?
Denote the noise-free output by wk!. Then, the difference equation is
ek!
uk!
B.q/
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
C.q/
1
A.q/
yk!
414
C.q; $/
B.q; $/
and H.q; $/ D
, and
A.q; $/
A.q; $/
ARX model
The ARX (auto regressive with exogenous (or extra) input) approach
models the system as shown in the diagram. That is,
A.q; $/yk! D B.q; $/uk! C ek!,
B.q; $/
and
or, G.q; $/ D
A.q; $/
1
H.q; $/ D
.
A.q; $/
ek!
uk!
B.q/
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
1
A.q/
yk!
a1 # # # ana b1 # # # bnb
415
B.q; $/
C.q; $/
uk! C
ek!,
C.q/
ek!
A.q; $/
D.q; $/
D.q/
B.q; $/
or G.q; $/ D
and
A.q; $/
B.q/
uk!
yk!
C.q; $/
A.q/
H.q; $/ D
.
D.q; $/
The BoxJenkins model is parametrized by
i
h
$ D a1 # # # ana b1 # # # bnb c1 # # # cnc d1 # # # dnd .
yk! D
416
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
417
Delay estimation
load dryer2
u2 = u2 - mean(u2); y2 = y2 - mean(y2);
x = (-length(u2)+1):(length(u2)-1);
Ryu = xcorr(y2,u2); stem(x,Ryu,'filled');
Cross correlation R
yu
1000
between y and u
Cross correlation R
600
400
200
100
600
400
200
800
Correlation
Correlation
800
200
200
yu
1000
100
200
10
We see a general noise floor of around 200. So, a time delay of two
or three samples looks about right. Should probably try both.
A second method is to look at unit-pulse- or step-response data (if
available), looking for the first non-negligible response point.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
418
Coefficient order
! Slope of magnitude plot of some use, but care must be taken due
load dryer2
u2 = u2 - mean(u2);
y2 = y2 - mean(y2);
z = iddata(y2,u2,0.08);
[m,p,w]=bode(spa(z));
w=squeeze(w); m=squeeze(m);
semilogx(w,20*log10(m));
Magnitude (dB)
0
10
20
30
40
1
10
10
10
1
Slope $ "40 dB per decade: at least two more poles than zeros.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
10
419
! But, how many zeros? Phase plot is pretty useless due to delays.
! Might try a range of model orders from 2 to 5 to see which fits best.
Polynomial form
KEY POINT :
"3
1
D
,
1 " 1:5"1 C 0:7"2
3 " 1:52 C 0:7
entered as num = 1 and den = [1 -1.5 0.7 0].
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
420
A very fundamental question we must consider is: what will the final
model be used for?
! Simulation: Given knowledge only of uk! up to the present time,
and yk! up to the prior time, k " 1, estimate present output yk!.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
421
! For simulation, %s k! D yk! " yOs k!, where yOs k! D G.q/uk!, the
! For n-step prediction, %nk! D yk! " yOnk j k " n!, where
The formulations for the predicted yOp k j k " 1! and yOnk j k " n!
deserve some attention.
We start with the assumed form yk! D G.q/uk! C H.q/ek!.
Note that the RHS requires knowledge of uk! up until the present,
ek! at the present time only, and yk! up until the prior time only.
To see this last point, lets look at the details of H.q/ more closely:
C.q/
1 C c1q "1 C c2 q "2 C :::
H.q/ D
D
D.q/
1 C d1q "1 C d2q "2 C # # #
1"H
"1
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
"
422
The numerator has a pure delay term, therefore, 1 " H "1.q/ yk!
contains only old values of the output fys!; s % k " 1g.
Therefore, we can use this relationship to predict yk! from past
values of yk! (etc).
! Assume that ek! is white, so best estimate of ek!, given
EXAMPLE :
n"1
X
j D0
EXAMPLE :
B.q/
uk!, which is not a function of past outputs.
A.q/
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
423
N
X
L.%k; $!/,
kD1
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
424
! Linear least squares puts too much emphasis on fitting the poles.
Logarithmic least squares is better for the fitting of the zeros since it
weights the ratio of model gain to measurement gain.
X
j!i
b
j!i 2
VN .$/ D
i log.G.e ; $// " log.G N .e // .
!i
EXAMPLE :
Measured data: b
G.!1/ D 10 and b
G.!2/ D 0:1.
0:0001'1
0:0111
0:0111
Optimization method
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
425
D $ T &k!,
$D
&k! D
a1 a2 # # # ana b1 b2 # # # bnb
iT
iT
N
X
!
kD1
N
X
!
kD1
"2
N
X
y 2k! "2$ T
kD1
yN
N
X
&k!yk! C$ T
kD1
D yN " 2$ T fN C $ T RN $.
fN
"
N
X
&k!& T k! $
kD1
RN
Note, the term regress here alludes to the fact that we try to calculate or describe
yk! by going back to &k!. Also, models such as ARX where &k! contains old values
yk " ' ! of the variable to be explained, yk!, are then partly auto-regression models.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
426
not a function of $
"1
We get the smallest possible VN .$/ when we select $ D $ON D RN
fN .
In MATLAB, theta = X \ Y.
The prediction ARX problem with quadratic cost function is the only
system ID that can be done using linear least squares.
Other problems require nonlinear optimization, which is quite tricky.
One fundamental principle is that we know we are at a minima (or
maxima) of an objective function when dVN .$/=d$ D 0.
Comparing to the matrix form of the prior case, we get
d
VN D 2RN $ " 2fN D 0
d$
$O D R"1fN ,
N
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
427
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
428
Some references for this section include: L.E. Scales, Introduction to Non-Linear
Optimization, Springer-Verlag, 1985, and M.A. Wolfe, Numerical Methods for
Unconstrained Optimization: An Introduction, Van Nostrand Reinhold, 1978.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
429
! Others use values of F , its gradient, and its Hessian G.xO k / (the
The basic idea is that the minimum lies (more or less) in the direction
of the negative gradient of F .x/ with respect to x.
Via gradient descent, we eventually get to (at least a local) minimum.
! However, for objective functions having both steep regions and flat
! Note that the scaling G "1 is large when the curvature is small (can
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
430
Can use fixed value for k D , but generally get better computational
performance if we do a line search to find the value for k that
minimizes F .x/ along the search direction pk . Line search:
! First looks for bracket a; b! in which there is a minimum; then
NOTE :
NOTE :
NOTE :
NOTE :
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
431
EXAMPLE :
function F=rosen(x)
global xpath;
F=100*(x(1)^2-x(2))^2+(1-x(1))^2;
xpath=[xpath;x'];
end
xpath=[];
options = optimset('LargeScale','off','HessUpdate','steepdesc',...
'MaxFunEvals',20000,'MaxIter',2000);
[X,FVAL,EXITFLAG,OUTPUT]=fminunc('rosen',[-1.2 1]',options);
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
432
Value
x(2)
3000
0
1000
x(1)
2000
0
2
1
x(1)
2 2
x(2)
xpath=[];
options = optimset('LargeScale','off','HessUpdate','dfp',...
'MaxFunEvals',5000,'MaxIter',2000);
[X,FVAL,EXITFLAG,OUTPUT] = fminunc('rosen',[-1.2 1]',options);
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
433
Value
x(2)
3000
0
1000
2000
0
2
1
x(1)
2 2
x(2)
xpath=[];
options = optimset('LargeScale','off'); % select quasi-Newton; BFGS
[X,FVAL,EXITFLAG,OUTPUT]=fminunc('rosen',[-1.2 1]',options);
figure; clf; contour(x1,x2,FF',[0:2:10 15:50:1000]); hold on
plot(xpath(:,1),xpath(:,2),'gd'); plot(xpath(:,1),xpath(:,2),'g-');
xlabel('x(1)'); ylabel('x(2)');
title('Rosenbrocks optimization path: BFGS')
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
434
plot(-1.2,1,'ro'); plot(1,1,'ro')
text(-1.8,-1.8,sprintf('Iterations: %d, Fn evals: %d',...
OUTPUT.iterations,OUTPUT.funcCount),'fontsize',16);
figure; clf; mesh(x1,x2,FF'); hold on
for ii=1:length(xpath);
plot3(xpath(ii,1),xpath(ii,2),0.1+rosen(xpath(ii,:)'),'gd',...
'markersize',5)
end
plot3(-1.2,1,1+rosen([-1.2 1]'),'r.','markersize',25)
plot3(1,1,1+rosen([1 1]'),'r.','markersize',25)
xlabel('x(1)'); ylabel('x(2)'); zlabel('Value');
title('Rosenbrocks optimization path: BFGS')
campos([-22.248, 21.81, 17143]); xlim([-2 2]); ylim([-2 2]);
Value
x(2)
3000
0
2000
1000
x(1)
0
2
1
x(1)
2 2
x(2)
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
435
EXAMPLE :
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
436
identification
input signal for identification
compute "clean" output of G
noise - note that this is white, gaussian
scale so energy in sensor noise 1/4 times
%
the energy in the "clean" signal
actual output y=Gu+v
data available to identification
0.5
0.5
1
0
10
20
30
40
50
Amplitude (unitless)
Amplitude (unitless)
measured y[k]
2
0
10
Time (s)
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
20
30
Time (s)
40
50
437
438
50
Phase (deg)
Magnitude
10
10
10 1
10
Actual
SPA
ARX221
ARX441
0
10
Frequency (rad s )
10
10
10
Frequency (rad s )
10
50
Phase (deg)
Magnitude
Actual
SPA
ARX221
ARX441
250
10
10
150
200
10 1
10
100
Actual
OE221
ARMAX2221
BJ22221
100
150
Actual
OE221
ARMAX2221
BJ22221
200
250
0
10
Frequency (rad s )
10
10
10
Frequency (rad s )
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
10
439
Ntime=30;
y_act
=
y_arx221 =
y_arx441 =
y_armax =
y_oe
=
y_bj
=
dimpulse(num,den,Ntime);
dimpulse(b_arx221,a_arx221,Ntime);
dimpulse(b_arx441,a_arx441,Ntime);
dimpulse(b_armax,a_armax,Ntime);
dimpulse(b_oe,f_oe,Ntime);
dimpulse(b_bj,f_bj,Ntime);
Output amplitude
0.4
0.3
0.2
0.1
0
0.1
0.2
0
0.5
0.4
Output amplitude
0.3
0.2
0.1
0
0.1
10
15
0.2
0
Time (sec)
10
Time (sec)
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
15
440
[['Act B
['ARX221 B
['ARMAX B
['BJ B
['OE B
',num2str(num,4)]
',num2str(b_arx221,4)]
',num2str(b_armax,4)]
',num2str(b_bj,4)]
',num2str(b_oe,4)]
['
['
['
['
['
Act A
',num2str(den,4)]; ...
ARX221 A ',num2str(a_arx221,4)];...
ARM A
',num2str(a_armax,4)]; ...
BJ A
',num2str(f_bj,4)]; ...
OE A
',num2str(f_oe,4)]]
O
! Other three provide good estimates of G.q/.
b0
b1
b2
a0
a1
a2
Actual
0:1129
0:1038
0:7788
ARX221
"1:562
0:1309
0:1538
ARMAX2221
0:1144
0:0941
BJ22221
0:1153
0:0940
OE221
0:1148
0:0948
"0:689
"1:569
"1:569
"1:568
0:0154
0:7874
0:7872
0:7869
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
"
441
e_arx2 = resid(M_arx221,Z);
e_arx4 = resid(M_arx441,Z);
e_arm = resid(M_armax,Z);
e_bj
= resid(M_bj,Z);
e_oe
= resid(M_oe,Z);
mean([v e_arx2 e_arx4 e_arm
[['sigma^2
['sigma^2
['sigma^2
['sigma^2
['sigma^2
e_arx2 = e_arx2.OutputData;
e_arx4 = e_arx4.OutputData;
e_arm = e_arm.OutputData;
e_bj
= e_bj.OutputData;
e_oe
= e_oe.OutputData;
e_bj e_oe])'
arx
arm
BJ
OE
act
',sprintf('%1.4f',M_arx221.NoiseVariance)];...
',sprintf('%1.4f',M_armax.NoiseVariance)];...
',sprintf('%1.4f',M_bj.NoiseVariance)];...
',sprintf('%1.4f',M_oe.NoiseVariance)];...
',sprintf('%1.4f',LL)]]
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
442
ARX221
ARMAX
BJ2221
OE221
"0:0087
"0:0272
"0:0281
"0:0249
0:2206
0:1085
0:1091
0:1083
0:1118
100
50
0.5
0.5
50
0
1.5
1.5
0.5
Value of residual
0.5
0.5
0.5
1.5
Value of residual
50
Model fit
Actual
150
Count
Count
0
1.5
1.5
Model fit
Actual
100
0
1.5
Value of residual
150
100
50
0.5
0.5
Value of residual
100
50
0
1.5
Model fit
Actual
150
Count
100
Model fit
Actual
150
Count
Count
Model fit
Actual
150
1.5
0
1.5
0.5
0.5
1.5
Value of residual
Again, ARX models do not match data very well; ARMAX, BJ and OE
much better.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
443
inputs in the residuals, then a part of yk! that originates from the
input was not captured well in our model (bad).
cN
where ' > 0 correlates %k! with old uk " '!, and so we desire R
%u '!
to be (roughly) zero for ' > 0.
WHITENESS:
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
444
#N;M D !
cN
R
% 0!
M
X
! c N "2
R% '!
"2
'D1
element of rN;M '! to decide if the autocorrelation is small for ' > 0.
kD"1
c
Can perform a normality test on R
%u '! by checking if
p
c
jR
%u'!j % 1:96 Pr =N for all '.
c
If R
%u '! is outside these bounds, then for those values of ', %k! and
uk " '! are probably dependent.
Dependency for small ' could imply the need for smaller nk .
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
OTHER TESTS :
445
Try at least one of these, never using same data to create and
validate a model.
! A larger model will always give a better fit (a lower value of VN .$/).
! Must use new data to compare. Good models will still give good
Problem with using same data for model creation and validation is
that model will attempt to fit the randomness of the noise.
! Since noise varies from run to run, this is counterproductive.
! If only one data set is available, split in half: one half for training,
EXAMPLE :
446
0.5
0.5
0.5
0.5
0
10
15
20
25
0.5
0
10
Lag
0.6
15
20
25
0.5
0
0.2
0.2
0.1
0.2
0.1
0.1
10
10
20
0.2
20
10
Lag
10
0.2
20
20
10
Lag
0.5
10
15
20
25
0.5
0
for BJ2221
10
Cross correlation R
0.1
0.1
0.1
10
15
0.2
0.1
20
20
20
25
Lag
Cross correlation R
0.2
10
Lag
0.2
25
0.5
0.5
0
20
Lag
15
10
Lag
0.1
20
Lag
0.4
0.2
10
20
0.2
Lag
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
20
10
Lag
for OE221
10
20
447
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
448
! For each d , select the model structure having the lowest VN .$/.
! Finally, plot J.$; d /; select model with overall lowest value.
EXAMPLE :
! Denominator: na 2 f1 : : : 15g,
! Delays: nk 2 f1 : : : 5g.
0.25
Cost function
! Numerator: nb 2 f1 : : : 15g,
0.3
0.2
0.15
0.1
0
10
15
20
25
Number of parameters
30
35
449
See right away that there are many poor choices! With only five
parameters, would get very poor fit.
Can also use selstruc
GUI to manually select best
model for each overall size.
! For MDL, fifteen
parameters: 8,6,1;
parameters: 8,7,1.
0
50
Phase (deg)
Magnitude
10
100
10
10 1
10
150
Actual
ARX221
ARX441
ARXAIC
ARXMDL
Actual
ARX221
ARX441
ARXAIC
ARXMDL
200
250
0
10
10
Frequency (rad s )
10
10
10
Frequency (rad s )
Model fit
Actual
150
Count
100
Count
100
50
0
1.5
Model fit
Actual
150
50
0.5
0.5
Value of residual
1.5
0
1.5
0.5
0.5
Value of residual
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
1.5
450
0.5
0.5
0.5
0
0.2
10
15
20
25
0.5
0
10
25
0.2
0.1
0.1
20
10
10
20
0.2
20
10
Lag
20
Lag
0.1
15
Lag
0.1
0.2
10
20
Lag
! So, will generally be better off trying this approach with multiple
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
451
For valid comparison, try same experiment, but with nonwhite noise.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
452
u[k]
v[k]
Amplitude (unitless)
Amplitude (unitless)
0.5
0.5
1
0
10
20
30
40
2
0
50
10
20
Time (s)
40
50
b1
b2
a0
a1
a2
Actual
0:1129
0:1038
0:7788
ARX221
"1:562
0:1200
0:1106
ARMAX2221
0:1181
0:0868
BJ22221
0:1125
0:1002
OE221
0:1053
0:0920
"1:199
0:4338
"1:581
0:7984
"1:566
0:7841
"1:573
0:7989
ARX441 gives reasonable fit to G.q/, but BJ, ARMAX, OE look better.
Bode mag. plots of several system id models
10
Magnitude
10
Magnitude
30
Time (s)
10
10 1
10
Actual
SPA
ARX221
ARX441
10
10
Frequency (rad s )
10
10 1
10
Actual
OE221
ARMAX2221
BJ22221
0
10
Frequency (rad s )
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
10
453
Phase (deg)
50
100
150
Actual
SPA
ARX221
ARX441
200
250
Actual
OE221
ARMAX2221
BJ22221
10
10
Frequency (rad s )
10
0.5
True system
ARX221
ARX441
0.4
0.3
0.1
0
0.1
Frequency (rad s )
0.3
0.2
0.1
0
0.1
0.2
0
10
0.2
0
15
10
Time (s)
10
0.4
0.2
10
Output amplitude
150
250
10
100
200
Output amplitude
Phase (deg)
50
15
Time (s)
c1
c2
d0
d1
d2
Actual
0:7567
"1:862
0:9851
ARX221
"1:634
ARMAX2221
0:5905
BJ22221
"1:122
0:7865
"1:872
0:9955
OE221
"1:673
"1:199
"1:581
0:4338
0:7984
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
454
10
Magnitude
50
Phase (deg)
10
10
50
1
10 1
10
10
Frequency (rad s )
200
10
Frequency (rad s )
250
Model fit
Actual
200
100
100
50
0.5
0.5
0.5
Value of residual
250
0.5
0
1.5
1.5
0.5
Value of residual
200
250
0.5
1.5
Value of residual
200
Count
150
Count
150
100
100
50
0
1.5
Model fit
Actual
200
50
0
1.5
1.5
100
50
0
1.5
250
150
Count
150
Count
150
50
0.5
0.5
1.5
0
1.5
Value of residual
0.5
0.5
1.5
Value of residual
Results for noise mean and noise-power estimation of ek! (not vk!)
Actual
Mean "0:0117
(2
10
250
10
Count
10
0:0310
ARX221
ARMAX
BJ2221
OE221
"0:0026
"0:0071
"0:0155
"0:0197
0:0750
0:0418
0:0302
0:1038
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
455
0.5
0.5
0.5
0.5
0
10
15
20
25
0.5
0
10
Lag
15
20
25
0.5
0
10
Lag
Cross correlation R
0.3
for ARX221
Cross correlation R
0.2
0.2
15
20
25
Lag
for ARX441
Cross correlation R
0.2
0.1
0.1
0.1
0.1
for ARMAX2221
0.1
0
0.1
0.2
20
10
10
0.2
20
20
10
Lag
10
0.2
20
20
10
Lag
Lag
0.5
0.5
0
0
0.5
1
0
10
15
20
25
0.5
0
10
Lag
0.2
0.1
0.1
0.1
20
10
Lag
20
25
10
0.2
0.1
0.2
15
Lag
20
0.2
20
10
Lag
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
10
20
10
20
456
All model types have bias/variance problems, but ARX easiest (only?)
case to analyze.
! Also most pronounced.
2. Bias errors:
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
Typical problems
SIMPLE EXAMPLE :
457
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
458
0.5
0.5
0.5
0
10
15
20
25
0.5
0
10
Lag
u
Cross correlation R
0.05
0.05
0.05
20
10
0.1
0.05
0.1
20
25
Lag
Cross correlation R
0.1
15
10
20
0.1
20
10
Lag
10
20
Lag
A.q/.yk! " y/
N D B.q/.uk! " u/
N C ek!.
Q
! Use the modified data .uk! " u;
N yk! " y/
N to estimate the
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
459
0.5
0.5
0
10
15
20
25
Lag
Cross correlation R
0.1
0.05
0.05
0.1
20
10
10
20
Lag
For the example given above, I solved the least-squares problem with
handwritten code, averaged over ten runs, and got aO D 0:9029,
bO D 0:2565, and eO0 D 1:0005.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
460
CALCULATION :
Compute
$ " $O D $ " .X T W X/"1X T W Y .
L
"XW
D "XW"L e,
so
"L
E$ " $O ! D "EXW
e!.
Big problem: For the WLSE we care about in the ARX case,
h
iT
so is explicitly a function of measured data.
X D &1! # # # &N !
So, we cannot pull out the XW"L term. We are stuck with the bias.
O D "E.X T W X/"1X T W e! 0
E$ " $!
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
in general:
461
One last exception: If ek! is white, it will not be correlated with past
data in X.
! Therefore, E$ " $O ! D "EXW !Ee! D 0.
"L
EXAMPLE :
analytical comparison.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
462
New cost
V D E% 2! D E .yk! " yk;
O $!/2 !
D E .yk! C ayk " 1! " buk " 1!/2 !
:::
(see appendix)
@V
D r0 .2a " 2a0/ C 2c0 D 0
@a
a( D a0 " c0=r0,
@V
D 2b " 2b0 D 0
@b
b ( D b0.
V .$0/ D 1 C c02,
But, for this assumed model class, $ ( gives a better predictor since
V .$ (/ < V .$0/.
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
NUMERICAL EXAMPLE :
463
System has yk! C 0:9yk " 1! D 0:25uk " 1! C ek! C 0:7ek " 1!. So,
" # "
#
a0
0:9
$0 D
D
.
0:25
b0
We can compute r0 D 1:54, so we expect a( D a0 " c0=r0 D 0:4453
and b ( D b0. That is,
"
# "
#
(
a
0:4453
$( D
D
.
(
b
0:25
Averaged ARX results over ten runs for several values of N .
"
#
"
#
"
#
0:4580
0:4501
0:4485
$O1024 D
, $O4096 D
, $O16384 D
.
0:2530
0:2508
0:2525
Note that V .$0/ D 1:49 and V .$ (/ D 1:17.
Algorithm gives us the best possible predictor, but this does not
necessarily mean we get a good model.
h
iT
(
Average ten ARMAX models: $
,
ARMAX D 0:9025 0:2520 0:7038
which gives a good fit with low bias.
Variance errors
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
464
d
.k; $0/ D
yk;
O $!
d$
$0
.k; $ON /
kD1
N
X
1
(O N2 D
% 2k; $ON !
N
kD1
1
"1
PON D (O N2 RO N
.
N
d
yk!
O
D &k!, so
d$
"
#
"yk " 1!
&k! D
.
uk " 1!
"
#
Ry 0! Ryu 0!
T
!D
.
Ryu 0! Ru 0!
This gives R D E
We now see that the selection of uk! explicitly plays a role in the
accuracy of our estimates through Ru and Ryu .
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
"1
is
465
Transfer functions great for analysis and design of predictors for and
control of LTI SISO systems.
But, much harder to use for MIMO systems, and much harder to
identify transfer-function models for MIMO systems.
Also, nonlinear optimization required for all but simplest
transfer-function approaches, which can get stuck in local minima,
yielding sub-optimal models.
State space models are an alternate way to describe system
dynamics.
! Work great for multi-input, multi-output systems.
! Allow new analysis and synthesis tools for estimation and control
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
466
D "2a0c0
b02Eu2k"1!! D b02
Euk"1!ek!! D Euk"1!ek"1!! D 0
Ee 2 k!! C 2c0 Eek!ek"1!! C c02Ee 2k"1!!
D 1 C c02
so, Ey 2k!! D a02Ey 2k!! " 2a0c0 C b02 C 1 C c02
Also
2
2aEyk!yk"1!! D 2aE."a0yk"1!Cb0uk"1!Cek!Cc0ek"1!/yk"1!!
D 2."aa0Ey 2k!! C ac0/2
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett
467
Lecture notes prepared by Dr. Gregory L. Plett. Copyright 2011, 2015, Gregory L. Plett