Dynamic Model Development: Methods, Theory and Applications

S.P. Asprey and S. Macchietto (editors)

2003 Elsevier Science B.V. All rights reserved 159

A Continuous-Time Hammerstein Approach Working with Statistical

Experimental Design
Derrick K. Rollins

Department of Chemical Engineering & Statistics, Iowa Sate University,

Ames, Iowa 50011

Hammerstein modelling is a basic approach to dynamic predictive modelling of non-linear

behaviour. The development of the continuous-time Hammerstein approach by Rollins, et al.
(2002) provides a natural compatibihty with statistical design of experiments (SDOE) and thus,
allows for optimal experimentation and the efficient treatment of all non-linear and interactive
effects. This approach is demonstrated on a theoretical Hammerstein model from literature and a
real house-hold dryer with four inputs and five outputs.

INTRODUCTION

When modeling dynamics process behavior, engineers will commonly choose a pseudo-random
binary sequence (PRBS) or a pseudo-random sequence (PRS) to 2excite" the process. A PRBS is
a series of input changes with random times for changes from one level to another level and then
back to the same level. A PRS will have multiple levels which may also be randomly set. If the
ultimate change behavior is truly additive (i.e., all interactive effects are zero) and linear, a PRBS
can provide information that is useful in estimating model parameters but it is not likely to be
optimal in terms of minimal changes to the process. (Interactive effects are cross product terms
or multi-linear terms or multi-linear terms.) In this situation, a low-resolution statistical
experimental design, that does not "confound" main effects with other main effects, is likely to
require fewer runs. (Two effects are confounded when they are perfectly correlated. Thus, when
two effects are confounded, their cause and effect relationship on the response cannot be
distinguished [i.e., separated]. "Partial confounding" is when two effects are correlated but not
perfectly correlated.) However, two critical reasons that an engineer may not choose a statistical
experimental design approach are the lack of recognition of this advantage or the lack of ability
to determine such a design. To overcome the later reason, engineering curricula have made
dramatic changes in recent years to include the teaching of statistical design of experiments
(SDOE). It is one of the objectives of this paper to help to overcome the former reason. If non-
linear effects are present, a PRBS is incapable of providing information to estimate this behavior
as pointed out by Pearson and Oguimaike (1977). In the cases, the
160

most popular approach taken by engineers to obtain models of process behavior is the use of a
PRS design (see Su and McAvoy, 1993). Multiple input change levels in this design will aUow
for the estimation of non-linear ultimate response behavior. However, the "non-intelligent"
(since it is random) setting of design points (i.e., treatment combinations) of a PRS design will
likely confound (at least partially) significant effects. Thus, in addition to ineflBciency, a PRS
design will also not allow the collection of information to estimate significant interactions. On
the other hand, SDOE does not suffer fi'om these limitations because of its superior
"intelligent" approach to data collection.
It would not make sense to use an experimental design to collect information to estimate
non-linear and interactive ultimate response behavior if the modeling method (i.e., the form of
the dynamic model) was incapable of addressing this behavior, as in the case of hnear transfer
fimction modeling. The only approach we have found in the literature meeting this criteria is
the class oiHammerstein models (see S. A. Billings, 1980) which can be a subset of Volterra
models (see Seinfeld and Lapidus, 1974) as pointed out by Pearson and Ogunnaike (1997).
The Hammerstein approach combines linear dynamics with non-linear steady state gains. A
description is shown in Figure 1 for a continuous multiple-input, multiple-output (MIMO)
system. Thus, its capability to address non-linear and interactive effects are possible by the
unrestrictive mapping of the steady state gains.
There are two basic classes of Hammerstein models, continuous-time and discrete-time.
However, discrete-time Hammerstein modeling, which can also be classified as NARMAX
(Nonlinear AutoRegressive Moving Average models with exogenous inputs) modeling,
appears to be the most popular one in the chemical engineering literature. It is interesting to
note that, even though discrete-time Hammerstein modeling has the ability to address
interaction terms, this is not commonly done in practice because of the enormous parameter
identification burden this would cause as pointed out by Eskinat, et al. (1991). Thus, in
practice, a PRS is made reasonable (but not necessarily practical) via the a priori assumption
that all interaction effects (cross product terms) are zero.

Static
Nonlinear
Map
> Linear
Dynamics
>
X f(X) G(s)
-^>
>

>

Figure 1. A description of the general "MIMO Hammerstein model" structure as it appears ki

Pearson and Ogunnaike (1997). The input vector X passes through a static map and
produces the gain vector f(X) which can be non-linear and then passes through the
linear dynamic map and produces the output vector y.
 161

In the statistics literatxire for experimental design, with no a priori assumption for the
model form, we have not found the treatment of dynamic systems (e.g., see Montgomery,
1984). This treatment of experimental design has been restricted to the paradigm of steady
state systems. In real dynamic applications the common practice by statisticians is to make a
process change as* specified by the design and allow the process to reach steady state before
collecting the response data associated with this change. We feel that there are two basic
reasons that statisticians have not extended SDOE to include the modeling of dynamic systems.
The first one is the lack of training in continuous-time dynamic systems and the second one is
the lack of a methodology for this extension.
So on the one hand, you have engineers that treat dynamic systems but not interactive
effects and thus, are able to "get by" without the need for SDOE. On the other hand, you have
statisticians that treat interactive effects but ignore the modeling of process dynamics. In this
paper we present a non-linear dynamic methodology that treats interactive effects as easily as
steady state modeling andfiiUyutilizes SDOE without adding any new requirements for
knowledge. This is done through the presentation of a continuous-time Hammerstein modeling
approach that was originally developed by Rollins, et al. (1998) for single iaput, single output
(SISO) processes but has recently been extended to multiple input, multiple output (MMO)
processes by Rollins, et al. (2002). Thus, this methodology seeks to connect eflScient and
sufScient experimental design with comprehensive and complete expressions for dynamic
model development.
This paper is written with the engineering community in mind. Rollins and Bascunana
(2001) have written an article directed towards the statisticians that contains considerable
detail on concepts of continuous-time dynamic model forms. In the next section, we describe
the proposed method in detail. Following this section, we present a theoretical Hammerstein
model from literature and demonstrate the ability of proposed method to fit this model. Next,
this paper demonstrates the application of the proposed method on a on a real process.

2. THE APPROACH

This section describes the attributes of the Rollins, et al. (1998) (also, Rollins, et al.,
2002) continuous-time, Hammerstein approach (CTHA) in detail. Figure 2 is block diagram
representing the structure of CTHA for a two input, two output system. In comparing Figure 1
with Figure 2 one sees that Figure 2 omits the block for the static map and shows the non-
linear gain fimctions entering directly into the linear dynamic block. Another difference is that
the linear dynamic fimction is shown in the "s" (Laplace) domain m Figure 1 and in the "t"
(time) domain in Figure 2. This was done to illustrate a critical achievement of CTHA. In the
context of continuous-time, Hammerstein modeling, we have not seen an explicit time-domain
expression for the outputs. That is, we have not seen a time domain expression for G(s) in
Figure 1. However, CHTA is only expressed in the time domain.
As illustrated by Figure 2, as in all Hammerstein approaches, the expressions for the
outputs separate the ultimate response fimctions from the dynamic response fimctions. An
example of a CTHA expression is given below for a two input system with first order
dynamics, undergoing a step change only at time 0, which is at steady state before this change.
162

fi(Ax Axj)

f^CAxi, AX2)

Figure 2. The block diagram representation of CTHA for a two input, two output system.
Inputs to the blocks are fimctions of process variables and the models in the ultimate
responses (i.e., gains) are able to address iateraction and other terms. Note, ia
contrast to Figure 1, the transfer functions are in the time domain.

.Vi(0 =J'i(0) +/(AxAx2)-g(/;T)-5(0

(1)
= 7i(0) + (Po + h^x -^ P2^2 + Pj^iAXj + P4AX2')-1 1 - e"^] Sit)

where S(t) is the unit step function. Note that at t = 0, Eq. 1 gives yi(t) = yi(0) and at t = oo^ it
gives yi() = yi(0) +y(Axi, Axj). Thus, Eq. 1 has the correct initial behavior and the correct
ultimate behavior ifX^Xj, AX2) is an accurate expression for the ultimate value of y for these
input changes. In CTHA this function is determined for each output by fitting the ultimate
response values against the input changes over the input space specified by the statistical
experimental design using multiple linear regression. Hence, for accurate fits, one would need
accurate ultimate response performance. Implementation of Eq. 1 for predictionfiromseveral
input changes is not simply a matter of inverting the transfer function given by g(t; x) and using
an input sequence fory(Axi, AX2) in the s domain. This is not possible because^Axj, Axj)
cannot be written the s domain. In a moment, we present a novel algorithm for implementing
models of the form given by Eq. 1. However, before this is presented, we describe our
procedure for estimatingX^x; p) and g(t;T).
The steps for obtaining the fitted model for CTHA are as follows:

1. Determine the statistical experimental design.

ii. Run the experimental design as a series of step tests, allowing steady state to
occur after each change while collecting the data dynamically over time.
m. Use the steady state data to determine the ultimate response function, y(Ax;P),
for each output.
IV. Use the dynamic data to determine the dynamic response function, g(t;T), for
each output.
 163

After obtaining the fitted equations for y(Ax;P) and g(t;T), they are incorporated into an
algorithm to predict output response for changes in inputs. The CTHA algorithm is a
procedure that predicts output response from the fits fory(Ax;P) and g(t;T) in a scheme that
depends only on the most recent change for each input. For an input change occurring at time
tj, a generic representation of the CTHA prediction algorithm is given by Eq. 2 below:

For t > t,: y{t) = yit,) + [/{Ax(0;P) - Kh) + y{0)]^t-t,'^)S[t-t,) (2)

where y(t) is the estimated output response at time t; y(0) is the measured value of the output
at the initial time, 0; Ax(t) is a vector that contains the deviation values of the process variables
from their initial values at time t; p is a vector that contains the estimates of the steady state
response parameters determined from the current input conditions; f(Ax(t);p) is the fimction
that computes the change in the ultimate response for the change Ax(t); x is a vector that
contains the estimates of the dynamic parameters that could depend on Ax(t); g(t -1^; T) is the
semi-empirical non-linear fimction that computes the dynamic portion of the response such that
as t -* >, the fimction - 1; and S(t - t j is the shifted unit step fimction. Note that at tj, y(t) =
y(ti), and as t - oo, y{X) - y(0) +fl;Ax(t);p).Next, we demonstrate the ability of CTHA to
accurately fit theoretical Hammerstein models.

3. AN APPLICATION OF CTHA TO A THEORETICAL HAMMERSTEIN PROCESS

In their survey paper of structure identification methods for non-linear dynamic systems,
Haber and Unbehauen (1990) presented the following Hammerstein model:
v(t) = 2.0 + u{t) + 0.5 uitf

ioM).xO=v(0 ^^^
dt
with u(0) = 0. In this section we wiU demonstrate the ability of our proposed method (i.e.,
CTHA) to identify Eq. 3 in integrated form and to predict the response y(t) continuously over
time quite effectively using Eq. 2. Note when placed in the context of Eq. 3, Eq. 2 terms are

f(Ax;^) =f(u;^)= u(t) ^ 0,5u(tf (4)

and

withy(O) = 2. One measure of the soxmdness of this approach v ^ be its ability to accurately
obtain Eqs. 4 and 5. The other critical measure will be to incorporate them into Eq. 2 and
obtain from Eq. 2 accurate predictions over time for changes in u(t).
The first step in applying CHTA is to determine the experimental design. Since this is a
single input, single output process, and the ultimate response is quadratic, the following four
input changes were chosen as the experiment design: u = -4, -1, 1, and 4. These input changes
were made and the fits shown in Figure 3 were obtained using first order models.
164

lO -

u=4
14 - True response
foru = -4
12 - - - Fitted response
foru=-4
True response
10 - foru = -l
Fitted response
> foru = -l
"'^-' True response
/ u = -4 for u = 1
"3 Fitted response
O 6- for u = 1

4-
/ ^^^^^'^'^ ""^ True response
for u == 4
- Fitted response
2- 1 ^ ^ - ..^^_^ for u = 4

0- ' 1 r ' 1 1 1

10 20 30 40 50 60

Time

Figure 3. The response of y and the fits of the first order models for the four changes in u to
obtain estimates of Eqs. 4 and 5.

As shown, the fit to each step test is quite accurate. From each trial (i.e., change in u),
the ultimate response value was obtained, and used to estimate Eq. 4. Figure 4 shows a plot of
the ultimate response data versus u. Five points are shown which include the fi^ur design points
and the center point (u = 0). This figure demonstrate the highly nonlinear behavior of the
ultimate response over this input space. Fitting a quadratic model to data in Figure 5 using
linear regression produced the following, excellent, approximation of Eq. 4,

/(w;P) =0.998w(0 + 0.499w(0^ (6)

The four values of x estimatedfiromeach of the first order fits in Figure 3 were averaged
to obtain the estimate for Eq. 5. This value was 9.93 giving a close estimate of Eq. 5 as shown
below:
t

g(/;T) - \ - e
9.93 (7)

Testing CTHA for this process consisted of incorporating Eqs. 6 and 7 into Eq. (2) and
examining its predictive behavior when making arbitrary changes in u over the fitted input
space. The input change sequence for this test is shown in Figure 5. The process response and
the fitted response of CTHA for this input sequence change is presented in Figure 6. As
shown, CHTA fits aknost perfectly to the real process. Thus, the proposed approach appears
to be very capable of accurately predicting non-linear dynamic process behavior having a
Hammerstein nature.
 165

-5 -3 -2 -1 0 1

Change i n U

Figure 4. A plot of the ultimate change in y against u with the fitted line (Eq. 6) for the
changes in the u.

5-

4-

3-

2-

D '-
n 0-
-1 -

-2 -

-3 -

-4-

-5 - -T

10 20 30 40 50 60 70 80 90 100
Time

Figure 5. The input sequence for testing CTHA for the first order process.
166

100

Figure 6. The fit of CTHA for the input testing sequence of Figure 5.

To further demonstrate the ability of CTHA to fit Hammerstein processes, a more

complex second order process is used. This process is described below:

d^yif) I \dy{t) ,,. dv{t) (8)

v(0
with all initial conditions and derivatives equal to zero, where a^ = 2.0, a2 = 1.0, ag = 1.5, a^ =
0.5, a5 = 1.0, T^ = 15.0, Ti = 2.0, and Xj = 5.0.
The experimental design for this case was a 3^ full factorial design to allow estimation of
all main effects, quadratic effects and the two factor interaction. The specific design points are
shown in Table 1 below. As an example of the fitting, the fit for Trial 1 is shown in Figure 7
below. As one can see, the fit to this more complicated process is quite good. Although not
only this case is shown for space consideration, this is typical of the performance for the other
cases. The fitted functions obtained for Eq. 2 are given below.

/(AJC;P) =/(i/;p) = 2.0wi(0 + 1.0^2(0 + L5u,it)u^{t) + 0,5u,{tf + hOu^(tf (9)

g(f^) = 1 + T^ ^ + i (10)
^x - h
withx^ = 14.98, t, = 2.01 and x^ = 4.98.
 167

Tablel. The experimental design used by CTHA for the process given by Eq. 8.

Trial Ul U2

1 5.0 -5.0
2 -5.0 5.0
3 0.0 5.0
4 0.0 -5.0
5 5.0 0.0
6 -5.0 0.0
7 -5.0 -5.0
8 5.0 5.0
9 0.0 0.0

Figure 7. Typical fit of the response of y for the second order models for the design in Table 1
to obtain Eqs. 9 and 10.
168

Testing CTHA for the second order process consisted of incorporating Eqs. 9 and 10
into Eq. (2) and examining its predictive behavior when making arbitrary changes in Uj and U2
over the fitted input space. The input change sequence for this test is shown in Figure 8. The
process response and the fitted response of CTHA for this sequence change is presented in
Figure 9. As in the first order case, this case also fits the real process almost perfectly. Thus,
this example fiuther illustrates the ability of the proposed approach to accurately model non-
linear dynamic processes meeting the form of Hammerstein processes. Next we illustrate
CTHA on a real process with five outputs and four inputs. The abiKty of CTHA to exploit
SDOE will be more appreciated in this example.

Ul
--U2

a 0

0 100 200 300 400 500 600 700 800 900 1000
Time

Figure 8. The input sequence for testing CTHA for the second order process.
 169

lOU n

I
True Process
- - - CTHA 1

100-

r
>4 50-

r
3
& (V_
3 1
O 0-

-50-

() 100 2CK) 300 400 500 600 700 800 900 10

Time

Figure 9, The fit of CTHA for the input testing sequence of Figure 8.

4. AN APPLICATION OF CTHA TO A REAL PROCESS

CTHA will be illustrated using a common household-style dryer that has been retro-fitted
with several sensors and recording instruments. For details of this process see Rollins, et al.
(2002). The input variables chosen for this study were the power supplied to the heater (P), the
inlet fan speed (N), the dry weight of the clothes in the dryer (w), and the initial moisture
content of the clothes (m). Although this study consisted of five outputs (see Rollins, et al.,
2002), we will present the results for only one here for space considerations, the temperature
of the air exitiag the heater (TJ.
The dynamic modeling goals of this study consisted of accurate output prediction over
the complete input space, whQe maintaining the ability to fit non-linear behavior as well as
significant two factor interactions (i.e., bilinear effects) in as few runs (i.e., step tests or trials)
as possible. As stated previously, a PRS design would not meet this criteria in terms of
estimability or eflSciency. However, the CTHA structure allows for thefiiUuse of SDOE,
which meets the data collection, experimental design and modeling criteria. The experimental
design that was selected, meeting the specified criteria, was a central composite design with
replicated center points (see Montgomery, 1984). For specific details on the levels of the
inputs and the design of this study see Rollins, et al. (2002).
Each experiment consisted of starting the dryer with the input variables set to values for
that run and then recording the outputs dynamically. Due to the batch nature of the dryer, the
process does not reach steady state, but the output variables tend to level off. The g(t;T) model
forms were selected by a visual inspection of the dynamic response of the output. The drying
170

process can be divided into two distinct phenomena; the constant rate drying and the falling
rate drying. This study considered only the constant-rate drying period. The dynamic response
of Ta for Rim 10 is shown in Figure 10. The other runs gave similar fitted performance.

200

Measured response

- - - Fitted response

10
Time, min

Figure 10. The dynamic response of the air exiting the heater for Run 10. The fitted curve is
obtained by using a second order model with a lead term and the parameters (x^, Xj,
and X2) are estimated by non-linear regression.

Using the steady state values for the 27 runs, the following fimction was obtained for the
ultimate change fimction using multiple linear regression.

/(AX(0;P) = 0.04239AP - 0M203AN + 4.393Aw - 0.2374Am (11)

Thus, non-linear or interacting effects, although statistically tested, did not indicate significance
at the 0.1 test level. For each of the 27 runs, non-linear regression was used to fit the dynamic
response behavior. By inspection, a second order continuous-time model form with a lead term
was determined to fit all 27 cases the best. The three dynamic parameters (xj, X2, and x j for the
final form were determined by averaging the 27 values for each one. The final equation for g(t;
T) is given by Eq. 12 below.

a 2 (12)
g(nr) = 1 + /^' + e '^
X, - X~ x^ - X,
V V 1 2; )
where Xj = 0.389, X2 = 3.62 and x^ = 3.04. After obtaining Eqs. 3 and 4, they were incorporated
into the prediction algorithm (i.e., Eq. 2) and CTHA was evaluated using the test sequence
shown in Figure 11.
 171

CTHA performance is shown in Figure 12 using the test sequence of Figure 11. As
shown, the CTHA predictions closely foUows the process at all times during the test run. Thus,
this approach appears to model this process for air temperature quite well. CTHA predictive
performance for the other four responses was similar as shown in RoUins, et al. (2002).

1800

Figure 11. The test input sequences, for power and fan speed, used for testing CTHA.

165 H
O 145 -\
- 125

Figure 12. The exit air temperature response to the iuput sequences shown ia Figure 11.
172

5. CLOSING REMARKS

This work presented a continuous-time Hammerstein modeling approach for dynamic

non-linear M M O predictive modeling that complements the use of the powerful jBeld of
statistical design of experiments (SDOE). It appears that Hammerstein modeling has seen
much activity in discrete-time situations but very limited application in continuous-time
modeling. The most common experimental design used for these modeling situations has been
a pseudo-random sequence (PRS). Discrete-time or NARMAX modeling typically omits fitting
interaction (i.e., cross product) terms because this causes huge parametrization, regardless of
whether a PRS design would provide the needed information for estimation of the parameters,
which is not likely when cross product terms are significant. A critical reason that the proposed
method is successfiil is its ability to formulate output prediction in explicit form with separate
terms for the steady and dynamic portions. Another key to its success is the ability of its
prediction algorithm to achieve high accuracy with serial input changes over time.
Parametrization for this approach is usually quite small in comparison to discrete-time
modeling.
Our main motivation for selecting the dryer process for this study was to demonstrate
the ability of the proposed approach to model a real system. However, this process, to our
surprise and disappointment, did not exhibit strong non-linear ultimate response behavior. We
demonstrated the ability of the proposed approach to model dynamic non-linear behavior in the
theoretical Hammerstein process studies.

6. ACKNOWLEDGMENTS

The author wishes to acknowledge the partial support for the research by the National
Science Foundation under grant number CTS-9453534 and by the Frigidaire Corp., lEC (Iowa
Energy Center), and CATD (Center for Advanced Technology Development). We are also
gratefiil to the Program for Women in Science and Engineering at Iowa State University and
Trisha Greiner for her assistance as well as the organizers of The Life of a Process Model:
From Conception to Action Workshop for providing support to attend.
 173

7. LITERATURE CITED

Billings, S. A., 1980, "Identification of Nonlinear Systems - a Survey," lEEEProc, 127, pp.
272-285.

Eskinat, E., Johnson, S. H. and Luyben, W. L., 1991, "Use of Hammerstein Models in
Identification of Nonlinear Systems," ^ / C M J : , 37(2), pp. 255-268.

Haber, R. and H. Unbehauen, 1990, "Structure Identification of Nonlinear Dynamic Systems -

A Survey on Input/Output Approaches, Automatica, 26(4), pp. 651-677.

Montgomery, D. C , 1984, "Design and Analysis of Experiments, " John Wiley and Sons, New
York.

Pearson, R. K., and Ogunnaike, B. A., 1997, "Nonlinear Process Identification," Nonlinear
Process Control Prentice-Hall PTR, Upper Saddle River, NJ, pp. 11-110.

Rollins, D. K., P. Smith, and J. M. Liang, 1998, "Accurate Simplistic Predictive Modeling of
Non-linear Dynamic Processes," ISA Transactions, Vol. 36, pp. 293-303.

Rollins, D. K, N. Bhandari, A. M. Bassily, and G.M. Colver, 2002, "A Continuous-Time

Hammerstein Approach With Application to a Real Dynamic Batch Process," submitted to Ind.
Eng. Chem. Res.

Su, H. T. and McAvoy, T. J., 1993, "Integration of Multilayer Perceptron Networks and
Linear Dynamic Models: A Hammerstein Modeling Approach," Ind. Eng. Chem. Res., 32, pp.
1927-1936.