NEURAL NETWORK PROCESS CONTROL

Michael J. Piovoso and Aaron J. Owens

E.I. du Pent de Nemours 8Z Co.

Allen Guez
Drexel University

Eva Nilssen
University of Delaware

Abstract. Neural Networks are increasingly incorporate neural networks are referred to as
finding engineering applications. Most early neuro-controllers [Guez et al.].
applications were in the areas of pattern
recognition and modeling. This paper shows The class of neurocontrollers is closely
how neural network models can be used in related to other classical forms of control. We
process control. Two separate techniques are first compare it to the adaptive and learning
illustrated, each with a specific example controllers and demonstrate their similarities.
application. One involves using the network
itself as the inverse model, by fixing the neural The problem of systems control can be
network weights and training on the inputs to defined as follows. Given the system S where
give the desired output pattern. The other
suggests using the pattern recognition ability of
a neural network to identify an appropriate S:x= f(x,u,t,p); X(to) = Xo; Y - 9(XNAP) (1.1)
lower order linear model to use for controller
design.

1. INTRODUCTION and where the time t & R. The initial time is %,

A neural network is an interconnected the state x & Sx & Rn, the control input u &
set of many neuron-like subsystems that are
dynamically coupled and exhibit through their Su & Rm, and the parameters vector p 8 Sp &
collective behavior some useful computational
features. These networks were extensively R]. The sets Sx, Su and Sp are admissible
studied by many researchers over the past 40 state, control and parameter sets respectively,
years. Of the many different neural network which account for general equality and
models, we will focus on feed-forward inequality constraints, and where n, m and 1 are
networks in which training is done using the appropriate integers. The control problem is to
back-propagation paradigm (BPN). Figure 1 find the system input; u(t) = c[v(t) ,z,z] for
illustrates this network, which is particulady
useful for model development [McClelland and open-loop control and U(t) = C[v(t) ,X ,z,z]
Rummelhart], [Piovoso and Owens]. for closed-loop control, such that certain
terminal and optimality conditions for the
Linear dynamic models are the basis of output y are satisfied. The signal v(t) is the
most modern control methods such as adaptive reference input vector, and z(t) is a vector
control, Dynamic Matrix Control [Cutler and describing the controller state variables.
Rarnaker] and Internal Model Control [Garcia
and Morari]. Such dynamic models are used to Under the above general definition, it is
predict system performance and to compare that easy to classify the various controller classes
performance with actual process outputs. We according to the type of control function c that
suggest the use of neural models in place of the they are able to provide. If c is independent of z
linear dynamic models. Control schemes that and Z, it defines the class of static controllers.
Permission to copy without fee alt or part of this material is granted
provided that the copies are not made or distributed for direct
commercial advantage, the ACM copyright notice and the title of the
publi~tion and its date appear, and notice is given that co ying is by
percussion of the Association for Computirrg Machinery. $ o copy 84
otherwise, or to republish requires a fee and/or specifk permission.

@ 1991 ACM 089791-432-5/91/0005/0084 $1.50

In particular, if c is also linear in x and v, we w,, in the sense usually associated with
obtain the class of linear state feedback adaptive controllers. Also, since the learning
controllers. When c is a function of z and Z, algorithm employed by neurocontrollers does
we obtain a dynamic controller, where z(t) not require periodic input, but rather can
accommodate any temporal behavior, the class
describes the controller’s state variables. If z is
of learning controllers is a proper subset of the
a function of all or some of the system
neurocontroller class.
parameters, p, we obtain the class of adaptive
controllers. (As there is no unique definition of
Although neurocontrollers are different
adaptive control { [Astrom], ~andau] ), the one
from the model based adaptive controllers
given above represents the authors’ choice.) If
described above, they do perform real time
z(t) is related to the states of the inverse of the
adaptation through their on-line learning
plant transfer function, then an Internal Model
capabilities. The robustness of neurocontrollers
Controlla or derivative thereof is formed.
stems from the fact that they are not model
based. It is expected, however, that when
An important feature of most adaptive
compared with classical controllers with the
controllers, including Model Reference
same throughput (processing power) their
Adaptive Controllers (MRAC), Self Tuning
accuracy may be inferior.
Regulators (STR) and Gain Schedulers (GS),
is that they are pmle 1 based . The control law c
Finally when some or all the nodes of
varies according to the best estimate of some
the neural network employed in the
system model parameters, p. Usually an
neurocontroller are specifically allocated for the
estimation scheme for some or all of the system
estimation and storage of all or a subset of the
parameters is employed ([Astrom], [Landau]).
system parameters, p, we obtain a neuro-
The parameter estimates are used to modify the
morphic realization of model-based adaptive
$tructu re of the control law (c) in an on-line controllers. In that sense the set of adaptive
fashion, automating in the process some
controllers may also be regarded as being
control design rules. A general block diagram
contained in the class of neurocontrollers.
is depicted in Figure 2.
Internal model control (IMC) [Garcia
In contrast is the class of learning
and Morari] is also a model based control
Controller, ([Arimoto et al.] [Bondi et al.],
scheme. Here a model of the process is
[Kawamura et al.]) also called repetitive
developed that gives a prediction of the output.
control, or betterment control, which should be
The controller Q(s) is the model inverse (F@re
distinguished from learning automata as
4). Unfortunately, for most problems of
described in [Narendra and Thatachar],
practical interest the model inverse is not
[Tsypkin], [Fu]. For a learning controller,
physically realizable. For example, if the
typically it is assumed that the reference signal process has a time delay, the inverse model
v(t) is periodic, the structure of the controller
would require a knowledge of the future input.
(i.e., the function c) is fixed, and learning is
Any system that has non-minimum phase
exhibited through the Iterative modification of
characteristics would have either an unstable or
the (open loop) Jime function u(t). A schematic
non-realizable inverse. The controller is chosen
block diagram is given in Figure 3.
to give the minimum squared error over the
class of linear controllers. However, such a
We define the class of neurocontrollers
design is not robust in the sense that errors in
as the set of all controllers whose
the assumed parameters may cause the control
structure (control law) c(x,v,z,Z) is to deteriorate significantly or possibly to
based on some neural network model become unstable. To overcome this problem, a
architecture and learning paradigm filter Fe(s) is added to the model inverse so that
[Guez and Kam]. It should be emphasized that the filter plus optimal controller has robust
the neural network model and the learning behavior in the face of model uncertainties. The
algorithm are defined independently from any models typically used in IMC are linear
specific process to be controlled (S). approximations of the typically nonlinear real
Neurocontrollers are, therefore, not model

85
system. Thus, a serious drawback occurs when
the process changes operating conditions. The
parameters of the linear approximation at the
previous state are often inappropriate in the
new state. The resulting control, if stable, is
still less than optimal. A neural net model, on
the other hand, captures the nonlinearities of
the process and should be valid over a larger
operating range than the linear IMC.
Neural networks can also be used to
generate models for doing process control.
Like IMC, the neural network must be, in some
sense, inverted to obtain the proper control
inputs to drive the output to desired result. A
conceptual problem exists with using a neural
net for the model of the process: since this form
of the model is nonlinear, the method for
generating the inverse (as done in IMC) is not
obvious. One approach would be to design a
neural network that would predict the system
input from measurement of the system output.
(I.e., have the neural network model the
inverse problem.) Care must be taken because
it is again the problem of modeling the inverse.
The system response time must be considered
(see below). Rather than trying to predict the
input needed to drive the process output error
to zero at the earliest possible time, one would
use an input to the system that would cause the
error to decrease in an acceptable way. The
dynamics for returning the setpoint plays the
role of the fflter in the IMC approach.
would have given a similar set of such
characteristics. The controller implemented
would then be one designed to control that
second order system. In this approach, the
neural network is used only for identifying the
second-order system needed to retune the
controller and not for continuous operation.
When the controller needs retuning, a step test
would be conducted and from that data, a new
second-order model defined. From this model,
a new controller is designed.
2. IMC
In Figure 4, P(s) is the process transfer
function, Q(s) is the controller transfer
function, and ~(s) is the transfer function of the
model. Disturbances enter the system at the
output of the plant. The closed-loop transfer
function between the reference input r and the
controlled output y of this system can be
shown to be:
P(S)Q(S)
(2.1)
- F(S))
I+ Q(s)(P(s)
The transfer function between the disturbance d
at the output and controlled output y is:

A technique for choosing the input

setpoint is to use the trained BPN model of the
forward process and a variant of the
l-~(s)Q(s)
backpropagation algorithm. Suppose that the (2.2)
forward process has been modeled by the 1 +Q(s)(P(s) - F(s))
network its weights have been fixed. The
inputs are manipulated to minimize the error
between the predicted response and the desired This controller is guaranteed to give zero
response. The gives the input setpoint to steadv-state error to step inputs if the product
produce the desired response. of th~ DC gains of the ~on~oller Q(s) “find the

Another method for using neural model of the process P(s) is forced to be 1.
networks to do control is by using their pattern
recognition capability. Suppose that the system Note also that if P(s) is a perfect model of
to be controlled is modeled as a second order P(s) and Q(s) is the perfect inverse of ;(S)
system. Apply a step input and monitor the
then the closed-loop transfer function of (2.1)
response. Characteristics of that response such is one and the disturbance transfer function of
as rise time, settling time and peak overshoot (2.2) is zero.
can be noted. This data so generated can be
associated with a second-order system that

86
 Because of the nature of physically and past plant inputs to predict the inverse
realizable system, the inverse of the plant P(s) model or the pnxent plant input. Such training
is seldom, if ever, realizable. Thus, the could take place either on-line or off-line.
inverse Q(s) is generally not the true inverse of
the plant. Sometimes Q(s) can be made This controller model would have all
physically realizable by adding a filter Fe(s) to the same problems as the linear, time-invariant
the true non-physically realizable inverse to model concerning realizability and operation.
generate a realizable one. For example, the For example, if the plant has a time delay of 3
system seconds, then the controller inverse should be
choosing inputs for the present time that will be
effecting the output 3 seconds later in time.
Also, to give robustness to the model, a
filtering function ought to be added to neural
controller output. This will allow good control
P(s) = -& (2.3) despite dead time drift and plant parameter
uncertainty.
is not physically realizable. By adding a filter
Fe(s) that is a simple RC integrator, the
3. NEURAL NETWORK CONTROL
combined idealized inverse plus the filter is
Many systems encountered in practice
tdizable.
can be modeled very accurately as a second-
order system with dead time. This model of
the system can then be used to design the
controller needed. Such practices are quite
common in industry and lead most often to
satisfactory controller design. The modeling
can be done using a step test or other signal
Q(s) = ~ (2.4)
testing method followed by a statistical estimate
of the model parameters ~jung].
Note that the closed system transfer function
will now be Fe(s) in this idealized case, as Neural Networks are good ways of
opposed to the value 1 for the case in which the modeling such a system. The models
plant inverse exists. developed are often determined by a feed-
forward, three-layered BPN network. These
The implementation of this technique networks can be trained in a number of ways.
would generally be done digitally. The One could develop the model in an analogous
controller and the plant models are digital fashion to the techniques of time series
approximations of the continuous-time analysis. Unfortunately, this can be a difficult
systems. The model is generally a linear way to train the network if the dynamics of the
approximation of the actual nonlinear, typically system are complex. The differing speeds of
time-delayed, system. From this linear response and differing gains make the identifi-
approximation, a controller is design to meet cation and training difficult. The heat
the constraints of realizability and robustness. exchanger example in Section 4 below shows
There is no reason to restrict consideration to a how the BPN can be used as a generalized
linear time-invariant model of the process. It model of time-series analysis.
could be a nonlinear, neural net model. The
controller could be a nonlinear neural net On the other hand, defining certain
system as well. The plant model would be features of the step response that define the
generated by taking present and past inputs and gains and speeds of response for the various
past outputs to predict the present inputs. The systems can produce more desirable results.
neural model would be a nonlinear These features include rise times, time to peak,
approximation of the real world system. The percent overshoot, and settling times. These
controller model would take past plant outputs features are easily measurable in a simple step
test. They are then applied to a network that

87
produces the approximate second-order model.
This model is then used for doing a pole-
placement design.
The pole-placement method is a design
technique where the user defines the desired
location of the closed-loop poles (equivalently
the desired closed-loop behavior). From this
information and the open-loop dynamics, a
controller is designed consisting of a feed-
forward component from the reference input
and a feedback component from the controlled
output [Astrom and Wittenmark].
A control system designed in this way
can be retuned upon demand. When retuning
is necessary, put the system into manual, input
a step disturbance, measure the performance
parameters, input them to the neural network to
determine the second-order model, and design
a pole-placement controller. Such an approach
can guarantee a very well- controlled response
without the problems associated with on-line
parameter identification for doing adaptive
control.
4. EXAMPLES
As an example of capturing the
dynamics of the system directly with a trained
BPN model, consider the heat exchanger
shown in Figure 5. Heat is applied by a
resistive heater. The control objective is to
adjust the voltage to the resistive heater to
maintain the outlet temperature constant. This
system is nonlinear and defined by the
differential equation:
dy
—.. ~F(t)y (t) i&(t) T(t)+ -&*(t) (4.1)
dt V
The differential euuation was exercised for ste~
changes in the i~puts (F, T, and u), and th~
results were used to train a neural network in
the forward direction. That is, given present
88
and past values of u, F, T, and y, predict the
future value of y.
For the backward or inverse problem,
you are given a desired output at the next time
step along with present and past values of y(t)
and the control variables (u, F, T). With fixed
weights and variable inputs, the neural network
is trained (on the inputs) to find the input set
needed now to produce the desired y(t+l) at the
next time step.
An internal model controller was used
to control this system. The desired output was
the output temperature value y desired. No
filter was used; that is, Fe(s) was 1. The
forward model was used to predict the plant
output. This predicted value was subtracted
from the desired output to produce an error
signal that was then fed back to network, and
the input was changed to minimize this error.
Because no filter was used, the input change
generated could be greater than is acceptable.
Inputs that we~ too large were clipped at their
maximum value. This produces a bang-bang
like control actual that is shown, for single-
variable (u) control, in figure 6. The actual
response of the closed-loop system (y) is given
in figure 7.
As a second example, consider the use
of a neural network for system identification.
A neural network was trained to output the
damping ratio, natural frequency and gain of a
second-order system. The inputs to the net
were several features that can be easily
measured off a step test: peak overshoot, time
to peak, 10-90% and 0-50% rise times, and
settling time. From this data set, generated
with many second order systems, the BPN was
trained to predict the parameters associated with
the time-series features. The technique was
applied to a high order system with the transfer
function given by:
G(s)= 75000
(4.2)
S5 + 162s4 + 6835ss + 65400s2 + 197500s + 75000
The neural network identified a second-order Astrom, K., J. (1988) “Adaptive Feedback
system with natural frequency of 1.27 and Control,” Proceedings of the IEEE, vol. 75,
damping ratio of 1.420 Thestep response of No. 2, pp. 185-217.
the second-order system and of the system
defined by equation 3.2 are shown in figure 8. Astrom, K., J., Wittenmark, B. (1990)
The match is quite acceptable. A controller was Comrmter-Co ntrolled Svstems. Prentice Hall,
designed using a pole placement approach. Englewood Cliffs, New Jersey.
The closed-loop poles were chosen so that the
closed-loop, reduced-order system would be Bondi, P., Casalino, G. and Gambardella, L.
optimally damped. Figure 9 shows the step (1988) “On the Iterative Learning Control
response of the system given by equation Theory,” IEEE J. Rob. and Auto., Vol. 4,
(3.2), being controlled by the controller No. 1.
designed for the second-order system. The
results are essentially indistinguishable from Cutler, C., R. and Ramaker, B. L. (1979)
those that would have been obtained if the “Dynamic Matrix Control - A Computer
system were actually the second-order approxi- Control Algorithm,” AICHE National Meeting,
mation. Figure 10 illustrates the control effort Houston, TX.
to achieve the step nxponse shown in Figure 9.
Fu, K (1987) “Learning Control Systems -
5. CONCLUSIONS Review and Outlook,” IEEE Trans. Automatic
Control, AC-15, pp. 210-221.
Neural networks have become an
increasing popular technique for modeling Garcia, C., E. and Morari, M. (1982) “Internal
problems. Models can be generated without a Model Control - 1. A Unifying Review and
priori assumptions of the model form. These Some New Results,” Int. Eng. Chem. Process
models can then be used in various model- Des & Dev., Vol. 21, pp 308-323.
based process control techniques. In
particular, internal model control (made popular Guez A., Elbert, J. and Kam, M. (1988)
by Morari) is well-suited for implementation “Neuromorphic Architecture for Control”,
using neural networks. Both the process IEEE Control System Magazine, Vol. 8, No.
model and the inverse process model can be 2.
learned by a neural network. The primary
advantage is that a complex process can be Guez, A. and Kam, M. (1988)
modeled by the neural network without an “Neurocomputing Architectures for Intelligent
exact knowledge of the form of the Control,” A tutorial, Third IEEE International
nonlinearities. An application to a nonlinear Symposium on Intelligent Control, Alexandria,
heat exchanger demonstrates the use of this use VA.
of this technique. A second example shows
how a neural network can be used to identify Kawamura, S., Miyazaki, F. and Arimoto, S. (
second-order model parameters to be used in 1984) “Iterative Learning Control for Robotic
controller design. Systems,” Proc. of IECON ’84, Tokyo, Japan.

Landau, Y., D. (1979) Adaptive Control - ~

Model Reference Adamive Control ADproac~,
References Marcel Dekker, New York.

Arimoto, S., Kawarnura, S., Miyazaki, F. and McClelland, J. L. and Rumelhart, D. E. (1986)
Tamaki. S. (1985) “Learning Control Theory Parallel Distributed Process ing, MIT Press,
for Dynamical Systems,” Proc. IEEE 24th Cambridge, MA.
CDC, Ft. Lauderdale, FL.
Narendra, K., S. and Thatachar, M., A., L.
(1974) “Learning Automata - A Survey,” IEEE
Trans. Syst., Man and Cybern., Vol SMC -4,
pp. 323-334.

89
Piovoso M. J. and Owens, A. J. (1991)
“Sensor Data Analysis Using Neural
Networks, ” Proceedings of the Fourth
International Conference on Chemical Process
Control, South Padre Island, TX.

Tyspkin, Z. (1971) Adamation and Learning in

Utomatic vste ms. Academic Press, New
York.

Hidden Nodes

3 2 1

o
0
●

inputs ● outputs
●

w i(3) ,j(2)

FIGURE 1

90
 u(t) x(t)
+ SYSTEM S

I I I

w
v(t)

Parameter estimates

ADAPTION
MECHANISM 4
*

FIGURE 2

Uk
MEMORY SYSTEM S ‘k

A
+
uk+l
BETTERMENT
PROCESS *
+

desired response

MEMORY
f

FIGURE 3

-&
1-Q(s)

w% Fe(s) P(s)
+

FIGURE 4

91
 Heat Exchanger

Volume V

I I
Incoming Fluid ~ R ~ mow Rate F(t)
Temperature T(t) Temperature y(t)

Vokage u(t)

dy/dt = -(IN) F(t) y(t)+ (I/V) F(t) T(t)+ (URCV) W**2

FIGURE 5

Control Strategy
Heat Exchanger Example

State

Time

FIGURE 6

9.2
 Dynamical Control
Heat Exchanger Example

0.3s

I Temperature

0.26
.
=al
~.
:: OM
~ki
*a I
;E
;: I
0.32
Initial
Temperature

0.30 20
0 10

Time

FIGURE 7

Tkne

FIGURE 8

Step responseof cosmolkd system

1.2

1-

0.8 -
a)
%!
~ 0.6 -
$
0.4 -

0.2 -

FIGURE 9

93
 Corrlmller Output
2

1.8 -

1.6 -

1.4

1.2

1-

0.8 L
o 2 4 6 8 10
time

FIGURE 10

!34