Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

Exercise 4 Linear System Identification Using Neural Networks OBJECTIVE

Artificial Neural Networks (ANN) are widely applied nowadays for classification, identification, control, diagnostics, recognition, etc. They can be implemented for identification of dynamic systems. The objective of this exercise is identification of a dynamic system based on its response to standard signals used in the case of deterministic, continuous, linear and stationary systems. We concentrate on linear and adaptive linear networks.

REQUIREMENTS
A. Prerequisite knowledge: You are expected to know the following topics: 1. Neuron model and transfer functions used in ANN, 2. Perceptron and network architectures, 3. Training and learning methods (including adaptive training), 4. ANN linear system design, 5. Linear classification. B. MATLAB/SIMULINK 1. Using Neural Network Toolbox under MATLAB. To learn more about ANN and its application using MATLAB, you are strongly encouraged to read Chapters 2, 3 and 4 of Neural Network Toolbox for Use with MATLAB, written by Howard Demuth and Mark Beale and published by The Mathworks, Inc. It is available online: http://www.mathworks.com/access/helpdesk/help/pdf_doc/nnet/nnet.pdf. The next very useful reference is Neural Network Design written by Martin Hagan (see: http://hagan.ecen.ceat.okstate.edu/nnd.html).

LAYOUT
Recall the dynamic system of the first exercise. You should make such a neural network that is able to identify your system. After identifying the delivered model by differential equation, state variables or transfer function should be substituted by the designed neural network. 1. Define the system which should be identified. 2. Define the input and output signals. Take the step response of the system as your target. 3. Design the neural network using two methods: Linear System Identification (LSI) and Adaptive Linear System Identification (ALSI). 4. Calculate the errors. 5. Modify and adjust your network.

MATLAB/SIMULINK
The following new MATLAB commands and functions are necessary to be understood and applied in this exercise: MATLAB 5 adaptwh delaysig errsurf initlin length MATLAB 7 nn2lin and adapt cell errsurf nn2lin and init length

29

Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

plotes plotep simulin solvelin sumsqr ----

plotes plotep nn2lin and sim newlind sumsqr nntool

REPORT
In addition to those points mentioned under Layout, your report should include: 1. Plots of all input and output signals (for system and network) 2. Plots of absolute and relative errors, 3. Investigation of learning rate influence on neuron response (for lr between 0.1 and 0.9) 4. Answers to the following questions: Which method gives a better solution? Why? Is there any other adaptive method? (Try to apply it).

EXAMPLE
Remark: MATLAB 5 commands and functions are used in the presented example. However, they can be easily adapted for MATLAB 7 using the table given under MATLAB/SIMULINK section of this exercise. Part One: Firstly, Linear System Identification (LSI) approach is presented under this example. Reconsider the presented example for Exercise 1. We take the output of that exercise as the input to the present exercise. Suppose that the system response to step function is given. Having the system response to step function we are able to train a linear neuron which models our system. In this case we do not need to introduce the transfer function/steady state representation of the system as it will be substituted by the linear neuron. Step 1: Defining the system should be identified: recalling the system of Exercise 1 we have: a=[ 0 1 ; -2 -0.5]; b=[0 ; 0.01]; c=[1 0]; d=0; sys=ss(a,b,c,d); Now, we define the system that should be identified: [x,time]=step(sys); T=x'; T defines the target which is system output signal. The input signal is step function: u(1:length(time))=1; The pattern, i.e. input to the neural network can be selected as the most recent three values of the system input signal: P=delaysig(u,0,2); Step 2: Choosing the weights and biases: to choose the weights and biases we have two ways: - plotting the error surface and contour:

30

Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

Using errsurf we can calculate errors for a neuron with a range of possible weight and bias values. Then by using plotes this error surface with a contour plot underneath will be plotted. The best weight and bias values are those that result in the lowest point on the error surface. rw=-1:0.1:1; % defines the range of weights rb=-1:0.1:1; % defines the range of biases es=errsurf(u,T,rw,rb,purelin); plotes(rw,rb,es);

- direct calculation of weight and bias that result the minimum error using solvelin: [w,b]=solvelin (P,T) w= 5.5285e-017 9.7570e-005 4.8942e-003 b= 0 Step 3: Testing the model: the model is tested by calculating the network error. For this purpose at first we use simulin to simulate the linear neuron which models the original system: resp=simulin(P,w,b); To show the results, the output signal can be plotted: plot(time,resp,time,T,r+); grid

31

Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

8 7 Network Output - System Output+ 6 5 4 3 2 1 0

x 10

-3

Network and System Output Signal

10 Time

15

20

25

Then the error signal is calculated: error=(100*(T-resp)./T); % relative error in % which can be also plotted to see the level of error signal:
Error Signal 200 0 -200 -400 -600 -800 -1000 -1200 -1400

Relative Error %

10 Time

15

20

25

Step 4: Solution: finally, the solution can be plotted on error surface by adding up the squared errors using sumsqr: sqe=sumsqr(T-resp); plotes(rw,rb,es) plotep(w(1,3),b,sqe)

32

Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

As it is visible, solvelin found the minimum error solution. Part Two: The second part presents the Adaptive Linear System Identification (ALSI). The linear neuron is adapted to changes in the model it is trying to predict. The following steps are taken into account: Step 1: Defining the system should be identified: the same as mentioned in Part One.
x 10
-3

Input Signal to System

8 7 6 5 Input Signal 4 3 2 1 0

10 Time

15

20

25

33

Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

8 7 6 5 4 3 2 1 0

x 10

-3

Output Signal of System

Output Signal

10 Time

15

20

25

Step 2: Defining the problem: Assume that the two most recent system inputs will be the inputs to the network as it tries to predict the system output T: P = delaysig(u,0,1); Step 3: Defining the network: initlin generates initial weights and biases for our neuron [w,b] = initlin(P,T); Step 4: Adapting the linear neuron: adaptwh simulates adaptive linear neurons. It takes initial weights and biases, an input signal and a target signal, and filters the signal adaptively. The output signal and the error signal are returned, along with new weights and biases. Let us use a learning rate equal 0.5: lr = 0.5; [resp,e,w,b] = adaptwh(w,b,P,T,lr); Now the output signal can be plotted with the target signal: plot(time,resp,time,T,'r') alabel('Time','Output (blue) Target (red)','Output and Target Signals') axis([0 10 0 1e-2])
Output and Target Signals 0.01 0.009 0.008 Output (blue) Target (red) 0.007 0.006 0.005 0.004 0.003 0.002 0.001 0 0 1 2 3 4 5 Time 6 7 8 9 10

34

Gdansk University of Technology, Faculty of Ocean Engineering and Ship Technology

Modeling and Control of Dynamic Systems, Laboratory, M. H. Ghaemi

Step 5: Calculating the error: A plot of the difference between the neurons output signal and the target signal shows how well the adaptive neuron works. plot(time,error,[min(time) max(time)],[0 0],':r') alabel('Time','Error','Error Signal') axis([0 15 0 1e-2])
Error Signal 0.01 0.009 0.008 0.007 0.006 Error 0.005 0.004 0.003 0.002 0.001 0 0 5 Time 10 15

Obviously, the adaptive method gives here more suitable solution with less error value. It is possible to calculate the energy of error in both cases to compare the results and selecting the proper network. Comment: The presented example has been prepared for running under MATLAB 5. If you use MATLAB 6 or 7, please use newlind and sim instead of solvelin and simulin, respectively, in the case of LSI. For ALSI application use newlin and adapt instead of initlin and adaptwh. You can also refer to the following m.files: Method LSI ALSI MATLAB 5 Ex4_1.m and Ex4_1_1.m Ex4_2.m MATLAB 7 Ex_4_1.m Ex_4_20.m (low level programming) and Ex_4_21.m (advanced level programming)

Additional work for willing students: Try to use a different step input signal and see how your defined ANN-based model responses to this new input. Make the same simulation using for example transfer function or steady space model. Then compare these two results and investigate how much your identified ANN-based model is reliable.

***

35