Time Series Prediction of Earthquake Input by using Soft Computing

Hitoshi FURUTA, Yasutoshi NOMURA Department of Informatics, Kansai University, Takatsuki, Osaka569-1095, Japan nomura@sc.kutc.kansai-u.ac.jp

Abstract
Time series analysis is one of important issues in science, engineering, and so on. Up to the present statistical methods[1] such as AR model[2] and Kalman filter[3] have been successfully applied, however, those statistical methods may have problems for solving highly nonlinear problems. In this paper, an attempt is made to develop practical methods of nonlinear time series by introducing such Soft Computing techniques[4][5][6] as Chaos theory[7], Neural Network[8][9], GMDH[10][11] and fuzzy modeling[12][13]. Using the earthquake input record obtained in Hyogo, the applicability and accuracy of the proposed methods are discussed with a comparison of those results.

In this paper, an attempt is made to develop practical prediction methods of earthquake input, which behaves irregularly time to time, by introducing such so-called soft computing techniques as Chaos theory (Ito,1993[7], Takens, 1981[14][15][16], Iokibe, 1994[17], Sakawa at all, 1998[18]) Neural Network (Chen at all, 1989[8], Funabashi, 1992[9]) and GMDH (Group Method of Data Handling)(Ivakhenemko, 1968[10], Hayashi, 1985[11]). Many researches have revealed that the Chaos theory is useful in dealing with complex systems, Neural Network is applicable to various problems like pattern recognition and function approximation, and GMDH can analyze highly nonlinear systems which have a few input and output variables. Numerical examples are presented to illustrate the applicability of the proposed methods, and to compare the characteristics of those methods.

1.Introduction
In this study, the prediction of external force such as earthquakes and wind loads is employed to discuss the accuracy and efficiency of the prediction methods, because of the importance of its prediction from the engineering point of view. In these days, monitoring and controlling play important roles to reduce the vibration of high-rise buildings due to earthquakes and wind loads. As buildings are getting higher, the vibration of high-rise buildings due to earthquakes and winds becomes a subject of discussion. At present, many high-rise buildings have vibration control systems on their own. However, the vibration control system works using measured earthquake input and acceleration. On the other hand, traffic flows on such bridges as Amarube Bridge, Kansai International Airport Bridge and Akashi Strait Bridge under strong winds are controlled with the intensity of earthquake input through measuring. Then, unsuitable control may be done due to the effects of time lag between real and measured wind velocities. In order to solve these problems, it is desirable not only to achieve wind-resistant structure of buildings and bridges but also to establish a practical method of predicting the earthquake input and wind loads.

2.Time Series Prediction by Chaos Theory

The definition of Chaos is done by several researchers, and generally speaking, Chaos is the phenomenon which is non-periodic vibration governed by a deterministic system. The deterministic system means the system governed by a definite constant rule. And the non-periodic vibration means the movement which entirely acts randomly. Thus, deterministic Chaos is considered as a phenomenon which behaves irregularly at a glance, but is governed by a definite deterministic rule. Orbital instability is a characteristic caused by the sensitivity to the initial state that two very near points in the state space become a long way off when the steps proceed. Lyapunov exponent is used to distinguish whether the time series are chaotic or not. Lyapunov exponent expresses the leaving velocity of the two orbits from near points time to time. If Lyapunov exponent is positive, then the behavior may be chaotic, else if it is negative, then it is not chaotic. Due to the characteristic, the Chaos theory can not be applied to long-term prediction, but is suitable for short-term prediction. There are some cases which behave chaotic; one is such a behavior as logistic mapping problem which is governed

Proceedings of the Fourth International Symposium on Uncertainty Modeling and Analysis (ISUMA03) 0-7695-1997-0/03 $17.00 2003 IEEE

by a clear recursive formula, and the others are the demands of water or traffic flows which are social problems. In order to provide us with useful information, it is desirable to predict it, and some researches have been made so far. It is necessary to discretize time series data at fixed sampling interval. If the data sampled is chaotic, the behavior is regard to be governed by a deterministic rule. Then, if a non-linear deterministic rule is estimated, it is able to predict the data of near future until the deterministic rule does not work due to the sensitivity to the initial state. This process is the most important to estimate the dynamics. It is needed to reconstruct time series data in n dimension state space with the embedding theorem of Takens (Takens, 1981[14][15][16]) and make an orbit. This orbit is called attractor, and the attractor which appears when the behavior is chaotic is especially called strange attractor. Then, the objective data vectors are predicted by a local reconstruction method with the nearest data vector plotted and the later data vectors. There are some methods of reconstruction such as the tethelation method, the local fuzzy reconstruction method and so on. And the objective time series data is searched with the transformation of the estimated data vector. When a time series shows a chaotic behavior, it may follow a deterministic rule. Then, it is possible to predict the near future behavior of the time series with the aid of Chaos theory. Here, it is attempted to predict the wind velocity given by a time series record, using Local Fuzzy Reconstruction Method (LFRM) (Iokibe at all, 1994[17]) and Deterministic Nonlinear Prediction Method using Neighborhoods Difference (NDM) (Sakawa at all, 1998[18]) The prediction process is performed as follows: At first, the embedding of the time series data is done by using the Takens theorem. The time lag is calculated according to the embedding results. The dimension of the state space is determined as the dimension showing the most strong chaotic behavior. Figure 1 shows the earthquake acceleration. Figure 2 shows the attractors obtained by changing the time lag. Figure 3 shows the variation of the Lyapunov exponent analysis according to the change of embedding dimension. Based upon those results, the optimal parameters can be obtained as shown in Table 1. In this study, prediction parameter is established in Table 1. Figure 4 and 5 present the results calculated by NDM and LFRM respectively. Although the both methods provide satisfactory results with good accuracy, NDM can

present slightly better agreement with the observed data in this example. Prediction start point is 100 step and Prediction end point is 5000 step.

Figure 1. Earthquake acceleration

Figure 2. Results of Embedding by Takens Theorem

Figure 3. Results of Lyapunov exponent analysis

Proceedings of the Fourth International Symposium on Uncertainty Modeling and Analysis (ISUMA03) 0-7695-1997-0/03 $17.00 2003 IEEE

Table 1. Prediction parameters of chaotic analysis Dimension 2 Neighborhoods vector 8 Time lag 1

multi-layer Neural Network methods for the time series prediction. As a learning method, the back-propagation method (Chen at all 1989, Funabashi, 1992) is employed, whose parameters are given in Table 2. Figureureure. 6 shows the convergence of the learning process. From Figureureure. 7, it can be seen that the Neural Network can provide a rather good prediction. However, comparing with the Chaos theory, the prediction of the Neural Network method still has such problems that the accuracy depends on the number of layer and node and the learning process. Table 4. Parameter of Neural Network Input Unit 4 Hidden Unit 5 Output Unit 1 Learning Coefficient 0.4 Momentum Coefficient 0.6 Slope of Sigmoid Function 1

Figure 4. Prediction results by NDM Table 2. Prediction Results by NDM Correlation Coeff Initialized MSE 0.97645 0.21644

Figure 6. Convergence of Learning Process

Figure 5. Prediction results by LFRM Table 3. Prediction Results by LFRM Correlation Coeff Initialized MSE 0.96729 0.25382

3.Time Series Prediction by Neural Network

Neural Network can be applied to various problems such as pattern recognition, approximation of function, classification, etc. Here, it is attempted to apply a Figure 7. Prediction results of Neural Network Table 5. Prediction Results by Neural Network

Proceedings of the Fourth International Symposium on Uncertainty Modeling and Analysis (ISUMA03) 0-7695-1997-0/03 $17.00 2003 IEEE

Correlation Coeff 0.96925

Initialized MSE 0.30292

If Inequality 4 is not satisfied, it is needed to calculate z with substitution input data for Eq. 2. Then, z is regarded as the input data for the next layer. And the operation is repeated until the termination condition is satisfied. If the algorithm is stopped, intermidiate parameter of previous layer is substituted into the expression at the layer where E* is gained. The estimated model is established with repeating this operation to the first layer. Figureureure. 8 presents the prediction results obtained by GMDH. This shows a good agreement. In this example, the calculation terminated at the first layer, because it satisfies the termination condition. Eq. 5 presents the quadratic function obtained through the method.

4. Time Series Prediction by GMDH

GMDH (Group Method of Data Handling) theory was developed by Ivakhnemko to analyze a nonlinear problem by introducing quadratic functions with the principle of Heuristic Self-Organization (Ivakhnemko, 1968, Hayashi, 1995) GMDH is regarded as one of the methods of Neural Network, but GMDH can deal with highly nonlinear complicated problems by a layer model of quadratic functions. In GMDH, multi-layer model is used, which is the most distinctive characteristic. The success of the model is dependent on the structure of multi-layer. At first, it is necessary to divide input and output variables into training data and checking data. Training data are used to analyze the model, and checking data are used to check the analyzed model. When there are N input and output data, Nt training and Nc checking data, it must be that the total of Nt and N is N. A quadratic expression is constructed with any two of n input data and output data, for variables of xp, xq and y (Eq. 1).

a bx p

cx q

dx 2 p

2 ex q

fx p x q

(1) Figureureureure 8. Prediction results by GMDH

where a, b, c, d ,e and f are constant coefficients. The six coefficientsa through f are estimated with training data by a linear recurrent analysis. Intermediate parameter Z is calculated with the substitution of estimated coefficients and input data of checking data into Eq. 2. Table 6. Prediction Results by GMDH Correlation Coeff Initialized MSE 0.96875 0.29738
y 0.637718 0.849673x n 0.001537 x
2 n 2 2

a bx p

cx q

dx 2 p

2 ex q

fx p x q

(2)

1.584467 x n
2 n 1

1 1

(5)

0.001386 x

0.000074 x n 2 x n

Then, square errors E are calculated as

E ( y z) 2

(3)

Square errors calculated for n checking data are sorted from small to large, and expressed as E1 to En(n-1)/2. If E1 and E* satisfy Inequality 4, the algorithm was terrminated. E* is the E1 obtained at the previous layer. E1>E* (4)

5.Comparison of Chaos, Neural Network, GMDH and Fuzzy modeling for Time Series Prediction
Table 8 presents the correlation coefficients and mean square errors obtained for the five methods. NDM Chaos method can provide the best prediction, namely, its errors of 1 % occurred 1321 cases in 4900 cases. Successively, LFRM Chaos method provides satisfactory results with 1% error for 1129 cases. Fuzzy modeling provides satisfactory results with 1% error for 1098 cases in 4900

Proceedings of the Fourth International Symposium on Uncertainty Modeling and Analysis (ISUMA03) 0-7695-1997-0/03 $17.00 2003 IEEE

cases. On the other hand, GMDH and Neural Network provide satisfactory results with 1 % error for 36, 79 cases in 4900 cases. As a result, it is concluded that the prediction methods based on chaotic time series analysis can provide enough results for the prediction of earthquake inputs.

NDM LFRM Neural Network GMDH Fuzzy Modeling

Correlation Coeff Initialized MSE 0.97645 0.21644 0.96729 0.25382 0.96925 0.30292 0.96875 0.29738 0.97291 0.23123

6.Conclusions
In this paper, the applicability and efficiency of the Chaos Prediction methods, Neural Network and GMDH were discussed and examined using the time series data of earthquake acceleration. Neural Network and GMDH methods could not present good results for such data with highly chaotic characteristics such as Logistic mapping. However, it requires the shortest time to predict the time series data, once the learning could be done. This is one of advantages of Neural Network from a practical point of view. In the development of predicting method of earthquake acceleration, the Chaotic Prediction methods and Fuzzy modeling are useful in short-term prediction. If a more long-term prediction is required, other methods may be superior, though the Chaotic Prediction methods can provide more accurate solutions for the short-term prediction. In this study, Chaotic Prediction methods, Neural Network, GMDH and Fuzzy modeling were compared and discussed with emphasis on accuracy and computing time. However, instead of using them independently, it is desirable to combine those methods to compensate their own defects. Table 7. Prediction error NDM LFRM NN GMDH 777 1035 2979 4418 97 140 30 25 121 174 37 17 163 238 26 13 272 403 27 19 497 586 41 26 824 543 38 10 556 400 31 6 345 263 32 19 205 198 25 12 184 114 27 11 859 806 1607 324 1321 1129 79 36

References
[1] T. Ozaki, H. Akaike, G. Akaike, Time Series, Asakura syoten, Japan, 1998. [2] T.Ozaki, G. Akaike, Methods of Time Series Analysis, Asakura syoten, Japan, 1998. [3] S, Arimoto, Kalman Filter, Asakura syoten, Japan, 1985. [4] L. A. Zadeh, Fuzzy Sets, Information and Control, Vol. 8, pp.338-353, 1965. [5] L. A. Zadeh, Outline of a new approach to the analysis of complex systems and decision processes, IEEE Trans. on System, Man and Cybernetics, Vol.SMC-3, 1973, pp.28-44. [6] L. A. Zadeh, Application of Fuzzy Technique and Soft Computing, Journal of The Japan Society for Fuzzy Theory and Systems, Vol. 5, No.2, 1993, pp.261-268. [7] K. Ito, What Is Chaos? Predict Unpredictable Chaos Diamond-sya, Japan, 1993. [8] Chen.S and Billings.S.A, Representations of non-linear systems the NARMAX, model-Int.J.Control, Vol.49, No.3, 1989, pp.1013-1032. [9] M. Funabashi, Neuro Computing, Ohmu-sya, Japan, 1992. [10] Ivakhenemko, A. G., The Group Method of Data Handling, A Rival of the Method Stochastic Approximation, Soviet Automatic Control, 13(3), 1968, pp.43-45. [11] I. Hayashi, GMDH, Journal of The Japan Society for Fuzzy Theory and Systems, Vol.7, No.2, Japan, 1995, pp.270-274. [12] H. Nomura, I. Hayashi and N. Wakami, A learning method of fuzzy inference rules by descent method, Proceeding of the 4th IFSA Congress, Vol. Engineering, Brussels, 1991, pp155-158. [13] Y. Shi, M. Mizumoto, N. Yubazaki and M. Otani, A Self-Tuning Method of Fuzzy Rules Based on Gradient Descent Methods, Journal of The Japan Society for Fuzzy Theory and Systems, Vol.8, No.4, Japan, 1996, pp.757-767. ,

Err<-5% -5%<Err<-4% -4%<Err<-3% -3%<Err<-2% -2%<Err<-1% -1%<Err<0% 0%<Err<1% 1%<Err<2% 2%<Err<3% 3%<Err<4% 4%<Err<5% 5%<Err -1%<Err<1%

Fuzzy 1136 124 150 217 304 510 588 329 215 184 166 977 1098

Table 8. Comparison of prediction results by All methods

Proceedings of the Fourth International Symposium on Uncertainty Modeling and Analysis (ISUMA03) 0-7695-1997-0/03 $17.00 2003 IEEE

[14] Takens. F(1981), In Dynamical Systems and Turbulence, Springer, Berlin, pp.366-381 [15] Takens, F.: Detecting Strange Attractors in Turbulence, in Dynamical Systems and Turbulence, Lecture Notes in Mathematics, Vol.898, pp.366-381, Springer, 1981. [16] M, Sano, Y, Sawada, Measurement of the Lyapunov spectrum from a chaotic time series, Phys. Rev. Lett., Vol.55, No.10, 1985, pp.1082-1085.

[17] T. Iokibe, M.Kanke, Y. Fujimoto, and S. Suzuki, Short-term Prediction on Chaotic Timeseries by Local Fuzzy Reconstruction Method, Proceeding of Brazil-Japan Joint Symposium on Fuzzy Systems, Japan, 1994, pp.136-139. [18] M. Sakawa, K. Kato and K. Ooura, A Deterministic Nonlinear Prediction Method through Fuzzy Reasoning Using Neighborhoods Difference and Its Application to Actual Time Series Data, Journal of The Japan Society for Fuzzy Theory and Systems, Japan, 1998,Vol.10, No.2, pp381-386.

Proceedings of the Fourth International Symposium on Uncertainty Modeling and Analysis (ISUMA03) 0-7695-1997-0/03 $17.00 2003 IEEE