Constraints in Prediction of Inflow to Reservoir Using Multi Layer Perceptron Artificial Neural Network

M. U. Kale Assistant Professor Department of Irrigation & Drainage Engineering, Dr. P. D. K.V., Akola

For prediction of inflow to reservoir, models such as

Knowledge driven models
Requires a huge data regarding underlying physical process e.g. conceptual and deterministic

Data driven models

do not provide any information on the physics of the hydrologic processes.

e.g. artificial neural network and fuzzy logic

Artificial Neural Network - emerging and challenging computational

technology. Advantages over conventional modeling ability to handle large amounts of noisy data from dynamic and nonlinear systems, especially when the underlying physical relationships are not fully understood.

The multi-layer perceptron (MLP) ANN is most widely used type of ANN in hydrological modelling (Wang et al. 2006).

Artificial Neural Network

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous system, such as brain, process information. An ANN is an adaptive, most often non-linear system that learns to perform a function (an input/output map) from data. A Computing system made of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external input.
given by R.Hecht-Nielsen (1989)

Structure of Multi-layer perceptron (MLP) ANN

Input Values

Input Layer Weight Matrix Hidden Layer Weight Matrix Output Layer

Output Values

Key elements required in formulation of ANN model

There is no systematic way to establish a suitable ANN architecture.

Number of hidden layers Number of nodes in hidden layers Selection of Training Algorithm Selection of Activation Function Selection of Stopping Criteria Selection of Input Parameters

Determination of number of hidden layers

Hornik et al. (1989) proved that a single hidden layer network containing a sufficiently large number of neurons can be used to approximate any measurable functional relationship between the input data and the output variable to any desired accuracy. De Villars and Barnard (1993) showed that an ANN comprised of two hidden layers tends to be less robust and converges with less accuracy than its single hidden layer counterpart. Minns and Hall (1996) and Abrahart and See (2000) indicate that the benefits of using a second hidden layer are marginal to the rainfall-runoff modeling problem. Dawson and Wilby (2001) stated that an ANN with only one hidden layer is enough to represent the nonlinear relationship between rainfall and the corresponding runoff or inflow to the reservoir.

Determination of number of hidden nodes in hidden layers

The performance of the ANN depends on the number of nodes in the hidden layer. Ways to determine optimum number of neurons in hidden layer(s) during training. Using pruning and constructive algorithms, Trial and error procedure (preferred choice of most users Shamseldin, 1997; Zealand et al., 1999; Abrahart and See, 2000). As no specific guidelines exist for choosing the optimum number of hidden nodes for a given problem, this network parameter should be optimized using a combination of empirical rules and trial and error procedure.

Selection of training algorithm

The ANN training - non-linear optimization. The aim of supervised learning is to determine a set of weights, which minimizes the error. Thats why a training algorithm is needed to solve a neural network problem. Algorithms available for training a network : Variable learning rate and momentum (BPvm), Resilient Back Propagation (RBP), Polak-Ribiere etc. Selection of an algorithm that provides the best fit to the data is required. Among various algorithms reported in literature Levenberg-Marquardt (L-M) algorithm is much more robust and outperformed other algorithms in terms of accuracy and convergence speed (Vos and Rientjes, 2005). Levenberg-Marquardt algorithm is a quasi-Newton method that proved to be quickest and was less easily trapped in local minima (Brath et al., 2002). Levenberg-Marquardt (L-M) algorithm should be used for modeling application for prediction of inflow to the reservoir.

Selection of activation function

Activation functions are needed for introducing non linearity into the network and it is non linearity that makes multilayer networks so powerful. Various transfer functions reported in literature are
sigmoidal types (logistic and hyperbolic tangent function), hard limit transfer functions (bounded to 0 or 1), linear, polynomial, rational function (ratios of polynomial) and fourier series (sum of cosines)

In literature, most commonly used transfer functions for rainfallrunoff (R-R) modeling are sigmoidal type transfer functions in the hidden layers and linear transfer functions in the output layer due to its advantage in extrapolation beyond the range of the training data (Hsu et al., 1995; Maier and Dandy, 2000; Zealand et al., 1999; Calvoa and Portelab, 2007).

Selection of stopping criteria

The stop criterion is a fundamental aspect of training. The simple ideas of capping the number of iterations or of letting the system train until a predetermined error value is not obtained. The reason is that the ANN has to perform well in the test set data; i.e. ANN should perform well in data it never saw before (good generalization) (Bishop, 1995). The alteration of network parameters in the training phase is commonly stopped before the training optimum is found, because the network will start learning the noise in the training data and lose its generalization capability (overtraining). Stopping too early means the ANN has not yet learnt all the information from the training data (undertraining). Both situations i.e. overtrainning and undertrainning, are likely to result in sub-optimal operational performance of an ANN model. It is for this reason that the available data are often split in three separate data sets (split-sampling method): (1) the training set, (2) the cross-validation set, and (3) the validation set. The first provides the data on which an ANN is trained. The second is used during the training phase to reduce the chance of overtraining of the network. The minimization of the training error is stopped as soon as the cross-validation error starts to increase. This point is considered to lie between undertraining and overtraining an ANN. The latter of the three data sets is used to validate the performance of a trained ANN. This is also called as early stopping approach. The use of early stopping reduced the training time four times and it provided better and more reliable generalization performance than the use of L-M algorithm alone (Coulibaly et al. 2000).

Selection of input parameters

In ANN modeling, choice of the input variables is an important issue. There is no general theory to solve this issue, and it is rather problem dependent. A trial and error procedure was generally adopted to finalize the input variables to the ANN model. As the hydrological state (i.e. the amount and distribution of stored water in a catchment) for a great part determines the catchments response to a rainfall event, it is critical as input to an ANN model. Previous discharge values are therefore often used as ANN inputs, since these are indirectly indicative for the hydrological state (ASCE, 2000). In addition to runoff values, current and previous days rainfall are used as inputs to the ANN models formulated for determination of inflow to the reservoir.

Conclusions
On the basis of above discussion it is suggested to adopt following parameters while building up ANN structure/model for prediction of inflow to the reservoir using MLP ANN technique, for accurate results. i) A single hidden layer network. ii) The trial and error method for determining optimum number of neurons in hidden layer. iii) Levenberg-Marquardt algorithm as a training algorithm iv) Sigmoidal type transfer function in input hidden layer and linear transfer function in output layer. v) An early stopping criterion for stopping of training. vi) Previous runoff/inflow to reservoir along with rainfall values as input parameters.

Thank You