Standard modelling techniques for time series vary from conceptual models based on physical system knowledge to black box approaches such as linear Auto-Regressive Moving Average models with eXogenous inputs (ARMAX), or non-linear neural networks. These models have in common that the output at time t depends on the present and past values of the model's forcings, as well as the outputs generated in one or more preceding time steps. A neural network with this characteristic is called a Regressive Neural Network vN). One of the advantages of an auto-regressive form is that much or all the information of the model's forcings of the past are accumulated into the model's output of preceding time step(s) so that only the most recent value(s) of the forcing need to be included into the model's input. As a result an auto-regressive model may be described with less parameters than a non-regressive model
The standard neural networks (NN) applied for the modelling of time series are often also in a regressive form, but not in strict sense. In fact, the feedback of outputs to the inputs of later time steps uses observations rather than computed outputs. This is done for a practical reason: In this way, the standard error back propagation rule (EBPR) can be applied during training, and a difficult problem is circumvented, namely the derivation/implementation of a generalized EBPR that takes the strict forrn of feed back into account.
Replacing computed outputs by their observed counterparts leads to a model calibration of the form 'data insertion' and it is known that this is not an optimal and consistent way of data assimilation. This can, amongst others, be seen from the fact that data insertion may inject observation noise into the evolving model solution and leave the model dynamically unbalanced. Replacing the model's outputs by observations suggests that the user has little or no confidence in the model's response. Moreover, it must be noted that even if the network is trained on the basis of data insertion, it must be applied in strict regressive form for predictions since no observations are available for feedback. In that case, the model's form during calibration differs significantly from its form during predictions.
At WLIDELFT SYDRAULICS regressive neural networks of type MLP have been developed that are strictly regressive both during training and during predictions. To derive the learning rule, i.e. the generalized EBPR, adjoint modelling was applied to compute the gradient of a cost function with respect to the weights. The gradient obtained via the adjoint neural network was combined with efficient gradient descent techniques. In this way, a generic non-linear technique for the modelling of time series was obtained and as such forms an important extension of the commonly applied linear ARMAX models.
In this thesis, the 'theoretical' aspects of RNN are considered as well as results of applications to the rainfall-runoff modelling for the Sieve basin in Italy and the modelling of water levels of Lake IJsselmeer in The Netherlands. In particular, the performances of RNNs, and NNs based on data insertion are compared. From the applications it is concluded that RNNs provide better predictions.
Back to the list of MSc abstracts