A collection of simple functions used within Generalised Likelihood Uncertainty Estimation (GLUE) including dotty plots, transformations and combination methods for likelihood measures, and plots for single variables and time series. The example application is for a simple hydrological model.
Libraries
No special library routines are required for these routines
Download
Download sample code and data file for GLUE routines [Here]
Procedure
GLUE: Likelihoods as weights on model predictions
GLUE is an extension of the HSY GSA in that it treats each parameter set considered “behavioural” within a model structure as a predictor associated with a likelihood measure. It is very flexible about what likelihood measures should be used. Formal statistical likelihoods can be used as special cases when the strong assumptions can be justified (in which case the results will be very similar to a formal Bayes analysis).
There is an on-going discussion about what types of likelihood measures should be used The main criticism of GLUE is that it allows a subjective choice of likelihood measure and therefore does not correctly estimate the confidence in predicted variables. However, in the face of input error and model structural error it is all too easy to overestimate confidence in the predictions (for a recent discussion of the issues see the Beven et al., JH, 2008 paper).
Within this framework, prediction quantiles are formed by likelihood weighting the predictions over all behavioural models (which can included more than one model structure). Residual errors are treated implicitly (unless a formal error model is added as a non-physical model component), i.e. each model carries along an implicit deviation from the observations, that can be reconstructed in calibration but is assumed to have similar characteristics in prediction.
GLUE: Transforming Likelihood Measures
The results of any GLUE analysis clearly has to depend on the decision on what is a behavioural model and the likelihood weighting given to the predictions from each model. It is very difficult to provide general rules for likelihood measures – what is required will depend on how good the model is and the quality of the input and output data. In “well-behaved” applications it may be possible to scale or transform a likelihood so that the prediction limits contained a specified number of observations (e.g. 95%), but this is not always possible. See Environmental Modelling: An Uncertain Future? for a discussion of some different likelihoods.
A number of likelihood transformations are provided in the routine
GLUE_transform(“filename”,likelihood,transform,parameter)
where likelihood is the index of the selected likelihood as before, transform is the type of transformation required,and parameter controls the transformation.
The transformations available are:
“thresh” : a simple threshold, with parameter = threshold
“norm” : minimum to maximum likelihood values are normalised to the range [0-1]
“power” : likelihoods >0 are raised to a power, with parameter = power
“log” : likelihoods > 0 are log transformed and added to parameter
“exp” : likelihoods > 0 are multiplied by parameter and then exponentiated
Results are presented as dotty plots so that effects of different transformations can be compared.
GLUE: Combining Likelihood Measures
A number ways of combining likelihoods are provided in the routine
GLUE_combine(“filename”,likelihood1,likelihood2,combin,w1,w2)
where likelihood1 and likellihood2 are the indices of the likelihoods to be combined, combin is the type of combination required, and w1 and w2 (default values = 1) control the combination as follows.
The combinations available are:
“bayes” : Bayes multiplication (w1 and w2 not required)
“weights” : weighted addition using weights w1 and w2
“fuzzyuni” : fuzzy union (maximum of the two likelihoods)
“fuzzyint” : fuzzy intersection (minimum of two likelihoods)
Results are presented as dotty plots so that effects of different combinations can be compared.
GLUE: Prediction Quantiles
GLUE_quantiles(“filename”,likelihood,variable)
where likelihood is the index of the likelihood weight, variable is the index of the index2 are the likelihoods to be combined.
Results are presented as a likelihood weighted histogram of the predictions, together with a likelihood weighted cumulative density function with 5 and 95% quantiles indicated.
GLUE: Prediction quantiles for time series
Download
Download sample code and data file for GLUE time series routines [Here]
Procedure
GLUE can also be tried with the example of the PDM rainfall-runoff model applied to the Leaf River catchment data set. Similar steps are involved:
- Initialise the process: load the GLUEsetup.R source
This routine loads in the hydrological data and sets minimum and maximum values for the parameters. These can be edited and the processes rerun.
- Make the GLUE runs: load the GLUErun.R source
This might again take a little while, depending on the number of parameter sets to be run (default is 1000 for this practical).
- Analyse the results: load the GLUEanalysis.R source
Once the runs are made the predictions can be calculated for different informal likelihood measures, NSE (Nash-Sutcliffe efficiency based on sum of squared errors) and NAE (normalised sum of absolute errors) where
NSE<-max(0,1-var(obshat-obs)/var(obs))
NAE<-max(0,1-sum(abs(obshat-obs))/sum(abs(obs-mean(obs)) ))
Behavioural models can be controlled by specifying a threshold value for a likelihood measure in calling the GLUEanalysis routine:
GLUEanalysis(“lik”, threshold)
where “lik” is “NSE” or “NAE” and threshold is the behavioural threshold. The analysis can be repeated with different parameter values for a given set of GLUErun outputs (I suggest starting with a threshold of zero to look at full dotty plots)
References
Beven, K.J., and Binley, A. The future of distributed models: Model calibration and uncertainty prediction. Hydrological Processes 6 (1992), 279-298.
Beven, K.J., A manifesto for the equifinality thesis. Journal of Hydrology 320 (2006), 18-36.
Beven, K J, Smith, P J, and Freer, J, 2008, So just why would a modeler choose to be incoherent?. J. Hydrology, 354,15-32.