A collection of simple functions used within Generalised Likelihood Uncertainty Estimation (GLUE) including dotty plots, transformations and combination methods for likelihood measures, and plots for single variables and time series. \u00a0The example application is for a simple hydrological model.<\/p>\n
Libraries<\/u><\/p>\n
No special library routines are required for these routines<\/p>\n
Download<\/u><\/p>\n
Download sample code and data file for GLUE routines\u00a0[Here<\/em><\/a>]<\/p>\n Procedure<\/u><\/p>\n GLUE:\u00a0 Likelihoods as weights on model predictions<\/strong><\/p>\n GLUE is an extension of the HSY GSA in that it treats each parameter set considered \u201cbehavioural\u201d within a model structure as a predictor associated with a likelihood measure.\u00a0 It is very flexible about what likelihood measures should be used.\u00a0\u00a0 Formal statistical likelihoods can be used as special cases when the strong assumptions can be justified (in which case the results will be very similar to a formal Bayes analysis).<\/p>\n There is an on-going discussion about what types of likelihood measures should be used The main criticism of GLUE is that it allows a subjective choice of likelihood measure and therefore does not correctly estimate the confidence in predicted variables.\u00a0 However, in the face of input error and model structural error it is all too easy to overestimate confidence in the predictions (for a recent discussion of the issues see the Beven et al., JH, 2008 paper).<\/p>\n Within this framework, prediction quantiles are formed by likelihood weighting the predictions over all behavioural models (which can included more than one model structure).\u00a0\u00a0 Residual errors are treated implicitly (unless a formal error model is added as a non-physical model component), i.e. each model carries along an implicit deviation from the observations, that can be reconstructed in calibration but is assumed to have similar characteristics in prediction.<\/p>\n <\/p>\n GLUE:\u00a0 Transforming Likelihood Measures<\/strong><\/p>\n The results of any GLUE analysis clearly has to depend on the decision on what is a behavioural model and the likelihood weighting given to the predictions from each model.\u00a0 It is very difficult to provide general rules for likelihood measures \u2013 what is required will depend on how good the model is and the quality of the input and output data.\u00a0\u00a0\u00a0 In \u201cwell-behaved\u201d applications it may be possible to scale or transform a likelihood so that the prediction limits contained a specified number of observations (e.g. 95%), but this is not always possible.\u00a0\u00a0 See\u00a0Environmental Modelling: An Uncertain Future?<\/strong>\u00a0for a discussion of some different likelihoods.<\/p>\n A number of likelihood transformations are provided in the routine<\/p>\n GLUE_transform(\u201cfilename\u201d,likelihood,transform,parameter)<\/p>\n where\u00a0likelihood\u00a0is the index of the selected likelihood as before,\u00a0transform\u00a0is the type of transformation required,and\u00a0parameter\u00a0controls the transformation.<\/p>\n The transformations available are:<\/p>\n \u201cthresh\u201d\u00a0 : a simple threshold, with parameter = threshold<\/p>\n \u201cnorm\u201d\u00a0\u00a0 :\u00a0 minimum to maximum likelihood values are normalised to the range [0-1]<\/p>\n \u201cpower\u201d\u00a0: likelihoods >0 are raised to a power, with parameter = power<\/p>\n \u201clog\u201d\u00a0\u00a0\u00a0\u00a0\u00a0 : likelihoods > 0 are log transformed and added to parameter<\/p>\n \u201cexp\u201d\u00a0\u00a0\u00a0\u00a0 : likelihoods > 0 are multiplied by parameter and then exponentiated<\/p>\n Results are presented as dotty plots so that effects of different transformations can be compared.<\/p>\n \u00a0<\/strong><\/p>\n GLUE:\u00a0 Combining Likelihood Measures<\/strong><\/p>\n A number ways of combining likelihoods are provided in the routine<\/p>\n GLUE_combine(\u201cfilename\u201d,likelihood1,likelihood2,combin,w1,w2)<\/p>\n where\u00a0likelihood1\u00a0and\u00a0likellihood2\u00a0are the indices of the likelihoods to be combined,\u00a0combin\u00a0is the type of combination required, and\u00a0w1\u00a0and\u00a0w2\u00a0(default values = 1)\u00a0control the combination as follows.<\/p>\n The combinations available are:<\/p>\n \u201cbayes\u201d\u00a0 : Bayes multiplication\u00a0 (w1 and w2 not required)<\/p>\n \u201cweights\u201d\u00a0\u00a0 :\u00a0 weighted addition using weights w1 and w2<\/p>\n \u201cfuzzyuni\u201d\u00a0: fuzzy union (maximum of the two likelihoods)<\/p>\n \u201cfuzzyint\u201d\u00a0: fuzzy intersection (minimum of two likelihoods)<\/p>\n Results are presented as dotty plots so that effects of different combinations can be compared.<\/p>\n <\/p>\n GLUE:\u00a0 Prediction Quantiles<\/strong><\/p>\n GLUE_quantiles(\u201cfilename\u201d,likelihood,variable)<\/p>\n where\u00a0likelihood\u00a0is the index of the likelihood weight, variable is the index of the\u00a0\u00a0index2\u00a0are the likelihoods to be combined.<\/p>\n Results are presented as a likelihood weighted histogram of the predictions, together with a likelihood weighted cumulative density function with 5 and 95% quantiles indicated.<\/p>\n \u00a0<\/strong><\/p>\n GLUE:\u00a0 Prediction quantiles for time series<\/strong><\/p>\n Download<\/u><\/p>\n Download sample code and data file for GLUE time series routines\u00a0[Here<\/em><\/a>]<\/p>\n Procedure<\/u><\/p>\n GLUE can also be tried with the example of the PDM rainfall-runoff model applied to the Leaf River catchment data set.\u00a0\u00a0 Similar steps are involved:<\/p>\n <\/p>\n <\/p>\n This routine loads in the hydrological data and sets minimum and maximum values for the parameters.\u00a0\u00a0 These can be edited and the processes rerun.<\/p>\n <\/p>\n <\/p>\n This might again take a little while, depending on the number of parameter sets to be run (default is 1000 for this practical).<\/p>\n <\/p>\n <\/p>\n Once the runs are made the predictions can be calculated for different informal likelihood measures, NSE\u00a0 (Nash-Sutcliffe efficiency based on sum of squared errors) and NAE (normalised sum of absolute errors) where<\/p>\n NSE<-max(0,1-var(obshat-obs)\/var(obs))<\/p>\n NAE<-max(0,1-sum(abs(obshat-obs))\/sum(abs(obs-mean(obs)) ))<\/p>\n Behavioural models can be controlled by specifying a threshold value for a likelihood measure in calling the\u00a0GLUEanalysis\u00a0routine:<\/p>\n GLUEanalysis(\u201clik\u201d, threshold)<\/p>\n where\u00a0\u201clik\u201d\u00a0is\u00a0\u201cNSE\u201d\u00a0or\u00a0\u201cNAE\u201d\u00a0and\u00a0threshold\u00a0is the behavioural threshold.\u00a0 The analysis can be repeated with different parameter values for a given set of\u00a0GLUErun\u00a0outputs (I suggest starting with a threshold of zero to look at full dotty plots)<\/p>\n References<\/u><\/p>\n Beven, K.J., and Binley, A. The future of distributed models: Model calibration and uncertainty prediction. Hydrological Processes 6 (1992), 279-298.<\/p>\n Beven, K.J., A manifesto for the equifinality thesis. Journal of Hydrology 320 (2006), 18-36.<\/p>\n Beven, K J,\u00a0 Smith, P J,\u00a0 and Freer, J, 2008, So just why would a modeler choose to be incoherent?.\u00a0J. Hydrology<\/em>, 354,15-32.<\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" A collection of simple functions used within Generalised Likelihood Uncertainty Estimation (GLUE) including dotty plots, transformations and combination methods for likelihood measures, and plots for single variables and time series. \u00a0The example application is for a simple hydrological model. Libraries No special library routines are required for these routines Download Download sample code and data … Continue reading Generalised Likelihood Uncertainty Estimation (GLUE)<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":28,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-131","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/pages\/131","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=131"}],"version-history":[{"count":4,"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/pages\/131\/revisions"}],"predecessor-version":[{"id":143,"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/pages\/131\/revisions\/143"}],"up":[{"embeddable":true,"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=\/wp\/v2\/pages\/28"}],"wp:attachment":[{"href":"http:\/\/www.uncertain-future.org.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=131"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
\n