Unpublished Paper
Addressing parameter uncertainty in SD models with fit-to-history and Monte-Carlo sensitivity methods
(2019)
Abstract
We present a practical guide, including a step-by-step flowchart, for establishing confidence intervals for key model outcomes in the face of uncertain parameters. The process starts with Powell optimization (e.g., using VensimTM) to find a set of uncertain parameters (the “optimum” parameter set or OPS) that minimize the model fitness error relative to available reference behavior data. The optimization process also helps in refinement of assumed parameter uncertainty ranges. Next, Markov Chain Monte Carlo (MCMC) or conventional Monte Carlo (MC) randomization is used to create a sample of parameter sets that fit the reference behavior data nearly as well as the OPS. Under the MC method, the entire parameter space is explored broadly (with a very large number of runs), and the results are sorted for selection of qualifying parameter sets (QPS) based on goodness-of-fit criteria. The statistical properties of the QPS parameter distributions are analyzed to ensure their centrality relative to the uncertainty ranges. Also, the full set of QPS outputs are graphed (as sensitivity graphs or box-and-whisker plots) for comparison with the reference behavior data. Finally, alternative policies and scenarios are run against the OPS and all QPS, and confidence intervals are found for key model outcomes. The method is demonstrated with a non-trivial model, and a narrative template is provided to illustrate how such analyses could be described to interested parties such as policy or decision makers.
Keywords
- MCMC,
- uncertainty analysis,
- optimization
Disciplines
Publication Date
Summer 2019
Citation Information
Wayne W. Wakeland and Jack Homer. "Addressing parameter uncertainty in SD models with fit-to-history and Monte-Carlo sensitivity methods" (2019) Available at: http://works.bepress.com/wayne_wakeland/117/