A methodology for sensitivity analysis of models fitted to data using statistical methods

A simple methodology is presented for sensitivity analysis ofmodels that have been fitted to data by statistical methods.Such analysis is a decision support tool that can focus theeffort of a modeller who wishes to further refine a model and/orto collect more data. A formula is given for the calculationof the proportional reduction in the variance of the model ‘output’that would be achievable with perfect knowledge of a subsetof the model parameters. This is a measure of the importanceof the set of parameters, and is shown to be asymptoticallyequal to the squared correlation between the model output andits best predictor based on the omitted parameters. The methodology is illustrated with three examples of OR problems,an age-based equipment replacement model, an ARIMA forecastingmodel and a cancer screening model. The sampling error of thecalculated percentage of variance reduction is studied theoretically,and a simulation study is then used to exemplify the accuracyof the method as a function of sample size.