Validation of decision-aiding spreadsheets: the influence of contingency factors

This paper describes an investigation into the influence of contingency factors on the validation of spreadsheet-based decision support systems (DSS), and develops a methodological framework for validation that takes account of the effect of contingency factors. The research extends and confirms previous research by one of the authors who identified relevant contingency factors. The perceived influence of these contingency factors on validation effort was investigated by an empirical study, and the results of this study were then used to derive contingency-related guidance for validation. This was evaluated by a sample of practitioners and academics with experience in the development of spreadsheet-based DSS, and was then incorporated into a methodological framework for validation. The framework is designed to enable managers, decision-makers, project leaders and other non-OR specialists to identify the extent of validation effort likely to be required to ensure that the spreadsheet models that they build are valid. The research has implications for the validation of other types of OR models.     

The Validation of Expert Systems — Contrasts with Traditional Methods

Recent research has demonstrated that little validation is carried out during and after the building of operational research models for decision support. In a subject area that has been of great interest for more than a decade, this reflects the fact that validation is not yet firmly embedded into the methodology of mathematical modelling that underpins conventional decision support systems. With the interest now being shown in expert systems, it is important to devise an appropriate methodology of validation for such systems. This paper contributes to this development by describing the types of validation that should be performed with decision support systems, and compares and contrasts the features found in the validation of mathematical models with those required of validation of expert systems.     

Financial modelling: Where to go? With an illustration for portfolio management

The definition of Financial Modelling chosen by the EURO working group on financial modelling is ‘the development and implementation of tools supporting firms, investors, intermediaries, governments and others in their financial-economic decision making, including the validation of the premises behind these tools and the measurement of the effectivity of the use of these tools’. Clearly, in this definition, the decision and its solution is central. Unlike financial modelling in our definition, the theory of finance is not so much concerned with individual decisions, but rather with the effects of the decisions and actions of many individuals on the formation of prices in financial markets. It is therefore no wonder that the assumptions underlying financial theory, which at best describe ‘average individuals’ and ‘average decision situations’, are not suited to describe specific individual decision problems. In our view it is the role of financial modelling to support individual decision making, taking account of the peculiarities of the actual case, where possible taking benefit from the results of the financial theory. This philosophy towards financial modelling is illustrated by a framework for portfolio management.     

A survey of contingency factors affecting the validation of end-user spreadsheet-based decision support systems

Decision support systems have become widespread in recent years. However, most of these systems are not carefully validated and this has led in some cases to serious errors. One of the principal reasons for the lack of validity is the lack of an operationalised validation methodology. A simplified validity framework has already been developed by the authors as the first part of a 3-phase approach towards devising a practical validation methodology. This paper describes the second phase, reporting as it does on a survey that has been undertaken to unearth the contingency factors that influence how the process of validation, employing the validity framework, will be carried out for a particular kind of decision support system—one involving a spreadsheet model.     

The development,validation and implementation of knowledge-based systems

It is now ten years since Operational Research, and other groups that support management decision making, seized upon Knowledge-based Systems (KBS) as an alternative set of techniques and methods for building systems that support, augment or automate decision making. Success, as with most information technologies, has been mixed. This paper briefly reviews where KBS have been successful, and what benefits have been derived. It notes that interest in KBS has been rekindled by the present interest in business process re-engineering. Given the mature state of development tools, most KBS developers are now focusing on methodological and life-cycle issues. The paper reviews progress in understanding and using development methodologies, validation methods (now generally considered as crucial to technical success) and implementation. Some ideas about where KBS ‘fits in’ to the array of tools, techniques and methodologies available to the modern analyst and system developer are presented.     

A framework for performance measurement during production ramp-up of assembly stations

Production ramp-up is an important phase in the lifecycle of a manufacturing system which still has significant potential for improvement and thereby reducing the time-to-market of new and updated products. Production systems today are mostly one-of-a-kind complex, engineered-to-order systems. Their ramp-up is a complex order of physical and logical adjustments which are characterised by try and error decision making resulting in frequent reiterations and unnecessary repetitions. Studies have shown that clear goal setting and feedback can significantly improve the effectiveness of decision-making in predominantly human decision processes such as ramp-up. However, few measurement-driven decision aides have been reported which focus on ramp-up improvement and no systematic approach for ramp-up time reduction has yet been defined. In this paper, a framework for measuring the performance during ramp-up is proposed in order to support decision making by providing clear metrics based on the measurable and observable status of the technical system. This work proposes a systematic framework for data preparation, ramp-up formalisation, and performance measurement. A model for defining the ramp-up state of a system has been developed in order to formalise and capture its condition. Functionality, quality and performance based metrics have been identified to formalise a clear ramp-up index as a measurement to guide and support the human decision making. For the validation of the proposed framework, two ramp-up processes of an assembly station were emulated and their comparison was used to evaluate this work.     

Modelling impacts of cropping systems: Demands and solutions for DEX methodology

Decision modelling of diverse groups of problems makes different requirements to the modelling methodologies and software. We present an actual decision problem and the required characteristics of corresponding decision models. The problem is from agronomy and addresses the ecological and economic impacts of cropping systems, with the focus on the differences between cropping systems with conventional crops and the ones with genetically modified crops. We describe the extensions of an existing DEX qualitative multi-attribute modelling methodology, which were made to cope with the challenges of the problem. The extensions address general hierarchical structures, probabilistic utility functions and numerical values of basic attributes. A new, freely available software tool called proDEX was implemented to support the extended methodology. In this paper we describe the problem of cropping system assessment, propose methodological extensions to DEX, and present the implementation of proDEX.     

Integrating Bayesian networks and decision trees in a sequential rule-based transportation model

Several activity-based transportation models are now becoming operational and are entering the stage of application for the modelling of travel demand. Some of these models use decision rules to support its decision-making instead of principles of utility maximization. Decision rules can be derived from different modelling approaches. In a previous study, it was shown that Bayesian networks outperform decision trees and that they are better suited to capture the complexity of the underlying decision-making. However, one of the disadvantages is that Bayesian networks are somewhat limited in terms of interpretation and efficiency when rules are derived from the network, while rules derived from decision trees in general have a simple and direct interpretation. Therefore, in this study, the idea of combining decision trees and Bayesian networks was explored in order to maintain the potential advantages of both techniques. The paper reports the findings of a methodological study that was conducted in the context of Albatross, which is a sequential rule based model of activity scheduling behaviour. To this end, the paper can be situated within the context of a series of previous publications by the authors to improve decision-making in Albatross. The results of this study suggest that integrated Bayesian networks and decision trees can be used for modelling the different choice facets of Albatross with better predictive power than CHAID decision trees. Another conclusion is that there are initial indications that the new way of integrating decision trees and Bayesian networks has produced a decision tree that is structurally more stable.     

MOIRA: A decision support system for decision making on aquatic ecosystems contaminated by radioactive fallout

Interventions to restore radionuclide contaminated aquatic ecosystems may reduce individual and collective radiation doses, but may also result in detrimental ecological, social and economic effects. Decision makers must carefully evaluate possible impacts before choosing a countermeasure, hence decision analysis methods constitute an important aid to rank intervention strategies after the contamination of an aquatic ecosystem. We describe MOIRA, a decision support system for the identification of optimal remedial strategies to restore water systems after accidental introduction of radioactive substances. MOIRA includes an evaluation module based on a multi-attribute value model to rank alternatives and a module to perform multiparametric sensitivity analyses, both with respect to weights and values, to allow us to gain insights into the problem. The problem is under certainty since the validation of models used to quantify countermeasure impacts suggests little uncertainty in policy effects.The system is implemented in a PC based decision support system which allows the inclusion of all relevant information.     

Influence of model management systems on decision making: empirical evidence and implications

Model management (MM) regards decision models as an important organisational resource deserving prudent management. Despite the remarkable volume of model management literature compiled over the past twenty-odd years, very little is known about how decision makers actually benefit from employing model management systems (MMS). In this paper, we report findings from an experiment designed to verify the idea that the adequacy of modeling support provided by a MMS influences the decision maker's problem solving performance and behaviour. We show that the decision makers who receive adequate modelling support from MMS outperform those without such support. Also, we provide empirical evidence that the MMS help turn the decision makers' perception of problem solving from a number crunching task into development of solution strategies, consequently changing their decision making behaviour.     

Computer-assisted Modelling and Analysis of Linear Programming Problems: Towards a Unified Framework

A framework for model formulation and analysis to support operationsand management of large-scale linear programs is developed fromthe combined capabilities of CAMPS and ANALYZE. Both the systemsare reviewed briefly and the interface which integrates thetwo systems is then described. The model formulation, matrixgeneration, and model management capability of CAMPS and thecomplementary model and solution analysis capability of ANALYZEare presented within a unified framework. Relevant generic functionsare highlighted, and an example is presented in detail to illustratethe level of integration achieved in the current prototype system.Some new results on discourse models and model management supportare given in a framework designed to move toward an ‘intelligent’system for linear programming modelling and analysis.     

On the asymptotic equilibrium of a population system with migration

In the analysis of economic and social issues of a country (or any larger or smaller socio-economic unit) the demographic dynamics of the considered population often play a crucial role. Very current emergencies in this respect are e.g. ageing, longevity risk, state-run healthcare etc. Over the last decade migration between EU countries also became an important issue, and in recent years the uncontrolled migration from non-EU countries is also a major concern. Therefore, the better theoretical understanding of the evolutionary mechanism of age-classified populations interacting via migration, is a timely modelling-methodological task. This paper is a preliminary demographic methodological contribution to a further research in support of socio-economic modelling and decision making concerning migration issues.It is known that in the framework of the classical age-specific Leslie model, under simple demographic conditions, a closed population in the long term tends to an equilibrium age distribution. As the main theoretical result of the paper, a similar convergence is proved for a system of several populations with migration between them, and this long-term behaviour (convergence theorem) is extended to systems of sex-structured populations. Based on the latter model, medium term projections are also analysed concerning the effect of migration among countries on the development of the old-age dependency ratio (the proportion of pensioner age classes to active ones), which is an aggregate scalar indicator of ageing, a major concern in most industrialized countries. Illustrative simulation analysis is carried out with data from three European countries.     

Interaction of trip decisions and traffic systems dynamics

A major difficulty in the study of large-scale complex systems involving human decision-making with non-linear collective effects is that of obtaining pertinent data at the desired level of detail. A controlled experiment involving real commuters in a hypothetical computer simulated traffic system is described as an alternative approach of conducting observational studies to support the modelling of such complex dynamic interactive decision systems. An overview of the principal characteristics of the traffic system's evolution obtained in this experiment is presented. Also illustrated are the features of a proposed behavioral framework where users are viewed as boundedly rational seekers of a satisfactory choice outcome.     

Validating an Expert Support System for Tender Enquiry Evaluation: A Case Study

An expert system was developed as a support tool for a large electro-mechanical engineering company. The system was developed to assist in tender enquiry evaluation and the bid versus no bid decision. It was developed using repertory grid techniques based on the expertise of two senior managers. The importance of the system's validation is stressed and an appropriate formal validation framework developed. The results from the validation study suggest that the system provides an appropriate model of the organization's consensual business perspective regarding the bid versus no bid decision. Furthermore, the study drew attention to some dissenting views amongst the organization's senior management which the management believed to be helpful in clarifying their understanding of the issues affecting the bid versus no bid decision.     

Modelling pedestrian dynamics into a metro station by thermostatted kinetic theory methods

This paper deals with the modelling of pedestrian dynamics at the entry of a metro station by means of the thermostatted kinetic theory framework. Specifically, the model depicts the time evolution of the pedestrian dynamics at the turnstiles under no panic conditions. The modelling of the microscopic interactions is based on the stochastic game theory and reflects the decision dynamics of the turnstiles pursued by pedestrians. A qualitative analysis is addressed to the equilibrium solutions by means of the classical stability theory of perturbations. Numerical simulations aim at showing the emerging behaviours captured by the model. In particular the model validation is obtained by performing a sensitivity analysis on the parameters and on the initial conditions. Further refinements and research perspective, including the modelling under panic conditions, are discussed in the last section of the paper.     

Hit or miss: A decision support system framework for signing new musical talent

In the music industry, the process of signing new musical talent is one of the most complex decision-making problems. The decision, which is generally made by an artist and repertoire (A&R) team, involves consideration of various quantitative and qualitative criteria, and usually results in a low success rate. We conducted a series of mental model interviews with the aim of developing a decision support framework for A&R teams. This framework was validated by creating a decision support system that utilises multi-criteria decision analysis to support decision-making. Our framework and subsequent implementation of the decision support system involving decision rule and weighted sum methods show an improvement in the ability to analyse and decide on greater amounts of talent. This paper serves as a building block for developing systems to aid in this complex decision-making problem.     

Cynefin,statistics and decision analysis

David Snowden's Cynefin framework, introduced to articulate discussions of sense-making, knowledge management and organisational learning, has much to offer discussion of statistical inference and decision analysis. I explore its value, particularly in its ability to help recognise which analytic and modelling methodologies are most likely to offer appropriate support in a given context. The framework also offers a further perspective on the relationship between scenario thinking and decision analysis in supporting decision makers.     

Linear,Integer, Separable and Fuzzy Programming Problems: A Unified Approach towards Reformulation

For mathematical programming (MP) to have greater impact as a decision tool, MP software systems must offer suitable support in terms of model communication and modelling techniques. In this paper, modelling techniques that allow logical restrictions to be modelled in integer programming terms are described, and their implications discussed. In addition, it is illustrated that many classes of non-linearities which are not variable separable may be, after suitable algebraic manipulation, put in a variable separable form. The methods of reformulating the fuzzy linear programming problem as a max-min problem is also introduced. It is shown that analysis of bounds plays a key role in the following four important contexts: model reduction, reformulation of logical restrictions as 0-1 mixed integer programmes, reformulation of non-linear programmes as variable separable programmes and reformulation of fuzzy linear programmes. It is observed that, as well as incorporating an interface between the modeller and the optimizer, there is a need to make available to the modeller software facilities which support the model reformulation techniques described here.     

Sets and indices in linear programming modelling and their integration with relational data models

LP models are usually constructed using index sets and data tables which are closely related to the attributes and relations of relational database (RDB) systems. We extend the syntax of MPL, an existing LP modelling language, in order to connect it to a given RDB system. This approach reuses existing modelling and database software, provides a rich modelling environment and achieves model and data independence. This integrated software enables Mathematical Programming to be widely used as a decision support tool by unlocking the data residing in corporate databases.     

On the organization of large shared Model bases

Central to the Model Management (MM) function is the creation and maintenance of a knowledge-based model repository. The Model Knowledge Base (MKB) provides the basis by which information about models can be shared to facilitate consistent and controlled utilization of existing models for decision making, as well as the development of new models. Various schemes for representing individual models have been proposed in the literature. This paper focuses on how best to structure, control, and administer a large MKB to support organization-wide modeling activities. Guided by a recently proposed systems framework for MM, we describe a number of concepts which are useful for capturing the semantics and structural relationships of models in an MKB. These concepts, and the nature of the MMS functions to be supported, are then used to derive specific information management requirements for model bases. Four major requirements are identified: (1) management of composite model configurations; (2) management of model version histories; (3) support for the model consultation and selection functions of an MMS; and (4) support for multiple logical MKBs (private, group, and public). We argue that traditional record-based approaches to data management appear to fall short of capturing the rich semantics present in an MM environment. The paper proposes an architecture for an MMS, focusing on its major component — the MKB Management Subsystem. An implementation of this architecture is briefly described.