A comprehensive method for comparing mental models of dynamic systems

Mental models are the basis on which managers make decisions even though external decision support systems may provide help. Research has demonstrated that more comprehensive and dynamic mental models seem to be at the foundation for improved policies and decisions. Eliciting and comparing such models can systematically explicate key variables and their main underlying structures. In addition, superior dynamic mental models can be identified. This paper reviews existing studies which measure and compare mental models. It shows that the methods used to compare such models lack to account for relevant aspects of dynamic systems, such as, time delays in causal links, feedback structures, and the polarities of feedback loops. Mental models without those properties are mostly static models. To overcome these limitations of the methods to compare mental models, we enhance the widely used distance ratio approach (Markóczy and Goldberg, 1995) so as to comprehend these dynamic characteristics and detect differences among mental models at three levels: the level of elements, the level of individual feedback loops, and the level of the complete model. Our contribution lies in a new method to compare explicated mental models, not to elicit such models. An application of the method shows that this previously non-existent information is essential for understanding differences between managers’ mental models of dynamic systems. Thereby, a further path is created to critically analyze and elaborate the models managers use in real world decision making. We discuss the benefits and limitations of our approach for research about mental models and decision making and conclude by identifying directions for further research for operational researchers.     

Structuring resource allocation decisions: A framework for building multi-criteria portfolio models with area-grouped options

Multi-criteria portfolio modelling has been extensively employed as an effective means to allocate scarce resources for investment in projects when considering costs, benefits and risks. Some of these modelling approaches allow the grouping of projects into organisational areas, thus also supporting the decision of resource allocation among organisational units in a way that is collectively efficient for the organisation. However, structuring in practice a portfolio model using this latter type of approach is not a trivial task. How should areas be defined? Where should new projects be included? How should one define the criteria to evaluate performance? As far as we know, there is very little indication in the operational research and decision sciences literatures on how to structure this type of model. This paper suggests different ways to structuring portfolio models where projects are divided into areas and evaluated by multiple criteria, and illustrates their use in two action-research projects. Drawing on these experiences it then suggests a general framework for the structuring of such models in practice. Directions for future research are also identified.     

Structuring decision problems: A case study and reflections for practitioners

This article reviews several approaches to problem structuring and, in particular, the three-step structuring process for decision analysis proposed by von Winterfeldt and Edwards: (1) identifying the problem; (2) selecting an appropriate analytical approach; (3) developing the a detailed analytic structure. This three-step process is re-examined in the context of a decision analysis of alternative policies to reduce electromagnetic field exposure from electric power lines. This decision analysis was conducted for a public health organization funded by the California Public Utilities Commission and it was scrutinized throughout by interested stakeholders. As a result a significant effort went into structuring this problem appropriately, with some successes and some missteps. The article extracts lessons from this experience, updating existing guidance on structuring problems for decision analysis, and concluding with some general insights for problem structuring.     

An MCDA approach for the selection of bike projects based on structuring and appraising activities

This paper presents an MCDA approach for the structuring and appraising activities of a large and complex decision problem. More specifically, the paper makes use of the three-step structuring process for decision analysis proposed by von Winterfeldt and Edwards: (1) identifying the problem; (2) selecting an appropriate analytic approach; and (3) developing a detailed analytic structure. For illustration of the approach a case study dealing with the assessment task of prioritising and selecting initiatives and projects from a public pool with limited funds is examined throughout the paper. The process is embedded in a Decision Support System (DSS) making use of the REMBRANDT technique for pair wise comparisons to determine project rankings. A procedure for limiting the number of pair wise comparisons to be made in the process is in this connection presented. Finally, strengths and weaknesses in the approach are discussed and conclusions are made.     

A study of high-level managerial decision processes, with implications for MCDM research

We study six real-world major strategic decisions and discuss the role that analytic Multiple Criteria Decision Making (MCDM) models could play in helping decision makers structure and solve such problems. We have interviewed successful and well-educated managers who had access to quantitative decision models, but did not use them as part of their decision process. Our approach is a clinical one that takes a close look at the decision processes. We believe that the normative MCDM framework is oversimplified and does not always fit well with complex, real-world organizational decision processes. This may be one reason why decision tools are not used more widely for solving high-level decision problems. We believe that it would be worthwhile to revise some of the MCDM mainstream postulates and practices to make existing models and tools more suitable for practical purposes. The MCDM mainstream research has until today focused on the choice among alternatives. One should realize that MCDM models could also be used in creating alternatives, in assessing the importance of criteria, in providing the decision makers with “post-commitment support”, and as part of a devil's advocate approach.     

Problem structuring methods ‘in the Dock’: Arguing the case for Soft OR

Problem structuring methods (‘soft’ OR) have been around for approximately 40 years and yet these methods are still very much overlooked in the OR world. Whilst there is almost certainly a number of explanations for this, two key stumbling blocks are: (1) the subjective nature of the modelling yielding insights rather than testable results, and (2) the demand on users to both manage content (through modelling) and processes (work with rather than ‘on behalf’ of groups). However, as evidenced from practice there are also a number of significant benefits. This paper therefore aims to examine the case of Soft OR through examining the case for and against problem structuring methods.     

A methodology for sensitivity analysis of maintenance and reliability models

Baker (2001 IMA Journal of Management Mathematics 12, 1–17)presented a methodology for sensitivity analysis of models thathad been fitted to data by statistical methods, e.g. likelihood-basedmethods, and where the purpose of modelling was to calculatea cost function or loss function. Use of this methodology isenvisaged as a quick way to get insight into how to furtherrefine a model, and into which additional data to collect. Itis essentially an exploratory method that would precede useof any more formal decision-theoretic methods. The field of maintenance and reliability offers a rich fieldfor application of the method. The Methodology is briefly summarizedand is illustrated with two examples drawn from this field.A simulation study using a three-parameter model of maintenanceis then used to exemplify the accuracy of the method as a functionof sample size.     

A comparative study of decomposition algorithms for stochastic combinatorial optimization

This paper presents comparative computational results using three decomposition algorithms on a battery of instances drawn from two different applications. In order to preserve the commonalities among the algorithms in our experiments, we have designed a testbed which is used to study instances arising in server location under uncertainty and strategic supply chain planning under uncertainty. Insights related to alternative implementation issues leading to more efficient implementations, benchmarks for serial processing, and scalability of the methods are also presented. The computational experience demonstrates the promising potential of the disjunctive decomposition (D ²) approach towards solving several large-scale problem instances from the two application areas. Furthermore, the study shows that convergence of the D ² methods for stochastic combinatorial optimization (SCO) is in fact attainable since the methods scale well with the number of scenarios.     

A methodology for sensitivity analysis of models fitted to data using statistical methods

A simple methodology is presented for sensitivity analysis ofmodels that have been fitted to data by statistical methods.Such analysis is a decision support tool that can focus theeffort of a modeller who wishes to further refine a model and/orto collect more data. A formula is given for the calculationof the proportional reduction in the variance of the model ‘output’that would be achievable with perfect knowledge of a subsetof the model parameters. This is a measure of the importanceof the set of parameters, and is shown to be asymptoticallyequal to the squared correlation between the model output andits best predictor based on the omitted parameters. The methodology is illustrated with three examples of OR problems,an age-based equipment replacement model, an ARIMA forecastingmodel and a cancer screening model. The sampling error of thecalculated percentage of variance reduction is studied theoretically,and a simulation study is then used to exemplify the accuracyof the method as a function of sample size.     

A framework for the probabilistic analysis of hierarchical planning systems

As we have argued in previous papers, multi-level decision problems can often be modeled as multi-stage stochastic programs, and hierarchical planning systems designed for their solution, when viewed as stochastic programming heuristics, can be subjected to analytical performance evaluation. The present paper gives a general formulation of such stochastic programs and provides a framework for the design and analysis of heuristics for their solution. The various ways to measure the performance of such heuristics are reviewed, and some relations between these measures are derived. Our concepts are illustrated on a simple two-level planning problem of a general nature and on a more complicated two-level scheduling problem.     

Dynamic facility location when the total number of facilities is uncertain: A decision analysis approach

Models developed to analyze facility location decisions have typically optimized one or more objectives, subject to physical, structural, and policy constraints, in a static or deterministic setting. Because of the large capital outlays that are involved, however, facility location decisions are frequently long-term in nature. Consequently, there may be considerable uncertainty regarding the way in which relevant parameters in the location decision will change over time. In this paper, we propose two approaches for analyzing these types of dynamic location problems, focussing on situations where the total number of facilities to be located in uncertain. We term this type of location problem NOFUN (Number Of Facilities Uncertain). We analyze the NOFUN problem using two well-established decision criteria: the minimization of expected opportunity loss (EOL), and the minimization of maximum regret. In general, these criteria assume that there are a finite number of decision options and a finite number of possible states of nature. The minisum EOL criterion assumes that one can assign probabilities for the occurrence of the various states of nature and, therefore, find the initial set of facility locations that minimize the sum of expected losses across all future states. The minimax regret criteria finds the pattern of initial facility locations whose maximum loss is minimized over all possible future states.     

A survey of the perturbation analysis of discrete event dynamic systems

This paper introduces an analysis and optimization technique for discrete event dynamic systems, such as flexible manufacturing systems (FMSs), and other discrete part production processes. It can also be used for enhancement of the simulation results of, or the monitoring of the operations of such systems in real time. Extensive references are given where readers may pursue futher details.     

A matrix approach to status quo analysis in the graph model for conflict resolution

An algebraic method is developed to carry out status quo analysis within the framework of the graph model for conflict resolution. As a form of post-stability analysis, status quo analysis aims at confirming that possible equilibria, or states stable for all decision-makers, are in fact reachable from the status quo or any other initial state. Although pseudo-codes for status quo analysis have been developed, they have never been implemented within a practical decision support system. The novel matrix approach to status quo analysis designed here is convenient for computer implementation and easy to employ, as is illustrated by an application to a real-world conflict case. Moveover, the proposed explicit matrix approach reveals an inherent link between status quo analysis and the traditional stability analysis and, hence, provides the possibility of establishing an integrated paradigm for stability and status quo analyses.     

Activity graphs: A language for flexible consultation systems

An intelligent decision system (IDS) uses artificial intelligence principles to deliver automated, interactive decision analysis (DA) consultations. Network methods adapted from operations research underlie two key IDS components: influence diagrams and activity graphs. Influence diagrams, which are familiar to DA researchers and practitioners, represent decision problems inevent space. Activity graphs, which are introduced in this paper, represent processes inaction space. While activity graphs can represent any process, we use them as a knowledge-engineering and programming language to represent the process knowledge of skilled decision analysts in the context of a specific class of decisions. This paper defines activity graphs as an extension of directed AND-OR graphs. Anactivity tree is a directed AND-OR tree consisting of nodes, which may contain activities (small computer programs) and connectors that establish logical relationships among nodes and define logical resolution agendas. Anactivity graph is a directed, multiply connected network of activity trees. Activity graphs may involve recursion. Development of the activity graph language is motivated by our desire to enable professional decision analysts — or other experts — with limited advanced programming experience to design and build consultation systems that combine the guidance offered by protocol systems with the flexibility and generality of transaction systems. This paper defines the activity graph language in detail. A simple example illustrates key concepts. The paper also discusses our experience using a computer system that implements activity graphs for developing commercial IDSs.     

An evolutionary approach to construction of outranking models for multicriteria classification: The case of the ELECTRE TRI method

Outranking methods constitute an important class of multicriteria classification models. Often, however, their implementation is cumbersome, due to the large number of parameters that the decision maker must specify. Past studies tried to address this issue using linear and nonlinear programming, to elicit the necessary preferential information from assignment examples. In this study, an evolutionary approach, based on the differential evolution algorithm, is proposed in the context of the ELECTRE TRI method. Computational results are given to test the effectiveness of the methodology and the quality of the obtained models.     

Cycles of violence: A dynamic control analysis

We introduce and analyze a simple model of cycles of violence in which oscillations are generated when surges in lethal violence shrink the pool of active violent offenders. Models with such endogenously induced variation may help explain why historically observed trends in violence are generally not well correlated with exogenous forcing functions, such as changes in the state of the economy. The analysis includes finding the optimal dynamic trajectory of incarceration and violence prevention interventions. Those trajectories yield some surprising results, including situations in which myopic decision makers will invest more in prevention than will far-sighted decision makers.     

DEA as a tool for bankruptcy assessment: A comparative study with logistic regression technique

This paper proposes data envelopment analysis (DEA) as a quick-and-easy tool for assessing corporate bankruptcy. DEA is a non-parametric method that measures weight estimates (not parameter estimates) of a classification function for separating default and non-default firms. Using a recent sample of large corporate failures in the United States, we examine the capability of DEA in assessing corporate bankruptcy by comparing it with logistic regression (LR). We find that DEA outperforms LR in evaluating bankruptcy out-of-sample. This feature of DEA is appealing and has practical relevance for investors. Another advantage of DEA over LR is that it does not have assumptions associated with statistical and econometric methods. Furthermore, DEA does not need a large sample size for bankruptcy evaluation, usually required by such statistical and econometric approaches. The need for such a large sample size is a significant disadvantage to practitioners when investment decisions are made using small samples. DEA can bypass such a difficulty related to a sample size. Thus, DEA is a practically appealing method for bankruptcy assessment.     

Models and software for improving the profitability of pharmaceutical research

The pharmaceutical industry is highly competitive, and the discovery and development of new drugs is extremely expensive and time consuming. This paper is a contribution to the task of improving the effectiveness of pre-clinical research. Our model investigates for any given project the number of lead series which should if necessary be optimised in the search for a development compound which is sufficiently promising to proceed to clinical trials. The numbers of scientists which should be allocated to each research stage are also investigated. Two widely-applied profitability criteria are considered. Computer software designed to implement the optimisation calculations is described and shown to produce reasonable results, leading to a potentially dramatic improvement in profitability.     

Security protection design for deception and real system regimes: A model and analysis

In this paper, we model a possible deception system with the explicit purpose of enticing unauthorized users and restricting their access to the real system. The proposed model represents a system designer’s defensive actions against intruders in a way that maximizes the difference between the intruders’ cost and the system designer’s cost of system protection. Under the assumption of a dual entity system, the proposed model shows that intruders differ in behavior depending on the system’s vulnerability at the time of intrusion as well as depending on their own economic incentives. The optimal results of the proposed model provide the system designer with insights on how to configure the level of protection for the two systems.     

Rejoinder on: Queueing models for the analysis of communication systems

In this rejoinder, we respond to the comments and questions of three discussants of our paper on queueing models for the analysis of communication systems. Our responses are structured around two main topics: discrete-time modeling and further extensions of the presented queueing analysis.