Validation of decision-aiding spreadsheets: the influence of contingency factors

This paper describes an investigation into the influence of contingency factors on the validation of spreadsheet-based decision support systems (DSS), and develops a methodological framework for validation that takes account of the effect of contingency factors. The research extends and confirms previous research by one of the authors who identified relevant contingency factors. The perceived influence of these contingency factors on validation effort was investigated by an empirical study, and the results of this study were then used to derive contingency-related guidance for validation. This was evaluated by a sample of practitioners and academics with experience in the development of spreadsheet-based DSS, and was then incorporated into a methodological framework for validation. The framework is designed to enable managers, decision-makers, project leaders and other non-OR specialists to identify the extent of validation effort likely to be required to ensure that the spreadsheet models that they build are valid. The research has implications for the validation of other types of OR models.     

Some observations on practising successful operational research

Much has been written about the decline of Operational Research. In the 1960s it had already been proclaimed dead and the decreasing number of OR departments in industrial enterprises seems to support this thesis. The question seems to be: ‘Why is OR apparently disappearing at a time when the major obstacles to OR applications in the past, namely lack of computer-readable data, lack of software and hardware have been overcome?’ I report on 30 years of successful external OR activities and offer some observations which I have made while being active at a university and in OR companies in Germany.     

Model validation in operations research

Numerous articles have appeared in the literature expressing different degrees of concern with the methodology of OR in general and with the validation of OR models in particular. Suggestions have been formulated to remove some of the shortcomings of the methodology as currently practised and to introduce modifications in the approach because of the changing nature of the problems tackled. Advances in modeling capabilities and solution techniques have also had considerable impact on the way validation is perceived. Large scale computer-based mathematical models and especially simulation models have brought new dimensions to the notion of validation. Terms like ‘confidence’, ‘credibility and reliability’, ‘model assessment and evaluation’, ‘usefulness and usability of the model’ have become rather common. This paper is an attempt, through an interpretation of the literature, to put model validation and related issues in a framework that may be of use both to model-builders and to decision-makers.     

A survey of contingency factors affecting the validation of end-user spreadsheet-based decision support systems

Decision support systems have become widespread in recent years. However, most of these systems are not carefully validated and this has led in some cases to serious errors. One of the principal reasons for the lack of validity is the lack of an operationalised validation methodology. A simplified validity framework has already been developed by the authors as the first part of a 3-phase approach towards devising a practical validation methodology. This paper describes the second phase, reporting as it does on a survey that has been undertaken to unearth the contingency factors that influence how the process of validation, employing the validity framework, will be carried out for a particular kind of decision support system—one involving a spreadsheet model.     

Soft with a hard centre: discrete-event simulation in facilitation

Discrete event simulation is normally described as a ‘hard’ OR technique. This may not, however, always be the case. An example of a simulation of a user support helpline is described which, it is argued, has many of the traits of a ‘soft’ OR intervention. In particular, the study involved a facilitated discussion around a simulation model about possible improvements to a problem situation. The nature of the intervention is considered from both a methodological and paradigmatic perspective, and conclusions are drawn about where the intervention lies on a ‘hard’ to ‘soft’ continuum. It is argued that ‘soft’ issues need to be subsumed into the prescribed methodology for discrete-event simulation.     

Analysing factors affecting the choice of emergent human resource capital

Ever since the announcement that UK higher education (HE) fees were to increase up to £9000, many universities have expressed concern about how to attract the best students while offering choice and promoting student mobility through broader access. This in turn has led to questioning how such complexity might be modelled using sophisticated operational research (OR) techniques. Because higher education institutions (HEIs) are now beginning to compete ‘against’ rather than ‘with’ each other, potential students are paying increased attention to where and what they will study, as well as graduate opportunities after their degree. Hence, the quality of education services becomes increasingly vital for HEIs in order to attract potential students. This study seeks to develop a framework of those factors affecting international (non-EU) students’ choice of institution. A number of factors were identified and collated from the existing literature providing a solid foundation on which to base this research. A survey approach was utilised to determine the importance of identified factors based on data collected from students of two different types of institutions (a university and a feeder institution). Through a better understanding of factors such as social influence, financial and career opportunities, universities should be well placed to construct models underpinned by OR principles that will promote scenario modelling and planning within HE.     

Experiences in using a contingency factor-based validation methodology for spreadsheet DSS

The use of spreadsheets has become routine in all aspects of business with usage growing across a range of functional areas and a continuing trend towards end user spreadsheet development. However, several studies have raised concerns about the accuracy of spreadsheet models in general, and of end user developed applications in particular, raising the risk element for users. High error rates have been discovered, even though the users/developers were confident that their spreadsheets were correct. The lack of an easy to use, context-sensitive validation methodology has been highlighted as a significant contributor to the problems of accuracy. This paper describes experiences in using a practical, contingency factor-based methodology for validation of spreadsheet-based DSS. Because the end user is often both the system developer and a stakeholder, the contingency factor-based validation methodology may need to be used in more than one way. The methodology can also be extended to encompass other DSS.     

On the Nature of OR — Entering the Fray

In 1980 a seminar of operational researchers and practitioners from a variety of different countries concluded that there was a remarkable consensus about the way that the process of operational research and systems analysis should be understood and practised. Since then the development of new methods has continued apace, stimulated by interaction with the social sciences and paradigms other than natural science. This paper enters the debate, started in this journal, about the nature of OR's status as science or technology. It recognizes the historic claims of science on OR, but it suggests that for ‘private’ consumption, technology will prove more fruitful, and for ‘public’ consumption practitioners will increasingly adopt the label ‘consultant’. Ackoff's original six-step method of OR is updated to reflect current OR practice. The process suggested acknowledges the importance of context, socio-political factors and the growing role of facilitation.     

Writing practitioner case studies to help behavioural OR researchers ground their theories: application of the mangle perspective

Behavioural research into the practice of OR needs to be grounded. Case studies written by practitioners can potentially help address this need but currently most do not. The paper explores a way of describing OR projects that place the emphasis on the ‘actors’ who provide the motivating force and the consequences of their actions. The ‘mangle’ perspective focuses on the dynamic intertwining of people, technology and concepts; this can provide the basis for an insightful narrative describing the reality of the project in terms of the planned approach, the problems met and the outcomes. Two examples are given, one of a conventional model building exercise, the second of a ‘soft OR’ intervention: both describe projects conducted by practitioners for commercial purposes. It is concluded that, by using the mangle perspective, the OR case writer can winnow the wheat from the chaff in order to write a succinct informative narrative, a narrative that could be utilized by behavioural OR (BOR) researchers. It is further concluded that BOR researchers should engage with ‘practice theory’ to deepen their understanding of what actually happens in projects.     

The development,validation and implementation of knowledge-based systems

It is now ten years since Operational Research, and other groups that support management decision making, seized upon Knowledge-based Systems (KBS) as an alternative set of techniques and methods for building systems that support, augment or automate decision making. Success, as with most information technologies, has been mixed. This paper briefly reviews where KBS have been successful, and what benefits have been derived. It notes that interest in KBS has been rekindled by the present interest in business process re-engineering. Given the mature state of development tools, most KBS developers are now focusing on methodological and life-cycle issues. The paper reviews progress in understanding and using development methodologies, validation methods (now generally considered as crucial to technical success) and implementation. Some ideas about where KBS ‘fits in’ to the array of tools, techniques and methodologies available to the modern analyst and system developer are presented.     

Recent Developments in Critical Management Science

For much of its history management science had a quantitative and technical emphasis. More recently, there has been a move towards more subjective approaches such as ‘soft OR’ and ‘soft systems’. Currently, there is interest in ‘critical management science’ drawing on critical theory, particularly the work of Habermas. This paper reviews developments in critical management science, in particular critiques of traditional and ‘soft’ management science; Jackson and Keys' system of systems methodology; critical management science methodologies and the problem of power in bringing about change; and the postmodernist critique.     

Problem structuring methods: theorizing the benefits of deconstructing sustainable development projects

Problem structuring methods or PSMs are widely applied across a range of variable but generally small-scale organizational contexts. However, it has been argued that they are seen and experienced less often in areas of wide ranging and highly complex human activity—specifically those relating to sustainability, environment, democracy and conflict (or SEDC). In an attempt to plan, track and influence human activity in SEDC contexts, the authors in this paper make the theoretical case for a PSM, derived from various existing approaches. They show how it could make a contribution in a specific practical context—within sustainable coastal development projects around the Mediterranean which have utilized systemic and prospective sustainability analysis or, as it is now known, Imagine. The latter is itself a PSM but one which is ‘bounded’ within the limits of the project to help deliver the required ‘deliverables’ set out in the project blueprint. The authors argue that sustainable development projects would benefit from a deconstruction of process by those engaged in the project and suggest one approach that could be taken—a breakout from a project-bounded PSM to an analysis that embraces the project itself. The paper begins with an introduction to the sustainable development context and literature and then goes on to illustrate the issues by grounding the debate within a set of projects facilitated by Blue Plan for Mediterranean coastal zones. The paper goes on to show how the analytical framework could be applied and what insights might be generated.     

Facilitation: some contributions to theorising the practice of operational research

The paper examines the contributions of two distinct areas of theory to the practice of facilitation in operational research. The first area examined is deconstruction, presented in terms of its theorising about the role of language in social interaction, and as a method for describing and critiquing different positions present in written or verbal texts, for example positions inherent in the views of different stakeholders or underlying alternative strategies or options under consideration. The second area examined is ‘re-evaluation counselling’ or ‘co-counselling’, presented as a body of theory about human emotion and the impact of distress, and associated with a range of tools and techniques for use in stimulating creative thinking and analysing and responding to emotional outbursts during the course of facilitated work sessions. Examples from OR practice in the field of group decision-making and action illustrate the discussion in both these sections.     

Information in the operations research process

The article deals with information in the OR process. The value of an ‘optimal’ model solution which is not implemented is not very high. Many errors are due to a too narrow formulation of the problem, often a formulation which has focused on purely technical and economic aspects, or a formulation which has been adapted for a standard model. The concepts and models treated in the article take their point of departure in the actual problem. The models can be used as a frame of reference by various individuals and groups which are involved in OR work.     

OR in strategic land-use planning

An important historic strategic application of OR has been in the field of land-use and development plan production. Changes in Government policy and legislation have led to varying levels of interest in plan production. Three post-war cycles of ‘enthusiasm for plans’ can be identified. Whilst the first was rooted very much in the Architectural Design tradition, the second led to significant developments in OR, with far wider application. Subsequent reduced Governmental enthusiasm for ‘Development Plan production’ led to considerable atrophy of relevant skills in the planning community, including those derived from OR. However, the current ‘third period of post-war enthusiasm for planning’, reinforced by environmental concerns, has revived the need for relevant skills. It is suggested that, whilst the deficit in skills and their application remains high, there are some encouraging signs. Moreover, substantial progress in the field of ‘soft OR’ offers opportunities to both the OR and planning communities.     

Causal study of low stakeholder engagement in healthcare simulation projects

Stakeholder engagement plays a fundamental role in the success of ‘operational research’ initiatives including simulation projects. However, there is little empirical evidence of real engagement in the context of healthcare simulation. This paper principally examines this issue and aims to provide insights into the possible causes. The paper reports on the results of a literature review and 10 field studies within the UK healthcare settings, supplemented with the authors’ experience in order to arrive at an initial list of the causes, which will then be tested through a survey of expert opinions. Twelve primary and 26 secondary causal factors, which received statistically significant level of agreement from the experts, are presented in a fish-bone diagram. The findings indicate that communication gap between simulation and stakeholder groups is the top primary factor contributing the most to the poor stakeholder engagement in healthcare simulation projects, followed by ‘poor management support’, ‘clinician’s high workload’ and ‘failure in producing tangible and quick results’.     

A survey of OR utilization in Greece

A recent survey concerning OR utilization in firms and public organizations in Greece showed a rather poor profile. The survey covered 20 big organizations including the National Defense General Staff, national enterprises of public utilities, banks and firms from the textile, cement, oil, construction and other industries. Only a few of these organizations are using OR to a significant degree. The rest reported ‘poor’ or ‘no’ use of OR.Research/implementation problems most frequently appearing in public sector organizations include lack of data, insufficient qualified personnel and unfavorable attitude towards cooperation by affected parties. In private firms, the need for quick solutions and the use of OR analysts in line functions quite often imply the use of a ‘rule of thumb’ rather than a mathematical OR technique. In addition, OR study results quite often do not get implemented. One of the reasons is resistance by the affected parties, considering any proposal for reorganization put forward by other parties as a threat.The use of OR has proven beneficial for the using firm or organization. These benefits, however, are not as great as one might expect, due to the problems mentioned above. The future growth of OR utilization in Greece will depend on developments concerning the education of managers in OR methods, creation of OR groups in firms that will seek solutions to real-world, worth-while problems, publicizing successful OR applications, active support by top management, reforms in the educational system etc. The real push for OR utilization in Greece will come from drastic changes in administrative practices, which are directly related to the socioeconomic developments in the country.     

The role of operational research in organizational decision making

An increase in the complexity of modern organisations and their interactions with the environment and each other has lead to a corresponding increase in the complexity of the organizational decision making process and the various factors on which decisions need to be based. Although there is currently much debate amongst both OR academics and practitioners as to how OR should respond to this increase in complexity the public manifestations of this debate have been confined largely to academics talking to each other.This paper is intended to contribute to a rectification of this situation by indicating how OR is currently used in one particular large, bureaucratic, high technology organization, and how it is spontaneously evolving to meet the demands of increased complexity. This discussion is based on an empirical study, and on experience of the author of OR in this and other similar organizations.It is concluded that very few major decisions appear to be directly influenced by OR studies although the latter are frequently used in a ‘justification’ role to support decisions already taken. An increase in awareness is however becoming apparent of the many indirect benefits that may accrue to the organization from the OR process. The nature of these indirect benefits and the ways in which they might be increased are discussed.     

On becoming expert in the use of problem structuring methods

While much has been written about the theory and practice of using problem structuring methods (PSMs) there is little formal discussion of how individuals may effectively learn about their use. In this paper it is argued that a foundation for designing and providing a suitable learning environment lies in understanding the nature of being an expert user of PSMs and the knowledge that such experts hold and deploy. An analysis of how knowledge and expertise are grounded in the work involved in using PSMs is offered. This leads to some proposals for what is required if processes to support the acquisition of expertise in the use of PSMs are to be implemented.