Evaluating soft OR: some reflections on an apparently ‘unsuccessful’ implementation using a Soft Systems Methodology (SSM) based approach

This paper reflects on a case study in which a soft OR problem structuring approach, based on Soft Systems Methodology (SSM), was used to help in the design of an information system for health service users providing care in the community in a part of the South and West Health Region in the UK. The paper reflects on why an apparently ‘successful’ structuring led to an apparently ‘unsuccessful’ implementation. As part of this reflective process, the paper makes a contribution to the soft OR literature by reviewing some of the approaches taken in the literature to the evaluation of success of problem structuring using soft OR. This review reveals a lack of evaluative criteria, a lack of clarity over the identification of users, and a failure to differentiate criteria associated with ‘structuring’ and ‘implementation’ issues. An evaluative framework is proposed, which is applied retrospectively to the case study.     

Mixing methodologies to enhance the implementation of healthcare operational research

Healthcare Operational Research (OR) has had limited success in achieving a sufficient level of stakeholder acceptance to lead to implementation of results. This research study is aimed at combining OR methodologies to achieve greater acceptance of results for organizational change. Patient flow delays in the Paediatric intensive care unit (PICU) at Children's Hospital of Wisconsin were identified using case studies. Patient flow was modelled using hard OR (simulation) using case study findings, with active stakeholder participation, and enhanced with outcomes research. Outcomes research allowed achieving greater clinical relevance of OR findings. Results from hard OR, particularly for politically sensitive issues, were persuasive but inadequate to result in change. Soft OR (cognitive mapping) was used to identify new issues and enhance results. Consequently, the planned PICU expansion was deferred, and resources focused on improving staffing strategies identified independently from both hard and soft OR. These findings suggest that: (1) Outcomes research can play a unique role to enhance results in healthcare OR. (2) After obtaining a holistic understanding of the system using hard and soft OR, stakeholders are willing to implement results from each independently, supporting the development of a common form of knowledge, which is consistent with Mixed-Mode Modelling. (3) Hard OR, enhanced with outcomes research, with active stakeholder participation, and combined with soft OR, results in greater acceptance and sustained organizational change.     

Multimethodology in series and parallel: strategic planning using hard and soft OR

This paper examines two distinct ways in which hard and soft operational research (OR) methodologies can be combined, in series and in parallel. Multimethodology in series is acknowledged as the simpler and more common approach. Multimethodology in parallel is identified as having the potential to provide significant benefits to projects in political, changing, or ‘wicked’ contexts that multimethodology in series cannot. Observations regarding these approaches to multimethodology are examined in light of an information systems strategic planning project in the Australian public sector. Two distinct methodologies were combined in the project: soft systems methodology and project management. These methodologies are based on the soft and hard paradigms, respectively. However, findings in this paper have the potential to be transferred to combinations of other hard and soft OR methodologies.     

Putting Soft OR Methods to Work: Information Systems Strategy Development at Richards Bay

The development of an information systems (IS) strategy for a mineral extraction and processing plant operated by Rio Tinto Zinc (RTZ) in South Africa is described. The participative approach used was developed from soft OR and systems thinking. The aim of the Richards Bay project was to test commercially whether the Participative Information Management Strategy (PIMS) methodology could be used as a standard method to be applied to the development of an information systems strategy at any mine site. The paper describes the implementation of the process and the problems encountered. More generally it discusses the diffusion of new ideas in an international company, illustrated by the use of the participative approach for other activities including systems development. Conclusions are drawn about the value of participative approaches based on soft OR in this context.     

Contemporary OR/MS in strategy development and policy-making: some reflections

It is now widely accepted that strategy-making and policy development require both rational analysis and an ability to work with insights that are sometimes hard to pin down. Can operational research (OR) contribute to this process in which soft and hard are interwoven? Simon's longstanding distinction between substantive and procedural rationality is helpful in addressing this question. Undoubtedly, OR has made major contributions to strategy development, although there has been a marked tendency to argue for even greater use of substantive rationality. In addition, some soft OR methods are also successfully used in strategy development as ways to provide procedural rationality. Add to this the suggestion of Sagasti that metaphor and language are powerful tools in strategizing, then there is a powerful case for the greater use of OR/management science is strategy development and policy-making.     

Widening requirements capture with soft methods: an investigation of hybrid M&S studies in health care

A simulation study consists of several stages: problem formulation, model implementation, verification and validation, experimentation and output data analysis. The application of multiple techniques in the model implementation stage is referred to as hybrid simulation, which we distinguish in this paper from a hybrid M&S study, the latter referring to studies that apply methods and techniques from disciplines like Operations Research (OR), Systems Engineering and Computer Science to one or more stages of a simulation study. Our paper focuses on the contribution of soft OR methods in the problem formulation stage of a simulation study (and by extension a hybrid M&S study). Soft Systems Methodology (SSM) has, arguably, been the most widely used qualitative approach for eliciting system requirements. In this paper, we present Qualitative System Dynamics (QSD), a soft systems method, as having potential use in the problem formulation stage of a healthcare M&S study. The contribution of this paper is thus twofold: (1) a review of the literature in SSM for healthcare operations management and (2) an examination of QSD as an additional soft OR method, complementing (rather than supplanting) existing approaches, which can further aid the understanding of the system in the problem formulation/conceptual modelling stage of a hybrid M&S study.     

Learning to improve reliability during system development

Research, conducted in collaboration with a leading aerospace manufacturer, aimed to facilitate learning in order to improve the reliability of engineering systems during their development phase. In particular, the processes and mathematical models used during reliability growth testing were investigated to assess how they might be better used to support this improvement. This required both soft and hard OR approaches to be adopted. For example, information flows were mapped and reengineered in order to provide a basis for more effective data collection and feed-back to decision-makers. A new mathematical model that combines failure data with engineering judgement was developed to estimate reliability growth. The paper presents a case study describing the problem, the modelling conducted, the recommendations made and the actions implemented. The ways in which the researchers and the manufacturer learnt to improve both the modelling and the reliability growth testing process are reflected upon.     

Soft with a hard centre: discrete-event simulation in facilitation

Discrete event simulation is normally described as a ‘hard’ OR technique. This may not, however, always be the case. An example of a simulation of a user support helpline is described which, it is argued, has many of the traits of a ‘soft’ OR intervention. In particular, the study involved a facilitated discussion around a simulation model about possible improvements to a problem situation. The nature of the intervention is considered from both a methodological and paradigmatic perspective, and conclusions are drawn about where the intervention lies on a ‘hard’ to ‘soft’ continuum. It is argued that ‘soft’ issues need to be subsumed into the prescribed methodology for discrete-event simulation.     

Fifty years of systems thinking for management

The point of this paper is to provide an account of the last 50 years of systems thinking applied to management that is insightful and useful to those interested in the theory and practice of operational research (OR). In seeking to fulfil this purpose, it employs Boulding's well-known ‘hierarchy of complexity’ to think through the reasons for the emergence of different strands of applied systems thinking and to detail their strengths. In theoretical terms, operational researchers will find a number of the key issues that have engaged their field (eg, hard versus soft approaches) mirrored in debates that have taken place between systems thinkers. They may discover new theoretical avenues to follow to advance their discipline. OR practitioners may also be surprised by the nature and scope of the systems applications described and conclude that systems approaches should be added to their own intervention strategies. At the least, the paper is designed to reinvigorate discussion around the relationship between OR and systems thinking that has occasionally surfaced over the last half century but has never been satisfactorily concluded.     

The role of OR in shaping the future: smart bits,helpful ways and things that matter

This paper attempts to set out a personal view of the future of OR in the UK. Against a context of a changing political, economic, social and technical environment those areas where OR can contribute are identified. Three areas are selected and illustrated with examples from personal experience. Firstly, in a world awash with data, the smart bits, the algorithms are often primitive. OR is successfully established in this area and can maintain itself as the pre-eminent provider of smart bits. Secondly, OR has developed helpful ways, consulting approaches, to guide successful interventions. Here OR competes with other professional groups. OR has developed unique approaches but will need to contest the ground, vigorously and with imagination. Thirdly, OR can contribute to things that matter, important strategic, political and social issues of the day. OR will have no automatic right to be heard. The contribution will depend on the motivation and skill of key individuals. The recent trend away from in-house OR groups to external consultancies and small firms provides OR with the opportunity to become the discipline that underpins consultancy, offering exciting opportunities for young academics and practitioners.     

The impact of contingency factors on validation of problem structuring methods

Although much has been written about validating Decision support systems andother ‘hard’ OR models, less has been written about validating‘soft’ OR models. This article seeks to determine which contingencyfactors known to influence DSS validation are also important factors influencingproblem structuring methods (PSM) validation. In addition, after consultationwith PSM experts other contingency factors influencing PSM validation areproposed. Evidence from these PSM experts concerning the levels of influence ofthese factors is used to support proposals for a contingency approach to PSMvalidation.     

Wavelet Shrinkage Denoising Using the Non-Negative Garrote

Abstract

In this article, we combine Donoho and Johnstone's wavelet shrinkage denoising technique (known as WaveShrink) with Breiman's non-negative garrote. We show that the non-negative garrote shrinkage estimate enjoys the same asymptotic convergence rate as the hard and the soft shrinkage estimates. Simulations are used to demonstrate that garrote shrinkage offers advantages over both hard shrinkage (generally smaller mean-square-error and less sensitivity to small perturbations in the data) and soft shrinkage (generally smaller bias and overall mean-square-error). The minimax thresholds for the non-negative garrote are derived and the threshold selection procedure based on Stein's unbiased risk estimate (SURE) is studied. We also propose a threshold selection procedure based on combining Coifman and Donoho's cycle-spinning and SURE. The procedure is called SPINSURE. We use examples to show that SPINSURE is more stable than SURE: smaller standard deviation and smaller range.     

A transdisciplinary approach for formalized integrated planning and decision-making in complex systems

Multi-sectoral systems, such as cities or regional economies, face strategic challenges of optimal development due to the complexity of interacting perspectives, interests, and preferences of decision-makers and stakeholders. In coping with these challenges, integrated approaches in strategic planning and decision support promise to generate more efficient and effective results than sectoral approaches. In this article, we follow the concept of operational research (OR) by applying a formalized approach to cross-sectoral integrated planning and decision-making in complex, i.e., multi-sectoral systems. The presented Transdisciplinary Integrated Planning and Synthesis (TIPS) approach relies on (soft) OR methods combined in a multi-methodological framework that is adapted to the cognitive skills and habits of the stakeholders and experts involved in mutual and joint learning processes (transdisciplinarity). This methodological contribution structures, formalizes, and empirically illustrates cross-sectoral integrated strategic planning step by step, and evaluates the TIPS methodology against requirements for integrated approaches derived from literature.     

Using OR for diagnosis and facilitation in change programmes: a university application

In this paper, a methodology for the ongoing diagnosis and facilitation (when required) of organizational change management programmes in an Australian University is described. The application of this methodology on an annual basis requires the assessment of research attitudes and behaviour, areas that have assumed considerable importance within universities around the world in the last decade. This increasing importance associated with research stems in part (a significant part) from the linkage of research quantum (output) of universities to their funding from government. The methodology developed in this paper embraces a wide range of OR-type techniques as well as a range of change management tools from human resources management. These two sets of techniques and tools work together as tools in their own right as well as providing the infrastructure to achieve the objectives. The developed methodology is an interesting blend of hard techniques and ‘soft’ approaches implemented through a soft heuristic, indeed this application is an example of mixed-mode modelling. The OR techniques (conventional in both their nature and application) are comprised of social judgement theory (for bench-marking research attitudes) and integer linear programming (for setting research targets). The results of the application of the developed methodology are discussed in terms of the effect on research quantum over a 5-year period. A similar methodology could be developed for a change process covering aspects of an organization other than research output in a number of countries.     

Recent Developments in Critical Management Science

For much of its history management science had a quantitative and technical emphasis. More recently, there has been a move towards more subjective approaches such as ‘soft OR’ and ‘soft systems’. Currently, there is interest in ‘critical management science’ drawing on critical theory, particularly the work of Habermas. This paper reviews developments in critical management science, in particular critiques of traditional and ‘soft’ management science; Jackson and Keys' system of systems methodology; critical management science methodologies and the problem of power in bringing about change; and the postmodernist critique.     

Availability and maintenance modeling for systems subject to dependent hard and soft failures

This paper develops availability and maintenance models for single‐unit systems subject to dependent hard and soft failures. A hard failure stops the system immediately, whereas a soft failure only reduces the performance capacity of the system. Dependence between these 2 types of failures is reflected in the fact that each soft failure directly increases the hazard rate of the hard failure. On the basis of such interaction, we derive recursive equations for the system reliability and availability functions. To detect both types of failures, inspections are executed periodically. Furthermore, we investigate the optimal inspection policy via the minimization of the expected cost per unit time. The applicability of the developed availability and maintenance models is validated by a case study on an electrical distribution system.     

Combining PSMs with hard OR methods: the philosophical and practical challenges

Combinations of problem structuring methods with hard OR methodologies are seldom described in the literature. This paper will reflect on the barriers to such combinations that can be seen at the philosophical level—paradigm incommensurability—and cognitive level—type of personality and difficulty of switching paradigm. This paper examines the combination of Soft Systems Methodology and Discrete Event Simulation within an Intermediate Care case study. The paper will argue, by way of the practical application, that these problems are not insurmountable and that the result can be seen as interplay of the soft and hard paradigms. The idea of yin and yang is proposed as a metaphor for this process.     

Wisdom,decision support and paradigms of decision making

Many decision support tools have been developed over the last 20 years and, in general, they support what Simon termed substantive rationality. However, such tools are rarely suited to helping people tackle wicked problems, for which a form of procedural rationality is better suited. Procedurally rational approaches have appeared in both management science and computer science, examples being the soft OR approach of cognitive mapping and the design rationale based on IBIS. These approaches are reviewed and the development of Wisdom, a procedurally rational decision support process and accompanying tool, is discussed and evaluated.