A flexible multicriteria decision-making methodology to support the strategic management of Science,Technology and Innovation research funding programs

Research funding programs are a policy instrument utilized by governments to influence the innovation process. They are usually elaborated, launched and managed by research funding agencies. In order to select the most adequate research projects, agencies often rely on the peer review process.This paper introduces a methodology to support funding decisions based on the peer review process. The methodology involves the use of a multicriteria decision model to support the assessment, evaluation, prioritization and selection of applications, under a multi-step decision-making process, which fits into a strategic management cycle within the agency. The Multiattribute Value Theory, being considered under a Value Focused Thinking approach, provides a basis for the construction of the multicriteria decision model. The good practices in peer review and also a logical framework for program management are considered by the methodology.A pilot study, presented in the paper, involved a retrospective implementation of a peer review process in the context of a program launched by the Ministry for Science, Technology, Innovations and Communications and the National Council of Technological and Scientific Development, in Brazil. The methodology allowed a clear distinction of roles. The agency staff in the role of decision-makers was responsible for making value judgments on behalf of the agency. The experts, in the role of committee members and ad hoc reviewers, contributed with their expertise by providing objective assessments. Such assessments served as a basis for evaluating the applications, characterizing the possible portfolios, and can be considered as data in future program evaluation studies.     

Development and implementation of a real time statistical control method to identify the start and end of the winter surge in demand for paediatric intensive care

Winter surge management in intensive care is hampered by the annual variability in the winter surge. We aimed to develop a real-time monitoring system that could promptly identify the start, and accurately predict the end, of the winter surge in a paediatric intensive care (PIC) setting. We adapted a statistical process control method from the stock market called “Bollinger bands” that compares current levels of demand for PIC services to thresholds based on the medium term average demand. Algorithms to identify the start and end of the surge were developed for a specific PIC service: the North Thames Children's Acute Transport Service (CATS) using eight winters of data (2005–12) to tune the algorithms and one winter to test the final method (2013/14). The optimal Bollinger band thresholds were 1.2 and 1 standard deviations above and below a 41-day moving average of demand respectively. A simple linear model was found to predict the end of the surge and overall demand volume as soon as the start had been identified. Applying the method to the validation winter of 2013/14 showed excellent performance, with the surge identified from 18th November 2013 to 4th January 2014.An Excel tool running the algorithms has been in use within CATS since September 2014. There were three factors which facilitated the successful implementation of this tool: the perceived problem was pressing and identified by the clinical team; there was close clinical engagement throughout and substantial effort was made to develop an easy-to-use Excel tool for sustainable use.     

Estimating the technical efficiency of health care systems: A cross-country comparison using the directional distance function

Economic activity produces not only desirable outputs but also undesirable outputs. Undesirable outputs are usually omitted from efficiency assessments (i.e., applications of Data Envelopment Analysis) which fail to express the true production process. The directional distance function model has been used for handling asymmetrically both desirable and undesirable outputs in the assessment process. In the present paper, we apply a generalized directional distance function to measure the efficiency of the health systems of 171 countries. We incorporate both desirable and undesirable outputs into the efficiency assessment without transforming the latter type of outputs into inputs or into their inverse form, as is done in most of the extant studies that deal with the measurement of health efficiency. The methodology that we apply introduces a modified definition of the efficiency score which yields results consistent with those obtained from radial DEA models. In addition, our results are independent of the length of the direction vector.     

Multicriteria decision support methodologies for auditing decisions: The case of qualified audit reports in the UK

All UK companies are required by company law to prepare financial statements that must comply with law and accounting standards. With the exception of very small companies, financial accounts must then be audited by UK registered auditors who must express an opinion on whether these statements are free from material misstatements, and have been prepared in accordance with legislation and relevant accounting standards (unqualified opinion) or not (qualified opinion). The objective of the present study is to explore the potentials of developing multicriteria decision aid models for reproducing, as accurately as possible, the auditors’ opinion on the financial statements of the firms. A sample of 625 company audited years with qualified statements and 625 ones with unqualified financial statements over the period 1998–2003 from 823 manufacturing private and public companies is being used in contrast to most of the previous works in the UK that have mainly focused on very small or very large public companies. Furthermore, the models are being developed and tested using the walk-forward approach as opposed to previous studies that employ simple holdout tests or resampling techniques. Discriminant analysis and logit analysis are also used for comparison purposes. The out-of-time and out-of-sample testing results indicate that the two multicriteria decision aid techniques achieve almost equal classification accuracies and are both more efficient than discriminant and logit analysis.     

An evolutionary approach to construction of outranking models for multicriteria classification: The case of the ELECTRE TRI method

Outranking methods constitute an important class of multicriteria classification models. Often, however, their implementation is cumbersome, due to the large number of parameters that the decision maker must specify. Past studies tried to address this issue using linear and nonlinear programming, to elicit the necessary preferential information from assignment examples. In this study, an evolutionary approach, based on the differential evolution algorithm, is proposed in the context of the ELECTRE TRI method. Computational results are given to test the effectiveness of the methodology and the quality of the obtained models.     

New approach for the assignment of new European agricultural subsidies using scores from data envelopment analysis: Application to olive-growing farms in Andalusia (Spain)

More than 60% of olive-growing farms in Andalusia (Spain) would have negative returns without European agricultural subsidies. Agenda 2000 criteria imply that agricultural subsidies currently play the role of enhancing the production quality and the environmental and social values of agriculture. Although the necessity of the modulation of subsidies is stated, the EU regulations do not state which objective criteria should be used or how they should be measured, despite the fact that regulations demand objectivity in this measurement.