A general formulation for some inconsistency indices of pairwise comparisons

Annals of Operations Research - We propose a unifying approach to the problem of measuring the inconsistency of judgments. More precisely, we define a general framework to allow several well-known...     

Inconsistency indices for pairwise comparisons and the Pareto dominance principle

We consider the problem of quantifying inconsistency in pairwise comparisons and valued-preferences. A wide range of indices have been proposed in the literature to perform this task, and two sets of conditions have been introduced to validate such indices. We summarize some criticisms from the literature and we add more evidence to show that neither of the two systems is adequate in its current formulation. Thanks to the widely accepted concept of weak Pareto dominance, we formulate a new property. We argue that a simple regularity condition and this new property can overcome the shortcomings of the two axiomatic systems, and represents a significantly simpler framework. Finally, we claim that, if we had resorted to strict Pareto dominance, we would have needed just one axiom.     

Judgement by pairwise comparisons

This article gives an alternative solution to the problem of W. Gaul and M. Schader (Vol. 4, no. 4 pp. 273–282).     

Information of relative pairwise comparisons

We study the amount of information obtained by a set of relative pairwise comparisons. From this point of view, two kinds of entropy are introduced for graphs.     

Ranking by pairwise comparisons for Swiss-system tournaments

Pairwise comparison matrices are widely used in multicriteria decision making. This article applies incomplete pairwise comparison matrices in the area of sport tournaments, namely proposing alternative rankings for the 2010 Chess Olympiad Open tournament. It is shown that results are robust regarding scaling technique. In order to compare different rankings, a distance function is introduced with the aim of taking into account the subjective nature of human perception. Analysis of the weight vectors implies that methods based on pairwise comparisons have common roots. Visualization of the results is provided by multidimensional scaling on the basis of the defined distance. The proposed rankings give in some cases intuitively better outcome than currently used lexicographical orders.     

On some convexity properties of the Least Squares Method for pairwise comparisons matrices without the reciprocity condition

The relaxation of the reciprocity condition for pairwise comparisons is revisited from the optimization point of view. We show that some special but not extreme cases of the Least Squares Method are easy to solve as convex optimization problems after suitable nonlinear change of variables. We also give some other, less restrictive conditions under which the convexity of a modified problem can be assured, and the global optimal solution of the original problem found by using local search methods. Mathematical and psychological justifications for the relaxation of the reciprocity condition as well as numerical examples are provided.     

Representing the strengths and directions of pairwise comparisons

We consider the problem of finding weights that well represent a set of pairwise multiplicative comparisons of a set of objects (as in the AHP and other methods). Our main contribution is a method for deriving such weights that takes into consideration not only the strengths of the pairwise comparisons, but also their directions. For example, if the comparison directions satisfy transitivity, then the weights produced by our method also satisfy transitivity (this is not always true for other methods). We also present a set of reasonable axioms for which our method is the (essentially) unique solution. Our method and axioms are closely related to those of Cook and Kress [Eur. J. Oper. Res. 37 (1988) 335]. Our method, like theirs, reduces to solving a linear program (hence it is different from the approach used in the AHP). For the special case that the comparison directions satisfy transitivity, our method is quite simple and reduces to performing a forward pass as in the critical path method.     

The categorical data analysis approach for ratio model of pairwise comparisons

In the Analytic Hierarchy Process (AHP), pairwise comparison values are elicited by a ratio scale representing a set of categories. In this article, we propose a method of statistical inference in AHP which pertains to cardinal representations of categorical variables. Relevant estimation and hypothesis testing procedures are addressed in this perspective.     

Incomplete pairwise comparisons in the analytic hierarchy process

The Analytic Hierarchy Process is a decision-analysis tool which was developed by T.L. Saaty in the 1970s and which has been applied to many different decision problems in corporate, governmental and other institutional settings. The most successful applications have come about in group decisionmaking sessions, where the group structures the problem in a hierarchical framework and pairwise comparisons are elicited from the group for each level of the hierarchy. However, the number of pairwise comparison necessary in a real problem often becomes overwhelming. For example, with 9 alternatives and 5 criteria, the group must answer 190 questions. This paper explores various methods for reducing the complexity of the preference eliciting process. The theory of a method based upon the graph-theoretic structure of the pairwise comparison matrix and the gradient of the right Perron vector is developed, and simulations of a series of random matrices are used to illustrate the properties of this approach.     

Inconsistency indices for pairwise comparison matrices: a numerical study

Evaluating the level of inconsistency of pairwise comparisons is often a crucial step in multi criteria decision analysis. Several inconsistency indices have been proposed in the literature to estimate the deviation of expert’s judgments from a situation of full consistency. This paper surveys and analyzes ten indices from the numerical point of view. Specifically, we investigate degrees of agreement between them to check how similar they are. Results show a wide range of behaviors, ranging from very strong to very weak degrees of agreement.     

A simplified implementation of the least squares solution for pairwise comparisons matrices

This is a follow up to “Solution of the least squares method problem of pairwise comparisons matrix” by Bozóki published by this journal in 2008. Familiarity with this paper is essential and assumed. For lower inconsistency and decreased accuracy, our proposed solutions run in seconds instead of days. As such, they may be useful for researchers willing to use the least squares method instead of the geometric means method.     

Convergence properties and practical estimation of the probability of rank reversal in pairwise comparisons for multi-criteria decision making problems

In this paper, we address the impact of uncertainty introduced when the experts complete pairwise comparison matrices, in the context of multi-criteria decision making. We first discuss how uncertainty can be quantified and modeled and then show how the probability of rank reversal scales with the number of experts. We consider the impact of various aspects which may affect the estimation of probability of rank reversal in the context of pairwise comparisons, such as the uncertainty level, alternative preference scales and different weight estimation methods. We also consider the case where the comparisons are carried out in a fuzzy manner. It is shown that in most circumstances, augmenting the size of the expert group beyond 15 produces a small change in the probability of rank reversal. We next address the issue of how this probability can be estimated in practice, from information gathered simply from the comparison matrices of a single expert group. We propose and validate a scheme which yields an estimate for the probability of rank reversal and test the applicability of this scheme under various conditions. The framework discussed in the paper can allow decision makers to correctly choose the number of experts participating in a pairwise comparison and obtain an estimate of the credibility of the outcome.     

FDR control for pairwise comparisons of normal means based on goodness of fit test

This study is undertaken to apply a bootstrap method of controlling the false discovery rate (FDR) when performing pairwise comparisons of normal means. Due to the dependency of test statistics in pairwise comparisons, many conventional multiple testing procedures can’t be employed directly. Some modified procedures that control FDR with dependent test statistics are too conservative. In the paper, by bootstrap and goodness-of-fit methods, we produce independent p-values for pairwise comparisons. Based on these independent p-values, plenty of procedures can be used, and two typical FDR controlling procedures are applied here. An example is provided to illustrate the proposed approach. Extensive simulations show the satisfactory FDR control and power performance of our approach. In addition, the proposed approach can be easily extended to more than two normal, or non-normal, balance or unbalance cases.     

The nearest adjoining order method for pairwise comparisons in the form of difference of ranks

The problem of ranking of elements from some finite set on the basis of nearest adjoining order method for pairwise comparisons is investigated in this paper. It is assumed that in the set under consideration there exists a weak preference relation, which is to be identified (estimated) on the basis of pairwise comparisons in the form of difference of ranks. Moreover, the results of comparisons may be disturbed with random errors; the assumptions made about error distributions are not restrictive. The paper comprises: the problem formulation (definitions, assumptions, and optimisation problem, which provides the NAO solution) and the theoretical background – the form of distributions of random variables which make it possible to determine the properties of NAO solution, in particular, evaluation of the probability, that the NAO solution is equivalent to the errorless one. The approach presented in the paper can be extended to the case of more than one comparison for each pair of elements, i.e., completely formalised multi-experts ranking procedure. This revised version was published online in June 2006 with corrections to the Cover Date.     

Compromising prioritization from pairwise comparisons considering type I and II errors

We explore an important problem in prioritizing product design alternatives, using a real-world case. Despite the importance of prioritization in the area of new product development, the development of systematic schemes has been limited and the concepts and methods developed in the decision analysis area do not seem to be used actively. Therefore, we propose a new method, referred to as the compromising prioritization technique, to prioritize the product design alternatives based on paired comparisons. It introduces type I and type II errors and compromises these two errors to arrive at a desirable order of alternatives. To accomplish this, the two indices of homogeneity and separation are developed together with a heuristic algorithm. A comparative study is also conducted to support our method for use in product development and analogous areas. We then demonstrate how to use the developed compromising prioritization technique using a case study on the asymmetric digital subscriber line (ADSL)-based high-speed internet service product.     

Rank reversals in multi-criteria decision analysis with statistical modelling of ratio-scale pairwise comparisons

In multi-criteria decision analysis, the overall performance of decision alternatives is evaluated with respect to several, generally conflicting decision criteria. One approach to perform the multi-criteria decision analysis is to use ratio-scale pairwise comparisons concerning the performance of decision alternatives and the importance of decision criteria. In this approach, a classical problem has been the phenomenon of rank reversals. In particular, when a new decision alternative is added to a decision problem, and while the assessments concerning the original decision alternatives remain unchanged, the new alternative may cause rank reversals between the utility estimates of the original decision alternatives. This paper studies the connections between rank reversals and the potential inconsistency of the utility assessments in the case of ratio-scale pairwise comparisons data. The analysis was carried out by recently developed statistical modelling techniques so that the inconsistency of the assessments was measured according to statistical estimation theory. Several type of decision problems were analysed and the results showed that rank reversals caused by inconsistency are natural and acceptable. On the other hand, rank reversals caused by the traditional arithmetic-mean aggregation rule are not in line with the ratio-scale measurement of utilities, whereas geometric-mean aggregation does not cause undesired rank reversals.