Verbal ordinal classification with multicriteria decision aiding

Professionals in neuropsychology usually perform diagnoses of patients’ behaviour in a verbal rather than in a numerical form. This fact generates interest in decision support systems that process verbal data. It also motivates us to develop methods for the classification of such data. In this paper, we describe ways of aiding classification of a discrete set of objects, evaluated on set of criteria that may have verbal estimations, into ordered decision classes. In some situations, there is no explicit additional information available, while in others it is possible to order the criteria lexicographically. We consider both of these cases. The proposed Dichotomic Classification (DC) method is based on the principles of Verbal Decision Analysis (VDA). Verbal Decision Analysis methods are especially helpful when verbal data, in criteria values, are to be handled. When compared to the previously developed Verbal Decision Analysis classification methods, Dichotomic Classification method performs better on the same data sets and is able to cope with larger sizes of the object sets to be classified. We present an interactive classification procedure, estimate the effectiveness and computational complexity of the new method and compare it to one of the previously developed Verbal Decision Analysis methods. The developed and studied methods are implemented in the framework of a decision support system, and the results of testing on artificial sets of data are reported.     

Basis Function Adaptation in Temporal Difference Reinforcement Learning

Reinforcement Learning (RL) is an approach for solving complex multi-stage decision problems that fall under the general framework of Markov Decision Problems (MDPs), with possibly unknown parameters. Function approximation is essential for problems with a large state space, as it facilitates compact representation and enables generalization. Linear approximation architectures (where the adjustable parameters are the weights of pre-fixed basis functions) have recently gained prominence due to efficient algorithms and convergence guarantees. Nonetheless, an appropriate choice of basis function is important for the success of the algorithm. In the present paper we examine methods for adapting the basis function during the learning process in the context of evaluating the value function under a fixed control policy. Using the Bellman approximation error as an optimization criterion, we optimize the weights of the basis function while simultaneously adapting the (non-linear) basis function parameters. We present two algorithms for this problem. The first uses a gradient-based approach and the second applies the Cross Entropy method. The performance of the proposed algorithms is evaluated and compared in simulations. This research was partially supported by the Fund for Promotion of Research at the Technion. The work of S.M. was partially supported by the National Science Foundation under grant ECS-0312921.     

A Dual-Objective Evolutionary Algorithm for Rules Extraction in Data Mining

This paper presents a dual-objective evolutionary algorithm (DOEA) for extracting multiple decision rule lists in data mining, which aims at satisfying the classification criteria of high accuracy and ease of user comprehension. Unlike existing approaches, the algorithm incorporates the concept of Pareto dominance to evolve a set of non-dominated decision rule lists each having different classification accuracy and number of rules over a specified range. The classification results of DOEA are analyzed and compared with existing rule-based and non-rule based classifiers based upon 8 test problems obtained from UCI Machine Learning Repository. It is shown that the DOEA produces comprehensible rules with competitive classification accuracy as compared to many methods in literature. Results obtained from box plots and t-tests further examine its invariance to random partition of datasets. An erratum to this article is available at .     

Mathematical programming models for piecewise-linear discriminant analysis

Mathematical programming (MP) discriminant analysis models are widely used to generate linear discriminant functions that can be adopted as classification models. Nonlinear classification models may have better classification performance than linear classifiers, but although MP methods can be used to generate nonlinear discriminant functions, functions of specified form must be evaluated separately. Piecewise-linear functions can approximate nonlinear functions, and two new MP methods for generating piecewise-linear discriminant functions are developed in this paper. The first method uses maximization of classification accuracy (MCA) as the objective, while the second uses an approach based on minimization of the sum of deviations (MSD). The use of these new MP models is illustrated in an application to a test problem and the results are compared with those from standard MCA and MSD models.     

Classification systems based on rough sets under the belief function framework

In this paper, we present two classification approaches based on Rough Sets (RS) that are able to learn decision rules from uncertain data. We assume that the uncertainty exists only in the decision attribute values of the Decision Table (DT) and is represented by the belief functions. The first technique, named Belief Rough Set Classifier (BRSC), is based only on the basic concepts of the Rough Sets (RS). The second, called Belief Rough Set Classifier, is more sophisticated. It is based on Generalization Distribution Table (BRSC-GDT), which is a hybridization of the Generalization Distribution Table and the Rough Sets (GDT-RS). The two classifiers aim at simplifying the Uncertain Decision Table (UDT) in order to generate significant decision rules for classification process. Furthermore, to improve the time complexity of the construction procedure of the two classifiers, we apply a heuristic method of attribute selection based on rough sets. To evaluate the performance of each classification approach, we carry experiments on a number of standard real-world databases by artificially introducing uncertainty in the decision attribute values. In addition, we test our classifiers on a naturally uncertain web usage database. We compare our belief rough set classifiers with traditional classification methods only for the certain case. Besides, we compare the results relative to the uncertain case with those given by another similar classifier, called the Belief Decision Tree (BDT), which also deals with uncertain decision attribute values.     

Lost in a black-box? Interpretable machine learning for assessing Italian SMEs default

Academic research and the financial industry have recently shown great interest in Machine Learning algorithms capable of solving complex learning tasks, although in the field of firms' default prediction the lack of interpretability has prevented an extensive adoption of the black-box type of models. In order to overcome this drawback and maintain the high performances of black-boxes, this paper has chosen a model-agnostic approach. Accumulated Local Effects and Shapley values are used to shape the predictors' impact on the likelihood of default and rank them according to their contribution to the model outcome. Prediction is achieved by two Machine Learning algorithms (eXtreme Gradient Boosting and FeedForward Neural Networks) compared with three standard discriminant models. Results show that our analysis of the Italian Small and Medium Enterprises manufacturing industry benefits from the overall highest classification power by the eXtreme Gradient Boosting algorithm still maintaining a rich interpretation framework to support decisions.     

Visualizing preferences on spheres for group decisions based on multiplicative preference relations

Decision makers’ choices are often influenced by visual background information. One of the difficulties in group decision is that decision makers may bias their judgment in order to increase the possibility of a preferred result. Hence, the method used to provide visual aids in helping decision making teams both to observe the background context and to perceive outliers is an important issue to consider. This study proposes an extended Decision Ball model to visualize a group’s decisions. By observing the Decision Balls, each decision maker can: see individual ranking as well as similarities between alternatives, identify the differences between individual judgments and the group’s collective opinion, observe the clusters of alternatives as well as clusters of decision makers, and discover outliers. Thus, this method can help decision makers make a more objective judgment.     

An extension of the FlowSort sorting method to deal with imprecision

We propose an extension of the FlowSort sorting method to the case when there is imprecision on the input data. Within multicriteria decision aid, a lot of attention has been paid to sorting problems where a set of actions has to be assigned to completely ordered categories. However, few methods suit when the data or the parameters of the model are not precisely defined. In this paper, instead of reducing the imprecise data to single values, we consider that the sorting parameters or the data are defined by intervals. We analyse the properties usually required for a sorting method and illustrate this extension on a practical example.     

Support vector machines maximizing geometric margins for multi-class classification

Machine learning is a very interesting and important branch of artificial intelligence. Among many learning models, the support vector machine is a popular model with high classification ability which can be trained by mathematical programming methods. Since the model was originally formulated for binary classification, various kinds of extensions have been investigated for multi-class classification. In this paper, we review some existing models, and introduce new models which we recently proposed. The models are derived from the viewpoint of multi-objective maximization of geometric margins for a discriminant function, and each model can be trained by solving a second-order cone programming problem. We show that discriminant functions with high generalization ability can be obtained by these models through some numerical experiments.     

A novel approach to cutting decision trees

In data mining, binary classification has a wide range of applications. Cutting Decision Tree (CDT) induction is an efficient mathematical programming based method that tries to discretize the data set on hand by using multiple separating hyperplanes. A new improvement to CDT model is proposed in this study by incorporating the second goal of maximizing the distance of the correctly classified instances to the misclassification region. Computational results show that developed model achieves better classification accuracy for Wisconsin Breast Cancer database and Japanese Banks data set when compared to existing piecewise-linear models in literature. Furthermore, remarkable results are obtained for the well-known benchmarking data sets (Buba Liver Disorders, Blood Tranfusion and Pima Indian Diabetes) when compared to the original CDT model.     

Discriminant versus rough sets approach to vague data analysis

This paper presents a comparative study of the use of two different methods of data analysis on a common set of data. The first is a method based on rough sets theory and the second is the location model method from the field of discriminant analysis. To investigate the comparative performance of these methods, a set of real medical data has been used. The data considered are of both discrete and continuous character. During the comparison, particular attention is paid to data reduction and to the derivation of decision rules and classification functions from the reduced set.     

Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study

In this study, we present a comprehensive comparative analysis of kernel-based fuzzy clustering and fuzzy clustering. Kernel based clustering has emerged as an interesting and quite visible alternative in fuzzy clustering, however, the effectiveness of this extension vis-à-vis some generic methods of fuzzy clustering has neither been discussed in a complete manner nor the performance of clustering quantified through a convincing comparative analysis. Our focal objective is to understand the performance gains and the importance of parameter selection for kernelized fuzzy clustering. Generic Fuzzy C-Means (FCM) and Gustafson–Kessel (GK) FCM are compared with two typical generalizations of kernel-based fuzzy clustering: one with prototypes located in the feature space (KFCM-F) and the other where the prototypes are distributed in the kernel space (KFCM-K). Both generalizations are studied when dealing with the Gaussian kernel while KFCM-K is also studied with the polynomial kernel. Two criteria are used in evaluating the performance of the clustering method and the resulting clusters, namely classification rate and reconstruction error. Through carefully selected experiments involving synthetic and Machine Learning repository (http://archive.ics.uci.edu/beta/) data sets, we demonstrate that the kernel-based FCM algorithms produce a marginal improvement over standard FCM and GK for most of the analyzed data sets. It has been observed that the kernel-based FCM algorithms are in a number of cases highly sensitive to the selection of specific values of the kernel parameters.     

Asymptotic Limit of the Bayes Actions Set Derived from a Class of Loss Functions

Similarly to the determination of a prior in Bayesian Decision theory, an arbitrarily precise determination of the loss function is unrealistic. Thus, analogously to global robustness with respect to the prior, one can consider a set of loss functions to describe the imprecise preferences of the decision maker. In this paper, we investigate the asymptotic behavior of the Bayes actions set derived from a class of loss functions. When the collection of additional observations induces a decrease in the range of the Bayes actions, robustness is improved. We give sufficient conditions for the convergence of the Bayes actions set with respect to the Hausdorff metric and we also give the limit set. Finally, we show that these conditions are satisfied when the set of decisions and the set of states of nature are subsets of ^p.     

Multicategory classification via discrete support vector machines

Discrete support vector machines (DSVM), originally proposed for binary classification problems, have been shown to outperform other competing approaches on well-known benchmark datasets. Here we address their extension to multicategory classification, by developing three different methods. Two of them are based respectively on one-against-all and round-robin classification schemes, in which a number of binary discrimination problems are solved by means of a variant of DSVM. The third method directly addresses the multicategory classification task, by building a decision tree in which an optimal split to separate classes is derived at each node by a new extended formulation of DSVM. Computational tests on publicly available datasets are then conducted to compare the three multicategory classifiers based on DSVM with other methods, indicating that the proposed techniques achieve significantly higher accuracies. This research was partially supported by PRIN grant 2004132117.     

MPOC: an agglomerative algorithm for multicriteria partially ordered clustering

In the field of multicriteria decision aid, considerable attention has been paid to supervised classification problems where the purpose is to assign alternatives into predefined ordered classes. In these approaches, often referred to as sorting methods, it is usually assumed that classes are either known a priori or can be identified by the decision maker. On the other hand, when the objective is to identify groups (clusters) of alternatives sharing similar characteristics, the problem is known as a clustering problem, also called an unsupervised learning problem. This paper proposes an agglomerative clustering method based on a crisp outranking relation. The method regroups alternatives into partially ordered classes, based on a quality of partition measure which reflects the percentage of pairs of alternatives that are compatible with a decision-maker’s multicriteria preference model.     

Project prioritization under policy restrictions. A combination of MCDA with 0–1 programming

Multiple Criteria Decision Aid methods are increasingly used in financial decision making in order to capture the multifaceted character of modern enterprises activated in a complex and versatile market environment. This paper presents a multiple criteria approach for the selection of firms applying for financial support from public funds. Besides the budget constraint, the specific decision situation imposes the consideration of additional policy constraints that prevent from directly exploiting rankings provided by a multiple criteria method. In such a case the problem solution is to find a set of alternatives satisfying the constraints and at the same time maximizing a measure of global performance. The proposed procedure relies on the PROMETHEE V method which belongs to the well-known PROMETHEE family of multiple criteria outranking methods and is combined with an integer programming formulation capable to effectively deal with the problem’s combinatorial character. This method is modified in order to avoid any bias in the selection of the optimal set that may arrive because of the apparent contradiction between the rate of resources consumption and the coefficients of the alternatives in the additive objective function.     

Robust multi-criteria ranking with additive value models and holistic pair-wise preference statements

We consider a problem of ranking alternatives based on their deterministic performance evaluations on multiple criteria. We apply additive value theory and assume the Decision Maker’s (DM) preferences to be representable with general additive monotone value functions. The DM provides indirect preference information in form of pair-wise comparisons of reference alternatives, and we use this to derive the set of compatible value functions. Then, this set is analyzed to describe (1) the possible and necessary preference relations, (2) probabilities of the possible relations, (3) ranges of ranks the alternatives may obtain, and (4) the distributions of these ranks. Our work combines previous results from Robust Ordinal Regression, Extreme Ranking Analysis and Stochastic Multicriteria Acceptability Analysis under a unified decision support framework. We show how the four different results complement each other, discuss extensions of the main proposal, and demonstrate practical use of the approach by considering a problem of ranking 20 European countries in terms of 4 criteria reflecting the quality of their universities.     

MEDSS: A Multi-Agent Environmental Decision Support System framework for model management

Model Management Systems (MMS) have become increasingly important in handling complicated problems in Decision Support Systems (DSS). The primary goal of MMS is to facilitate the development and the utilization of quantitative models to improve decision performance. Much current research focuses on model construction. Where early research used deductive reasoning approaches to construct new models, more recent efforts use inductive reasoning mechanisms. Both approaches have their drawbacks. Deductive reasoning methods require a strong domain theory (which may not exist or may be too complex to apply) and ignore previous solving experience. Inductive reasoning methods can take advantage of precedents or prototypical cases, but do not employ domain knowledge. Both methods are limited in learning capacity. This study proposes a Multi-Agent Environmental Decision Support System, which integrates an Inductive Reasoning Agent, and an Environmental Learning Agent to perform new model formation and problem solving. New models can be generated by the coordination of both the Inductive Agent and the Deductive Agent. At the same time, a model repair process is undertaken by the Environmental Learning Agent when the prediction resulting from existing knowledge fails.     

Composite alternatives in group decision support

Decision theory models of group decision processes usually assume a given set of alternatives, from which the group has to choose. In realistic group decision situations, however, alternatives are often not specified a priori, but are created during the group process from different components introduced by the group members. This paper develops methods for systematically creating such composite alternatives, also taking into account the necessity to keep both the computational effort and the cognitive load to group members within reasonable limits.Paper presented at the International Conference on Support Systems for Decision and Negotiation Processes DNS-92, Warsaw, 1992.     

基于变精度加权平均粗糙度决策树的财务预警研究

运用聚类方法把公司财务状况分为5个等级,分别为财务状况健康,良好,一般,预警和危机,与以往将研究样本分为ST和非ST两类的财务预警模型相比,5分类模型更加精确合理,贴近实际。同时基于指标相关性和指标重要度对33个财务指标进行了约简,得到9个能够反映企业财务状况的财务指标。以约简后的9个指标及5个等级的财务状况来建立决策树,指标体系和财务等级更加合理。树的生成过程运用粗糙集中的变精度加权平均粗糙度作为选择测试属性的方法,每次选择变精度加权平均粗糙度值最小的属性作为分支结点。变精度加权平均粗糙度的应用提高了决策树的防噪声能力,复杂性较低且能有效提高分类效果。实证研究表明将它应用到财务预警领域,提高了财务预警的分类精度。