Dynamic projection network for supervised pattern classification

This paper describes the development of the utility of a dynamic neural network known as projection network for pattern classification. It first gives the derivation of the projection network, and then describes the network architecture and analyzes properties such as equilibrium points and their stability condition. The procedures for utilizing the projection network for pattern classification are established and the benefits are discussed. The proposed classification system is then tested with well-known benchmark data sets, namely the Fisher’s iris data, the heart disease data and the credit screening data and the results are compared to other classifiers including Neural Network Rule Base (NNRB), Genetic Algorithm Rule Base (GARB), Rough Set, and C4.5 decision tree. The projection network was proven to be a viable alternative to existing methods.     

Dynamic projection network for supervised pattern classification

This paper describes the development of the utility of a dynamic neural network known as projection network for pattern classification. It first gives the derivation of the projection network, and then describes the network architecture and analyzes properties such as equilibrium points and their stability condition. The procedures for utilizing the projection network for pattern classification are established and the benefits are discussed. The proposed classification system is then tested with well-known benchmark data sets, namely the Fisher’s iris data, the heart disease data and the credit screening data and the results are compared to other classifiers including Neural Network Rule Base (NNRB), Genetic Algorithm Rule Base (GARB), Rough Set, and C4.5 decision tree. The projection network was proven to be a viable alternative to existing methods.     

Classification systems based on rough sets under the belief function framework

In this paper, we present two classification approaches based on Rough Sets (RS) that are able to learn decision rules from uncertain data. We assume that the uncertainty exists only in the decision attribute values of the Decision Table (DT) and is represented by the belief functions. The first technique, named Belief Rough Set Classifier (BRSC), is based only on the basic concepts of the Rough Sets (RS). The second, called Belief Rough Set Classifier, is more sophisticated. It is based on Generalization Distribution Table (BRSC-GDT), which is a hybridization of the Generalization Distribution Table and the Rough Sets (GDT-RS). The two classifiers aim at simplifying the Uncertain Decision Table (UDT) in order to generate significant decision rules for classification process. Furthermore, to improve the time complexity of the construction procedure of the two classifiers, we apply a heuristic method of attribute selection based on rough sets. To evaluate the performance of each classification approach, we carry experiments on a number of standard real-world databases by artificially introducing uncertainty in the decision attribute values. In addition, we test our classifiers on a naturally uncertain web usage database. We compare our belief rough set classifiers with traditional classification methods only for the certain case. Besides, we compare the results relative to the uncertain case with those given by another similar classifier, called the Belief Decision Tree (BDT), which also deals with uncertain decision attribute values.     

Employment of neural network and rough set in meta-learning

The selection of the optimal ensembles of classifiers in multiple-classifier selection technique is un-decidable in many cases and it is potentially subjected to a trial-and-error search. This paper introduces a quantitative meta-learning approach based on neural network and rough set theory in the selection of the best predictive model. This approach depends directly on the characteristic, meta-features of the input data sets. The employed meta-features are the degree of discreteness and the distribution of the features in the input data set, the fuzziness of these features related to the target class labels and finally the correlation and covariance between the different features. The experimental work that consider these criteria are applied on twenty nine data sets using different classification techniques including support vector machine, decision tables and Bayesian believe model. The measures of these criteria and the best result classification technique are used to build a meta data set. The role of the neural network is to perform a black-box prediction of the optimal, best fitting, classification technique. The role of the rough set theory is the generation of the decision rules that controls this prediction approach. Finally, formal concept analysis is applied for the visualization of the generated rules.     

Using table lens to interactively build classifiers

Rather than induce classification rules by sophisticated algorithms, we introduce a fully interactive approach for building classifiers from large multivariate datasets based on the table lens, a multidimensional visualization technique, and appropriate interaction capabilities. Constructing classifiers is an interaction with a feedback loop. The domain knowledge and human perception can be profitably included. In our approach, both continuous and categorical attributes are processed uniformly, and continuous attributes are partitioned on the fly. Our performance evaluation with data sets from the UCI repository demonstrates that this interactive approach is useful to easily build understandable classifiers with high prediction accuracy and no required a prior knowledge about the datasets.     

A Dual-Objective Evolutionary Algorithm for Rules Extraction in Data Mining

This paper presents a dual-objective evolutionary algorithm (DOEA) for extracting multiple decision rule lists in data mining, which aims at satisfying the classification criteria of high accuracy and ease of user comprehension. Unlike existing approaches, the algorithm incorporates the concept of Pareto dominance to evolve a set of non-dominated decision rule lists each having different classification accuracy and number of rules over a specified range. The classification results of DOEA are analyzed and compared with existing rule-based and non-rule based classifiers based upon 8 test problems obtained from UCI Machine Learning Repository. It is shown that the DOEA produces comprehensible rules with competitive classification accuracy as compared to many methods in literature. Results obtained from box plots and t-tests further examine its invariance to random partition of datasets. An erratum to this article is available at .     

A fuzzy random forest

When individual classifiers are combined appropriately, a statistically significant increase in classification accuracy is usually obtained. Multiple classifier systems are the result of combining several individual classifiers. Following Breiman’s methodology, in this paper a multiple classifier system based on a “forest” of fuzzy decision trees, i.e., a fuzzy random forest, is proposed. This approach combines the robustness of multiple classifier systems, the power of the randomness to increase the diversity of the trees, and the flexibility of fuzzy logic and fuzzy sets for imperfect data management. Various combination methods to obtain the final decision of the multiple classifier system are proposed and compared. Some of them are weighted combination methods which make a weighting of the decisions of the different elements of the multiple classifier system (leaves or trees). A comparative study with several datasets is made to show the efficiency of the proposed multiple classifier system and the various combination methods. The proposed multiple classifier system exhibits a good accuracy classification, comparable to that of the best classifiers when tested with conventional data sets. However, unlike other classifiers, the proposed classifier provides a similar accuracy when tested with imperfect datasets (with missing and fuzzy values) and with datasets with noise.     

T3C: improving a decision tree classification algorithm’s interval splits on continuous attributes

This paper proposes, describes and evaluates T3C, a classification algorithm that builds decision trees of depth at most three, and results in high accuracy whilst keeping the size of the tree reasonably small. T3C is an improvement over algorithm T3 in the way it performs splits on continuous attributes. When run against publicly available data sets, T3C achieved lower generalisation error than T3 and the popular C4.5, and competitive results compared to Random Forest and Rotation Forest.     

基于多Agent的车间任务招投标粗模糊集建模研究

首先通过相识集、招标集、投标集和任务集的概念 ,描述制造执行系统中的调度 Agent与资源 A-gent间任务招投标过程模型 ;基于任务的属性和资源 Agent完成任务的成本、质量、负荷和时间等属性 ,定义论域上的模糊集 ,将模糊集中的隶属度函数作为粗集的属性 ,在模糊集上作截集 ,从而获得系统的分类知识 ;收集样本数据 ,构造并分析决策表 ,进而获得调度 Agent调度决策知识 ;应用调度知识进行推理 ,从争取获得招标任务的若干个资源 Agent中 ,选出最适合招标任务的中标者 .     

GA-Ensemble: a genetic algorithm for robust ensembles

Many simple and complex methods have been developed to solve the classification problem. Boosting is one of the best known techniques for improving the accuracy of classifiers. However, boosting is prone to overfitting with noisy data and the final model is difficult to interpret. Some boosting methods, including AdaBoost, are also very sensitive to outliers. In this article we propose a new method, GA-Ensemble, which directly solves for the set of weak classifiers and their associated weights using a genetic algorithm. The genetic algorithm utilizes a new penalized fitness function that limits the number of weak classifiers and controls the effects of outliers by maximizing an appropriately chosen $p$ th percentile of margins. We compare the test set error rates of GA-Ensemble, AdaBoost, and GentleBoost (an outlier-resistant version of AdaBoost) using several artificial data sets and real-world data sets from the UC-Irvine Machine Learning Repository. GA-Ensemble is found to be more resistant to outliers and results in simpler predictive models than AdaBoost and GentleBoost.     

Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms

This paper presents a hybrid method for identification of Pareto-optimal fuzzy classifiers (FCs). In contrast to many existing methods, the initial population for multiobjective evolutionary algorithms (MOEAs) is neither created randomly nor a priori knowledge is required. Instead, it is created by the proposed two-step initialization method. First, a decision tree (DT) created by C4.5 algorithm is transformed into an FC. Therefore, relevant variables are selected and initial partition of input space is performed. Then, the rest of the population is created by randomly replacing some parameters of the initial FC, such that, the initial population is widely spread. That improves the convergence of MOEAs into the correct Pareto front. The initial population is optimized by NSGA-II algorithm and a set of Pareto-optimal FCs representing the trade-off between accuracy and interpretability is obtained. The method does not require any a priori knowledge of the number of fuzzy sets, distribution of fuzzy sets or the number of relevant variables. They are all determined by it. Performance of the obtained FCs is validated by six benchmark data sets from the literature. The obtained results are compared to a recently published paper [H. Ishibuchi, Y. Nojima, Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning, International Journal of Approximate Reasoning 44 (1) (2007) 4–31] and the benefits of our method are clearly shown.     

Multi-criteria classification – A new scheme for application of dominance-based decision rules

We are considering the problem of multi-criteria classification. In this problem, a set of “if … then …” decision rules is used as a preference model to classify objects evaluated by a set of criteria and regular attributes. Given a sample of classification examples, called learning data set, the rules are induced from dominance-based rough approximations of preference-ordered decision classes, according to the Variable Consistency Dominance-based Rough Set Approach (VC-DRSA). The main question to be answered in this paper is how to classify an object using decision rules in situation where it is covered by (i) no rule, (ii) exactly one rule, (iii) several rules. The proposed classification scheme can be applied to both, learning data set (to restore the classification known from examples) and testing data set (to predict classification of new objects). A hypothetical example from the area of telecommunications is used for illustration of the proposed classification method and for a comparison with some previous proposals.     

Learning multicriteria classification models from examples: Decision rules in continuous space

The classification problem statement of multicriteria decision analysis is to model the classification of the alternatives/actions according to the decision maker's preferences. These models are based on outranking relations, utility functions or (linear) discriminant functions. Model parameters can be given explicitly or learnt from a preclassified set of alternatives/actions.In this paper we propose a novel approach, the Continuous Decision (CD) method, to learn parameters of a discriminant function, and we also introduce its extension, the Continuous Decision Tree (CDT) method, which describes the classification more accurately.The proposed methods are results of integration of Machine Learning methods in Decision Analysis. From a Machine Learning point of view, the CDT method can be considered as an extension of the C4.5 decision tree building algorithm that handles only numeric criteria but applies more complex tests in the inner nodes of the tree. For the sake of easier interpretation, the decision trees are transformed to rules.     

Support vector machine classifiers with uncertain knowledge sets via robust optimization

In this article we study support vector machine (SVM) classifiers in the face of uncertain knowledge sets and show how data uncertainty in knowledge sets can be treated in SVM classification by employing robust optimization. We present knowledge-based SVM classifiers with uncertain knowledge sets using convex quadratic optimization duality. We show that the knowledge-based SVM, where prior knowledge is in the form of uncertain linear constraints, results in an uncertain convex optimization problem with a set containment constraint. Using a new extension of Farkas' lemma, we reformulate the robust counterpart of the uncertain convex optimization problem in the case of interval uncertainty as a convex quadratic optimization problem. We then reformulate the resulting convex optimization problems as a simple quadratic optimization problem with non-negativity constraints using the Lagrange duality. We obtain the solution of the converted problem by a fixed point iterative algorithm and establish the convergence of the algorithm. We finally present some preliminary results of our computational experiments of the method.     

Credit rating analysis using adaptive fuzzy rule-based systems: an industry-specific approach

This paper presents an analysis of credit rating using fuzzy rule-based systems. The disadvantage of the models used in previous studies is that it is difficult to extract understandable knowledge from them. The root of this problem is the use of natural language that is typical for the credit rating process. This problem can be solved using fuzzy logic, which enables users to model the meaning of natural language words. Therefore, the fuzzy rule-based system adapted by a feed-forward neural network is designed to classify US companies (divided into the finance, manufacturing, mining, retail trade, services, and transportation industries) and municipalities into the credit rating classes obtained from rating agencies. Features are selected using a filter combined with a genetic algorithm as a search method. The resulting subsets of features confirm the assumption that the rating process is industry-specific (i.e. specific determinants are used for each industry). The results show that the credit rating classes assigned to bond issuers can be classified with high classification accuracy using low numbers of features, membership functions, and if-then rules. The comparison of selected fuzzy rule-based classifiers indicates that it is possible to increase classification performance by using different classifiers for individual industries.     

Pareto-optimality of oblique decision trees from evolutionary algorithms

This paper investigates the performance of evolutionary algorithms in the optimization aspects of oblique decision tree construction and describes their performance with respect to classification accuracy, tree size, and Pareto-optimality of their solution sets. The performance of the evolutionary algorithms is analyzed and compared to the performance of exhaustive (traditional) decision tree classifiers on several benchmark datasets. The results show that the classification accuracy and tree sizes generated by the evolutionary algorithms are comparable with the results generated by traditional methods in all the sample datasets and in the large datasets, the multiobjective evolutionary algorithms generate better Pareto-optimal sets than the sets generated by the exhaustive methods. The results also show that a classifier, whether exhaustive or evolutionary, that generates the most accurate trees does not necessarily generate the shortest trees or the best Pareto-optimal sets.     

Optimal selection of retailers for a manufacturing vendor in a vendor managed inventory system

A Vendor Managed Inventory (VMI) system consists of a manufacturing vendor and a number of retailers. In such a system, it is essential for the vendor to optimally determine retailer selection and other related decisions, such as the product’s replenishment cycle time and the wholesale price, in order to maximize his profit. Meanwhile, each retailer’s decisions on her willingness to enter the system and retail price are simultaneously considered in the retailer selection process. However, the above interactive decision making is complex and the available studies on interactive retailer selection are scarce. In this study, we formulate the retailer selection problem as a Stackelberg game model to help the manufacturer, as a vendor, optimally select his retailers to form a VMI system. This model is non-linear, mixed-integer, game-theoretic, and analytically intractable. Therefore, we further develop a hybrid algorithm for effectively and efficiently solving the developed model. The hybrid algorithm combines dynamic programming (DP), genetic algorithm (GA) and analytical methods. As demonstrated by our numerical studies, the optimal retailer selection can increase the manufacturer’s profit by up to 90% and the selected retailers’ profits significantly compared to non-selection strategy. The proposed hybrid algorithm can solve the model within a minute for a problem with 100 candidate retailers, whereas a pure GA has to take more than 1 h to solve a small sized problem of 20 candidate retailers achieving an objective value no worse than that obtained by the hybrid algorithm.     

MEPAR-miner: Multi-expression programming for classification rule mining

Classification and rule induction are two important tasks to extract knowledge from data. In rule induction, the representation of knowledge is defined as IF-THEN rules which are easily understandable and applicable by problem-domain experts. In this paper, a new chromosome representation and solution technique based on Multi-Expression Programming (MEP) which is named as MEPAR-miner (Multi-Expression Programming for Association Rule Mining) for rule induction is proposed. Multi-Expression Programming (MEP) is a relatively new technique in evolutionary programming that is first introduced in 2002 by Oltean and Dumitrescu. MEP uses linear chromosome structure. In MEP, multiple logical expressions which have different sizes are used to represent different logical rules. MEP expressions can be encoded and implemented in a flexible and efficient manner. MEP is generally applied to prediction problems; in this paper a new algorithm is presented which enables MEP to discover classification rules. The performance of the developed algorithm is tested on nine publicly available binary and n-ary classification data sets. Extensive experiments are performed to demonstrate that MEPAR-miner can discover effective classification rules that are as good as (or better than) the ones obtained by the traditional rule induction methods. It is also shown that effective gene encoding structure directly improves the predictive accuracy of logical IF-THEN rules.     

A survey of fuzzy decision tree classifier

Decision-tree algorithm provides one of the most popular methodologies for symbolic knowledge acquisition. The resulting knowledge, a symbolic decision tree along with a simple inference mechanism, has been praised for comprehensibility. The most comprehensible decision trees have been designed for perfect symbolic data. Over the years, additional methodologies have been investigated and proposed to deal with continuous or multi-valued data, and with missing or noisy features. Recently, with the growing popularity of fuzzy representation, some researchers have proposed to utilize fuzzy representation in decision trees to deal with similar situations. This paper presents a survey of current methods for Fuzzy Decision Tree (FDT) designment and the various existing issues. After considering potential advantages of FDT classifiers over traditional decision tree classifiers, we discuss the subjects of FDT including attribute selection criteria, inference for decision assignment and stopping criteria. To be best of our knowledge, this is the first overview of fuzzy decision tree classifier.