An interpretability improvement for fuzzy rule bases obtained by the iterative rule learning approach

Interpretability is one of the key concepts in many of the applications using the fuzzy rule-based approach. It is well known that there are many different criteria around this concept, the complexity being one of them. In this paper, we focus our efforts in reducing the complexity of the fuzzy rule sets. One of the most interesting approaches for learning fuzzy rules is the iterative rule learning approach. It is mainly characterized by obtaining rules covering few examples in final stages, being in most cases useless to represent the knowledge. This behavior is due to the specificity of the extracted rules, which eventually creates more complex set of rules. Thus, we propose a modified version of the iterative rule learning algorithm in order to extract simple rules relaxing this natural trend. The main idea is to change the rule extraction process to be able to obtain more general rules, using pruned searching spaces together with a knowledge simplification scheme able to replace learned rules. The experimental results prove that this purpose is achieved. The new proposal reduces the complexity at both, the rule and rule base levels, maintaining the accuracy regarding to previous versions of the algorithm.     

Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms

This paper presents a hybrid method for identification of Pareto-optimal fuzzy classifiers (FCs). In contrast to many existing methods, the initial population for multiobjective evolutionary algorithms (MOEAs) is neither created randomly nor a priori knowledge is required. Instead, it is created by the proposed two-step initialization method. First, a decision tree (DT) created by C4.5 algorithm is transformed into an FC. Therefore, relevant variables are selected and initial partition of input space is performed. Then, the rest of the population is created by randomly replacing some parameters of the initial FC, such that, the initial population is widely spread. That improves the convergence of MOEAs into the correct Pareto front. The initial population is optimized by NSGA-II algorithm and a set of Pareto-optimal FCs representing the trade-off between accuracy and interpretability is obtained. The method does not require any a priori knowledge of the number of fuzzy sets, distribution of fuzzy sets or the number of relevant variables. They are all determined by it. Performance of the obtained FCs is validated by six benchmark data sets from the literature. The obtained results are compared to a recently published paper [H. Ishibuchi, Y. Nojima, Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning, International Journal of Approximate Reasoning 44 (1) (2007) 4–31] and the benefits of our method are clearly shown.     

Learning concurrently partition granularities and rule bases of Mamdani fuzzy systems in a multi-objective evolutionary framework

In this paper we propose a multi-objective evolutionary algorithm to generate Mamdani fuzzy rule-based systems with different good trade-offs between complexity and accuracy. The main novelty of the algorithm is that both rule base and granularity of the uniform partitions defined on the input and output variables are learned concurrently. To this aim, we introduce the concepts of virtual and concrete rule bases: the former is defined on linguistic variables, all partitioned with a fixed maximum number of fuzzy sets, while the latter takes into account, for each variable, a number of fuzzy sets as determined by the specific partition granularity of that variable. We exploit a chromosome composed of two parts, which codify the variables partition granularities, and the virtual rule base, respectively. Genetic operators manage virtual rule bases, whereas fitness evaluation relies on an appropriate mapping strategy between virtual and concrete rule bases. The algorithm has been tested on two real-world regression problems showing very promising results.     

Combining boosting and evolutionary algorithms for learning of fuzzy classification rules

This paper presents a novel boosting algorithm for genetic learning of fuzzy classification rules. The method is based on the iterative rule learning approach to fuzzy rule base system design. The fuzzy rule base is generated in an incremental fashion, in that the evolutionary algorithm optimizes one fuzzy classifier rule at a time. The boosting mechanism reduces the weight of those training instances that are classified correctly by the new rule. Therefore, the next rule generation cycle focuses on fuzzy rules that account for the currently uncovered or misclassified instances. The weight of a fuzzy rule reflects the relative strength the boosting algorithm assigns to the rule class when it aggregates the casted votes. The approach is compared with other classification algorithms for a number problem sets from the UCI repository.     

A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems

The need for trading off interpretability and accuracy is intrinsic to the use of fuzzy systems. The obtaining of accurate but also human-comprehensible fuzzy systems played a key role in Zadeh and Mamdani’s seminal ideas and system identification methodologies. Nevertheless, before the advent of soft computing, accuracy progressively became the main concern of fuzzy model builders, making the resulting fuzzy systems get closer to black-box models such as neural networks. Fortunately, the fuzzy modeling scientific community has come back to its origins by considering design techniques dealing with the interpretability-accuracy tradeoff. In particular, the use of genetic fuzzy systems has been widely extended thanks to their inherent flexibility and their capability to jointly consider different optimization criteria. The current contribution constitutes a review on the most representative genetic fuzzy systems relying on Mamdani-type fuzzy rule-based systems to obtain interpretable linguistic fuzzy models with a good accuracy.     

A fuzzy rule-based system for handwritten Chinese characters recognition based on radical extraction

In this paper, a fuzzy rule-based system for handwritten Chinese characters recognition (HCCR) based on radical extraction is proposed. Since the writings of handwritten Chinese characters vary a lot, we adopt fuzzy set theory to deal with the recognition of these fuzzy patterns. Candidates of strokes are provided with confidence values to obtain more reliable and accurate results. Furthermore, hierarchical fuzzy rule sets that represent the character structures are used to combine the extracted strokes into compound strokes or radicals. The flexible expansion ability thus provided is very promising. Also, since the number of rules in a fuzzy system is much less than that in a general rule-based system, the computation effort is not difficult. An average of 99.63% recognition rate of 542 test categories that are selected from the 100th sample set of HCCRBASE (character image database provided by CCL, ITRI, Taiwan) is obtained. The experimental results not only verify the feasibility of the proposed system, but also suggest that applying fuzzy set theory to HCCR is an efficient and promising approach.     

An efficient multi-objective evolutionary fuzzy system for regression problems

During the last years, multi-objective evolutionary algorithms (MOEAs) have been extensively employed as optimization tools for generating fuzzy rule-based systems (FRBSs) with different trade-offs between accuracy and interpretability from data. Since the size of the search space and the computational cost of the fitness evaluation depend on the number of input variables and instances, respectively, managing high-dimensional and large datasets is a critical issue.In this paper, we focus on MOEAs applied to learn concurrently the rule base and the data base of Mamdani FRBSs and propose to tackle the issue by exploiting the synergy between two different techniques. The first technique is based on a novel method which reduces the search space by learning rules not from scratch, but rather from a heuristically generated rule base. The second technique performs an instance selection by exploiting a co-evolutionary approach where cyclically a genetic algorithm evolves a reduced training set which is used in the evolution of the MOEA.The effectiveness of the synergy has been tested on twelve datasets. Using non-parametric statistical tests we show that, although achieving statistically equivalent solutions, the adoption of this synergy allows saving up to 97.38% of the execution time with respect to a state-of-the-art multi-objective evolutionary approach which learns rules from scratch.     

Evolution of fuzzy classifiers using genetic programming

In this paper, we propose a genetic programming (GP) based approach to evolve fuzzy rule based classifiers. For a c-class problem, a classifier consists of c trees. Each tree, T _i , of the multi-tree classifier represents a set of rules for class i. During the evolutionary process, the inaccurate/inactive rules of the initial set of rules are removed by a cleaning scheme. This allows good rules to sustain and that eventually determines the number of rules. In the beginning, our GP scheme uses a randomly selected subset of features and then evolves the features to be used in each rule. The initial rules are constructed using prototypes, which are generated randomly as well as by the fuzzy k-means (FKM) algorithm. Besides, experiments are conducted in three different ways: Using only randomly generated rules, using a mixture of randomly generated rules and FKM prototype based rules, and with exclusively FKM prototype based rules. The performance of the classifiers is comparable irrespective of the type of initial rules. This emphasizes the novelty of the proposed evolutionary scheme. In this context, we propose a new mutation operation to alter the rule parameters. The GP scheme optimizes the structure of rules as well as the parameters involved. The method is validated on six benchmark data sets and the performance of the proposed scheme is found to be satisfactory.     

Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms

The identification of a model is one of the key issues in the field of fuzzy system modeling and function approximation theory. An important characteristic that distinguishes fuzzy systems from other techniques in this area is their transparency and interpretability. Especially in the construction of a fuzzy system from a set of given training examples, little attention has been paid to the analysis of the trade-off between complexity and accuracy maintaining the interpretability of the final fuzzy system. In this paper a multi-objective evolutionary approach is proposed to determine a Pareto-optimum set of fuzzy systems with different compromises between their accuracy and complexity. In particular, two fundamental and competing objectives concerning fuzzy system modeling are addressed: fuzzy rule parameter optimization and the identification of system structure (i.e. the number of membership functions and fuzzy rules), taking always in mind the transparency of the obtained system. Another key aspect of the algorithm presented in this work is the use of some new expert evolutionary operators, specifically designed for the problem of fuzzy function approximation, that try to avoid the generation of worse solutions in order to accelerate the convergence of the algorithm.     

基于知识的模糊神经网络的旋转机械故障诊断

提出了一种基于知识的模糊神经网络并用于故障诊断．首先基于粗糙集对样本数据进行初步规则获取,并计算规则的依赖度和条件覆盖度,然后根据规则数目进行模糊神经网络结构部分设计,规则的依赖度和条件覆盖度用于设定网络初始权重,而用遗产算法对神经网络输出参数进行优化．这样的模糊神经网络称为基于知识的模糊神经网络．使用该网络对旋转机械常见故障进行诊断,结果表明,和一般模糊神经网络相比,该网络具有训练时间短而诊断率高的特点．     

Interpretability assessment of fuzzy knowledge bases: A cointension based approach

Computing with words (CWW) relies on linguistic representation of knowledge that is processed by operating at the semantical level defined through fuzzy sets. Linguistic representation of knowledge is a major issue when fuzzy rule based models are acquired from data by some form of empirical learning. Indeed, these models are often requested to exhibit interpretability, which is normally evaluated in terms of structural features, such as rule complexity, properties on fuzzy sets and partitions. In this paper we propose a different approach for evaluating interpretability that is based on the notion of cointension. The interpretability of a fuzzy rule-based model is measured in terms of cointension degree between the explicit semantics, defined by the formal parameter settings of the model, and the implicit semantics conveyed to the reader by the linguistic representation of knowledge. Implicit semantics calls for a representation of user’s knowledge which is difficult to externalise. Nevertheless, we identify a set of properties - which we call “logical view” - that is expected to hold in the implicit semantics and is used in our approach to evaluate the cointension between explicit and implicit semantics. In practice, a new fuzzy rule base is obtained by minimising the fuzzy rule base through logical properties. Semantic comparison is made by evaluating the performances of the two rule bases, which are supposed to be similar when the two semantics are almost equivalent. If this is the case, we deduce that the logical view is applicable to the model, which can be tagged as interpretable from the cointension viewpoint. These ideas are then used to define a strategy for assessing interpretability of fuzzy rule-based classifiers (FRBCs). The strategy has been evaluated on a set of pre-existent FRBCs, acquired by different learning processes from a well-known benchmark dataset. Our analysis highlighted that some of them are not cointensive with user’s knowledge, hence their linguistic representation is not appropriate, even though they can be tagged as interpretable from a structural point of view.     

Fuzzy rule generation methods for high-level computer vision

In many decision making systems involving multiple sources, the decisions made may be considered as the result of a rule-based system in which the decision rules are usually enumerated by experts or generated by a learning process. In this paper, we discuss the various issues involved in the generation of fuzzy rules automatically from training data for high-level computer vision. Features are treated as linguistic variables that appear in the antecedent clauses of the rules. We present methods to generate the corresponding linguistic labels (values) and their membership functions. Rules are generated by constructing a minimal approximate fuzzy aggregation network and then training the network using gradient descent methods. Several examples are given.     

Extracting compact and information lossless sets of fuzzy association rules

Applying classical association rule extraction framework on fuzzy datasets leads to an unmanageably highly sized association rule sets. Moreover, the discretization operation leads to information loss and constitutes a hamper towards an efficient exploitation of the mined knowledge. To overcome such a drawback, this paper proposes the extraction and the exploitation of compact and informative generic basis of fuzzy association rules. The presented approach relies on the extension, within the fuzzy context, of the notion of closure and Galois connection, that we introduce in this paper. In order to select without loss of information a generic subset of all fuzzy association rules, we define three fuzzy generic basis from which remaining (redundant) FARs are generated. This generic basis constitutes a compact nucleus of fuzzy association rules, from which it is possible to informatively derive all the remaining rules. In order to ensure a sound and complete derivation process, we introduce an axiomatic system allowing the complete derivation of all the redundant rules. The results obtained from experiments carried out on benchmark datasets are very encouraging. They highlight a very important reduction of the number of the extracted fuzzy association rules without information loss.     

Some relationships between fuzzy and random set-based classifiers and models

When designing rule-based models and classifiers, some precision is sacrificed to obtain linguistic interpretability. Understandable models are not expected to outperform black boxes, but usually fuzzy learning algorithms are statistically validated by contrasting them with black-box models. Unless performance of both approaches is equivalent, it is difficult to judge whether the fuzzy one is doing its best, because the precision gap between the best understandable model and the best black-box model is not known.In this paper we discuss how to generate probabilistic rule-based models and classifiers with the same structure as fuzzy rule-based ones. Fuzzy models, in which features are partitioned into linguistic terms, will be compared to probabilistic rule-based models with the same number of terms in every linguistic partition. We propose to use these probabilistic models to estimate a lower precision limit which fuzzy rule learning algorithms should surpass.     

Intelligent data analysis and model interpretation with spectral analysis fuzzy symbolic modeling

This paper proposes fuzzy symbolic modeling as a framework for intelligent data analysis and model interpretation in classification and regression problems. The fuzzy symbolic modeling approach is based on the eigenstructure analysis of the data similarity matrix to define the number of fuzzy rules in the model. Each fuzzy rule is associated with a symbol and is defined by a Gaussian membership function. The prototypes for the rules are computed by a clustering algorithm, and the model output parameters are computed as the solutions of a bounded quadratic optimization problem. In classification problems, the rules’ parameters are interpreted as the rules’ confidence. In regression problems, the rules’ parameters are used to derive rules’ confidences for classes that represent ranges of output variable values. The resulting model is evaluated based on a set of benchmark datasets for classification and regression problems. Nonparametric statistical tests were performed on the benchmark results, showing that the proposed approach produces compact fuzzy models with accuracy comparable to models produced by the standard modeling approaches. The resulting model is also exploited from the interpretability point of view, showing how the rule weights provide additional information to help in data and model understanding, such that it can be used as a decision support tool for the prediction of new data.     

Discrete optimization algorithms and problems of decision making in a fuzzy environment

An approach to solving optimization problems with fuzzy coefficients is described. It consists in formulating and analyzing one and the same problem within the framework of mutually related models by constructing equivalent analogs with fuzzy coefficients in objective functions alone. Since the approach is applied within the context of fuzzy discrete optimization problems, modified algorithms of discrete optimization are discussed. These algorithms are based on a combination of formal and heuristic procedures and allow one to obtain quasi-optimal solutions after a small number of steps, thus overcoming the computational complexity posed by the NP-completeness of discrete optimization problems. The subsequent contraction of the decision uncertainty regions is associated with reduction of the problem to multiobjective decision making in a fuzzy environment using techniques based on fuzzy preference relations. The results of the paper are of a universal character and are already being used to solve practical problems in several fields.     

Indirect adaptive fuzzy observer and controller design based on interval type-2 T-S fuzzy model

This paper presents the design scheme of the indirect adaptive fuzzy observer and controller based on the interval type-2 (IT2) T-S fuzzy model. The nonlinear systems can be well approximated by IT2 T-S fuzzy model, in which the fuzzy rules’ antecedents are interval type-2 fuzzy sets and consequents are linear state equations. The proposed IT2 T-S fuzzy model is a combination of IT2 fuzzy system and T-S fuzzy model, and also inherits the benefits of type-2 fuzzy logic systems, which is able to directly handle uncertainties and can minimize the effects of uncertainties in rule-based fuzzy system. These characteristics can improve the accuracy of the system modeling and reduce the number of system rules. The proposed method using feedback control, adaptive laws, and on-line object parameters are adjusted to ensure observation error bounded. In addition, using Lyapunov synthesis approach and Lipschitz condition, the stability analysis is conducted. The simulation results show that the proposed method can handle unpredicted disturbance and data uncertainties very well in advantage of the effectiveness of observation and control.     

Comparison of Heuristic Criteria for Fuzzy Rule Selection in Classification Problems

This paper compares heuristic criteria used for extracting a pre-specified number of fuzzy classification rules from numerical data. We examine the performance of each heuristic criterion through computational experiments on well-known test problems. Experimental results show that better results are obtained from composite criteria of confidence and support measures than their individual use. It is also shown that genetic algorithm-based rule selection can improve the classification ability of extracted fuzzy rules by searching for good rule combinations. This observation suggests the importance of taking into account the combinatorial effect of fuzzy rules (i.e., the interaction among them).