A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems

The need for trading off interpretability and accuracy is intrinsic to the use of fuzzy systems. The obtaining of accurate but also human-comprehensible fuzzy systems played a key role in Zadeh and Mamdani’s seminal ideas and system identification methodologies. Nevertheless, before the advent of soft computing, accuracy progressively became the main concern of fuzzy model builders, making the resulting fuzzy systems get closer to black-box models such as neural networks. Fortunately, the fuzzy modeling scientific community has come back to its origins by considering design techniques dealing with the interpretability-accuracy tradeoff. In particular, the use of genetic fuzzy systems has been widely extended thanks to their inherent flexibility and their capability to jointly consider different optimization criteria. The current contribution constitutes a review on the most representative genetic fuzzy systems relying on Mamdani-type fuzzy rule-based systems to obtain interpretable linguistic fuzzy models with a good accuracy.     

Rule generation for hierarchical collaborative fuzzy system

A new method of rule generation for the hierarchical collaborative fuzzy system, HCFS, is proposed. This HCFS is structured like various parallel fuzzy subsystems and it overcomes the dimensionality problem and the lack of interpretability of most of the traditional fuzzy systems, when dealing with complex real-world problems. An association process of different fuzzy systems is presented in this work, through the use of a relevance concept of a fuzzy system. The result of this aggregation is a collaborative structure where all sub-models have the ability to gradually improve the overall accuracy of approximation by adding their own contributions. For this structure we propose a new algorithm to be used in the procedures of the three learning phases: the structure building, the parametric identification and the division of the learning data among the various levels of the hierarchical structure. This new fuzzy modelling technique automatically generates and tunes the sets of fuzzy rules in the hierarchical collaborative structure (HCS). The effectiveness of the proposed HCFS model in handling high-dimensional and complex problems is demonstrated through various numerical simulations.     

Extracting interpretable fuzzy models for nonlinear systems using gradient-based continuous ant colony optimization

This paper exploits the ability of a novel ant colony optimization algorithm called gradient-based continuous ant colony optimization, an evolutionary methodology, to extract interpretable first-order fuzzy Sugeno models for nonlinear system identification. The proposed method considers all objectives of system identification task, namely accuracy, interpretability, compactness and validity conditions. First, an initial structure of model is obtained by means of subtractive clustering. Then, an iterative two-step algorithm is employed to produce a simplified fuzzy model in terms of number of fuzzy sets and rules. In the first step, the parameters of the model are adjusted by utilizing the gradient-based continuous ant colony optimization. In the second step, the similar membership functions of an obtained model merge. The results obtained on three case studies illustrate the applicability of the proposed method to extract accurate and interpretable fuzzy models for nonlinear system identification.     

边缘线性化方法构造的Fuzzy系统及其逼近性能分析

给出了基于边缘线性化方法构造的Fuzzy系统输出函数的一般表达式,揭示了该方法的插值机理,证明了由其构造的Fuzzy系统输出函数可归结为插值函数的形式,在此基础上,分析了该方法所构造的Fuzzy系统的逼近误差.仿真实验表明,边缘线性化方法构造的Fuzzy系统对非线性连续函数具有很高的逼近精度.     

Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation

One of the problems that focus the research in the linguistic fuzzy modeling area is the trade-off between interpretability and accuracy. To deal with this problem, different approaches can be found in the literature. Recently, a new linguistic rule representation model was presented to perform a genetic lateral tuning of membership functions. It is based on the linguistic 2-tuples representation that allows the lateral displacement of a label considering an unique parameter. This way to work involves a reduction of the search space that eases the derivation of optimal models and therefore, improves the mentioned trade-off.Based on the 2-tuples rule representation, this work proposes a new method to obtain linguistic fuzzy systems by means of an evolutionary learning of the data base a priori (number of labels and lateral displacements) and a simple rule generation method to quickly learn the associated rule base. Since this rule generation method is run from each data base definition generated by the evolutionary algorithm, its selection is an important aspect. In this work, we also propose two new ad hoc data-driven rule generation methods, analyzing the influence of them and other rule generation methods in the proposed learning approach. The developed algorithms will be tested considering two different real-world problems.     

Fuzzy modelling through logic optimization

This study proposes a new logic-driven approach to the development of fuzzy models. We introduce a two-phase design process realizing adaptive logic processing in the form of structural and parametric optimization. By recognizing the fundamental links between binary (two-valued) and fuzzy (multi-valued) logic, effective structural learning is achieved through the use of well-established methods of Boolean minimization encountered in digital systems. This blueprint structure is then refined by adjusting connections of fuzzy neurons, helping to capture the numeric details of the target system’s behavior. The introduced structure along with the learning mechanisms helps achieve high accuracy and interpretability (transparency) of the resulting model.     

Looking for a good fuzzy system interpretability index: An experimental approach

Interpretability is acknowledged as the main advantage of fuzzy systems and it should be given a main role in fuzzy modeling. Classical systems are viewed as black boxes because mathematical formulas set the mapping between inputs and outputs. On the contrary, fuzzy systems (if they are built regarding some constraints) can be seen as gray boxes in the sense that every element of the whole system can be checked and understood by a human being. Interpretability is essential for those applications with high human interaction, for instance decision support systems in fields like medicine, economics, etc. Since interpretability is not guaranteed by definition, a huge effort has been done to find out the basic constraints to be superimposed during the fuzzy modeling process. People talk a lot about interpretability but the real meaning is not clear. Understanding of fuzzy systems is a subjective task which strongly depends on the background (experience, preferences, and knowledge) of the person who makes the assessment. As a consequence, although there have been a few attempts to define interpretability indices, there is still not a universal index widely accepted. As part of this work, with the aim of evaluating the most used indices, an experimental analysis (in the form of a web poll) was carried out yielding some useful clues to keep in mind regarding interpretability assessment. Results extracted from the poll show the inherent subjectivity of the measure because we collected a huge diversity of answers completely different at first glance. However, it was possible to find out some interesting user profiles after comparing carefully all the answers. It can be concluded that defining a numerical index is not enough to get a widely accepted index. Moreover, it is necessary to define a fuzzy index easily adaptable to the context of each problem as well as to the user quality criteria.     

Looking for a good fuzzy system interpretability index: An experimental approach

Interpretability is acknowledged as the main advantage of fuzzy systems and it should be given a main role in fuzzy modeling. Classical systems are viewed as black boxes because mathematical formulas set the mapping between inputs and outputs. On the contrary, fuzzy systems (if they are built regarding some constraints) can be seen as gray boxes in the sense that every element of the whole system can be checked and understood by a human being. Interpretability is essential for those applications with high human interaction, for instance decision support systems in fields like medicine, economics, etc. Since interpretability is not guaranteed by definition, a huge effort has been done to find out the basic constraints to be superimposed during the fuzzy modeling process. People talk a lot about interpretability but the real meaning is not clear. Understanding of fuzzy systems is a subjective task which strongly depends on the background (experience, preferences, and knowledge) of the person who makes the assessment. As a consequence, although there have been a few attempts to define interpretability indices, there is still not a universal index widely accepted. As part of this work, with the aim of evaluating the most used indices, an experimental analysis (in the form of a web poll) was carried out yielding some useful clues to keep in mind regarding interpretability assessment. Results extracted from the poll show the inherent subjectivity of the measure because we collected a huge diversity of answers completely different at first glance. However, it was possible to find out some interesting user profiles after comparing carefully all the answers. It can be concluded that defining a numerical index is not enough to get a widely accepted index. Moreover, it is necessary to define a fuzzy index easily adaptable to the context of each problem as well as to the user quality criteria.     

Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms

This paper presents a hybrid method for identification of Pareto-optimal fuzzy classifiers (FCs). In contrast to many existing methods, the initial population for multiobjective evolutionary algorithms (MOEAs) is neither created randomly nor a priori knowledge is required. Instead, it is created by the proposed two-step initialization method. First, a decision tree (DT) created by C4.5 algorithm is transformed into an FC. Therefore, relevant variables are selected and initial partition of input space is performed. Then, the rest of the population is created by randomly replacing some parameters of the initial FC, such that, the initial population is widely spread. That improves the convergence of MOEAs into the correct Pareto front. The initial population is optimized by NSGA-II algorithm and a set of Pareto-optimal FCs representing the trade-off between accuracy and interpretability is obtained. The method does not require any a priori knowledge of the number of fuzzy sets, distribution of fuzzy sets or the number of relevant variables. They are all determined by it. Performance of the obtained FCs is validated by six benchmark data sets from the literature. The obtained results are compared to a recently published paper [H. Ishibuchi, Y. Nojima, Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning, International Journal of Approximate Reasoning 44 (1) (2007) 4–31] and the benefits of our method are clearly shown.     

Robust Adaptive Fuzzy Identification of Time-Varying Processes with Uncertain Data. Handling Uncertainties in the Physical Fitness Fuzzy Approximation with Real World Medical Data: An Application

This study considers the problem of Robust Fuzzy approximation of a time-varying nonlinear process in the presence of uncertainties in the identification data using a Sugeno Fuzzy System while maintaining the interpretability of the fuzzy model during identification. A recursive procedure for the estimation of fuzzy parameters is proposed based on solving local optimization problem that attempt to minimize the worst-case effect of data uncertainties on approximation performance. To illustrate the approach, several simulation studies on numerical examples are provided. The developed scheme was applied to handle the vagueness, ambiguity and uncertainty inherently present in the general notion of a Medical Expert about Physical Fitness based on a set of various Physiological parameters measurements.     

Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning

This paper examines the interpretability-accuracy tradeoff in fuzzy rule-based classifiers using a multiobjective fuzzy genetics-based machine learning (GBML) algorithm. Our GBML algorithm is a hybrid version of Michigan and Pittsburgh approaches, which is implemented in the framework of evolutionary multiobjective optimization (EMO). Each fuzzy rule is represented by its antecedent fuzzy sets as an integer string of fixed length. Each fuzzy rule-based classifier, which is a set of fuzzy rules, is represented as a concatenated integer string of variable length. Our GBML algorithm simultaneously maximizes the accuracy of rule sets and minimizes their complexity. The accuracy is measured by the number of correctly classified training patterns while the complexity is measured by the number of fuzzy rules and/or the total number of antecedent conditions of fuzzy rules. We examine the interpretability-accuracy tradeoff for training patterns through computational experiments on some benchmark data sets. A clear tradeoff structure is visualized for each data set. We also examine the interpretability-accuracy tradeoff for test patterns. Due to the overfitting to training patterns, a clear tradeoff structure is not always obtained in computational experiments for test patterns.     

Intelligent data analysis and model interpretation with spectral analysis fuzzy symbolic modeling

This paper proposes fuzzy symbolic modeling as a framework for intelligent data analysis and model interpretation in classification and regression problems. The fuzzy symbolic modeling approach is based on the eigenstructure analysis of the data similarity matrix to define the number of fuzzy rules in the model. Each fuzzy rule is associated with a symbol and is defined by a Gaussian membership function. The prototypes for the rules are computed by a clustering algorithm, and the model output parameters are computed as the solutions of a bounded quadratic optimization problem. In classification problems, the rules’ parameters are interpreted as the rules’ confidence. In regression problems, the rules’ parameters are used to derive rules’ confidences for classes that represent ranges of output variable values. The resulting model is evaluated based on a set of benchmark datasets for classification and regression problems. Nonparametric statistical tests were performed on the benchmark results, showing that the proposed approach produces compact fuzzy models with accuracy comparable to models produced by the standard modeling approaches. The resulting model is also exploited from the interpretability point of view, showing how the rule weights provide additional information to help in data and model understanding, such that it can be used as a decision support tool for the prediction of new data.     

三层前向神经网络的一致逼近性

近年来，前向神经网络泛逼近的一致性分析一直为众多学者所重视。本文系统分析三层前向网络对于拟差值保序函数族的一致逼近性，其中，转换函数σ是广义Sigmoidal函数。并将此一致性结果用于建立一类新的模糊神经网络(FNN)，即折线FNN．研究这类网络对于两个给定的模糊函数的逼近性，相关结论在分析折线FNN的泛逼近性时起关键作用。     

An efficient multi-objective evolutionary fuzzy system for regression problems

During the last years, multi-objective evolutionary algorithms (MOEAs) have been extensively employed as optimization tools for generating fuzzy rule-based systems (FRBSs) with different trade-offs between accuracy and interpretability from data. Since the size of the search space and the computational cost of the fitness evaluation depend on the number of input variables and instances, respectively, managing high-dimensional and large datasets is a critical issue.In this paper, we focus on MOEAs applied to learn concurrently the rule base and the data base of Mamdani FRBSs and propose to tackle the issue by exploiting the synergy between two different techniques. The first technique is based on a novel method which reduces the search space by learning rules not from scratch, but rather from a heuristically generated rule base. The second technique performs an instance selection by exploiting a co-evolutionary approach where cyclically a genetic algorithm evolves a reduced training set which is used in the evolution of the MOEA.The effectiveness of the synergy has been tested on twelve datasets. Using non-parametric statistical tests we show that, although achieving statistically equivalent solutions, the adoption of this synergy allows saving up to 97.38% of the execution time with respect to a state-of-the-art multi-objective evolutionary approach which learns rules from scratch.     

A fuzzy clustering based technique for piecewise affine approximation of a class of nonlinear systems

In recent years piecewise affine (PWA) modeling has developed as an attractive tool for the approximation of various complex nonlinear systems. In spite of the wide application of PWA modeling, the optimal approximation of a continuous time nonlinear system with scalar functions by the minimum number of affine systems has not been addressed properly in literature. This paper deals with a fuzzy clustering based approach for the optimal PWA approximation of a class of continuous time nonlinear systems. The technique is based on the trade-off between increasing the approximation accuracy of the various nonlinear functions and simplifying the approximation by the minimum number of subsystems. As an application, the technique is utilized to obtain a PWA approximation of the glucose regulation system. Numerical simulations depicted that, for a given number of subsystems, the derived glucose regulation model provides an optimal approximation of the original nonlinear system. The model also provided some biological insight about the interactions involved in glucose regulation.     

On the efficiency of evolutionary fuzzy clustering

This paper tackles the problem of showing that evolutionary algorithms for fuzzy clustering can be more efficient than systematic (i.e. repetitive) approaches when the number of clusters in a data set is unknown. To do so, a fuzzy version of an Evolutionary Algorithm for Clustering (EAC) is introduced. A fuzzy cluster validity criterion and a fuzzy local search algorithm are used instead of their hard counterparts employed by EAC. Theoretical complexity analyses for both the systematic and evolutionary algorithms under interest are provided. Examples with computational experiments and statistical analyses are also presented.     

A dominance intuitionistic fuzzy-rough set approach and its applications

Although the rough set and intuitionistic fuzzy set both capture the same notion, imprecision, studies on the combination of these two theories are rare. Rule extraction is an important task in a type of decision systems where condition attributes are taken as intuitionistic fuzzy values and those of decision attribute are crisp ones. To address this issue, this paper makes a contribution of the following aspects. First, a ranking method is introduced to construct the neighborhood of every object that is determined by intuitionistic fuzzy values of condition attributes. Moreover, an original notion, dominance intuitionistic fuzzy decision tables (DIFDT), is proposed in this paper. Second, a lower/upper approximation set of an object and crisp classes that are confirmed by decision attributes is ascertained by comparing the relation between them. Third, making use of the discernibility matrix and discernibility function, a lower and upper approximation reduction and rule extraction algorithm is devised to acquire knowledge from existing dominance intuitionistic fuzzy decision tables. Finally, the presented model and algorithms are applied to audit risk judgment on information system security auditing risk judgement for CISA, candidate global supplier selection in a manufacturing company, and cars classification.     

Robust Adaptive Identification of Fuzzy Systems with Uncertain Data

This study presents a method of adaptive identification of parameters describing Sugeno fuzzy inference system in presence of bounded disturbances while maintaining the readability and interpretability of the fuzzy model during and after identification. This method do not require any a priori knowledge of a bound on the disturbance and noise and of a bound on the unknown parameters values. The method can be used for the robust and adaptive identification of slowly time varying nonlinear systems using fuzzy inference systems. The suggested method was used to build a fuzzy expert system that approximates the functional relationship between physical fitness and some of the measurable physiological parameters by their real measurements and opinion (human-experiences) of a medical expert.     

A new approach to fuzzy wavelet system modeling

In this paper, we propose simple but effective two different fuzzy wavelet networks (FWNs) for system identification. The FWNs combine the traditional Takagi–Sugeno–Kang (TSK) fuzzy model and discrete wavelet transforms (DWT). The proposed FWNs consist of a set of if–then rules and, then parts are series expansion in terms of wavelets functions. In the first system, while the only one scale parameter is changing with it corresponding rule number, translation parameter sets are fixed in each rule. As for the second system, DWT is used completely by using wavelet frames. The performance of proposed fuzzy models is illustrated by examples and compared with previously published examples. Simulation results indicate the remarkable capabilities of the proposed methods. It is worth noting that the second FWN achieves high function approximation accuracy and fast convergence.     

Maximum likelihood least squares identification for systems with autoregressive moving average noise

Maximum likelihood methods are important for system modeling and parameter estimation. This paper derives a recursive maximum likelihood least squares identification algorithm for systems with autoregressive moving average noises, based on the maximum likelihood principle. In this derivation, we prove that the maximum of the likelihood function is equivalent to minimizing the least squares cost function. The proposed algorithm is different from the corresponding generalized extended least squares algorithm. The simulation test shows that the proposed algorithm has a higher estimation accuracy than the recursive generalized extended least squares algorithm.