A genetic design of linguistic terms for fuzzy rule based classifiers

The determination of fuzzy information granules including the estimation of their membership functions play a significant role in fuzzy system design as well as in the design of fuzzy rule based classifiers (FRBCSs). However, although linguistic terms are fundamental elements in the process of elucidating expert’s knowledge, the problem of linguistic term design along with their fuzzy-set-based semantics has not been fully addressed, since term-sets of attributes have not been interpreted as a formalized structure. Thus, the essential relationship between linguistic terms, as syntax, and the constructed fuzzy sets, as their quantitative semantics, or in other words, the problem of the natural semantics of terms behind the linguistic literal has not been addressed. In this paper, we introduce the problem of the design of optimal linguistic terms and propose a method of the design of FRBCSs which may incorporate with the design of linguistic terms to ensure that the presence of linguistic literals are supported not only by data but also by their natural semantics. It is shown that this problem plays a primordial role in enhancing the performance and the interpretability of the designed FRBCSs and helps striking a better balance between the generality and the specificity of the desired fuzzy rule bases for fuzzy classification problems. A series of experiments concerning 17 Machine Learning datasets is reported.     

Some relationships between fuzzy and random set-based classifiers and models

When designing rule-based models and classifiers, some precision is sacrificed to obtain linguistic interpretability. Understandable models are not expected to outperform black boxes, but usually fuzzy learning algorithms are statistically validated by contrasting them with black-box models. Unless performance of both approaches is equivalent, it is difficult to judge whether the fuzzy one is doing its best, because the precision gap between the best understandable model and the best black-box model is not known.In this paper we discuss how to generate probabilistic rule-based models and classifiers with the same structure as fuzzy rule-based ones. Fuzzy models, in which features are partitioned into linguistic terms, will be compared to probabilistic rule-based models with the same number of terms in every linguistic partition. We propose to use these probabilistic models to estimate a lower precision limit which fuzzy rule learning algorithms should surpass.     

A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems

The need for trading off interpretability and accuracy is intrinsic to the use of fuzzy systems. The obtaining of accurate but also human-comprehensible fuzzy systems played a key role in Zadeh and Mamdani’s seminal ideas and system identification methodologies. Nevertheless, before the advent of soft computing, accuracy progressively became the main concern of fuzzy model builders, making the resulting fuzzy systems get closer to black-box models such as neural networks. Fortunately, the fuzzy modeling scientific community has come back to its origins by considering design techniques dealing with the interpretability-accuracy tradeoff. In particular, the use of genetic fuzzy systems has been widely extended thanks to their inherent flexibility and their capability to jointly consider different optimization criteria. The current contribution constitutes a review on the most representative genetic fuzzy systems relying on Mamdani-type fuzzy rule-based systems to obtain interpretable linguistic fuzzy models with a good accuracy.     

Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation

One of the problems that focus the research in the linguistic fuzzy modeling area is the trade-off between interpretability and accuracy. To deal with this problem, different approaches can be found in the literature. Recently, a new linguistic rule representation model was presented to perform a genetic lateral tuning of membership functions. It is based on the linguistic 2-tuples representation that allows the lateral displacement of a label considering an unique parameter. This way to work involves a reduction of the search space that eases the derivation of optimal models and therefore, improves the mentioned trade-off.Based on the 2-tuples rule representation, this work proposes a new method to obtain linguistic fuzzy systems by means of an evolutionary learning of the data base a priori (number of labels and lateral displacements) and a simple rule generation method to quickly learn the associated rule base. Since this rule generation method is run from each data base definition generated by the evolutionary algorithm, its selection is an important aspect. In this work, we also propose two new ad hoc data-driven rule generation methods, analyzing the influence of them and other rule generation methods in the proposed learning approach. The developed algorithms will be tested considering two different real-world problems.     

A construction of sound semantic linguistic scales using 4-tuple representation of term semantics

Data semantics plays a fundamental role in computer science, in general, and in computing with words, in particular. The semantics of words arises as a sophisticated problem, since words being actually vague linguistic terms are pieces of information characterized by impreciseness, incompleteness, uncertainty and/or vagueness. The qualitative semantics and the quantitative semantics are two aspects of vague linguistic information, which are closely related. However, the qualitative semantics of linguistic terms, and even the qualitative semantics of the symbolic approaches, seem to be not elaborated on directly in the literature. In this study, we propose an interpretation of the inherent order-based semantics of terms through their qualitative semantics modeled by hedge algebra structures. The quantitative semantics of terms are developed based on the quantification of hedge algebras. With this explicit approach, we propose two concepts of assessment scales to address decision problems: linguistic scales used for representing expert linguistic assessments and semantic linguistic scales based on 4-tuple linguistic representation model, which forms a formalized structure useful for computing with words. An example of a simple multi-criteria decision problem is examined by running a comparative study. We also analyze the main advantages of the proposed approach.     

Mining axiomatic fuzzy set association rules for classification problems

In this paper, we propose a novel method to mine association rules for classification problems namely AFSRC (AFS association rules for classification) realized in the framework of the axiomatic fuzzy set (AFS) theory. This model provides a simple and efficient rule generation mechanism. It can also retain meaningful rules for imbalanced classes by fuzzifying the concept of the class support of a rule. In addition, AFSRC can handle different data types occurring simultaneously. Furthermore, the new model can produce membership functions automatically by processing available data. An extensive suite of experiments are reported which offer a comprehensive comparison of the performance of the method with the performance of some other methods available in the literature. The experimental result shows that AFSRC outperforms most of other methods when being quantified in terms of accuracy and interpretability. AFSRC forms a classifier with high accuracy and more interpretable rule base of smaller size while retaining a sound balance between these two characteristics.     

Construction of rule-based assignment models

Assignment refers to the problem of assigning objects or alternatives described on multiple dimensions into predefined categories.Most assignment models use analytical mechanisms to aggregate multiple dimensions in order to select a category for each candidate object. We investigate another approach based on the use of simple “if…then…” rules.We propose a general approach for a progressive construction of a rule-based assignment model. The process consists of testing iteratively the consistency of the rule base to transform it progressively into a consistent assignment model. Consistency tests are based on a correspondence between the logical representation of rules and an equivalent algebraic representation. This allows us to express rules by linear constraints and then to test the consistency of rule-based assignment models by solving a series of linear programs.     

Learning concurrently partition granularities and rule bases of Mamdani fuzzy systems in a multi-objective evolutionary framework

In this paper we propose a multi-objective evolutionary algorithm to generate Mamdani fuzzy rule-based systems with different good trade-offs between complexity and accuracy. The main novelty of the algorithm is that both rule base and granularity of the uniform partitions defined on the input and output variables are learned concurrently. To this aim, we introduce the concepts of virtual and concrete rule bases: the former is defined on linguistic variables, all partitioned with a fixed maximum number of fuzzy sets, while the latter takes into account, for each variable, a number of fuzzy sets as determined by the specific partition granularity of that variable. We exploit a chromosome composed of two parts, which codify the variables partition granularities, and the virtual rule base, respectively. Genetic operators manage virtual rule bases, whereas fitness evaluation relies on an appropriate mapping strategy between virtual and concrete rule bases. The algorithm has been tested on two real-world regression problems showing very promising results.     

MEPAR-miner: Multi-expression programming for classification rule mining

Classification and rule induction are two important tasks to extract knowledge from data. In rule induction, the representation of knowledge is defined as IF-THEN rules which are easily understandable and applicable by problem-domain experts. In this paper, a new chromosome representation and solution technique based on Multi-Expression Programming (MEP) which is named as MEPAR-miner (Multi-Expression Programming for Association Rule Mining) for rule induction is proposed. Multi-Expression Programming (MEP) is a relatively new technique in evolutionary programming that is first introduced in 2002 by Oltean and Dumitrescu. MEP uses linear chromosome structure. In MEP, multiple logical expressions which have different sizes are used to represent different logical rules. MEP expressions can be encoded and implemented in a flexible and efficient manner. MEP is generally applied to prediction problems; in this paper a new algorithm is presented which enables MEP to discover classification rules. The performance of the developed algorithm is tested on nine publicly available binary and n-ary classification data sets. Extensive experiments are performed to demonstrate that MEPAR-miner can discover effective classification rules that are as good as (or better than) the ones obtained by the traditional rule induction methods. It is also shown that effective gene encoding structure directly improves the predictive accuracy of logical IF-THEN rules.     

Extracting interpretable fuzzy models for nonlinear systems using gradient-based continuous ant colony optimization

This paper exploits the ability of a novel ant colony optimization algorithm called gradient-based continuous ant colony optimization, an evolutionary methodology, to extract interpretable first-order fuzzy Sugeno models for nonlinear system identification. The proposed method considers all objectives of system identification task, namely accuracy, interpretability, compactness and validity conditions. First, an initial structure of model is obtained by means of subtractive clustering. Then, an iterative two-step algorithm is employed to produce a simplified fuzzy model in terms of number of fuzzy sets and rules. In the first step, the parameters of the model are adjusted by utilizing the gradient-based continuous ant colony optimization. In the second step, the similar membership functions of an obtained model merge. The results obtained on three case studies illustrate the applicability of the proposed method to extract accurate and interpretable fuzzy models for nonlinear system identification.     

A parametric representation of linguistic hedges in Zadeh’s fuzzy logic

This paper proposes a model for the parametric representation of linguistic hedges in Zadeh’s fuzzy logic. In this model each linguistic truth-value, which is generated from a primary term of the linguistic truth variable, is identified by a real number r depending on the primary term. It is shown that the model yields a method of efficiently computing linguistic truth expressions accompanied with a rich algebraic structure of the linguistic truth domain, namely De Morgan algebra. Also, a fuzzy logic based on the parametric representation of linguistic truth-values is introduced.     

The OWA-based consensus operator under linguistic representation models using position indexes

When using linguistic approaches to solve decision problems, we need linguistic representation models. The symbolic model, the 2-tuple fuzzy linguistic representation model and the continuous linguistic model are three existing linguistic representation models based on position indexes. Together with these three linguistic models, the corresponding ordered weighted averaging operators, such as the linguistic ordered weighted averaging operator, the 2-tuple ordered weighted averaging operator and the extended ordered weighted averaging operator, have been developed, respectively. In this paper, we analyze the internal relationship among these operators, and propose a consensus operator under the continuous linguistic model (or the 2-tuple fuzzy linguistic representation model). The proposed consensus operator is based on the use of the ordered weighted averaging operator and the deviation measures. Some desired properties of the consensus operator are also presented. In particular, the consensus operator provides an alternative consensus model for group decision making. This consensus model preserves the original preference information given by the decision makers as much as possible, and supports consensus process automatically, without moderator.     

Data-driven fuzzy models for nonlinear identification of a complex heat exchanger

This paper presents and discusses experimental results on nonlinear model identification method applied to a real pilot thermal plant. The aim of this work is to develop a moderately complex model with interpretable structure for a complex parallel flow heat exchanger which is the main component of the thermal plant using a fuzzy clustering technique. The proposed Takagi–Sugeno-type (TS) fuzzy rule-based model is derived through an iterative fuzzy clustering algorithm using a set of input–output measurements. It is shown that the identified multivariable fuzzy rule-based model captures well the key dynamical properties of the physical plant over a wide operating range and under varying operating conditions. For validation, the model is run in parallel and series-parallel configurations to the real process. The experimental results show clearly the high performance of the proposed fuzzy model in achieving good prediction of the main process variables.     

Bisimulations and bisimulation games between Verbrugge models

Interpretability logic is a modal formalization of relative interpretability between first-order arithmetical theories. Verbrugge semantics is a generalization of Veltman semantics, the basic semantics for interpretability logic. Bisimulation is the basic equivalence between models for modal logic. We study various notions of bisimulation between Verbrugge models and develop a new one, which we call w-bisimulation. We show that the new notion, while keeping the basic property that bisimilarity implies modal equivalence, is weak enough to allow the converse to hold in the finitary case. To do this, we develop and use an appropriate notion of bisimulation games between Verbrugge models.     

An interpretability improvement for fuzzy rule bases obtained by the iterative rule learning approach

Interpretability is one of the key concepts in many of the applications using the fuzzy rule-based approach. It is well known that there are many different criteria around this concept, the complexity being one of them. In this paper, we focus our efforts in reducing the complexity of the fuzzy rule sets. One of the most interesting approaches for learning fuzzy rules is the iterative rule learning approach. It is mainly characterized by obtaining rules covering few examples in final stages, being in most cases useless to represent the knowledge. This behavior is due to the specificity of the extracted rules, which eventually creates more complex set of rules. Thus, we propose a modified version of the iterative rule learning algorithm in order to extract simple rules relaxing this natural trend. The main idea is to change the rule extraction process to be able to obtain more general rules, using pruned searching spaces together with a knowledge simplification scheme able to replace learned rules. The experimental results prove that this purpose is achieved. The new proposal reduces the complexity at both, the rule and rule base levels, maintaining the accuracy regarding to previous versions of the algorithm.     

Fuzzy rough sets and multiple-premise gradual decision rules

We propose a new fuzzy rough set approach which, differently from most known fuzzy set extensions of rough set theory, does not use any fuzzy logical connectives (t-norm, t-conorm, fuzzy implication). As there is no rationale for a particular choice of these connectives, avoiding this choice permits to reduce the part of arbitrary in the fuzzy rough approximation. Another advantage of the new approach is that it is based on the ordinal properties of fuzzy membership degrees only. The concepts of fuzzy lower and upper approximations are thus proposed, creating a base for induction of fuzzy decision rules having syntax and semantics of gradual rules. The proposed approach to rule induction is also interesting from the viewpoint of philosophy supporting data mining and knowledge discovery, because it is concordant with the method of concomitant variations by John Stuart Mill. The decision rules are induced from lower and upper approximations defined for positive and negative relationships between credibility degrees of multiple premises, on one hand, and conclusion, on the other hand.     

Fuzzy rule generation methods for high-level computer vision

In many decision making systems involving multiple sources, the decisions made may be considered as the result of a rule-based system in which the decision rules are usually enumerated by experts or generated by a learning process. In this paper, we discuss the various issues involved in the generation of fuzzy rules automatically from training data for high-level computer vision. Features are treated as linguistic variables that appear in the antecedent clauses of the rules. We present methods to generate the corresponding linguistic labels (values) and their membership functions. Rules are generated by constructing a minimal approximate fuzzy aggregation network and then training the network using gradient descent methods. Several examples are given.     

Improving the scalability of rule-based evolutionary learning

Evolutionary learning techniques are comparable in accuracy with other learning methods such as Bayesian Learning, SVM, etc. These techniques often produce more interpretable knowledge than, e.g. SVM; however, efficiency is a significant drawback. This paper presents a new representation motivated by our observations that Bioinformatics and Systems Biology often give rise to very large-scale datasets that are noisy, ambiguous and usually described by a large number of attributes. The crucial observation is that, in the most successful rules obtained for such datasets, only a few key attributes (from the large number of available ones) are expressed in a rule, hence automatically discovering these few key attributes and only keeping track of them contributes to a substantial speed up by avoiding useless match operations with irrelevant attributes. Thus, in effective terms this procedure is performing a fine-grained feature selection at a rule-wise level, as the key attributes may be different for each learned rule. The representation we propose has been tested within the BioHEL machine learning system, and the experiments performed show that not only the representation has competent learning performance, but that it also manages to reduce considerably the system run-time. That is, the proposed representation is up to 2–3 times faster than state-of-the-art evolutionary learning representations designed specifically for efficiency purposes.