Combining boosting and evolutionary algorithms for learning of fuzzy classification rules

This paper presents a novel boosting algorithm for genetic learning of fuzzy classification rules. The method is based on the iterative rule learning approach to fuzzy rule base system design. The fuzzy rule base is generated in an incremental fashion, in that the evolutionary algorithm optimizes one fuzzy classifier rule at a time. The boosting mechanism reduces the weight of those training instances that are classified correctly by the new rule. Therefore, the next rule generation cycle focuses on fuzzy rules that account for the currently uncovered or misclassified instances. The weight of a fuzzy rule reflects the relative strength the boosting algorithm assigns to the rule class when it aggregates the casted votes. The approach is compared with other classification algorithms for a number problem sets from the UCI repository.     

An interpretability improvement for fuzzy rule bases obtained by the iterative rule learning approach

Interpretability is one of the key concepts in many of the applications using the fuzzy rule-based approach. It is well known that there are many different criteria around this concept, the complexity being one of them. In this paper, we focus our efforts in reducing the complexity of the fuzzy rule sets. One of the most interesting approaches for learning fuzzy rules is the iterative rule learning approach. It is mainly characterized by obtaining rules covering few examples in final stages, being in most cases useless to represent the knowledge. This behavior is due to the specificity of the extracted rules, which eventually creates more complex set of rules. Thus, we propose a modified version of the iterative rule learning algorithm in order to extract simple rules relaxing this natural trend. The main idea is to change the rule extraction process to be able to obtain more general rules, using pruned searching spaces together with a knowledge simplification scheme able to replace learned rules. The experimental results prove that this purpose is achieved. The new proposal reduces the complexity at both, the rule and rule base levels, maintaining the accuracy regarding to previous versions of the algorithm.     

Evolving fuzzy rule based controllers using genetic algorithms

The synthesis of genetics-based machine learning and fuzzy logic is beginning to show promise as a potent tool in solving complex control problems in multi-variate non-linear systems. In this paper an overview of current research applying the genetic algorithm to fuzzy rule based control is presented. A novel approach to genetics-based machine learning of fuzzy controllers, called a Pittsburgh Fuzzy Classifier System # 1 (P-FCS1) is proposed. P-FCS1 is based on the Pittsburgh model of learning classifier systems and employs variable length rule-sets and simultaneously evolves fuzzy set membership functions and relations. A new crossover operator which respects the functional linkage between fuzzy rules with overlapping input fuzzy set membership functions is introduced. Experimental results using P-FCS 1 are reported and compared with other published results. Application of P-FCS1 to a distributed control problem (dynamic routing in computer networks) is also described and experimental results are presented.     

New fuzzy wavelet network for modeling and control: The modeling approach

In this paper, a fuzzy wavelet network is proposed to approximate arbitrary nonlinear functions based on the theory of multiresolution analysis (MRA) of wavelet transform and fuzzy concepts. The presented network combines TSK fuzzy models with wavelet transform and ROLS learning algorithm while still preserve the property of linearity in parameters. In order to reduce the number of fuzzy rules, fuzzy clustering is invoked. In the clustering algorithm, those wavelets that are closer to each other in the sense of the Euclidean norm are placed in a group and are used in the consequent part of a fuzzy rule. Antecedent parts of the rules are Gaussian membership functions. Determination of the deviation parameter is performed with the help of gold partition method. Here, mean of each function is derived by averaging center of all wavelets that are related to that particular rule. The overall developed fuzzy wavelet network is called fuzzy wave-net and simulation results show superior performance over previous networks.The present work is complemented by a second part which focuses on the control aspects and to be published in this journal([17]). This paper proposes an observer based self-structuring robust adaptive fuzzy wave-net (FWN) controller for a class of nonlinear uncertain multi-input multi-output systems.     

A fast genetic method for inducting descriptive fuzzy models

Under certain inference mechanisms, fuzzy rule bases can be regarded as extended additive models. This relationship can be applied to extend some statistical techniques to learn fuzzy models from data. The interest in this parallelism is twofold: theoretical and practical. First, extended additive models can be estimated by means of the matching pursuit algorithm, which has been related to Support Vector Machines, Boosting and Radial Basis neural networks learning; this connection can be exploited to better understand the learning of fuzzy models. In particular, the technique we propose here can be regarded as the counterpart to boosting fuzzy classifiers in the field of fuzzy modeling. Second, since matching pursuit is very efficient in time, we can expect to obtain faster algorithms to learn fuzzy rules from data. We show that the combination of a genetic algorithm and the backfitting process learns faster than ad hoc methods in certain datasets.     

Reasoning under uncertainty with FIR methodology

The aim of this research is to develop a new methodology called UNFIR (uncertainty in FIR) as an extension of the fuzzy inductive reasoning (FIR) technique. The main idea behind UNFIR is to expand the modeling capacity of the FIR methodology allowing it to work with classical fuzzy rules. On the one hand, UNFIR is able to automatically construct fuzzy rules starting from a set of pattern rules obtained by FIR. On the other hand, UNFIR affords the prediction of systems behavior by using a mixed pattern/fuzzy inference system that takes advantage of the uncertainty inherent to the data. The pattern rule base that the FIR methodology generates can be very large, obstructing the prediction process and reducing its efficiency. The new methodology preserves as much as possible the knowledge of the pattern rules in a compact fuzzy rule base. In this process some precision is lost but the robustness is considerably increased.The performance of UNFIR methodology as a systems’ prediction tool is also studied in this work. Three different applications are used for this purpose, i.e., a linear system, a non-linear system and an industrial process.     

Comparison of Heuristic Criteria for Fuzzy Rule Selection in Classification Problems

This paper compares heuristic criteria used for extracting a pre-specified number of fuzzy classification rules from numerical data. We examine the performance of each heuristic criterion through computational experiments on well-known test problems. Experimental results show that better results are obtained from composite criteria of confidence and support measures than their individual use. It is also shown that genetic algorithm-based rule selection can improve the classification ability of extracted fuzzy rules by searching for good rule combinations. This observation suggests the importance of taking into account the combinatorial effect of fuzzy rules (i.e., the interaction among them).     

Evolution of fuzzy classifiers using genetic programming

In this paper, we propose a genetic programming (GP) based approach to evolve fuzzy rule based classifiers. For a c-class problem, a classifier consists of c trees. Each tree, T _i , of the multi-tree classifier represents a set of rules for class i. During the evolutionary process, the inaccurate/inactive rules of the initial set of rules are removed by a cleaning scheme. This allows good rules to sustain and that eventually determines the number of rules. In the beginning, our GP scheme uses a randomly selected subset of features and then evolves the features to be used in each rule. The initial rules are constructed using prototypes, which are generated randomly as well as by the fuzzy k-means (FKM) algorithm. Besides, experiments are conducted in three different ways: Using only randomly generated rules, using a mixture of randomly generated rules and FKM prototype based rules, and with exclusively FKM prototype based rules. The performance of the classifiers is comparable irrespective of the type of initial rules. This emphasizes the novelty of the proposed evolutionary scheme. In this context, we propose a new mutation operation to alter the rule parameters. The GP scheme optimizes the structure of rules as well as the parameters involved. The method is validated on six benchmark data sets and the performance of the proposed scheme is found to be satisfactory.     

Real-time stable self-learning FNN controller using genetic algorithm

A kind of real-time stable self-learning fuzzy neural network (FNN) control system is proposed in this paper. The control system is composed of two parts: (1) A FNN controller which use genetic algorithm (GA) to search optimal fuzzy rules and membership functions for the unknown controlled plant; (2) A supervisor which can guarantee the stability of the control system during the real-time learning stage, since the GA has some random property which may cause control system unstable. The approach proposed in this paper combine a priori knowledge of designer and the learning ability of FNN to achieve optimal fuzzy control for an unknown plant in real-time. The efficiency of the approach is verified by computer simulation.     

Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning

This paper examines the interpretability-accuracy tradeoff in fuzzy rule-based classifiers using a multiobjective fuzzy genetics-based machine learning (GBML) algorithm. Our GBML algorithm is a hybrid version of Michigan and Pittsburgh approaches, which is implemented in the framework of evolutionary multiobjective optimization (EMO). Each fuzzy rule is represented by its antecedent fuzzy sets as an integer string of fixed length. Each fuzzy rule-based classifier, which is a set of fuzzy rules, is represented as a concatenated integer string of variable length. Our GBML algorithm simultaneously maximizes the accuracy of rule sets and minimizes their complexity. The accuracy is measured by the number of correctly classified training patterns while the complexity is measured by the number of fuzzy rules and/or the total number of antecedent conditions of fuzzy rules. We examine the interpretability-accuracy tradeoff for training patterns through computational experiments on some benchmark data sets. A clear tradeoff structure is visualized for each data set. We also examine the interpretability-accuracy tradeoff for test patterns. Due to the overfitting to training patterns, a clear tradeoff structure is not always obtained in computational experiments for test patterns.     

基于模糊集相容性的模糊控制规则优化方法

在简要介绍模糊模型的完备性和相容性的基础上 ,根据模糊集贴近度 ,提出模糊集相容性的概念 ;通过对模糊控制规则表特征的分析 ,进一步完善规则相容性概念及其定量评价方法 ;最后给出模糊控制规则模型自寻优优化方法。仿真结果表明 ,该方法可以大大提高控制系统的控制性能。     

Extracting compact and information lossless sets of fuzzy association rules

Applying classical association rule extraction framework on fuzzy datasets leads to an unmanageably highly sized association rule sets. Moreover, the discretization operation leads to information loss and constitutes a hamper towards an efficient exploitation of the mined knowledge. To overcome such a drawback, this paper proposes the extraction and the exploitation of compact and informative generic basis of fuzzy association rules. The presented approach relies on the extension, within the fuzzy context, of the notion of closure and Galois connection, that we introduce in this paper. In order to select without loss of information a generic subset of all fuzzy association rules, we define three fuzzy generic basis from which remaining (redundant) FARs are generated. This generic basis constitutes a compact nucleus of fuzzy association rules, from which it is possible to informatively derive all the remaining rules. In order to ensure a sound and complete derivation process, we introduce an axiomatic system allowing the complete derivation of all the redundant rules. The results obtained from experiments carried out on benchmark datasets are very encouraging. They highlight a very important reduction of the number of the extracted fuzzy association rules without information loss.     

Rule generation for hierarchical collaborative fuzzy system

A new method of rule generation for the hierarchical collaborative fuzzy system, HCFS, is proposed. This HCFS is structured like various parallel fuzzy subsystems and it overcomes the dimensionality problem and the lack of interpretability of most of the traditional fuzzy systems, when dealing with complex real-world problems. An association process of different fuzzy systems is presented in this work, through the use of a relevance concept of a fuzzy system. The result of this aggregation is a collaborative structure where all sub-models have the ability to gradually improve the overall accuracy of approximation by adding their own contributions. For this structure we propose a new algorithm to be used in the procedures of the three learning phases: the structure building, the parametric identification and the division of the learning data among the various levels of the hierarchical structure. This new fuzzy modelling technique automatically generates and tunes the sets of fuzzy rules in the hierarchical collaborative structure (HCS). The effectiveness of the proposed HCFS model in handling high-dimensional and complex problems is demonstrated through various numerical simulations.     

Decomposition of a complex fuzzy controller for the truck-and-trailer reverse parking problem

The use of fuzzy logic has, in the last twenty years, become standard practice in the field of control. The reason lies in the fuzzy logic’s ability to relatively quickly transfer uncertain experience and knowledge about the observed object’s behaviour into the process of decision making. Nevertheless, one of the biggest problems that arises when using a fuzzy approach is the large number of fuzzy rules that have to be processed in order to produce one decision (i.e. one control output). The number of rules in a fuzzy controller primarily originates from the number of input variables that are entering the decision process and one possible solution for decreasing it is to use the method of decomposition. Its main goal is to implement the equivalent control functionality with a hierarchy of simpler fuzzy controllers. Their main characteristic is a lower number of input variables, which as a consequence leads to a smaller number of fuzzy rules. In our paper we apply the decomposition approach to the classical complex control case of the Truck-and-Trailer (T&T) reverse parking control problem. In such cases the implementation of control using only one fuzzy controller is very complex and the existing solutions, in some details, even deviate from the classical fuzzy approach. Our solution is, on the other hand, based only on the uncertain knowledge about the behaviour of the T&T driver and the results achieved are even better than those achieved by using the existing solutions.     

A neural fuzzy control system with structure and parameter learning

A general connectionist model, called neural fuzzy control network (NFCN), is proposed for the realization of a fuzzy logic control system. The proposed NFCN is a feedforward multilayered network which integrates the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure which has distributed learning abilities. The NFCN can be constructed from supervised training examples by machine learning techniques, and the connectionist structure can be trained to develop fuzzy logic rules and find membership functions. Associated with the NFCN is a two-phase hybrid learning algorithm which utilizes unsupervised learning schemes for structure learning and the backpropagation learning scheme for parameter learning. By combining both unsupervised and supervised learning schemes, the learning speed converges much faster than the original backpropagation algorithm. The two-phase hybrid learning algorithm requires exact supervised training data for learning. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a reinforcement neural fuzzy control network (RNFCN) is further proposed. The RNFCN is constructed by integrating two NFCNs, one functioning as a fuzzy predictor and the other as a fuzzy controller. By combining a proposed on-line supervised structure-parameter learning technique, the temporal difference prediction method, and the stochastic exploratory algorithm, a reinforcement learning algorithm is proposed, which can construct a RNFCN automatically and dynamically through a reward-penalty signal (i.e., “good” or “bad” signal). Two examples are presented to illustrate the performance and applicability of the proposed models and learning algorithms.     

Redundant fuzzy rules exclusion by genetic algorithms

A genetic-algorithm-based method for exclusion of the potential redundant if-then fuzzy rules that have been extracted from numerical input-output data is proposed. The main idea is the input-space separation into activation rectangles, corresponding to certain output intervals. The generation of fuzzy rules and the membership functions are based on these activation rectangles and appropriate fuzzy rules inference mechanism is proposed. As the method usually produces too many rules, it is necessary to exclude the potential redundant if-then rules. The concept for varying the family of sensitivity parameters, defining the overlapping of the fuzzy regions is proposed. The genetic algorithms are used to resolve the following combinatorial optimization problem: the generation of families of sensitivity parameters. In this way the potential redundant if-then fuzzy rules are excluded.

The method formalizes the synthesis of the fuzzy system and could be used for function approximation, classification and control purposes. An illustrative example for implementation of the method for traffic fuzzy control is given.     

Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation

One of the problems that focus the research in the linguistic fuzzy modeling area is the trade-off between interpretability and accuracy. To deal with this problem, different approaches can be found in the literature. Recently, a new linguistic rule representation model was presented to perform a genetic lateral tuning of membership functions. It is based on the linguistic 2-tuples representation that allows the lateral displacement of a label considering an unique parameter. This way to work involves a reduction of the search space that eases the derivation of optimal models and therefore, improves the mentioned trade-off.Based on the 2-tuples rule representation, this work proposes a new method to obtain linguistic fuzzy systems by means of an evolutionary learning of the data base a priori (number of labels and lateral displacements) and a simple rule generation method to quickly learn the associated rule base. Since this rule generation method is run from each data base definition generated by the evolutionary algorithm, its selection is an important aspect. In this work, we also propose two new ad hoc data-driven rule generation methods, analyzing the influence of them and other rule generation methods in the proposed learning approach. The developed algorithms will be tested considering two different real-world problems.     

A soft computing-based approach to spatio-temporal prediction

This paper aims to incorporate intelligent mechanisms based on Soft Computing in Geographical Information Systems (GIS). The proposal here is to present a spatio-temporal prediction method of forestry evolution for a sequence of binary images by means of fuzzy inference systems (FIS), genetic algorithm (GA) and genetic programming (GP). The main inference is based on a fuzzy system which processes a set of crisp/fuzzy relations and infers a crisp relation representing the predicted image at a predefined date. The fuzzy system is formed by a fixed fuzzy rule base and a partition set that may be defined by an expert or optimized by means of a GA. Genetic programming may also be adopted to generate the size of predicted area used in the final stage of the inference process. The developed methodology is applied in regions of Venezuela, France and Guatemala to identify their forestry evolution trends. The proposed approaches are compared with other techniques to validate the system.     

Decision-making in testing process performance with fuzzy data

Over the years, numerous process capability indices (PCIs) have been proposed to the manufacturing industry to provide numerical measures of process performance. Most research efforts have focused on developing and investigating PCIs that assess process capability by precise measurements of output quality. However, real observations of continuous quantities are not precise numbers; in practice, they are more or less imprecise. Since observations of continuous random variables are imprecise the values of related test statistics become imprecise. Therefore, decision rules for statistical tests have to be adapted to this situation. This article presents a set of confidence intervals that produces triangular fuzzy numbers for the estimation of C_pk index using Buckley’s approach with some modification. Additionally, a three-decision testing rule and step-by-step procedure are developed to assess process performance based on fuzzy critical values and fuzzy p-values. This concept is also illustrated with an example for testing process performance.