A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems

The need for trading off interpretability and accuracy is intrinsic to the use of fuzzy systems. The obtaining of accurate but also human-comprehensible fuzzy systems played a key role in Zadeh and Mamdani’s seminal ideas and system identification methodologies. Nevertheless, before the advent of soft computing, accuracy progressively became the main concern of fuzzy model builders, making the resulting fuzzy systems get closer to black-box models such as neural networks. Fortunately, the fuzzy modeling scientific community has come back to its origins by considering design techniques dealing with the interpretability-accuracy tradeoff. In particular, the use of genetic fuzzy systems has been widely extended thanks to their inherent flexibility and their capability to jointly consider different optimization criteria. The current contribution constitutes a review on the most representative genetic fuzzy systems relying on Mamdani-type fuzzy rule-based systems to obtain interpretable linguistic fuzzy models with a good accuracy.     

The principal-agent problem with evolutionary learning

We extend agency theory to incorporate bounded rationality of both principals and agents. In this study we define a simple version of the principal-agent game and examine it using object-oriented computer simulation. Player learning is simulated with a genetic algorithm model. Our results show that players of incentive games in highly uncertain environments may take on defensive strategies. These defensive strategies lead to equilibria which are inferior to Nash equilibria. If agents are risk averse, the principal may not be able to provide enough monetary compensation to encourage them to take risks. But principals may be able to improve system performance by identifying good performers and facilitating information exchange among agents.The authors would like to thank the anonymous referees for their helpful suggestions.     

Autonomous vehicle navigation using evolutionary reinforcement learning

Reinforcement learning schemes perform direct on-line search in control space. This makes them appropriate for modifying control rules to obtain improvements in the performance of a system. The effectiveness of a reinforcement learning strategy is studied here through the training of a learning classifier system (LCS) that controls the movement of an autonomous vehicle in simulated paths including left and right turns. The LCS comprises a set of condition-action rules (classifiers) that compete to control the system and evolve by means of a genetic algorithm (GA). Evolution and operation of classifiers depend upon an appropriate credit assignment mechanism based on reinforcement learning. Different design options and the role of various parameters have been investigated experimentally. The performance of vehicle movement under the proposed evolutionary approach is superior compared with that of other (neural) approaches based on reinforcement learning that have been applied previously to the same benchmark problem.     

Epistemological change through peer apprenticeship learning: From rule-based to idea-based social constructivism

The thrust of peer apprenticeship learning is in how an individual's personal beliefs, disposition, and epistemology can be transformed through an apprenticeship-like learning process. This paper describes the peer apprenticeship learning situation between two students, Dom (age: 13) and Ming (age: 14), in distributed computer-mediated co-construction of mathematical meanings. Ming was initially procedural or rule-based in his problem solving methods, encountering numerous difficulties with his approaches. Dom, however, was constantly engaged in playing with ideas through conjecturing and perceiving mathematical relationships. In the initial stages of co-construction efforts, Dom and Ming were solving the problems rather independently as each appropriated a radically different epistemology of mathematics – Ming was rule-based whereas Dom was idea-based. In the cause of increasing difficulties, Ming soon recognized that his methods were inadequate and, after a considerable struggle, positioned himself in an apprentice-like manner in order to follow Dom's conceptualizations. Through monitoring of Dom's conceptualizations and with personal experimentations to concretize his understanding, Ming was gradually able to assimilate Dom's mathematical meaning perspectives. We depicted such a learning situation as peer apprenticeship learning. As a result of assimilating the disposition towards playing with ideas, Dom and Ming were able to engage in meaningful idea-based social constructivism.This revised version was published online in September 2005 with corrections to the Cover Date.     

Investment using evolutionary learning methods and technical rules

In this paper, I propose a genetic learning approach to generate technical trading systems for stock timing. The most informative technical indicators are selected from a set of almost 5000 signals by a multi-objective genetic algorithm with variable string length. Successively, these signals are combined into a unique trading signal by a learning method. I test the expert weighting solution obtained by the plurality voting committee, the Bayesian model averaging and Boosting procedures with data from the S&P 500 Composite Index, in three market phases, up-trend, down-trend and sideways-movements, covering the period 2000–2006. Computational results indicate that the near-optimal set of rules varies among market phases but presents stable results and is able to reduce or eliminate losses in down-trend periods.     

Support vector machine learning with an evolutionary engine

The paper presents a novel evolutionary technique constructed as an alternative of the standard support vector machines architecture. The approach adopts the learning strategy of the latter but aims to simplify and generalize its training, by offering a transparent substitute to the initial black-box. Contrary to the canonical technique, the evolutionary approach can at all times explicitly acquire the coefficients of the decision function, without any further constraints. Moreover, in order to converge, the evolutionary method does not require the positive (semi-)definition properties for kernels within nonlinear learning. Several potential structures, enhancements and additions are proposed, tested and confirmed using available benchmarking test problems. Computational results show the validity of the new approach in terms of runtime, prediction accuracy and flexibility.     

Integrating memetic search into the BioHEL evolutionary learning system for large-scale datasets

Local search methods are widely used to improve the performance of evolutionary computation algorithms in all kinds of domains. Employing advanced and efficient exploration mechanisms becomes crucial in complex and very large (in terms of search space) problems, such as when employing evolutionary algorithms to large-scale data mining tasks. Recently, the GAssist Pittsburgh evolutionary learning system was extended with memetic operators for discrete representations that use information from the supervised learning process to heuristically edit classification rules and rule sets. In this paper we first adapt some of these operators to BioHEL, a different evolutionary learning system applying the iterative learning approach, and afterwards propose versions of these operators designed for continuous attributes and for dealing with noise. The performance of all these operators and their combination is extensively evaluated on a broad range of synthetic large-scale datasets to identify the settings that present the best balance between efficiency and accuracy. Finally, the identified best configurations are compared with other classes of machine learning methods on both synthetic and real-world large-scale datasets and show very competent performance.     

A hybrid evolutionary learning algorithm for TSK-type fuzzy model design

In this paper, a TSK-type fuzzy model (TFM) with a hybrid evolutionary learning algorithm (HELA) is proposed. The proposed HELA method combines the compact genetic algorithm (CGA) and the modified variable-length genetic algorithm (MVGA). Both the number of fuzzy rules and the adjustable parameters in the TFM are designed concurrently by the HELA method. In the proposed HELA method, individuals of the same length constitute the same group, and there are multiple groups in a population. Moreover, the proposed HELA adopts the compact genetic algorithm (CGA) to carry out the elite-based reproduction strategy. The CGA represents a population as a probability distribution over the set of solutions and is operationally equivalent to the order-one behavior of the simple GA. The evolution processes of a population consist of three major operations: group reproduction using the compact genetic algorithm, variable two-part individual crossover, and variable two-part mutation. Computer simulations have demonstrated that the proposed HELA method gives a better performance than some existing methods.     

Improving student learning in calculus through applications

Nationally only 40% of the incoming freshmen Science, Technology, Engineering and Mathematics (STEM) majors are successful in earning a STEM degree. The University of Central Florida (UCF) EXCEL programme is a National Science Foundation funded STEM Talent Expansion Programme whose goal is to increase the number of UCF STEM graduates. One of the key requirements for STEM majors is a strong foundation in Calculus. To improve student learning in calculus, the EXCEL programme developed two special courses at the freshman level called Applications of Calculus I (Apps I) and Applications of Calculus II (Apps II). Apps I and II are one-credit classes that are co-requisites for Calculus I and II. These classes are teams taught by science and engineering professors whose goal is to demonstrate to students where the calculus topics they are learning appear in upper level science and engineering classes as well as how faculty use calculus in their STEM research programmes. This article outlines the process used in producing the educational materials for the Apps I and II courses, and it also discusses the assessment results pertaining to this specific EXCEL activity. Pre- and post-tests conducted with experimental and control groups indicate significant improvement in student learning in Calculus II as a direct result of the application courses.     

Combining boosting and evolutionary algorithms for learning of fuzzy classification rules

This paper presents a novel boosting algorithm for genetic learning of fuzzy classification rules. The method is based on the iterative rule learning approach to fuzzy rule base system design. The fuzzy rule base is generated in an incremental fashion, in that the evolutionary algorithm optimizes one fuzzy classifier rule at a time. The boosting mechanism reduces the weight of those training instances that are classified correctly by the new rule. Therefore, the next rule generation cycle focuses on fuzzy rules that account for the currently uncovered or misclassified instances. The weight of a fuzzy rule reflects the relative strength the boosting algorithm assigns to the rule class when it aggregates the casted votes. The approach is compared with other classification algorithms for a number problem sets from the UCI repository.     

Construction of rule-based assignment models

Assignment refers to the problem of assigning objects or alternatives described on multiple dimensions into predefined categories.Most assignment models use analytical mechanisms to aggregate multiple dimensions in order to select a category for each candidate object. We investigate another approach based on the use of simple “if…then…” rules.We propose a general approach for a progressive construction of a rule-based assignment model. The process consists of testing iteratively the consistency of the rule base to transform it progressively into a consistent assignment model. Consistency tests are based on a correspondence between the logical representation of rules and an equivalent algebraic representation. This allows us to express rules by linear constraints and then to test the consistency of rule-based assignment models by solving a series of linear programs.     

Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms

The identification of a model is one of the key issues in the field of fuzzy system modeling and function approximation theory. An important characteristic that distinguishes fuzzy systems from other techniques in this area is their transparency and interpretability. Especially in the construction of a fuzzy system from a set of given training examples, little attention has been paid to the analysis of the trade-off between complexity and accuracy maintaining the interpretability of the final fuzzy system. In this paper a multi-objective evolutionary approach is proposed to determine a Pareto-optimum set of fuzzy systems with different compromises between their accuracy and complexity. In particular, two fundamental and competing objectives concerning fuzzy system modeling are addressed: fuzzy rule parameter optimization and the identification of system structure (i.e. the number of membership functions and fuzzy rules), taking always in mind the transparency of the obtained system. Another key aspect of the algorithm presented in this work is the use of some new expert evolutionary operators, specifically designed for the problem of fuzzy function approximation, that try to avoid the generation of worse solutions in order to accelerate the convergence of the algorithm.     

A rule-based development of incremental models

In the study, we propose a concept of incremental fuzzy models in which fuzzy rules are aimed at compensating discrepancies resulting because of the use of a certain global yet simple model of general nature (such as e.g., a constant or linear regression). The structure of input data and error discovered through fuzzy clustering is captured in the form of a collection of fuzzy clusters, which helps eliminate (compensate) error produced by the global model. We discuss a detailed architecture of the proposed rule-based model and present its design based on an augmented version of Fuzzy C-Means (FCM). An extended suite of experimental studies offering some comparative analysis is covered as well.     

Robust design of rule-based fuzzy controllers

This paper deals with the design of stable and robust rule-based fuzzy control systems. New expressions to compute indices which provide a measure of the stability and robustness of fuzzy control systems are presented. The relation between the modification of the rules and the stability is studied through the so-called sensitivity indices. The paper presents procedures that make use of these indices to improve the design of fuzzy control systems, including the modification of the rules to obtain the global stability of an unstable system with multiple attractors, and to improve the dynamic behavior or the robustness of a non-linear plant. An example with a fuzzy controller for a system with non-linear damping and saturation in the actuation is presented to illustrate the design procedure.     

Speeding up chaos and limit cycles in evolutionary language and learning processes

Evolution of human language and learning processes have their foundation built on grammar that sets rules for construction of sentences and words. These forms of replicator–mutator (game dynamical with learning) dynamics remain however complex and sometime unpredictable because they involve children with some predispositions. In this paper, a system modeling evolutionary language and learning dynamics is investigated using the Crank–Nicholson numerical method together with the new differentiation with non‐singular kernel. Stability and convergence are comprehensively proven for the system. In order to seize the effects of the non‐singular kernel, an application to game dynamical with learning dynamics for a population with five languages is given together with numerical simulations. It happens that such dynamics, as functions of the learning accuracy μ, can exhibit unusual bifurcations and limit cycles followed by chaotic behaviors. This points out the existence of fickle and unpredictable variations of languages as time goes on, certainly due to the presence of learning errors. More interestingly, this chaos is shown to be dependent on the order of the non‐singular kernel derivative and speeds up as this derivative order decreases. Hence, can people use that order to control chaotic behaviors observed in game dynamical systems with learning! Copyright © 2016 John Wiley & Sons, Ltd.     

Quadrature rule-based bounds for functions of adjacency matrices

Bounds for entries of matrix functions based on Gauss-type quadrature rules are applied to adjacency matrices associated with graphs. This technique allows to develop inexpensive and accurate upper and lower bounds for certain quantities (Estrada index, subgraph centrality, communicability) that describe properties of networks.     

Improving biorefinery planning: Integration of spatial data using exact optimization nested in an evolutionary strategy

Biorefineries can provide a product portfolio from renewable biomass similar to that of crude oil refineries. To operate biorefineries of any kind, however, the availability of biomass inputs is crucial and must be considered during planning. Here, we develop a planning approach that uses Geographic Information Systems (GIS) to account for spatially scattered biomass when optimizing a biorefinery’s location, capacity, and configuration. To deal with the challenges of a non-smooth objective function arising from the geographic data, higher dimensionality, and strict constraints, the planning problem is repeatedly decomposed by nesting an exact nonlinear program (NLP) inside an evolutionary strategy (ES) heuristic, which handles the spatial data from the GIS. We demonstrate the functionality of the algorithm and show how including spatial data improves the planning process by optimizing a synthesis gas biorefinery using this new planning approach.     

The use of mathematical programming to verify rule-based knowledge

The purpose of this paper is to use mathematical programming, including linear programming, dynamic programming, integer programming and goal programming to verify rule-based knowledge. We investigate both domain independent verification, exploiting the general structure of rules, and domain dependent verification, exploiting structure in the domain. Mathematical programming software is readily available and is very efficient. As a result, verification using mathematical programming can be very efficient at finding errors. Mathematical programming can be used to more than just find errors in knowledge representation. Once an error has been found, mathematical programming can be used to recommend an alternative. The recommendation can take into account the previous verified knowledge to mitigate the potential introduction of redundant knowledge and to help guide the choice process. Normally the development of recommendations to fix errors has been ignored in the verification literature, and treated as a separate knowledge acquisition task. Accordingly, this paper also extends the verification effort by providing a recommendation on how to fix errors.