Solvable models of layered neural networks based on their differential structure

We introduce the notion of solvable models of artificial neural networks, based on the theory of ordinary differential equations. It is shown that a solvable, three layer, neural network can be realized as a solution of an ordinary differential equation. Several neural networks in standard use are shown to be solvable. This leads to a new, two-step, non-recursive learning paradigm: estimate the differential equation which the target function satisfies approximately, and then approximate the target function in the solution space of that differential equation. It is shown experimentally that the proposed algorithm is useful for analyzing the generalization problem in artificial neural networks. Connections with wavelet analysis are also pointed out.     

GA-BP嵌套算法的理论及应用

分析了BP算法、遗传算法以及GA-BP-APARTING算法的特点,提出了GA-BP-NESTING算法.在人工神经网络的在线学习和离线学习方式下,分别对BP算法、GA算法、GA-BP-APARTING算法和GA-BP-NESTING算法进行了比较研究,研究发现:第一,网络初始权值的赋值对人工神经网络训练影响很大;第二,离线学习方式下GA-BP-NESTING算法效果最佳.     

Reducing network and computation complexities in neural based real-time scheduling scheme

Multiprocessor real-time scheduling is an important issue in many applications. A neural network provides a highly effective method to obtain good solutions for real-time scheduling problems. However, multiprocessor real-time scheduling problems include multiple variables; processor, process and time, and the neural networks have to be presented in three dimensions with these variables. Hence, the corresponding neural networks have more neurons, and synaptic weights, and thus associated network and computational complexities increase. Meanwhile, a neural network using the competitive scheme can provide a highly effective method with less network complexity. Therefore, in this study, a simplified two-dimensional Hopfield-type neural network using competitive rule is introduced for solving three-dimensional multiprocessor real-time scheduling problems. Restated, a two-dimensional network is proposed to lower the neural network dimensions and decrease the number of neurons and hence reduce the network complexity; an M-out-of-N competitive scheme is suggested to greatly reduce the computational complexity. Simulation results reveal that the proposed scheme imposed on the derived energy function with respect to process time and deadline constraints is an appropriate approach to solving these class scheduling problems. Moreover, the computational complexity of the proposed scheme is greatly lowered to O(N × T²).     

Real-time stable self-learning FNN controller using genetic algorithm

A kind of real-time stable self-learning fuzzy neural network (FNN) control system is proposed in this paper. The control system is composed of two parts: (1) A FNN controller which use genetic algorithm (GA) to search optimal fuzzy rules and membership functions for the unknown controlled plant; (2) A supervisor which can guarantee the stability of the control system during the real-time learning stage, since the GA has some random property which may cause control system unstable. The approach proposed in this paper combine a priori knowledge of designer and the learning ability of FNN to achieve optimal fuzzy control for an unknown plant in real-time. The efficiency of the approach is verified by computer simulation.     

Selecting informative rules with parallel genetic algorithm in classification problem

The classification system is very important for making decision and it has been attracted much attention of many researchers. Usually, the traditional classifiers are either domain specific or produce unsatisfactory results over classification problems with larger size and imbalanced data. Hence, genetic algorithms (GA) are recently being combined with traditional classifiers to find useful knowledge for making decision. Although, the main concerns of such GA-based system are the coverage of less search space and increase of computational cost with the growth of population. In this paper, a rule-based knowledge discovery model, combining C4.5 (a Decision Tree based rule inductive algorithm) and a new parallel genetic algorithm based on the idea of massive parallelism, is introduced. The prime goal of the model is to produce a compact set of informative rules from any kind of classification problem. More specifically, the proposed model receives a base method C4.5 to generate rules which are then refined by our proposed parallel GA. The strength of the developed system has been compared with pure C4.5 as well as the hybrid system (C4.5 + sequential genetic algorithm) on six real world benchmark data sets collected from UCI (University of California at Irvine) machine learning repository. Experiments on data sets validate the effectiveness of the new model. The presented results especially indicate that the model is powerful for volumetric data set.     

BP-GA混合优化策略在人力资源战略规划中的应用

采用混合优化策略训练神经网络,进而实现地区人力资源数据的时间序列预测.神经网络,尤其是应用反向传播(back propagation,简称BP)算法训练的神经网络,被广泛应用于预测中.但是BP神经网络训练速度慢、容易陷入局部极值.遗传算法(genetic algorithm,简称GA)具有很好的全局寻优性.因而提出将BP和GA结合起来的混合优化策略训练神经网络,来实现人力资源数据预测.与BP算法相比,数值计算结果表明预测精度高、速度快,为地区人力资源数据的时间序列预测研究提供了一条新的途径.     

A neural fuzzy control system with structure and parameter learning

A general connectionist model, called neural fuzzy control network (NFCN), is proposed for the realization of a fuzzy logic control system. The proposed NFCN is a feedforward multilayered network which integrates the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure which has distributed learning abilities. The NFCN can be constructed from supervised training examples by machine learning techniques, and the connectionist structure can be trained to develop fuzzy logic rules and find membership functions. Associated with the NFCN is a two-phase hybrid learning algorithm which utilizes unsupervised learning schemes for structure learning and the backpropagation learning scheme for parameter learning. By combining both unsupervised and supervised learning schemes, the learning speed converges much faster than the original backpropagation algorithm. The two-phase hybrid learning algorithm requires exact supervised training data for learning. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a reinforcement neural fuzzy control network (RNFCN) is further proposed. The RNFCN is constructed by integrating two NFCNs, one functioning as a fuzzy predictor and the other as a fuzzy controller. By combining a proposed on-line supervised structure-parameter learning technique, the temporal difference prediction method, and the stochastic exploratory algorithm, a reinforcement learning algorithm is proposed, which can construct a RNFCN automatically and dynamically through a reward-penalty signal (i.e., “good” or “bad” signal). Two examples are presented to illustrate the performance and applicability of the proposed models and learning algorithms.     

A memetic algorithm based on multiple learning procedures for global optimal design of composite structures

The mechanical properties of composite structures are sensitive to the choice of the material systems, the distribution of laminates on the structure, and the sizing at the laminate level. This sensitivity depends on many variables. To overcome the difficulties in attaining a global solution for the optimisation problem of composite structures, a new strategy is proposed based on the hybridisation of the memetic algorithm (MA) and the selfish gene (SG) algorithm. Instead of a local search, as performed in MAs, the selfish gene (SG) theory is applied, which follows a different learning scheme in which the conventional population of the individuals is replaced by a virtual population of alleles. The proposed approach, which is called the memetic-based selfish gene algorithm (MA + SG), is a mixed model that applies multiple learning procedures to explore the synergy of different cultural transmission rules in the evolutionary process. The principal aspects of the approach are as follows: co-evolution of multiple populations, species conservation, migration rules, self-adaptive multiple crossovers, local search in hybrid crossover with local genetic improvements, controlled mutation, individual age control, and feature-based allele statistical analysis. To discuss the capabilities of the proposed approach, numerical examples are represented to compare the results of MA + SG with those obtained using genetic algorithms (GA) and MA. The numerical results of the comparison tests showed that the GA and the MA maintained long periods without the evolution of the best-fitted individual/solutions. This behaviour during evolution is associated with the slow maturation of the elite group of populations in the GA and MA approaches. This behaviour is avoided when the MA + SG proposed approach is used with computational cost benefits.     

Optimal selection of retailers for a manufacturing vendor in a vendor managed inventory system

A Vendor Managed Inventory (VMI) system consists of a manufacturing vendor and a number of retailers. In such a system, it is essential for the vendor to optimally determine retailer selection and other related decisions, such as the product’s replenishment cycle time and the wholesale price, in order to maximize his profit. Meanwhile, each retailer’s decisions on her willingness to enter the system and retail price are simultaneously considered in the retailer selection process. However, the above interactive decision making is complex and the available studies on interactive retailer selection are scarce. In this study, we formulate the retailer selection problem as a Stackelberg game model to help the manufacturer, as a vendor, optimally select his retailers to form a VMI system. This model is non-linear, mixed-integer, game-theoretic, and analytically intractable. Therefore, we further develop a hybrid algorithm for effectively and efficiently solving the developed model. The hybrid algorithm combines dynamic programming (DP), genetic algorithm (GA) and analytical methods. As demonstrated by our numerical studies, the optimal retailer selection can increase the manufacturer’s profit by up to 90% and the selected retailers’ profits significantly compared to non-selection strategy. The proposed hybrid algorithm can solve the model within a minute for a problem with 100 candidate retailers, whereas a pure GA has to take more than 1 h to solve a small sized problem of 20 candidate retailers achieving an objective value no worse than that obtained by the hybrid algorithm.     

遗传算法结合神经网络在油气产量预测中的应用

基于遗传算法的全局搜索能力和BP算法的局部精确搜索特性,通过采用遗传算法优化神经网络的方法,将遗传算法和BP算法有机结合,做到优势互补,在提高油气产量预测精度的研究中得到了很好的应用.在对国内某中小型气田油气产量的预测中,以历史产量资料进行检验,其结果表明,提出的预测方法,预测精度明显优于BP算法,证明了这种方法的有效性和可靠性.     

变尺度快速二阶非线性优化算法在神经网络辨识中的应用

神经网络是非线性系统建模与辨识的重要方法 ,反向传播 (BP)算法常常用在神经网络的权值训练中 ,但是 BP算法的收敛速度慢 .本文提出一种变尺度二阶快速优化方法 ,在这种方法中用二阶插值法来优化搜索学习速率 ,然后将这一方法应用于神经网络的辨识中 ,仿真研究表明新算法有更快的收敛速度和更好的收敛精度 .     

Machine-part cell formation in group technology using a modified ART1 method

Group Technology (GT) is a useful way of increasing the productivity for manufacturing high quality products and improving the flexibility of manufacturing systems. Cell formation (CF) is a key step in GT. It is used in designing good cellular manufacturing systems using the similarities between parts in relation to the machines in their manufacture. It can identify part families and machine groups. Recently, neural networks (NNs) have been widely applied in GT due to their robust and adaptive nature. NNs are very suitable in CF with a wide variety of real applications. Although Dagli and Huggahalli adopted the ART1 network with an application in machine-part CF, there are still several drawbacks to this approach. To address these concerns, we propose a modified ART1 neural learning algorithm. In our modified ART1, the vigilance parameter can be simply estimated by the data so that it is more efficient and reliable than Dagli and Huggahalli’s method for selecting a vigilance value. We then apply the proposed algorithm to machine-part CF in GT. Several examples are presented to illustrate its efficiency. In comparison with Dagli and Huggahalli’s method based on the performance measure proposed by Chandrasekaran and Rajagopalan, our modified ART1 neural learning algorithm provides better results. Overall, the proposed algorithm is vigilance parameter-free and very efficient to use in CF with a wide variety of machine/part matrices.     

Theorem of learning for a competition algorithm

This paper is an extension of [1], where a new decisive algorithm was proposed. In its operation, the unit resembles artificial neural networks. However, the functioning of the algorithm proposed is based on different concepts. It does not use the concept of a net or a neuron. The theorem of learning for the new competition algorithm is proved.     

A hybrid evolutionary learning algorithm for TSK-type fuzzy model design

In this paper, a TSK-type fuzzy model (TFM) with a hybrid evolutionary learning algorithm (HELA) is proposed. The proposed HELA method combines the compact genetic algorithm (CGA) and the modified variable-length genetic algorithm (MVGA). Both the number of fuzzy rules and the adjustable parameters in the TFM are designed concurrently by the HELA method. In the proposed HELA method, individuals of the same length constitute the same group, and there are multiple groups in a population. Moreover, the proposed HELA adopts the compact genetic algorithm (CGA) to carry out the elite-based reproduction strategy. The CGA represents a population as a probability distribution over the set of solutions and is operationally equivalent to the order-one behavior of the simple GA. The evolution processes of a population consist of three major operations: group reproduction using the compact genetic algorithm, variable two-part individual crossover, and variable two-part mutation. Computer simulations have demonstrated that the proposed HELA method gives a better performance than some existing methods.     

Autonomous vehicle navigation using evolutionary reinforcement learning

Reinforcement learning schemes perform direct on-line search in control space. This makes them appropriate for modifying control rules to obtain improvements in the performance of a system. The effectiveness of a reinforcement learning strategy is studied here through the training of a learning classifier system (LCS) that controls the movement of an autonomous vehicle in simulated paths including left and right turns. The LCS comprises a set of condition-action rules (classifiers) that compete to control the system and evolve by means of a genetic algorithm (GA). Evolution and operation of classifiers depend upon an appropriate credit assignment mechanism based on reinforcement learning. Different design options and the role of various parameters have been investigated experimentally. The performance of vehicle movement under the proposed evolutionary approach is superior compared with that of other (neural) approaches based on reinforcement learning that have been applied previously to the same benchmark problem.     

Image compression with a dynamic autoassociative neural network

Image compression using neural networks has been attempted with some promise. Among the architectures, feedforward backpropagation networks (FFBPN) have been used in several attempts. Although it is demonstrated that using the mean quadratic error function is equivalent to applying the Karhunen-Loeve transformation method, promise still arises from directed learning possibilities, generalization abilities and performance of the network once trained. In this paper we propose an architecture and an improved training method to attempt to solve some of the shortcomings of traditional data compression systems based on feedforward neural networks trained with backpropagation—the dynamic autoassociation neural network (DANN).The successful application of neural networks to any task requires proper training of the network. In this research, this issue is taken as the main consideration in the design of DANN. We emphasize the convergence of the learning process proposed by DANN. This process provides an escape mechanism, by adding neurons in a random state, to avoid the local minima trapping seen in traditional PFBPN. Also, DANN's training algorithm constrains the error for every pattern to an allowed interval to balance the training for every pattern, thus improving the performance rates in recognition and generalization. The addition of these two mechanisms to DANN's training algorithm has the result of improving the final quality of the images processed by DANN.The results of several tasks presented to DANN-based compression are compared and contrasted with the performance of an FFBPN-based system applied to the same task. These results indicate that DANN is superior to FFBPN when applied to image compression.     

New stability criteria for neural networks with time-varying delays

This paper is concerned with the stability analysis problem of neural networks with time delays. The delay intervals [−d(t), 0] and [−h, 0] are divided into m subintervals with equal length. Some free matrices are introduced to build the relationship among the elements of the resultant matrix inequalities. With the above operations, the new stability criteria are built for the general class of neural networks. The conditions are presented in the form of linear matrix inequalities (LMIs), which can be solved by the numerically efficient Matlab LMI toolbox. Several examples are provided to show that our methods are much less conservative than recently reported ones.     

Dynamical analysis of fuzzy BAM neural networks with variable delays

In this work, employing Lyapunov functional and elemental inequality ($2ab \leqslant ra^2 + \tfrac{1} {r}b^2$2ab \leqslant ra^2 + \tfrac{1} {r}b^2, r > 0), some sufficient conditions are derived for the existence and uniqueness of periodic solution of fuzzy bidirectional associated memory (BAM) neural networks with variable delays. Some new and simple criteria are obtained to ensure global exponential stability of periodic solution, which are important in design and applications of fuzzy BAM neural networks.     

Multi-criteria classification – A new scheme for application of dominance-based decision rules

We are considering the problem of multi-criteria classification. In this problem, a set of “if … then …” decision rules is used as a preference model to classify objects evaluated by a set of criteria and regular attributes. Given a sample of classification examples, called learning data set, the rules are induced from dominance-based rough approximations of preference-ordered decision classes, according to the Variable Consistency Dominance-based Rough Set Approach (VC-DRSA). The main question to be answered in this paper is how to classify an object using decision rules in situation where it is covered by (i) no rule, (ii) exactly one rule, (iii) several rules. The proposed classification scheme can be applied to both, learning data set (to restore the classification known from examples) and testing data set (to predict classification of new objects). A hypothetical example from the area of telecommunications is used for illustration of the proposed classification method and for a comparison with some previous proposals.     

Simulation and evaluation of dual fully fuzzy linear systems by fuzzy neural network

In this paper, a novel hybrid method based on fuzzy neural network for approximate solution of fuzzy linear systems of the form Ax = Bx + d, where A and B are two square matrices of fuzzy coefficients, x and d are two fuzzy number vectors, is presented. Here a neural network is considered as a part of a large field called neural computing or soft computing. Moreover, in order to find the approximate solution, a simple and fast algorithm from the cost function of the fuzzy neural network is proposed. Finally, we illustrate our approach by some numerical examples.