变尺度快速二阶非线性优化算法在神经网络辨识中的应用

神经网络是非线性系统建模与辨识的重要方法 ,反向传播 (BP)算法常常用在神经网络的权值训练中 ,但是 BP算法的收敛速度慢 .本文提出一种变尺度二阶快速优化方法 ,在这种方法中用二阶插值法来优化搜索学习速率 ,然后将这一方法应用于神经网络的辨识中 ,仿真研究表明新算法有更快的收敛速度和更好的收敛精度 .     

Emergence of order from chaos by cooperative non-biased learning between random bit-strings

Numerical experiments show that non-biased learning between families of independent and random bit-strings causes order. A parallel distributed learning between these bit-strings is performed by a neural network of the type pattern associator. The system allows emergence of some order in the learning matrix when a non-linear process is used, while a linear learning is unable to break the learning-matrix randomness. This neural network is in fact a complex and non-linear dynamical system, and consequently is able to self-organize order from chaos. It is also a model of collective proto-cognition that would describe biological evolution in species by cooperative learning. Our model suggests that the cause of evolution towards order in complex systems, can be just the learning process.     

Universality of deep convolutional neural networks

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. Deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation or generalization ability of deep learning methods generated by the network architectures such as deep convolutional neural networks. Here we show that a deep convolutional neural network (CNN) is universal, meaning that it can be used to approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough. This answers an open question in learning theory. Our quantitative estimate, given tightly in terms of the number of free parameters to be computed, verifies the efficiency of deep CNNs in dealing with large dimensional data. Our study also demonstrates the role of convolutions in deep CNNs.     

Stability Analysis of Gradient-Based Neural Networks for Optimization Problems

The paper introduces a new approach to analyze the stability of neural network models without using any Lyapunov function. With the new approach, we investigate the stability properties of the general gradient-based neural network model for optimization problems. Our discussion includes both isolated equilibrium points and connected equilibrium sets which could be unbounded. For a general optimization problem, if the objective function is bounded below and its gradient is Lipschitz continuous, we prove that (a) any trajectory of the gradient-based neural network converges to an equilibrium point, and (b) the Lyapunov stability is equivalent to the asymptotical stability in the gradient-based neural networks. For a convex optimization problem, under the same assumptions, we show that any trajectory of gradient-based neural networks will converge to an asymptotically stable equilibrium point of the neural networks. For a general nonlinear objective function, we propose a refined gradient-based neural network, whose trajectory with any arbitrary initial point will converge to an equilibrium point, which satisfies the second order necessary optimality conditions for optimization problems. Promising simulation results of a refined gradient-based neural network on some problems are also reported.     

Neural network as a simulation metamodel in economic analysis of risky projects

An artificial neural network (ANN) model for economic analysis of risky projects is presented in this paper. Outputs of conventional simulation models are used as neural network training inputs. The neural network model is then used to predict the potential returns from an investment project having stochastic parameters. The nondeterministic aspects of the project include the initial investment, the magnitude of the rate of return, and the investment period. Backpropagation method is used in the neural network modeling. Sigmoid and hyperbolic tangent functions are used in the learning aspect of the system. Analysis of the outputs of the neural network model indicates that more predictive capability can be achieved by coupling conventional simulation with neural network approaches. The trained network was able to predict simulation output based on the input values with very good accuracy for conditions not in its training set. This allowed an analysis of the future performance of the investment project without having to run additional expensive and time-consuming simulation experiments.     

Multicriteria order acceptance decision support in over-demanded job shops: A neural network approach

Order acceptance is an important issue in job shop production systems where demand exceeds capacity. In this paper, a neural network approach is developed for order acceptance decision support in job shops with machine and manpower capacity constraints. First, the order acceptance decision problem is formulated as a sequential multiple criteria decision problem. Then a neural network based preference model for order prioritization is described. The neural network based preference model is trained using preferential data derived from pairwise comparisons of a number of representative orders. An order acceptance decision rule based on the preference model is proposed. Finally, a numerical example is discussed to illustrate the use of the proposed neural network approach. The proposed neural network approach is shown to be a viable method for multicriteria order acceptance decision support in over-demanded job shops.     

Tidal level forecasting using functional and sequential learning neural networks

Prediction of tides is very much essential for human activities and to reduce the construction cost in marine environment. This paper presents two methods (1) an application of the functional networks (FN) and (2) sequential learning neural network (SLNN) procedures for the accurate prediction of tides using very short-term observation. This functional network model predicts the time series data of hourly tides directly while using an efficient learning process by minimizing the error based on the observed data for 30 days. Using the functional network, a very simple equation in the form of finite difference equation using the tidal levels at two previous time steps is arrived at. Sequential learning neural network uses one hidden neuron to predict the current tidal level using the previous four levels quite accurately. Hourly tidal data measured at Taichung harbor and Mirtuor coast along the Taiwan coastal region have been used for testing the functional network and sequential neural network model. Results show that the hourly data on tides for even a month can be predicted efficiently with a very high correlation coefficient.     

基于粒子群的模糊神经网络的短时交通流量组合预测

为了进一步提高短时交通流量预测的精度,提出了一种粒子群算法的模糊神经网络组合预测模型,模糊神经网络融合了神经网络的学习机制和模糊系统的语言推理能力等优点,弥补各自不足,将自回归求和滑动平均(ARIMA)和灰色Verhulst模型进行初步预测,并将两种初步预测的结果作为模糊神经网络的输入,构建基于改进模神经网络的组合预测模型,在此基础上进行训练和预测,其中模糊神经网络的相关参数由改进粒子群来优化,利用本方法来对南京市汉中路短时交通流量进行预测,结论表明:方法充分发挥了单一模型的优势,比单一的预测模型更加精确,是短时交通流量预测的一个有效方法。     

基于遗传算法和半参数回归的神经网络集成股市预测研究

股票时间序列预测在经济和管理领域具有重要的应用前景,也是很多商业和金融机构成功的基础.首先利用奇异谱分析对股市时间序列重构,降低噪声并提取趋势序列.再利用C-C算法确定股市时间序列的嵌入维数和延迟阶数,对股市时间序列进行相空间重构,生成神经网络的学习矩阵.进一步利用Boosting技术和不同的神经网络模型,生成神经网络集成个体.最后采用带有惩罚项的半参数回归模型进行集成,并利用遗传算法选择最优的光滑参数,以此建立遗传算法和半参数回归的神经网络集成股市预测模型.通过上证指数开盘价进行实例分析,与传统的时间序列分析和其他集成方法对比,发现该方法能获得更准确的预测结果.计算结果表明该方法能充分反映股票价格时间序列趋势,为金融时间序列预测提供一个有效方法.     

Fuzzy neural network modeling for product development

A fairly general product development model is formulated and analyzed based on multiple attribute decision making with emphasis on the treatment of the linguistic and vague aspects by fuzzy logic and up-dating or learning by neural network. Due to the representative ability of fuzzy set theory and the learning or intelligent ability of neural network, the proposed approaches appear to be an effective tool for handling vague and not well-defined systems.     

二阶Hopfield神经网络周期解的存在性

本文讨论了一类带有时滞的二阶Hopfield神经网络周期解的存在性问题。首先利用Brouwer不动点定理证明了平衡点的存在性,通过平衡点和拉格朗日中值定理,将高阶神经网络模型转换为一阶模型,然后利用重合度理论给出了周期解存在的充分条件。     

Micro-level reserving for general insurance claims using a long short-term memory network

Detailed information about individual claims are completely ignored when insurance claims data are aggregated and structured in development triangles for loss reserving. In the hope of extracting predictive power from the individual claims characteristics, researchers have recently proposed to use micro-level loss reserving approaches. We introduce a discrete-time individual reserving framework incorporating granular information in a deep learning approach named Long Short-Term Memory (LSTM) neural network. At each time period, the network has two tasks: first, classifying whether there is a payment or a recovery, and second, predicting the corresponding non-zero amount, if any. Based on a generalized Pareto model for excess payments over a threshold, we adjust the LSTM reserve prediction to account for extreme payments. We illustrate the estimation procedure on a simulated and a real general insurance dataset. We compare our approach with the chain-ladder aggregate method using the predictive outstanding loss estimates and their actual values.     

基于多分辨率分析的股票市场预测模型研究

采用多分辨率分析技术将深证成指收盘数据序列分解为多个子序列,然后采用神经网络技术对每个子序列分别建立预测模型,将各个预测结果叠加后得到最终预测结果.研究首先发现多分辨率技术可以有效提高预测模型的预测精度,表明分析我国股市波动时应该按照不同因素对股市影响大小及周期的差异分别研究,才能更有效分析股市运行状况及对其预测;其次结果表明不同类型神经网络预测模型预测性能差异明显,在选择股市预测模型的神经网络类型时应该注意其学习算法及收敛过程,以便能更好捕获股市变化规律.     

Identification of nonlinear system using extreme learning machine based Hammerstein model

In this paper, a new method for nonlinear system identification via extreme learning machine neural network based Hammerstein model (ELM-Hammerstein) is proposed. The ELM-Hammerstein model consists of static ELM neural network followed by a linear dynamic subsystem. The identification of nonlinear system is achieved by determining the structure of ELM-Hammerstein model and estimating its parameters. Lipschitz quotient criterion is adopted to determine the structure of ELM-Hammerstein model from input–output data. A generalized ELM algorithm is proposed to estimate the parameters of ELM-Hammerstein model, where the parameters of linear dynamic part and the output weights of ELM neural network are estimated simultaneously. The proposed method can obtain more accurate identification results with less computation complexity. Three simulation examples demonstrate its effectiveness.     

A columnar competitive model for solving multi-traveling salesman problem

This paper studies an optimization problem: multi-traveling salesman problem (MTSP), which is an extension of the well known TSP. A columnar competitive model (CCM) of neural networks incorporates with a winner-take-all learning rule is employed to solve the MTSP. Stability conditions of CCM for MTSP is exploited by mathematical analysis. Parameters settings of the network for guaranteeing the network converges to valid solutions are discussed in detail. Simulations are carried out to illustrate the performance of the columnar competitive model compare to the heuristic algorithms: Tabu Search.     

The optimal combination: Grammatical swarm,particle swarm optimization and neural networks

Social behaviour is mainly based on swarm colonies, in which each individual shares its knowledge about the environment with other individuals to get optimal solutions. Such co-operative model differs from competitive models in the way that individuals die and are born by combining information of alive ones. This paper presents the particle swarm optimization with differential evolution algorithm in order to train a neural network instead the classic back propagation algorithm. The performance of a neural network for particular problems is critically dependant on the choice of the processing elements, the net architecture and the learning algorithm. This work is focused in the development of methods for the evolutionary design of artificial neural networks. This paper focuses in optimizing the topology and structure of connectivity for these networks.     

Evaluación de la habitabilidad de edificios afectados por sismo utilizando la teoría de conjuntos difusos y las redes neuronales artificiales

The fuzzy sets theory and the artificial neural networks are computational intelligence tools which are nowadays widely used in earthquake engineering. This paper develops a method and a computer program which use these computational intelligence tools in order to support the damage and safety evaluation of buildings after strong earthquakes. The model uses an artificial neural network with three layers and a Kohonen learning algorithm; it also uses fuzzy sets in order to manage subjective information such as linguistic qualification of the damage levels in buildings and a fuzzy rule base to support the decision making process. All these techniques are incorporated in the developed computer program. The input data is the subjective and incomplete information about the building state obtained by possibly non experienced evaluators in the field of the seismic performance of buildings. The proposed method is implemented in a tool especially useful in the emergency response phase, when it supports the decision making regarding the building habitability and reparability. In order to show its effectiveness, two examples are included for two different types of buildings.     

A Neural Network Application in Personnel Scheduling

In this paper, we present and evaluate a neural network model for solving a typical personnel-scheduling problem, i.e. an airport ground staff rostering problem. Personnel scheduling problems are widely found in servicing and manufacturing industries. The inherent complexity of personnel scheduling problems has normally resulted in the development of integer programming-based models and various heuristic solution procedures. The neural network approach has been admitted as a promising alternative to solving a variety of combinatorial optimization problems. While few works relate neural network to applications of personnel scheduling problems, there is great theoretical and practical value in exploring the potential of this area. In this paper, we introduce a neural network model following a relatively new modeling approach to solve a real rostering case. We show how to convert a mixed integer programming formulation to a neural network model. We also provide the experiment results comparing the neural network method with three popular heuristics, i.e. simulated annealing, Tabu search and genetic algorithm. The computational study reveals some potential of neural networks in solving personnel scheduling problems.     

人工神经网络BP算法的改进和结构的自调整

本文解决了BP神经网络结构参数和学习速率的选取问题,并对传统的BP算法进行了改进,提出了BP神经网络动态全参数自调整学习算法,又将其编制成计算机程序,使得隐层节点和学习速率的选取全部动态实现,减少了人为因素的干预,改善了学习速率和网络的适应能力。计算结果表明：BP神经网络动态全参数自调整算法较传统的方法优越。训练后的神经网络模型不仅能准确地拟合训练值,而且能较精确地预测未来趋势。     

A CS-AdaBoost-BP model for product quality inspection

Quality inspection is essential in preventing defective products from entering the market. Due to the typically low percentage of defective products, it is generally challenging to detect them using algorithms that aim for the overall classification accuracy. To help solve this problem, we propose an ensemble learning classification model, where we employ adaptive boosting (AdaBoost) to cascade multiple backpropagation (BP) neural networks. Furthermore, cost-sensitive (CS) learning is introduced to adjust the loss function of the basic classifier of the BP neural network. For clarity, this model is called a CS-AdaBoost-BP model. To empirically verify its effectiveness, we use data from home appliance production lines from Bosch. We carry out tenfold cross-validation to evaluate and compare the performance between the CS-AdaBoost-BP model and three existing models: BP neural network, BP neural network based on sampling, and AdaBoost-BP. The results show that our proposed model not only performs better than the other models but also significantly improves the ability to identify defective products. Furthermore, based on the mean value of the Youden index, our proposed model has the highest stability.