分类问题的在线顺序双并联快速学习机

双并联前馈神经网络模型是单层感知机和单隐层前馈神经网络的混合结构,本文构造了一种双并联快速学习机算法,与其他类似算法比较,提出的算法能利用较少的隐层单元及更少的待定参数,获得近似的学习性能.数值实验表明,对很多实际分类问题,提出的算法具备更佳的泛化能力,因而可以作为快速学习机算法的有益补充.     

基于组合神经网络的时间分数阶扩散方程计算方法

该文首次采用一种组合神经网络的方法，求解了一维时间分数阶扩散方程.组合神经网络是由径向基函数（RBF）神经网络与幂激励前向神经网络相结合所构造出的一种新型网络结构.首先，利用该网络结构构造出符合时间分数阶扩散方程条件的数值求解格式，同时设置误差函数，使原问题转化为求解误差函数极小值问题；然后，结合神经网络模型中的梯度下降学习算法进行循环迭代，从而获得神经网络的最优权值以及各项最优参数，最终得到问题的数值解.数值算例验证了该方法的可行性、有效性和数值精度.该文工作为时间分数阶扩散方程的求解开辟了一条新的途径.     

Lower bound for connectivity in local-learning neural networks

How does the connectivity of a neural network (number of synapses per neuron) relate to the complexity of the problems it can handle? Switching theory would suggest no relation at all, since all Boolean functions can be implemented using a circuit with very low connectivity (e.g., using two-input NAND gates). However, for a network that learns a problem from examples using a local learning rule, we prove that the entropy of the problem becomes a lower bound for the connectivity of the network. The current result generalizes a previous result by removing a restriction on the features that are loaded into the neurons during the learning phase.     

Response surface methodology: A neural network approach

Response surface methodology is used to optimize the parameters of a process when the function that describes it is unknown. The procedure involves fitting a function to the given data and then using optimization techniques to obtain the optimal parameters. This procedure is usually difficult due to the fact that obtaining the right model may not be possible or at best very time consuming.In this paper, a two-stage procedure for obtaining the best parameters for a process with an unknown model is developed. The procedure is based on implementing response surface methodology via neural networks. Two neural networks are trained: one for the unknown function and the other for derivatives of this function which are computed using the first neural network. These neural networks are then used iteratively to compute parameters for an equation which is ultimately used for optimizing the function. Results of some simulation studies are also presented.     

A Performance evaluation of neural network models in traffic volume forecasting

Forecasting traffic volume is an important task in controlling urban highways, guiding drivers' routes, and providing real-time transportation information. Previous research on traffic volume forecasting has concentrated on a single forecasting model and has reported positive results, which has been frequently better than those of other models. In addition, many previous researchers have claimed that neural network models are better than linear statistical models in terms of prediction accuracy. However, the forecasting power of a single model is limited to the typical cases to which the model fits best. In other words, even though many research efforts have claimed the general superiority of a single model over others in predicting future events, we believe it depends on the data characteristics used, the composition of the training data, the model architecture, and the algorithm itself.In this paper, we have studied the relationship in forecasting traffic volume between data characteristics and the forecasting accuracy of different models, particularly neural network models. To compare and test the forecasting accuracy of the models, three different data sets of traffic volume were collected from interstate highways, intercity highways, and urban intersections. The data sets show very different characteristics in terms of volatility, period, and fluctuations as measured by the Hurst exponent, the correlation dimension. The data sets were tested using a back-propagation network model, a FIR model, and a time-delayed recurrent model.The test results show that the time-delayed recurrent model outperforms other models in forecasting very randomly moving data described by a low Hurst exponent. In contrast, the FIR model shows better forecasting accuracy than the time-delayed recurrent network for relatively regular periodic data described by a high Hurst exponent. The interpretation of these results shows that the feedback mechanism of the previous error, through the temporal learning technique in the time-delayed recurrent network, naturally absorbs the dynamic change of any underlying nonlinear movement. The FIR and back-propagation model, which have claimed a nonlinear learning mechanism, may not be very good in handling randomly fluctuating events.     

Simulation of the effect of learning on the topology of the functional connectivity of neural networks

We introduce a procedure for simulating adaptive learning in neural networks and the effect this learning has on the way in which the functional connections between the nodes of the network are established. The procedure combines two mechanisms: firstly, the gradual dilution of the network through the elimination of synaptic weights in increasing order of magnitude, thus reducing the costs of the network structure. Secondly, to train the network as it is diluted so as not to compromise its performance pursuant to the proposed task. Considering different levels of learning difficulty, we compare the topology of the functional connectivities that result from the application of this procedure with those obtained using fMRI in healthy volunteers. According to our results, the topology of functional connectivities in healthy subjects can be interpreted as the product of a learning process with a specific degree of difficulty.     

BP神经网络在农机总动力预测中的应用

准确的预测黑龙江省农机总动力,可为黑龙江省的农业机械化发展趋势和农机产品市场分析提供理论指导,为制定黑龙江省农业机械化发展规划和预测近阶段农业机械化发展水平提供参考依据.利用黑龙江省1980-2007年农机总动力数据,运用标准BP神经网络和改进BP神经网络模型进行预测,预测结果表明,改进BP神经网络模型比标准BP神经网络模型在预测精度、运行时间、学习次数等方面更具优越性.     

Bayesian semiparametric Markov switching stochastic volatility model

This paper proposes a novel Bayesian semiparametric stochastic volatility model with Markov switching regimes for modeling the dynamics of the financial returns. The distribution of the error term of the returns is modeled as an infinite mixture of Normals; meanwhile, the intercept of the volatility equation is allowed to switch between two regimes. The proposed model is estimated using a novel sequential Monte Carlo method called particle learning that is especially well suited for state‐space models. The model is tested on simulated data and, using real financial times series, compared to a model without the Markov switching regimes. The results show that including a Markov switching specification provides higher predictive power for the entire distribution, as well as in the tails of the distribution. Finally, the estimate of the persistence parameter decreases significantly, a finding consistent with previous empirical studies.     

The Generalized-Trend-Diffusion modeling algorithm for small data sets in the early stages of manufacturing systems

The statistical theories are not expected to generate significant conclusions, when applied to very small data sets. Knowledge derived from limited data gathered in the early stages is considered too fragile for long term production decisions. Unfortunately, this work is necessary in the competitive industry and business environments. Our previous researches have been aimed at learning from small data sets for scheduling flexible manufacturing systems, and this article will focus development of an incremental learning procedure for small sequential data sets. The main consideration concentrates on two properties of data: that the data size is very small and the data are time-dependent. For this reason, we propose an extended algorithm named the Generalized-Trend-Diffusion (GTD) method, based on fuzzy theories, developing a unique backward tracking process for exploring predictive information through the strategy of shadow data generation. The extra information extracted from the shadow data has proven useful in accelerating the learning task and dynamically correcting the derived knowledge in a concurrent fashion.     

Learning management knowledge for manufacturing systems in the early stages using time series data

Acquiring knowledge in manufacturing systems in the early stages always has a challenging task due to the lack of sufficient data. This makes it hard for the derived management model to reach a reliable and stable level. Li and Lin (2006) developed a useful method that can deal with the problem of knowledge acquisition based on a small data set. However, this method assumes all data are collected at the same time, since they treat the data set as a source (from one population) of a priori knowledge for learning. In fact, instead of being a random data set, these collected data can be time dependent, that is, they tend to be a sequence of observations, occurring at different times. The consideration of this dependence property in the data will benefit the knowledge acquisition in the early stages by expanding the learning model from an independent model to a dependent model. This research expanded the intervalized kernel density estimator (IKDE) presented in Li and Lin (2006) to a more general form to improve the learning model in the early stages. The general model, called GIKDE, joints the concepts of time series and stochastic processes in order to deal with both independent and dependent data sets. The Virtual Sample Generation process based on GIKDE was also developed to produce extra information for expediting the learning. Results obtained from the application of the model to a control charts data, using a back-propagation neural network as the learning tool, show that this unique approach is an effective method of knowledge acquisition for a manufacturing system in the early stages.     

Artificial Bee Colony training of neural networks: comparison with back-propagation

The Artificial Bee Colony (ABC) is a swarm intelligence algorithm for optimization that has previously been applied to the training of neural networks. This paper examines more carefully the performance of the ABC algorithm for optimizing the connection weights of feed-forward neural networks for classification tasks, and presents a more rigorous comparison with the traditional Back-Propagation (BP) training algorithm. The empirical results for benchmark problems demonstrate that using the standard “stopping early” approach with optimized learning parameters leads to improved BP performance over the previous comparative study, and that a simple variation of the ABC approach provides improved ABC performance too. With both improvements applied, the ABC approach does perform very well on small problems, but the generalization performances achieved are only significantly better than standard BP on one out of six datasets, and the training times increase rapidly as the size of the problem grows. If different, evolutionary optimized, BP learning rates are allowed for the two layers of the neural network, BP is significantly better than the ABC on two of the six datasets, and not significantly different on the other four.     

基于深度学习LSTM对交通流状态的预测

当今道路交通状态对城市管理和人们出行愈加重要,影响着人类生活的方方面面.以深圳交通为研究对象,由基础车辆数据和道路坐标构建了路网系统,从车辆速度和密度两个方面导出了交通流状态评价指数TSI.利用深度学习长短期记忆神经网络(LSTM)对车辆速度和密度两个指标进行预测,并通过对比极限学习机(ELM),时间序列(ARMA)和BP神经网络,进行仿真实验,结果表明相对于传统预测模型,所采用的LSTM网络具有更优的预测精确度和对远期预测的稳定性.最后利用预测结果计算出更能直观反映出道路交通拥堵情况的TSI指数,为人们提供了准确的交通状态预测.     

基于遗传算法和半参数回归的神经网络集成股市预测研究

股票时间序列预测在经济和管理领域具有重要的应用前景,也是很多商业和金融机构成功的基础.首先利用奇异谱分析对股市时间序列重构,降低噪声并提取趋势序列.再利用C-C算法确定股市时间序列的嵌入维数和延迟阶数,对股市时间序列进行相空间重构,生成神经网络的学习矩阵.进一步利用Boosting技术和不同的神经网络模型,生成神经网络集成个体.最后采用带有惩罚项的半参数回归模型进行集成,并利用遗传算法选择最优的光滑参数,以此建立遗传算法和半参数回归的神经网络集成股市预测模型.通过上证指数开盘价进行实例分析,与传统的时间序列分析和其他集成方法对比,发现该方法能获得更准确的预测结果.计算结果表明该方法能充分反映股票价格时间序列趋势,为金融时间序列预测提供一个有效方法.     

AN ADAPTIVE LEARNING FRAMEWORK FOR FORECASTING SEASONAL WATER ALLOCATIONS IN IRRIGATED CATCHMENTS

Abstract This paper describes an adaptive learning framework for forecasting end‐season water allocations using climate forecasts, historic allocation data, and results of other detailed hydrological models. The adaptive learning framework is based on artificial neural network (ANN) method, which can be trained using past data to predict future water allocations. Using this technique, it was possible to develop forecast models for end‐irrigation‐season water allocations from allocation data available from 1891 to 2005 based on the allocation level at the start of the irrigation season. The model forecasting skill was further improved by the incorporation of a set of correlating clusters of sea surface temperature (SST) and the Southern oscillation index (SOI) data. A key feature of the model is to include a risk factor for the end‐season water allocations based on the start of the season water allocation. The interactive ANN model works in a risk‐management context by providing probability of availability of water for allocation for the prediction month using historic data and/or with the incorporation of SST/SOI information from the previous months. All four developed ANN models (historic data only, SST incorporated, SOI incorporated, SST‐SOI incorporated) demonstrated ANN capability of forecasting end‐of‐season water allocation provided sufficient data on historic allocation are available. SOI incorporated ANN model was the most promising forecasting tool that showed good performance during the field testing of the model.     

Multicriteria order acceptance decision support in over-demanded job shops: A neural network approach

Order acceptance is an important issue in job shop production systems where demand exceeds capacity. In this paper, a neural network approach is developed for order acceptance decision support in job shops with machine and manpower capacity constraints. First, the order acceptance decision problem is formulated as a sequential multiple criteria decision problem. Then a neural network based preference model for order prioritization is described. The neural network based preference model is trained using preferential data derived from pairwise comparisons of a number of representative orders. An order acceptance decision rule based on the preference model is proposed. Finally, a numerical example is discussed to illustrate the use of the proposed neural network approach. The proposed neural network approach is shown to be a viable method for multicriteria order acceptance decision support in over-demanded job shops.     

Extended and Unscented Kalman filtering based feedforward neural networks for time series prediction

With the ability to deal with high non-linearity, artificial neural networks (ANNs) and support vector machines (SVMs) have been widely studied and successfully applied to time series prediction. However, good fitting results of ANNs and SVMs to nonlinear models do not guarantee an equally good prediction performance. One main reason is that their dynamics and properties are changing with time, and another key problem is the inherent noise of the fitting data. Nonlinear filtering methods have some advantages such as handling additive noises and following the movement of a system when the underlying model is evolving through time. The present paper investigates time series prediction algorithms by using a combination of nonlinear filtering approaches and the feedforward neural network (FNN). The nonlinear filtering model is established by using the FNN’s weights to present state equation and the FNN’s output to present the observation equation, and the input vector to the FNN is composed of the predicted signal with given length, then the extended Kalman filtering (EKF) and Unscented Kalman filtering (UKF) are used to online train the FNN. Time series prediction results are presented by the predicted observation value of nonlinear filtering approaches. To evaluate the proposed methods, the developed techniques are applied to the predictions of one simulated Mackey-Glass chaotic time series and one real monthly mean water levels time series. Generally, the prediction accuracy of the UKF-based FNN is better than the EKF-based FNN when the model is highly nonlinear. However, comparing from prediction accuracy and computational effort based on the prediction model proposed in our study, we draw the conclusion that the EKF-based FNN is superior to the UKF-based FNN for the theoretical Mackey-Glass time series prediction and the real monthly mean water levels time series prediction.     

A comparison of nonlinear optimization methods for supervised learning in multilayer feedforward neural networks

One impediment to the use of neural networks in pattern classification problems is the excessive time required for supervised learning in larger multilayer feedforward networks. The use of nonlinear optimization techniques to perform neural network training offers a means of reducing that computing time. Two key issues in the implementation of nonlinear programming are the choice of a method for computing search direction and the degree of accuracy required of the subsequent line search. This paper examines these issues through a designed experiment using six different pattern classification tasks, four search direction methods (conjugate gradient, quasi-Newton, and two levels of limited memory quasi-Newton), and three levels of line search accuracy. It was found that for the simplest pattern classification problems, the conjugate gradient performed well. For more complicated pattern classification problems, the limited memory BFGS or the BFGS should be preferred. For very large problems, the best choice seems to be the limited memory BFGS. It was also determined that, for the line search methods used in this study, increasing accuracy did not improve efficiency.     

Discriminant Analysis in Schizophrenia based on Neural Network

In this paper, we use neural network to classify schizophrenia patients and healthy control subjects. Based on 4005 high dimensions feature space consist of functional connectivity about 63 schizophrenic patients and 57 healthy control as the original data, attempting to try different dimensionality reduction methods, different neural network model to find the optimal classification model. The results show that using the Mann-Whitney U test to select the more discrimination features as input and using Elman neural network model for classification to get the best results, can reach a highest accuracy of 94.17%, with the sensitivity being 92.06% and the specificity being 96.49%. For the best classification neural network model, we identified 34 consensus functional connectivities that exhibit high discriminative power in classification, which includes 26 brain regions, particularly in the thalamus regions corresponding to the maximum number of functional connectivity edges, followed by the cingulate gyrus and frontal region.     

A CS-AdaBoost-BP model for product quality inspection

Quality inspection is essential in preventing defective products from entering the market. Due to the typically low percentage of defective products, it is generally challenging to detect them using algorithms that aim for the overall classification accuracy. To help solve this problem, we propose an ensemble learning classification model, where we employ adaptive boosting (AdaBoost) to cascade multiple backpropagation (BP) neural networks. Furthermore, cost-sensitive (CS) learning is introduced to adjust the loss function of the basic classifier of the BP neural network. For clarity, this model is called a CS-AdaBoost-BP model. To empirically verify its effectiveness, we use data from home appliance production lines from Bosch. We carry out tenfold cross-validation to evaluate and compare the performance between the CS-AdaBoost-BP model and three existing models: BP neural network, BP neural network based on sampling, and AdaBoost-BP. The results show that our proposed model not only performs better than the other models but also significantly improves the ability to identify defective products. Furthermore, based on the mean value of the Youden index, our proposed model has the highest stability.