前向神经网络的神经元分层逐个线性优化快速学习算法

本文提出了一种新的前向神经网络快速分层学习算法.在此学习算法中,其优化策略为对输出层和隐层神经元的连接权值交替优化.对输出层权值优化算法采用基于广义逆的最小二乘递推算法,对隐层神经元的连接权值采取则对每个神经元逐个进行优化,而且采用正交变换加快每一步学习的计算速度和提高算法的数值稳定性.当学习过程停滞时采用随机扰动的方法摆脱过早收敛.数值实验表明,与BP动量因子法、牛顿型方法和现有的分层优化算法相比,新算法不仅学习速度快学习时间短,而且当网络规模增大时仍然比较有效.

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

A new rapid layer-wise learning algorithm for feed-forward neural networks is proposed in this paper.In the proposed learning algorithm,the strategy is optimizhing the weights of output layer and hidden layer alternately.A least square recursive algorithm based on generalized inverse is applied to optimize output layer weights,while hidden layer weights are optimized neuron by neuron and an orthogonal transformation is applied to accelerate compute speed for every step and to improve numerical stability.When learning process stop prematurely a stochastic disturbance is applied.Numerical experiments show that the new algorithm is better than other learning algorithms both in learning speed and learning time,and is also efficient for large scale neural networks.