Online Gradient Methods with a Punishing Term for Neural Networks

1 IntroductionOnline gradient methods (OGM, for short) are widely used for training neuraJ networks (cf.Il,2,3,4]). Its iterative convergence for linear models is proved in e.g. I5,6,71. A nonlinearn1odel is considered in [8]. During the iterative training procedure, sometimes (see the nextsection of this paper)the weight of the network may become very laxge, causing difficultiesin the implementation of the network by electronic circuits. A revised error fUnction ispresented in [gl to prev…     

二层前传神经网络中在线梯度法的收敛性

本文给出了一般情况下二层前传神经网络中的在线梯度法的收敛性定理,并将其应用于一些常用的活化函数和能量函数。