CONVERGENCE OF ONLINE GRADIENT METHOD WITH A PENALTY TERM FOR FEEDFORWARD NEURAL NETWORKS WITH STOCHASTIC INPUTS

Online gradient algorithm has been widely used as a learning algorithm for feedforward neural network training. In this paper, we prove a weak convergence theorem of an online gradient algorithm with a penalty term, assuming that the training examples are input in a stochastic way. The monotonicity of the error function in the iteration and the boundedness of the weight are both guaranteed. We also present a numerical experiment to support our results.     

用在线梯度法训练积单元神经网络的收敛性分析

<正>1引言仅由加和单元构成的传统前向神经网络已经广泛应用于模式识别及函数逼近等领域.但在处理比较复杂的问题时,这种网络往往需要补充大量的隐节点,这样就不可避免地增     

解线性规划问题的梯度投影法

本文叙述了一个求解线性规划问题的梯度投影法,导出了投影矩阵的递推公式,利用此公式可大大减少每次迭代所需的计算量。实例计算表明,本文给出的算法是一有效的算法,在某些方面它要优于Karmarkar算法和单纯形法。     

CONVERGENCE OF SOLUTIONS FOR RLC-NONLINEAR NETWORKS WITH TIME-VARYING ELEMENTS

ＣＯＮＶＥＲＧＥＮＣＥＯＦＳＯＬＵＴＩＯＮＳ　ＦＯＲＲＬＣ－ＮＯＮＬＩＮＥＡＲＮＥＴＷＯＲＫＳＷＩＴＨＴＩＭＥ－ＶＡＲＹＩＮＧＥＬＥＭＥＮＴＳ￥ＪｉａｎｇＪｉｊａ（蒋断发）（Ｄａｐｔ．ｏｆＭａｔｈ．，ＵｎｉｖｅｒｓｉｔｙｏｆＳｃｉｅｎｃｅａｎｄＴｅｃ...     

A CLASS OF REDUCED GRADIENT METHODS FOR HANDLING OPTIMIZATION PROBLEMS WITH LINEAR INEQUALITY CONSTRAINTS

A class of reduced gradient methods for handling general optimization problems with linear equality and inequality constraints is suggested in this paper. Although a slack vector is introduced, the dimension of the problem is not increased, which is unlike the conventional way of transferring the inequality constraints into the equality constraints by introducing slack variables. When an iterate x(k) is not a K-T point of the problem under consideration, different feasible descent directions can be obtained by different choices of the slack vectors. The suggested method is globally convergent and the numerical experiment given in the paper shows that the method is efficient.     

NEW CRITERIA OF EXPONENTIAL STABILITY FOR BAM NEURAL NETWORKS WITH DELAYS

In this Letter,a novel Lyapunov functional is constructed to investigate the exponential stability of the BAM neural networks.New sufficient conditions of the uniqueness and global exponential stability for the equilibrium of BAM neural networks with delays are obtained.The results improve those existing ones.     

一种新的记忆梯度法及其收敛性

本文提出一种新的无约束优化记忆梯度算法,算法在每步迭代时利用了前面迭代点的信息,增加了参数选择的自由度,适于求解大规模无约束优化问题.分析了算法的全局收敛性.数值试验表明算法是有效的.     

变时滞细胞神经网络的指数稳定的一个充分条件

本文我们研究了变时滞细胞神经网络的指数稳定性。利用分析技巧，获得了变时滞细胞神经网络的指数稳定的充分条件，改进了已有文献中的相应结果。     

PERIODIC SOLUTIONS AND STABILITY ANALYSIS FOR NEURAL NETWORKS WITH TIME-VARYING DELAY

1 IntroductionIt is well known that both in biological and man-made neural systems, inte-gl.ation afld communication delays are ubiquitous, and often become sources ofinstabilitY' The de1ays in electronic neural networks are usually time varying,and sometimes vary vio1ently with time due to the finite switching speed ofamplifiers and faults in the electrical circuit. They s1ow down the transmissionrate and tend to introduce some degree of instability in circuits. Therefore,fast response must …     

SCALE-TYPE STABILITY FOR NEURAL NETWORKS WITH UNBOUNDED TIME-VARYING DELAYS

This paper studies scale-type stability for neural networks with unbounded time-varying delays and Lipschitz continuous activation functions. Several sufficient conditions for the global exponential stability and global asymptotic stability of such neural networks on time scales are derived. The new results can extend the existing relevant stability results in the previous literatures to cover some general neural networks.     

THE CONVERGENCE OF DURAND-KERNER METHOD FOR SIMULTANEOUSLY FINDING ALL ZEROS OF THE POLYNOMIAL

1. IntroductionConsider a monic polynomial of degree nTo solve all zeros of this polynomial, the common--used is Durand--Kerner methodwhich has been considered simple and effective. For this reason) there are many works(Docev (1962), hemer (1966), Yamamoto (1990) and Zheng (1982)) to discuss theproperties of this method. In this paper, we propose the two kinds of criterion anduse them to judge the convergence of Durand-Kerner method or to compare with othermethods.Durand--Kerner method is …     

ON CONVERGENCE OF MULTIGRID METHOD FOR NONNEGATIVE DEFINITE SYSTEMS

In this paper, we consider multigrid methods for solving symmetric nonnegative definite matrix equations. We present some interesting features of the multigrid method and prove that the method is convergent in L2 space and the convergent solution is unique for such nonnegative definite system and given initial guess.     

EXPONENTIAL STABILITY OF DISCRETE-TIME CELLULAR NEURAL NETWORKS WITH DELAYS

In this paper, a new sufficient condition for the global exponential stability of a unique equilibrium point of discrete-time cellular neural networks is given. It is shown that the condition relies on the feedback matrices and is independent of the delay parameter. Furthermore, this condition is less restrictive than those given in the literature.     

变时滞Hopfield神经网络的全局指数稳定性

本文研究了一类具有变时滞Hopfield神经网络的稳定性问题.利用时滞微分不等式方法,获得了几个关于该网络的全局指数稳定性与时滞无关的充分条件,并且给出了此类网络的收敛指数的估计,推广了已知文献的结果.最后给出数值例子证明结论的有效性.     

THE EXISTENCE OF ALMOST PERIODIC SOLUTION FOR CELLULAR NEURAL NETWORKS WITH VARIABLE COEFFICIENTS AND DELAYS

In this paper, we study the problem of almost periodic solution for the cellular neural networks with variable coefficients and delays. By ingeniously importing real parameters di > 0(i = 1,2,…n) which can be adjusted, we establish some sufficient conditions which ensure the neural networks exist a unique almost periodic solution, and all other solutions exponentially converge to such an almost periodic solution.     

结合广义Armijo步长搜索的一类新的三项共轭梯度算法及其收敛特征

In this paper, we consider the convergence properties of a new class of three terms conjugate gradient methods with generalized Armijo step size rule for minimizing a continuously differentiable function f on R^π without assuming that the sequence {xk} of iterates is bounded. We prove that the limit infimum of ‖↓△f(xk)‖ is Zero. Moreover, we prove that, when f(x) is pseudo-convex (quasi-convex) function, this new method has strong convergence results: either xk→x* and x* is a minimizer (stationary point); or ‖xk‖→∞, arg min{f(x) :x∈R^n} =φ, and.f(xk) ↓ inf(f(x) : x∈R^n}. Combining FR, PR, HS methods with our new method, FR, PR, HS methods are modified to have global convergence property.Numerical result show that the new algorithms are efficient by comparing with FR,PR, HS conjugate gradient methods with Armijo step size rule.     

ON GLOBAL ROBUST STABILITY FOR INTERVAL HOPFIELD NEURAL NETWORKS WITH TIME DELAY

The theorem obtained by Liao was not true (see [2]). So, this paper presents some criteria of global robust stability for interval Hopfield neural networks with time delay. The methods to judge the robust stability are practical and easily verifiable.     

具有i个强分支的有向图的幂敛指数集

本文给出了ｎ阶具有ｉ个强分支的有向图的幂敛指数集Ｒ（ｎ，ｉ）的一个明显表达式。     

一种求解非线性方程组的3p阶迭代方法

本文将一种改进的二步迭代算法作为预测,将高斯-勒让德求积公式作为校正,提出了一种求解非线性方程组的具有3p收敛阶的迭代方法.最后给出了一些数值实例,将本文的实验结果与现有的几种迭代方法的实验结果作了比较分析,验证了本文所提出的结果.     

GLOBAL ATTRACTIVITY OF THE HOPFIELD NEURAL NETWORKS

1 IntroductionHopfield neural networks is an associative memory model, it is also a neuraloptimization networks. This model has been studied extensively (see [1--12]).But in most of the literature, the signal function gi(x) is differentiable and isa sigmoid curve, and the sufficient conditions about the global attractivity ofthe Hopfield neural networks have no relevance to the delay. The aim of thispaper is to give a delayed sufficient condition of the global attractivity of theHopfield neur…