Stability Analysis of Gradient-Based Neural Networks for Optimization Problems |
| |
Authors: | Qiaoming Han Li-Zhi Liao Houduo Qi Liqun Qi |
| |
Institution: | (1) School of Mathematics and Computer Science, Nanjing Normal University, Nanjing, 210097, P.R. China;(2) Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Kowloon, Hong Kong;(3) School of Mathematics, The University of New South Wales, Sydney, New South Wales, 2052, Australia;(4) Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hum, Kowloon, Hong Kong |
| |
Abstract: | The paper introduces a new approach to analyze the stability of neural network models without using any Lyapunov function. With the new approach, we investigate the stability properties of the general gradient-based neural network model for optimization problems. Our discussion includes both isolated equilibrium points and connected equilibrium sets which could be unbounded. For a general optimization problem, if the objective function is bounded below and its gradient is Lipschitz continuous, we prove that (a) any trajectory of the gradient-based neural network converges to an equilibrium point, and (b) the Lyapunov stability is equivalent to the asymptotical stability in the gradient-based neural networks. For a convex optimization problem, under the same assumptions, we show that any trajectory of gradient-based neural networks will converge to an asymptotically stable equilibrium point of the neural networks. For a general nonlinear objective function, we propose a refined gradient-based neural network, whose trajectory with any arbitrary initial point will converge to an equilibrium point, which satisfies the second order necessary optimality conditions for optimization problems. Promising simulation results of a refined gradient-based neural network on some problems are also reported. |
| |
Keywords: | Gradient-based neural network Equilibrium point Equilibrium set Asymptotic stability Exponential stability |
本文献已被 SpringerLink 等数据库收录! |
|