Long-time asymptotic for the Hirota equation via nonlinear steepest descent method

We present the Riemann–Hilbert problem formalism for the initial value problem for the Hirota equation on the line. We show that the solution of this initial value problem can be obtained from that of associated Riemann–Hilbert problem, which allows us to use nonlinear steepest descent method/Deift–Zhou method to analyze the long-time asymptotic for the Hirota equation.     

Steepest descent preconditioning for nonlinear GMRES optimization

Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N‐GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, whereas the second employs a predefined small step. A simple global convergence proof is provided for the N‐GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioning for N‐GMRES optimization is also motivated by relating it to standard non‐preconditioned GMRES for linear systems in the case of a standard quadratic optimization problem with symmetric positive definite operator. Numerical tests on a variety of model problems show that the N‐GMRES optimization algorithm is able to very significantly accelerate convergence of stand‐alone steepest descent optimization. Moreover, performance of steepest‐descent preconditioned N‐GMRES is shown to be competitive with standard nonlinear conjugate gradient and limited‐memory Broyden–Fletcher–Goldfarb–Shanno methods for the model problems considered. These results serve to theoretically and numerically establish steepest‐descent preconditioned N‐GMRES as a general optimization method for unconstrained nonlinear optimization, with performance that appears promising compared with established techniques. In addition, it is argued that the real potential of the N‐GMRES optimization framework lies in the fact that it can make use of problem‐dependent nonlinear preconditioners that are more powerful than steepest descent (or, equivalently, N‐GMRES can be used as a simple wrapper around any other iterative optimization process to seek acceleration of that process), and this potential is illustrated with a further application example. Copyright © 2012 John Wiley & Sons, Ltd.     

Convergence of the steepest descent method for minimizing quasiconvex functions

To minimize a continuously differentiable quasiconvex functionf: ⁿ , Armijo's steepest descent method generates a sequencex ^k+1 =x ^k –t _k f(x ^k ), wheret _k >0. We establish strong convergence properties of this classic method: either , s.t. ; or arg minf = , x ^k andf(x ^k ) inff. We also discuss extensions to other line searches.The research of the first author was supported by the Polish Academy of Sciences. The second author acknowledges the support of the Department of Industrial Engineering, Hong Kong University of Science and Technology.We wish to thank two anonymous referees for their valuable comments. In particular, one referee has suggested the use of quasiconvexity instead of convexity off.     

半监督度量学习内蕴最速下降算法的收敛性分析

主要研究对称正定矩阵群上的内蕴最速下降算法的收敛性问题.首先针对一个可转化为对称正定矩阵群上无约束优化问题的半监督度量学习模型,提出对称正定矩阵群上一种自适应变步长的内蕴最速下降算法.然后利用李群上的光滑函数在任意一点处带积分余项的泰勒展开式,证明所提算法在对称正定矩阵群上是线性收敛的.最后通过在分类问题中的数值实验说明算法的有效性.     

Steepest descent method with a generalized Armijo search for quasiconvex functions on Riemannian manifolds

This paper extends the full convergence of the steepest descent method with a generalized Armijo search and a proximal regularization to solve minimization problems with quasiconvex objective functions on complete Riemannian manifolds. Previous convergence results are obtained as particular cases and some examples in non-Euclidian spaces are given. In particular, our approach can be used to solve constrained minimization problems with nonconvex objective functions in Euclidian spaces if the set of constraints is a Riemannian manifold and the objective function is quasiconvex in this manifold.     

A combination of potential reduction steps and steepest descent steps for solving convex programming problems

This paper studies the possibility of combining interior point strategy with a steepest descent method when solving convex programming problems, in such a way that the convergence property of the interior point method remains valid but many iterations do not request the solution of a system of equations. Motivated by this general idea, we propose a hybrid algorithm which combines a primal–dual potential reduction algorithm with the use of the steepest descent direction of the potential function. The complexity of the potential reduction algorithm remains valid but the overall computational cost can be reduced. Our numerical experiments are also reported. Copyright © 2002 John Wiley & Sons, Ltd.     

On steepest descent curves for quasi convex families in Rn

A connected, linearly ordered subset $\gamma ?{\mathbb{R}}^n$ satisfying $$x_1,x_2,x_3\in \gamma ,\text{and}{x}_1?x_2?x_3\to |x_2?x_1|\le |x_3?x_1|$$ is shown to be a rectifiable curve; a priori bounds for its length are given; moreover, these curves are generalized steepest descent curves of suitable quasi convex functions. Properties of quasi convex families are considered; special curves related to quasi convex families are defined and studied; they are generalizations of steepest descent curves for quasi convex functions and satisfy the previous property. Existence, uniqueness, stability results and length bounds are proved for them.     

A sufficient descent three-term conjugate gradient method via symmetric rank-one update for large-scale optimization

In this paper, we propose a three-term conjugate gradient method via the symmetric rank-one update. The basic idea is to exploit the good properties of the SR1 update in providing quality Hessian approximations to construct a conjugate gradient line search direction without the storage of matrices and possess the sufficient descent property. Numerical experiments on a set of standard unconstrained optimization problems showed that the proposed method is superior to many well-known conjugate gradient methods in terms of efficiency and robustness.     

A generalized steepest descent method for continuously subdifferentiable functions

A class of methods for unconstrained minimization of quasidifferentiable, especially subdifferentiable functions is described, which includes well-known algorithms as special cases. Moreover, it is shown that an algorithm previously published fails to converge to an e-inf-stationary point in general. Some preliminary numerical results are reported on.     

Stochastic algorithms with armijo stepsizes for minimization of functions

Stochastic algorithms for optimization problems, where function evaluations are done by Monte Carlo simulations, are presented. At each iteratex _i, they draw a predetermined numbern(i) of sample points from an underlying probability space; based on these sample points, they compute a feasible-descent direction, an Armijo stepsize, and the next iteratex _i+1. For an appropriate optimality function , corresponding to an optimality condition, it is shown that, ifn(i) , then (x _i) 0, whereJ is a set of integers whose upper density is zero. First, convergence is shown for a general algorithm prototype: then, a steepest-descent algorithm for unconstrained problems and a feasible-direction algorithm for problems with inequality constraints are developed. A numerical example is supplied.     

A characteristic condition for convergence of steepest descent approximation to accretive operator equations

In the present paper a sufficient and necessary condition for convergence of steepest descent approximation to accretive operator equations is established, and for the sufficiency part a specific error estimation is also given.     

A projected‐steepest‐descent potential‐reduction algorithm for convex programming problems

A recent work of Shi (Numer. Linear Algebra Appl. 2002; 9 : 195–203) proposed a hybrid algorithm which combines a primal‐dual potential reduction algorithm with the use of the steepest descent direction of the potential function. The complexity of the potential reduction algorithm remains valid but the overall computational cost can be reduced. In this paper, we make efforts to further reduce the computational costs. We notice that in order to obtain the steepest descent direction of the potential function, the Hessian matrix of second order partial derivatives of the objective function needs to be computed. To avoid this, we in this paper propose another hybrid algorithm which uses a projected steepest descent direction of the objective function instead of the steepest descent direction of the potential function. The complexity of the original potential reduction algorithm still remains valid but the overall computational cost is further reduced. Our numerical experiments are also reported. Copyright © 2004 John Wiley & Sons, Ltd.     

Local convergence of the steepest descent method in Hilbert spaces

The aim of this paper is to establish the local convergence of the steepest descent method for C¹-functionals defined on an infinite-dimensional Hilbert space H, under a Palais–Smale-type condition. The functionals f under consideration are also assumed to have a locally Lipschitz continuous gradient operator f. Our approach is based on the solutions of the ordinary differential equation .     

A novel hybrid immune algorithm and its convergence based on the steepest descent algorithm

This paper proposes a novel hybrid immune algorithm (HIA) that can overcome the typical drawback of the artificial immune algorithm (AIA), which runs slowly and experiences slow convergence. The HIA combines the adaptive AIA based on the steepest descent algorithm. The HIA fully displays global search ability and the global convergence of the immune algorithm. At the same time, it inserts a quasi-descent operator to strengthen its local search ability. A good convergence of the HIA with the quasi-descent idea is shown as well. Numerical experiment results show that the HIA successfully improves running speed and convergence performance.     

On nonoscillating integrals for computing inhomogeneous Airy functions

Integral representations are considered of solutions of the inhomogeneous Airy differential equation . The solutions of these equations are also known as Scorer functions. Certain functional relations for these functions are used to confine the discussion to one function and to a certain sector in the complex plane. By using steepest descent methods from asymptotics, the standard integral representations of the Scorer functions are modified in order to obtain nonoscillating integrals for complex values of . In this way stable representations for numerical evaluations of the functions are obtained. The methods are illustrated with numerical results.

     

A sufficient descent LS conjugate gradient method for unconstrained optimization problems

In this paper, we make a modification to the Liu-Storey (LS) conjugate gradient method and propose a descent LS method. The method can generate sufficient descent directions for the objective function. This property is independent of the line search used. We prove that the modified LS method is globally convergent with the strong Wolfe line search. The numerical results show that the proposed descent LS method is efficient for the unconstrained problems in the CUTEr library.     

A New Gradient Method with an Optimal Stepsize Property

The gradient method for the symmetric positive definite linear system is as follows