A survey of various tactics for generating Lagrangian multipliers in the context of Lagrangian duality

This study concerns itself with Lagrangian duality for continuous and discrete mathematical programming problems. Properties of the dual function, including subdifferentiability, differentiability, ascent, and steepest ascent directions are discussed. We show the relationship between directions of steepest ascent and shortest subgradients under different normalization constraints. We then discuss various strategies for generating and updating the Lagrangian multiplier vectors in the course of dual optimization.     

Improvement of extreme controls and the steepest ascent method in the norm maximization problem on the reachable set

The maximization of the terminal state norm of a linear system is considered in the sense of searching for and improving extreme points of the reachable set. A sufficient optimality condition is formulated in terms of a special maximum function. A steepest ascent method for level surfaces of the objective function is constructed, and related procedures for improving extreme controls are described.     

Interactive algorithm for multiobjective optimization

A man-machine interactive algorithm is given for solving multiobjective optimization problems involving one decision maker. The algorithm, a modification of the Frank-Wolfe steepest ascent method, gives at each iteration a significant freedom and ease for the decision-maker's self-expression, and requires a minimal information on his local estimate of the steepest-ascent direction. The convergence of the iterative algorithm is proved under natural assumptions on the convergence and stability of the basic Frank-Wolfe algorithm.     

Resolving degeneracy in combinatorial linear programs: Steepest edge,steepest ascent,and parametric ascent

While variants of the steepest edge pivot rule are commonly used in linear programming codes they are not known to have the theoretically attractive property of avoiding an infinite sequence of pivots at points of degeneracy. An example is presented demonstrating that the steepest edge pivot rule can fail to terminate finitely. It is then shown that a natural extension of the steepest edge pivot rule based on steepest ascent is guaranteed to determine a direction of ascent or a proof that no such direction exists after a finite number of pivots, although without modification the extension may not terminate with an ascent direction corresponding to an edge. Finally, it is demonstrated that a computationally more efficient pivot rule proposed by Magnanti and Orlin has similar theoretical properties to steepest ascent with probability 1independent of the linear program being solved. Unlike alternative methods such as primal lexicographic rules and Bland's rule, the algorithms described here have the advantage that they choose the pivot element without explicit knowledge of the set of all active constraints at a point of degeneracy, thus making them attractive in combinatorial settings where the linear program is represented implicitly.This work was sponsored in part by the National Science Foundation and the Office of Naval Research under NSF grant number DDM-9101578. The author gratefully acknowledges the support of IMSL, Inc.     

Successive unconstrained dual optimization method for?rank-one approximation to tensors

A successive unconstrained dual optimization (SUDO) method is developed to solve the high order tensors?? best rank-one approximation problems, in the least-squares sense. The constrained dual program of tensors?? rank-one approximation is transformed into a sequence of unconstrained optimization problems, for where a fast gradient method is proposed. We introduce the steepest ascent direction, a initial step length strategy and a backtracking line search rule for each iteration. A proof of the global convergence of the SUDO algorithm is given. Preliminary numerical experiments show that our method outperforms the alternating least squares (ALS) method.     

Proper exhausters and coexhausters in nonsmooth analysis

There exist many tools to analyze nonsmooth functions. For convex and max-type functions, the notion of subdifferential is used, for quasidifferentiable functions – that of quasidifferential. By means of these tools, one is able to solve, e.g. the following problems: to get an approximation of the increment of a functional, to formulate conditions for an extremum, to find steepest descent and ascent directions and to construct numerical methods. For arbitrary directionally differentiable functions, these problems are solved by employing the notions of upper and lower exhausters and coexhausters, which are generalizations of such notions of nonsmooth analysis as sub- and superdifferentials, quasidifferentials and codifferentials. Exhausters allow one to construct homogeneous approximations of the increment of a functional while coexhausters provide nonhomogeneous approximations. It became possible to formulate conditions for an extremum in terms of exhausters and coexhausters. It turns out that conditions for a minimum are expressed by an upper exhauster, and conditions for a maximum are formulated via a lower one. This is why an upper exhauster is called a proper one for the minimization problem (and adjoint for the maximization problem) while a lower exhauster is called a proper one for the maximization problem (and adjoint for the minimization problem). The conditions obtained provide a simple geometric interpretation and allow one to find steepest descent and ascent directions. In this article, optimization problems are treated by means of proper exhausters and coexhausters.     

A parameter-free multiplier method for constrained minimization problems

This paper is concerned with the development of a parameter-free method, closely related to penalty function and multiplier methods, for solving constrained minimization problems. The method is developed via the quadratic programming model with equality constraints. The study starts with an investigation into the convergence properties of a so-called “primal-dual differential trajectory”, defined by directions given by the direction of steepest descent with respect to the variables x of the problem, and the direction of steepest ascent with respect to the Lagrangian multipliers λ, associated with the Lagrangian function. It is shown that the trajectory converges to a stationary point (x*, λ*) corresponding to the solution of the equality constrained problem. Subsequently numerical procedures are proposed by means of which practical trajectories may be computed and the convergence of these trajectories are analyzed. A computational algorithm is presented and its application is illustrated by means of simple but representative examples. The extension of the method to inequality constrained problems is discussed and a non-rigorous argument, based on the Kuhn—Tucker necessary conditions for a constrained minimum, is put forward on which a practical procedure for determining the solution is based. The application of the method to inequality constrained problems is illustrated by its application to a couple of simple problems.     

Cluster robustness of preconditioned gradient subspace iteration eigensolvers

The paper presents convergence estimates for a class of iterative methods for solving partial generalized symmetric eigenvalue problems whereby a sequence of subspaces containing approximations to eigenvectors is generated by combining the Rayleigh-Ritz and the preconditioned steepest descent/ascent methods. The paper uses a novel approach of studying the convergence of groups of eigenvalues, rather than individual ones, to obtain new convergence estimates for this class of methods that are cluster robust, i.e. do not involve distances between computed eigenvalues.     

Steepest ascent can be exponential in bounded treewidth problems

We investigate the complexity of local search based on steepest ascent. We show that even when all variables have domains of size two and the underlying constraint graph of variable interactions has bounded treewidth (in our construction, treewidth 7), there are fitness landscapes for which an exponential number of steps may be required to reach a local optimum. This is an improvement on prior recursive constructions of long steepest ascents, which we prove to need constraint graphs of unbounded treewidth.     

求解log—最优组合投资问题的一个自适应算法及其理论分析

本文给出了一个求解log－最优组合投资问题的自适应算法,它是一个变型的随机逼近方法。该问题是一个约束优化问题,因此,采用基于约束流形的梯度上升方向替代常规梯度上升方向,在一些合理的假设下证明了算法的收敛性并进行了渐近稳定性分析。最后,本文将该算法应用于上海证券交易所提供的实际数据的log-最优组合投资问题求解,获得了理想的数值模拟结果。     

Justification of the Steepest Descent Method for the Integral Statement of an Inverse Problem for a Hyperbolic Equation

Under consideration is the steepest descent method for solving the problem of determination of a coefficient in a hyperbolic equation in integral statement. The properties of solutions to the direct and inverse problems are studied. Estimates for the objective functional and its gradient are obtained. Convergence in the mean is proved for the steepest descent method for minimizing the residual functional.     

Measure of Ordering in Statistical Systems

In this paper, we introduce a phenomenological measure of ordering in statistical systems. Using this characteristic, we construct a formal model of a system evolving according to the law of steepest ascent as the control parameter increases.     

The relationship between theorems of the alternative,least norm problems,steepest descent directions,and degeneracy: A review

Recently, it has been observed that several nondifferentiable minimization problems share the property that the question of whether a given point is optimal can be answered by solving a certain bounded least squares problem. If the resulting residual vector,r, vanishes then the current point is optimal. Otherwise,r is a descent direction. In fact, as we shall see,r points at the steepest descent direction. On the other hand, it is customary to characterize the optimality conditions (and the steepest descent vector) of a convex nondifferentiable function via its subdifferential. Also, it is well known that optimality conditions are usually related to theorems of the alternative. One aim of our survey is to clarify the relations between these subjects. Another aim is to introduce a new type of theorems of the alternative. The new theorems characterize the optimality conditions of discretel ₁ approximation problems and multifacility location problems, and provide a simple way to obtain the subdifferential and the steepest descent direction in such problems. A further objective of our review is to demonstrate that the ability to compute the steepest descent direction at degenerate dead points opens a new way for handling degeneracy in active set methods.     

A finite steepest-ascent algorithm for maximizing piecewise-linear concave functions

In this note, we describe a finitely convergent steepest-ascent scheme for maximizing piecewise-linear concave functions. Given any point, the algorithm moves along the direction of steepest ascent, that is, along the shortest subgradient, until a new ridge is reached. The overall process is then repeated by moving along the new steepest-ascent direction.     

An algorithm for finding the shortest element of a polyhedral set with application to lagrangian duality

An algorithm is developed which finds the point in a compact polyhedral set with smallest Euclidean norm. At each iteration the algorithm requires knowledge of only those vertices of the set which are adjacent to a current reference vertex. This feature is shown to permit the adoption of the technique to find iteratively the shortest subgradient (i.e. the direction of steepest ascent) of the lagrangian dual function for large scale linear programs. Procedures are presented for finding the direction of steepest ascent in both the equality constraint and the inequality constraint cases of lagrangian duality.     

Steepest descent method with a generalized Armijo search for quasiconvex functions on Riemannian manifolds

This paper extends the full convergence of the steepest descent method with a generalized Armijo search and a proximal regularization to solve minimization problems with quasiconvex objective functions on complete Riemannian manifolds. Previous convergence results are obtained as particular cases and some examples in non-Euclidian spaces are given. In particular, our approach can be used to solve constrained minimization problems with nonconvex objective functions in Euclidian spaces if the set of constraints is a Riemannian manifold and the objective function is quasiconvex in this manifold.     

Static control optimization in multivariable technological systems

The static operation of a technological process is optimized by several variables. An optimal control algorithm using the fastest descent (steepest ascent) method is described. Some test results are presented."Stroimaterialy" Ukrainian Scientific-Industrial Association. Translated from Vychislitel'naya i Prikladnaya Matematika, No. 74, pp. 68–77, 1992;     

求解低秩矩阵填充的改进的交替最速下降法

矩阵填充是指利用矩阵的低秩特性而由部分观测元素恢复出原矩阵,在推荐系统、信号处理、医学成像、机器学习等领域有着广泛的应用。采用精确线搜索的交替最速下降法由于每次迭代计算量小因而对大规模问题的求解非常有效。本文在其基础上采用分离地精确线搜索,可使得每次迭代下降更多但计算量相同,从而可望进一步提高计算效率。本文分析了新算法的收敛性。数值结果也表明所提出的算法更加有效。     

Steepest descent preconditioning for nonlinear GMRES optimization

Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N‐GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, whereas the second employs a predefined small step. A simple global convergence proof is provided for the N‐GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioning for N‐GMRES optimization is also motivated by relating it to standard non‐preconditioned GMRES for linear systems in the case of a standard quadratic optimization problem with symmetric positive definite operator. Numerical tests on a variety of model problems show that the N‐GMRES optimization algorithm is able to very significantly accelerate convergence of stand‐alone steepest descent optimization. Moreover, performance of steepest‐descent preconditioned N‐GMRES is shown to be competitive with standard nonlinear conjugate gradient and limited‐memory Broyden–Fletcher–Goldfarb–Shanno methods for the model problems considered. These results serve to theoretically and numerically establish steepest‐descent preconditioned N‐GMRES as a general optimization method for unconstrained nonlinear optimization, with performance that appears promising compared with established techniques. In addition, it is argued that the real potential of the N‐GMRES optimization framework lies in the fact that it can make use of problem‐dependent nonlinear preconditioners that are more powerful than steepest descent (or, equivalently, N‐GMRES can be used as a simple wrapper around any other iterative optimization process to seek acceleration of that process), and this potential is illustrated with a further application example. Copyright © 2012 John Wiley & Sons, Ltd.