Gauss Quadrature Applied to Trust Region Computations

This paper discusses the application of the Lanczos process to the solution of large-scale trust-region subproblems. Techniques pioneered by Golub based on Gauss quadrature are applied to derive inexpensively computable upper and lower bounds for quantities of interest. These bounds help determine how many steps of the Lanczos process should be carried out.     

Spectral Gradient Methods for Linearly Constrained Optimization

Linearly constrained optimization problems with simple bounds are considered in the present work. First, a preconditioned spectral gradient method is defined for the case in which no simple bounds are present. This algorithm can be viewed as a quasi-Newton method in which the approximate Hessians satisfy a weak secant equation. The spectral choice of steplength is embedded into the Hessian approximation and the whole process is combined with a nonmonotone line search strategy. The simple bounds are then taken into account by placing them in an exponential penalty term that modifies the objective function. The exponential penalty scheme defines the outer iterations of the process. Each outer iteration involves the application of the previously defined preconditioned spectral gradient method for linear equality constrained problems. Therefore, an equality constrained convex quadratic programming problem needs to be solved at every inner iteration. The associated extended KKT matrix remains constant unless the process is reinitiated. In ordinary inner iterations, only the right-hand side of the KKT system changes. Therefore, suitable sparse factorization techniques can be applied and exploited effectively. Encouraging numerical experiments are presented.This research was supported by FAPESP Grant 2001-04597-4 and Grant 903724-6, FINEP and FAEP-UNICAMP, and the Scientific Computing Center of UCV. The authors thank two anonymous referees whose comments helped us to improve the final version of this paper.     

Enriched Methods for Large-Scale Unconstrained Optimization

This paper describes a class of optimization methods that interlace iterations of the limited memory BFGS method (L-BFGS) and a Hessian-free Newton method (HFN) in such a way that the information collected by one type of iteration improves the performance of the other. Curvature information about the objective function is stored in the form of a limited memory matrix, and plays the dual role of preconditioning the inner conjugate gradient iteration in the HFN method and of providing an initial matrix for L-BFGS iterations. The lengths of the L-BFGS and HFN cycles are adjusted dynamically during the course of the optimization. Numerical experiments indicate that the new algorithms are both effective and not sensitive to the choice of parameters.     

Extension of Quasi-Newton Methods to Mathematical Programs with Complementarity Constraints

Quasi-Newton methods in conjunction with the piecewise sequential quadratic programming are investigated for solving mathematical programming with equilibrium constraints, in particular for problems with complementarity constraints. Local convergence as well as superlinear convergence of these quasi-Newton methods can be established under suitable assumptions. In particular, several well-known quasi-Newton methods such as BFGS and DFP are proved to exhibit the local and superlinear convergence.     

非单调广义对角拟牛顿算法

本文研究了无约束最优化的求解问题.利用新的对角拟牛顿校正和非单调技术,获得了一种非单调广义对角拟牛顿算法.新算法具有低存储、低计算量的特点,非常适合大规模问题的求解,推广了文献[8]的结果.     

A family of quasi-Newton methods for unconstrained optimization problems

ABSTRACT

In this paper, we present a class of approximating matrices as a function of a scalar parameter that includes the Davidon-Fletcher-Powell and Broyden-Fletcher-Goldfarb-Shanno methods as special cases. A powerful iterative descent method for finding a local minimum of a function of several variables is described. The new method maintains the positive definiteness of the approximating matrices. For a region in which the function depends quadratically on the variables, no more than n iterations are required, where n is the number of variables. A set of computational results that verifies the superiority of the new method are presented.     

发散维数SICA惩罚Cox回归模型的一种修正BIC调节参数选择器

本文研究了发散维数SICA惩罚Cox回归模型的调节参数选择问题,提出了一种修正的BIC调节参数选择器.在一定的正则条件下,证明了方法的模型选择相合性.数值结果表明提出的方法表现要优于GCV准则.     

OPTIMAL MOTION PLANNING FOR A RIGID SPACECRAFT WITH TWO MOMENTUM WHEELS USING QUASI-NEWTON METHOD

An optimal motion planning scheme based on the quasi-Newton method is proposedfor a rigid spacecraft with two momentum wheels. A cost functional is introduced to incorporatethe control energy, the final state errors and the constraints on states. The motion planning fordetermining control inputs to minimize the cost functional is formulated as a nonlinear optimalcontrol problem. Using the control parametrization, one can transform the infinite dimensionaloptimal control problem to a finite dimensional one that is solved via the quasi-Newton methodsfor a feasible trajectory which satisfies the nonholonomic constraint. The optimal motion planningscheme was applied to a rigid spacecraft with two momentum wheels. The simulation results showthe effectiveness of the proposed optimal motion planning scheme.     

Least-change quasi-Newton updates for equality-constrained optimization

This paper investigates quasi-Newton updates for equality-constrained optimization. Using a least-change argument we derive a class of rank-3 updates to approximations of the one-sided projection of the Hessian of the Lagrangian which keeps the appropriate part symmetric (and possibly positive definite). By imposing the usual assumptions we are able to prove 1-step superlinear convergence for one of these updates. Encouraging numerical results and comparisons with other previously analyzed updates are presented. Received: May 3, 1999 / Accepted: January 28, 2000?Published online March 15, 2000