A quasi-Newton trust-region method

The classical trust-region method for unconstrained minimization can be augmented with a line search that finds a point that satisfies the Wolfe conditions. One can use this new method to define an algorithm that simultaneously satisfies the quasi-Newton condition at each iteration and maintains a positive-definite approximation to the Hessian of the objective function. This new algorithm has strong global convergence properties and is robust and efficient in practice.     

On quasi-Newton and pseudo-Newton algorithms

This paper surveys some of the existing approaches to quasi-Newton methods and introduces a new way for constructing inverse Hessian approximations for such algorithms. This new approach is based on restricting Newton's method to subspaces over which the inverse Hessian is assumed to be known, while expanding this subspace using gradient information. It is shown that this approach can lead to some well-known formulas for updating the inverse Hessian approximation. Deriving such updates through this approach provides new understanding of these formulas and their relation to the pseudo-Newton-Raphson algorithm.     

On Lagrange multipliers of trust-region subproblems

Trust-region methods are globally convergent techniques widely used, for example, in connection with the Newton’s method for unconstrained optimization. One of the most commonly-used iterative approaches for solving trust-region subproblems is the Steihaug–Toint method which is based on conjugate gradient iterations and seeks a solution on Krylov subspaces. This paper contains new theoretical results concerning properties of Lagrange multipliers obtained on these subspaces. AMS subject classification (2000)  65F20     

Convergence to a second-order point of a trust-region algorithm with a nonmonotonic penalty parameter for constrained optimization

In a recent paper (Ref. 1), the author proposed a trust-region algorithm for solving the problem of minimizing a nonlinear function subject to a set of equality constraints. The main feature of the algorithm is that the penalty parameter in the merit function can be decreased whenever it is warranted. He studied the behavior of the penalty parameter and proved several global and local convergence results. One of these results is that there exists a subsequence of the iterates generated by the algorithm that converges to a point that satisfies the first-order necessary conditions.In the current paper, we show that, for this algorithm, there exists a subsequence of iterates that converges to a point that satisfies both the first-order and the second-order necessary conditions.This research was supported by the Rice University Center for Research on Parallel Computation, Grant R31853, and the REDI Foundation.     

A hybrid of adjustable trust-region and nonmonotone algorithms for unconstrained optimization

This study devotes to incorporating a nonmonotone strategy with an automatically adjusted trust-region radius to propose a more efficient hybrid of trust-region approaches for unconstrained optimization. The primary objective of the paper is to introduce a more relaxed trust-region approach based on a novel extension in trust-region ratio and radius. The next aim is to employ stronger nonmonotone strategies, i.e. bigger trust-region ratios, far from the optimizer and weaker nonmonotone strategies, i.e. smaller trust-region ratios, close to the optimizer. The global convergence to first-order stationary points as well as the local superlinear and quadratic convergence rates are also proved under some reasonable conditions. Some preliminary numerical results and comparisons are also reported.     

A model trust-region modification of Newton's method for nonlinear two-point boundary-value problems

The method of quasilinearization for nonlinear two-point boundary-value problems is Newton's method for a nonlinear differential operator equation. A model trust-region approach to globalizing the quasilinearization algorithm is presented. A double-dogleg implementation yields a globally convergent algorithm that is robust in solving difficult problems.This work was supported in part by DOE Contract DE-AS05-82-ER13016 and NSF Grant RII-89-17691 and was part of the author's doctoral thesis at Rice University. It is a pleasure to thank the author's thesis advisors, Professor J. E. Dennis, Jr., and Professor R. A. Tapia.     

一类新的非单调信赖域算法及其收敛性

利用非单调性,邓乃扬等提出了一类具有强收敛性质的非单调信赖型算法,为了保证算法的收敛性,他们假定以下两个条件成立;（１）信赖域半径（△ｋ）有上界;（２）对所有ｋ有∥ｓｋ∥≤ｃ∥ｇｋ∥,其中ｓｋ＝ｘｋ＋１－ｘｋ,ｇｋ为ｆ（ｔ）在ｘｋ处的梯度,ｃ〉０随后,柯小伍,韩继业从另一角度了提出了一类非调信赖域型算法,尽管他们未利用条件,但仍假定条件（２）成立,在本文中,我们提出了一类新的非单调信赖域算法,在没     

A trust-region strategy for minimization on arbitrary domains

We present a trust-region method for minimizing a general differentiable function restricted to an arbitrary closed set. We prove a global convergence theorem. The trust-region method defines difficult subproblems that are solvable in some particular cases. We analyze in detail the case where the domain is a Euclidean ball. For this case we present numerical experiments where we consider different Hessian approximations.Work partially supported by FAPESP (Grants 90-3724-6 and 91-2441-3), FINEP, CNPq and FAEP-UNICAMP.     

Global convergence without the assumption of linear independence for a trust-region algorithm for constrained optimization

A trust-region algorithm for solving the equality constrained optimization problem is presented. This algorithm uses the Byrd and Omojokun way of computing the trial steps, but it differs from the Byrd and Omojokun algorithm in the way steps are evaluated. A global convergence theory for this new algorithm is presented. The main feature of this theory is that the linear independence assumption on the gradients of the constraints is not assumed.This research was supported in part by the Center for Research on Parallel Computation, by Grant NSF-CCR-91-20008, and by the REDI Foundation.     

Perspectives on self-scaling variable metric algorithms

Recent attempts to assess the performance of SSVM algorithms for unconstrained minimization problems differ in their evaluations from earlier assessments. Nevertheless, the new experiments confirm earlier observations that, on certain types of problems, the SSVM algorithms are far superior to other variable metric methods. This paper presents a critical review of these recent assessments and discusses some current interpretations advanced to explain the behavior of SSVM methods. The paper examines the new empirical results, in light of the original self-scaling theory, and introduces a new interpretation of these methods based on anL-function model of the objective function. This interpretation sheds new light on the performance characteristics of the SSVM methods, which contributes to the understanding of their behavior and helps in characterizing classes of problems which can benefit from the self-scaling approach.The subject of this paper was presented at the ORSA/TIMS National Meeting in New York, 1978.This work was done while the author was with the Analysis Research Group, Xerox Palo Alto Research Center, Palo Alto, California.     

Numerical experience with a polyhedral-norm CDT trust-region algorithm

In this paper, we study a modification of the Celis-Dennis-Tapia trust-region subproblem, which is obtained by replacing thel ₂-norm with a polyhedral norm. The polyhedral norm Celis-Dennis-Tapia (CDT) subproblem can be solved using a standard quadratic programming code.We include computational results which compare the performance of the polyhedral-norm CDT trust-region algorithm with the performance of existing codes. The numerical results validate the effectiveness of the approach. These results show that there is not much loss of robustness or speed and suggest that the polyhedral-norm CDT algorithm may be a viable alternative. The topic merits further investigation.The first author was supported in part by the REDI foundation and State of Texas Award, Contract 1059 as Visiting Member of the Center for Research on Parallel Computation, Rice University, Houston, Texas, He thanks Rice University for the congenial scientific atmosphere provided. The second author was supported in part by the National Science Foundation, Cooperative Agreement CCR-88-09615, Air Force Office of Scientific Research Grant 89-0363, and Department of Energy Contract DEFG05-86-ER25017.     

New least-square algorithms

New algorithms are presented for approximating the minimum of the sum of squares ofM real and differentiable functions over anN-dimensional space. These algorithms update estimates for the location of a minimum after each one of the functions and its first derivatives are evaluated, in contrast with other least-square algorithms which evaluate allM functions and their derivatives at one point before using any of this information to make an update. These new algorithms give estimates which fluctuate about a minimum rather than converging to it. For many least-square problems, they give an adequate approximation for the solution more quickly than do other algorithms.It is a pleasure to thank J. Chesick of Haverford College for suggesting and implementing the application of this algorithm to x-ray crystallography.     

A nonmonotone trust-region method of conic model for unconstrained optimization

In this paper, we present a nonmonotone trust-region method of conic model for unconstrained optimization. The new method combines a new trust-region subproblem of conic model proposed in [Y. Ji, S.J. Qu, Y.J. Wang, H.M. Li, A conic trust-region method for optimization with nonlinear equality and inequality 4 constrains via active-set strategy, Appl. Math. Comput. 183 (2006) 217–231] with a nonmonotone technique for solving unconstrained optimization. The local and global convergence properties are proved under reasonable assumptions. Numerical experiments are conducted to compare this method with the method of [Y. Ji, S.J. Qu, Y.J. Wang, H.M. Li, A conic trust-region method for optimization with nonlinear equality and inequality 4 constrains via active-set strategy, Appl. Math. Comput. 183 (2006) 217–231].     

On the convergence of two branch-and-bound algorithms for nonconvex programming problems

This note presents a new convergence property for each of two branch-and-bound algorithms for nonconvex programming problems (Falk-Soland algorithms and Horst algorithms). For each algorithm, it has been shown previously that, under certain conditions, whenever the algorithm generates an infinite sequence of points, at least one accumulation point of this sequence is a global minimum. We show here that, for each algorithm, in fact, under these conditions, every accumulation point of such a sequence is a global minimum.The author would like to thank Professor R. M. Soland for his helpful comments concerning this paper.     

Acceleration of conjugate gradient algorithms for unconstrained optimization

Conjugate gradient methods are important for large-scale unconstrained optimization. This paper proposes an acceleration of these methods using a modification of steplength. The idea is to modify in a multiplicative manner the steplength α_k, computed by Wolfe line search conditions, by means of a positive parameter η_k, in such a way to improve the behavior of the classical conjugate gradient algorithms. It is shown that for uniformly convex functions the convergence of the accelerated algorithm is still linear, but the reduction in function values is significantly improved. Numerical comparisons with some conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that the accelerated computational scheme outperform the corresponding conjugate gradient algorithms.     

A Trust-Region Method for Nonlinear Bilevel Programming: Algorithm and Computational Experience

We consider the approximation of nonlinear bilevel mathematical programs by solvable programs of the same type, i.e., bilevel programs involving linear approximations of the upper-level objective and all constraint-defining functions, as well as a quadratic approximation of the lower-level objective. We describe the main features of the algorithm and the resulting software. Numerical experiments tend to confirm the promising behavior of the method.     

Interior-point algorithms for semi-infinite programming

In order to study the behavior of interior-point methods on very large-scale linear programming problems, we consider the application of such methods to continuous semi-infinite linear programming problems in both primal and dual form. By considering different discretizations of such problems we are led to a certain invariance property for (finite-dimensional) interior-point methods. We find that while many methods are invariant, several, including all those with the currently best complexity bound, are not. We then devise natural extensions of invariant methods to the semi-infinite case. Our motivation comes from our belief that for a method to work well on large-scale linear programming problems, it should be effective on fine discretizations of a semi-infinite problem and it should have a natural extension to the limiting semi-infinite case.Research supported in part by NSF, AFORS and ONR through NSF grant DMS-8920550.     

Sensitivity of a shallow-water model to parameters

An adjoint based technique is applied to a shallow water model in order to estimate the influence of the model’s parameters on the solution. Among parameters, the bottom topography, initial conditions, boundary conditions on rigid boundaries, viscosity coefficients, Coriolis parameter and the amplitude of the wind stress tension are considered. Their influence is analyzed from three points of view:

•

flexibility of the model with respect to a parameter that is related to the lowest value of the cost function that can be obtained in the data assimilation experiment that controls this parameter;

•

possibility to improve the model by the parameter’s control, i.e., whether the solution with the optimal parameter remains close to observations after the end of control;

•

sensitivity of the model solution to the parameter in a classical sense. That implies the analysis of the sensitivity estimates and their comparison with each other and with the local Lyapunov exponents that characterize the sensitivity of the model to initial conditions.

Two configurations have been analyzed: an academic case of the model in a square box and a more realistic case simulating Black sea currents. It is shown in both experiments that the boundary conditions near a rigid boundary highly influence the solution. This fact points out the necessity to identify optimal boundary approximation during a model development.     

Sizing the BFGS and DFP updates: Numerical study

In this study, we develop and test a strategy for selectively sizing (multiplying by an appropriate scalar) the approximate Hessian matrix before it is updated in the BFGS and DFP trust-region methods for unconstrained optimization. Our numerical results imply that, for use with the DFP update, the Oren-Luenberger sizing factor is completely satisfactory and selective sizing is vastly superior to the alternatives of never sizing or first-iteration sizing and is slightly better than the alternative of always sizing. Numerical experimentation showed that the Oren-Luenberger sizing factor is not a satisfactory sizing factor for use with the BFGS update. Therefore, based on our newly acquired understanding of the situation, we propose a centered Oren-Luenberger sizing factor to be used with the BFGS update. Our numerical experimentation implies that selectively sizing the BFGS update with the centered Oren-Luenberger sizing factor is superior to the alternatives. These results contradict the folk axiom that sizing should be done only at the first iteration. They also show that, without sufficient sizing, DFP is vastly inferior to BFGS; however, when selectively sized, DFP is competitive with BFGS.This research was supported in part by NSF Cooperative Agreement No. CCR-88-09615, AFOSR Grant 89-0363, DOE Grant DEFG05-86-ER25017, and AR0 Grant 9DAAL03-90-G-0093. This paper was presented at the ICIAM 91 International Conference, Washington DC, July 1991.The authors thank all those individuals who took part in the lively discussions concerning this material following the ICIAM 91 presentation. These discussions influenced the current version of the paper. The first author also thanks Maria Cristina Maciel for assistance and support during the earlier stages of the research.This author is currently a Graduate Student in the Mathematics Department, University of California at Riverside, Riverside, California, and has participated in the Summer Student Visitor Program at the Center for Research in Parallel Computation of Rice University for the past two years.     

Efficient generalized conjugate gradient algorithms,part 2: Implementation

In Part 1 of this paper (Ref. 1), a new, generalized conjugate gradient algorithm was proposed and its convergence investigated. In this second part, the new algorithm is compared numerically with other modified conjugate gradient methods and with limited-memory quasi-Newton methods.