A Trust Region Method for Optimization Problem with Singular Solutions

In this paper, we propose a trust region method for minimizing a function whose Hessian matrix at the solutions may be singular. The global convergence of the method is obtained under mild conditions. Moreover, we show that if the objective function is LC ² function, the method possesses local superlinear convergence under the local error bound condition without the requirement of isolated nonsingular solution. This is the first regularized Newton method with trust region technique which possesses local superlinear (quadratic) convergence without the assumption that the Hessian of the objective function at the solution is nonsingular. Preliminary numerical experiments show the efficiency of the method. This work is partly supported by the National Natural Science Foundation of China (Grant Nos. 70302003, 10571106, 60503004, 70671100) and Science Foundation of Beijing Jiaotong University (2007RC014).     

An Example of Only Linear Convergence of Trust Region Algorithms for Non-smooth Optimization

Most superlinear convergence results about trust region algorithmsfor non-smooth optimization are dependent on the inactivityof trust region restrictions. An example is constructed to showthat it is possible that at every iteration the trust regionbound is active and the rate of convergence is only linear,though strict complementarity and second order sufficiency conditionsare satisfied. Presented at the 1983 Dundee Conference on Numerical Analysis     

A new modified nonmonotone adaptive trust region method for unconstrained optimization

In this paper, we present an adaptive trust region method for solving unconstrained optimization problems which combines nonmonotone technique with a new update rule for the trust region radius. At each iteration, our method can adjust the trust region radius of related subproblem. We construct a new ratio to adjust the next trust region radius which is different from the ratio in the traditional trust region methods. The global and superlinear convergence results of the method are established under reasonable assumptions. Numerical results show that the new method is efficient for unconstrained optimization problems.     

A Projected Indefinite Dogleg Path Method for Equality Constrained Optimization

In this paper, we propose a 2-step trust region indefinite dogleg path method for the solution of nonlinear equality constrained optimization problems. The method is a globally convergent modification of the locally convergent Fontecilla method and an indefinite dogleg path method is proposed to get approximate solutions of quadratic programming subproblems even if the Hessian in the model is indefinite. The dogleg paths lie in the null space of the Jacobian matrix of the constraints. An ₁ exact penalty function is used in the method to determine if a trial point is accepted. The global convergence and the local two-step superlinear convergence rate are proved. Some numerical results are presented.     

A Riemannian subspace limited-memory SR1 trust region method

In this paper, we present a new trust region algorithm on any compact Riemannian manifolds using subspace techniques. The global convergence of the method is proved and local \(d+1\)-step superlinear convergence of the algorithm is presented, where d is the dimension of the Riemannian manifold. Our numerical results show that the proposed subspace algorithm is competitive to some recent developed methods, such as the LRTR-SR1 method, the LRTR-BFGS method, the Riemannian CG method.     

A globally and superlinearly convergent primal-dual interior point trust region method for large scale constrained optimization

This paper proposes a primal-dual interior point method for solving large scale nonlinearly constrained optimization problems. To solve large scale problems, we use a trust region method that uses second derivatives of functions for minimizing the barrier-penalty function instead of line search strategies. Global convergence of the proposed method is proved under suitable assumptions. By carefully controlling parameters in the algorithm, superlinear convergence of the iteration is also proved. A nonmonotone strategy is adopted to avoid the Maratos effect as in the nonmonotone SQP method by Yamashita and Yabe. The method is implemented and tested with a variety of problems given by Hock and Schittkowskis book and by CUTE. The results of our numerical experiment show that the given method is efficient for solving large scale nonlinearly constrained optimization problems.Acknowledgement The authors would like to thank anonymous referees for their valuable comments to improve the paper.     

Combining nonmonotone conic trust region and line search techniques for unconstrained optimization

In this paper, we propose a trust region method for unconstrained optimization that can be regarded as a combination of conic model, nonmonotone and line search techniques. Unlike in traditional trust region methods, the subproblem of our algorithm is the conic minimization subproblem; moreover, our algorithm performs a nonmonotone line search to find the next iteration point when a trial step is not accepted, instead of resolving the subproblem. The global and superlinear convergence results for the algorithm are established under reasonable assumptions. Numerical results show that the new method is efficient for unconstrained optimization problems.     

A new trust region method for unconstrained optimization

In this paper, we propose a new trust region method for unconstrained optimization problems. The new trust region method can automatically adjust the trust region radius of related subproblems at each iteration and has strong global convergence under some mild conditions. We also analyze the global linear convergence, local superlinear and quadratic convergence rate of the new method. Numerical results show that the new trust region method is available and efficient in practical computation.     

An improved SQP algorithm for inequality constrained optimization

In this paper, the feasible type SQP method is improved. A new algorithm is proposed to solve nonlinear inequality constrained problem, in which a new modified method is presented to decrease the computational complexity. It is required to solve only one QP subproblem with only a subset of the constraints estimated as active per single iteration. Moreover, a direction is generated to avoid the Maratos effect by solving a system of linear equations. The theoretical analysis shows that the algorithm has global and superlinear convergence under some suitable conditions. In the end, numerical experiments are given to show that the method in this paper is effective.This work is supported by the National Natural Science Foundation (No. 10261001) and Guangxi Science Foundation (No. 0236001 and 0249003) of China. Acknowledgement.We would like to thank one anonymous referee for his valuable comments and suggestions, which greatly improved the quality of this paper.     

Local analysis of Newton-type methods for variational inequalities and nonlinear programming

This paper presents some new results in the theory of Newton-type methods for variational inequalities, and their application to nonlinear programming. A condition of semistability is shown to ensure the quadratic convergence of Newton's method and the superlinear convergence of some quasi-Newton algorithms, provided the sequence defined by the algorithm exists and converges. A partial extension of these results to nonsmooth functions is given. The second part of the paper considers some particular variational inequalities with unknowns (x, ), generalizing optimality systems. Here only the question of superlinear convergence of {x ^k } is considered. Some necessary or sufficient conditions are given. Applied to some quasi-Newton algorithms they allow us to obtain the superlinear convergence of {x ^k }. Application of the previous results to nonlinear programming allows us to strengthen the known results, the main point being a characterization of the superlinear convergence of {x ^k } assuming a weak second-order condition without strict complementarity.     

A NEW TRUST-REGION ALGORITHM FOR FINITE MINIMAX PROBLEM

In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different from the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the efficiency of the new method.     

一类带线搜索的非单调信赖域新算法

结合非单调信赖域方法,和非单调线搜索技术,提出了一种新的无约束优化算法.信赖域方法的每一步采用线搜索,使得迭代每一步都充分下降加快了迭代速度.在一定条件下,证明了算法具有全局收敛性和局部超线性.收敛速度.数值试验表明算法是十分有效的.     

关于不等式约束的信赖域算法

对于具有不等式约束的非线性优化问题，本文给出一个依赖域算法，由于算法中依赖区域约束采用向量的∞范数约束的形式，从而使子问题变二次规划，同时使算法变得更实用。在通常假设条件下，证明了算法的整体收敛性和超线性收敛性。     

A self-adaptive trust region method for the extended linear complementarity problems

By using some NCP functions, we reformulate the extended linear complementarity problem as a nonsmooth equation. Then we propose a self-adaptive trust region algorithm for solving this nonsmooth equation. The novelty of this method is that the trust region radius is controlled by the objective function value which can be adjusted automatically according to the algorithm. The global convergence is obtained under mild conditions and the local superlinear convergence rate is also established under strict complementarity conditions. This work is supported by National Natural Science Foundation of China (No. 10671126) and Shanghai Leading Academic Discipline Project (S30501).     

Convergence Rate of The Trust Region Method for Nonlinear Equations Under Local Error Bound Condition

In this paper, we present the new trust region method for nonlinear equations with the trust region converging to zero. The new method preserves the global convergence of the traditional trust region methods in which the trust region radius will be larger than a positive constant. We study the convergence rate of the new method under the local error bound condition which is weaker than the nonsingularity. An example given by Y.X. Yuan shows that the convergence rate can not be quadratic. Finally, some numerical results are given. This work is supported by Chinese NSFC grants 10401023 and 10371076, Research Grants for Young Teachers of Shanghai Jiao Tong University, and E-Institute of Computational Sciences of Shanghai Universities. An erratum to this article is available at .     

A new hybrid method for nonlinear complementarity problems

In this paper, a new hybrid method is proposed for solving nonlinear complementarity problems (NCP) with P ₀ function. In the new method, we combine a smoothing nonmonotone trust region method based on a conic model and line search techniques. We reformulate the NCP as a system of semismooth equations using the Fischer-Burmeister function. Using Kanzow’s smooth approximation function to construct the smooth operator, we propose a smoothing nonmonotone trust region algorithm of a conic model for solving the NCP with P ₀ functions. This is different from the classical trust region methods, in that when a trial step is not accepted, the method does not resolve the trust region subproblem but generates an iterative point whose steplength is defined by a line search. We prove that every accumulation point of the sequence generated by the algorithm is a solution of the NCP. Under a nonsingularity condition, the superlinear convergence of the algorithm is established without a strict complementarity condition.     

A nonmonotone trust region method based on nonincreasing technique of weighted average of the successive function values

In this paper we propose a nonmonotone trust region method. Unlike traditional nonmonotone trust region method, the nonmonotone technique applied to our method is based on the nonmonotone line search technique proposed by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14(4) (2004) 1043–1056] instead of that presented by Grippo et al. [A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23(4) (1986) 707–716]. So the method requires nonincreasing of a special weighted average of the successive function values. Global and superlinear convergence of the method are proved under suitable conditions. Preliminary numerical results show that the method is efficient for unconstrained optimization problems.     

Methods of conjugate directions versus quasi-Newton methods

It is shown that algorithms for minimizing an unconstrained functionF(x), x E ⁿ , which are solely methods of conjugate directions can be expected to exhibit only ann or (n–1) step superlinear rate of convergence to an isolated local minimizer. This is contrasted with quasi-Newton methods which can be expected to exhibit every step superlinear convergence. Similar statements about a quadratic rate of convergence hold when a Lipschitz condition is placed on the second derivatives ofF(x). Research was supported in part by Army Research Office, Contract Number DAHC 19-69-C-0017 and the Office of Naval Research, Contract Number N00014-71-C-0116 (NR 047-99).     

A new adaptive trust region algorithm for optimization problems

It is well known that trust region methods are very effective for optimization problems. In this article, a new adaptive trust region method is presented for solving unconstrained optimization problems. The proposed method combines a modified secant equation with the BFGS updated formula and an adaptive trust region radius, where the new trust region radius makes use of not only the function information but also the gradient information. Under suitable conditions, global convergence is proved, and we demonstrate the local superlinear convergence of the proposed method. The numerical results indicate that the proposed method is very efficient.     

A new successive quadratic programming algorithm

The main difficulties encountered in the successive quadratic programming methods are the choice of penalty parameter, the choice of steplenth, and the Maratos effect. An algorithm without penalty parameters is presented in this paper. The choice of steplength parameters is based on the method of trust region. Global convergence and local superlinear convergence are proved under suitable assumption.