Two New Dai–Liao-Type Conjugate Gradient Methods for Unconstrained Optimization Problems

In this paper, we present two new Dai–Liao-type conjugate gradient methods for unconstrained optimization problems. Their convergence under the strong Wolfe line search conditions is analysed for uniformly convex objective functions and general objective functions, respectively. Numerical experiments show that our methods can outperform some existing Dai–Liao-type methods by using Dolan and Moré’s performance profile.     

Hybrid Conjugate Gradient Method for a Convex Optimization Problem over the Fixed-Point Set of a Nonexpansive Mapping

The main aim of the paper is to accelerate the existing method for a convex optimization problem over the fixed-point set of a nonexpansive mapping. To achieve this goal, we present an algorithm (Algorithm 3.1) by using the conjugate gradient direction. We present also a convergence analysis (Theorem 3.1) under some assumptions. Finally, to demonstrate the effectiveness and performance of the proposed method, we present numerical comparisons of the existing method with the proposed method.     

New Hybrid Conjugate Gradient and Broyden–Fletcher–Goldfarb–Shanno Conjugate Gradient Methods

Three hybrid methods for solving unconstrained optimization problems are introduced. These methods are defined using proper combinations of the search directions and included parameters in conjugate gradient and quasi-Newton methods. The convergence of proposed methods with the underlying backtracking line search is analyzed for general objective functions and particularly for uniformly convex objective functions. Numerical experiments show the superiority of the proposed methods with respect to some existing methods in view of the Dolan and Moré’s performance profile.     

A Modified Nonmonotone Hestenes–Stiefel Type Conjugate Gradient Methods for Large-Scale Unconstrained Problems

In this article, by slightly modifying the search direction of the nonmonotone Hestenes–Stiefel method, a variant Hestenes–Stiefel conjugate gradient method is proposed that satisfies the su?cient descent condition independent of any line search. This algorithm also possesses information about the gradient value and the function value. We establish the global convergence of our methods without the assumption that the steplength is bounded away from zero. Numerical results illustrate that our method can e?ciently solve the test problems, and therefore is promising.     

A New Nonmonotonic Trust Region Algorithm for A Class of Unconstrained Nonsmooth Optimization

This paper presents a new trust region algorithm for solving a class of composite nonsmooth optimizations. It is distinguished by the fact that this method does not enforce strict monotonicity of the objective function values at successive iterates and that this method extends the existing results for this type of nonlinear optimization with smooth, or piecewise smooth, or convex objective functions or their composition. It is proved that this algorithm is globally convergent under certain conditions. Finally, some numerical results for several optimization problems are reported which show that the nonmonotonic trust region method is competitive with the usual trust region method.     

Convergence Properties of the Regularized Newton Method for the Unconstrained Nonconvex Optimization

The regularized Newton method (RNM) is one of the efficient solution methods for the unconstrained convex optimization. It is well-known that the RNM has good convergence properties as compared to the steepest descent method and the pure Newton’s method. For example, Li, Fukushima, Qi and Yamashita showed that the RNM has a quadratic rate of convergence under the local error bound condition. Recently, Polyak showed that the global complexity bound of the RNM, which is the first iteration k such that ‖∇ f(x _k )‖≤ε, is O(ε ⁻⁴), where f is the objective function and ε is a given positive constant. In this paper, we consider a RNM extended to the unconstrained “nonconvex” optimization. We show that the extended RNM (E-RNM) has the following properties. (a) The E-RNM has a global convergence property under appropriate conditions. (b) The global complexity bound of the E-RNM is O(ε ⁻²) if ∇ ² f is Lipschitz continuous on a certain compact set. (c) The E-RNM has a superlinear rate of convergence under the local error bound condition.     

Subspace Barzilai-Borwein Gradient Method for Large-Scale Bound Constrained Optimization

An active set subspace Barzilai-Borwein gradient algorithm for large-scale bound constrained optimization is proposed. The active sets are estimated by an identification technique. The search direction consists of two parts: some of the components are simply defined; the other components are determined by the Barzilai-Borwein gradient method. In this work, a nonmonotone line search strategy that guarantees global convergence is used. Preliminary numerical results show that the proposed method is promising, and competitive with the well-known method SPG on a subset of bound constrained problems from CUTEr collection. This work was supported by the 973 project granted 2004CB719402 and the NSF project of China granted 10471036.     

Comments on “Surrogate Gradient Algorithm for Lagrangian Relaxation”

This note presents not only a surrogate subgradient method, but also a framework of surrogate subgradient methods. Furthermore, the framework can be used not only for separable problems, but also for coupled subproblems. The note delineates such a framework and shows that the algorithm can converges for a larger stepsize. The author thanks Professor Ching-An Lin from the Department of Electrical and Control Engineering of National Chiao Tung University, Hsinchu, Taiwan for valuable discussions.     

A Filter-Based Pattern Search Method for Unconstrained Optimization

We discuss a filter-based pattern search method for unconstrained optimization in this paper. For the purpose to broaden the search range we use both filter technique and frames, which are fragments of grids, to provide a new criterion of iterate acceptance. The convergence can be ensured under some conditions. The numerical result shows that this method is practical and efficient.     

The Global Convergence of Self-Scaling BFGS Algorithm with Nonmonotone Line Search for Unconstrained Nonconvex Optimization Problems

The self-scaling quasi-Newton method solves an unconstrained optimization problem by scaling the Hessian approximation matrix before it is updated at each iteration to avoid the possible large eigenvalues in the Hessian approximation matrices of the objective function. It has been proved in the literature that this method has the global and superlinear convergence when the objective function is convex （or even uniformly convex）. We propose to solve unconstrained nonconvex optimization problems by a self-scaling BFGS algorithm with nonmonotone linear search. Nonmonotone line search has been recognized in numerical practices as a competitive approach for solving large-scale nonlinear problems. We consider two different nonmonotone line search forms and study the global convergence of these nonmonotone self-scale BFGS algorithms. We prove that, under some weaker condition than that in the literature, both forms of the self-scaling BFGS algorithm are globally convergent for unconstrained nonconvex optimization problems.     

Convergence of Conjugate Gradient Methods with a Closed-Form Stepsize Formula

Conjugate gradient methods are efficient methods for minimizing differentiable objective functions in large dimension spaces. However, converging line search strategies are usually not easy to choose, nor to implement. Sun and colleagues (Ann. Oper. Res. 103:161–173, 2001; J. Comput. Appl. Math. 146:37–45, 2002) introduced a simple stepsize formula. However, the associated convergence domain happens to be overrestrictive, since it precludes the optimal stepsize in the convex quadratic case. Here, we identify this stepsize formula with one iteration of the Weiszfeld algorithm in the scalar case. More generally, we propose to make use of a finite number of iterates of such an algorithm to compute the stepsize. In this framework, we establish a new convergence domain, that incorporates the optimal stepsize in the convex quadratic case. The authors thank the associate editor and the reviewer for helpful comments and suggestions. C. Labat is now in postdoctoral position, Johns Hopkins University, Baltimore, MD, United States.     

A Simple Multistart Algorithm for Global Optimization

1.IntroductionConsidertheunconstrainedoptimizationproblem:findx*suchthatf(x*)~caf(x),(1)wheref(x)isanonlinearfllnctiondefinedonW"andXCR".Ourobjectiveistofindtheglobalminimizeroff(x)inthefeasibleset.Withoutassuminganyconditionsonf(x)globaloptimizationproblemsareunsolvableinthefollowingsensefnoalgorithmcanbeguaranteedtofindaglobalminimizerofageneralnonlinearfunctionwithinfinitelymanyiterations.Supposethatanalgorithmappliedtoanonlinearfunctionf(x)producesiteratesxlandterminatesafterKiterations.…     

A Restarted Conjugate Gradient Method for Ill—posed Problems

This paper presents a restarted conjugate gradient iterative algorithm for solving ill-posed problems.The damped Morozov‘s discrepancy principle is used as a stopping rule,Numerical experiments are given to illustrate the efficiency of the method.     

An Algorithm for the Global Optimization of a Class of Continuous Minimax Problems

We propose an algorithm for the global optimization of continuous minimax problems involving polynomials. The method can be described as a discretization approach to the well known semi-infinite formulation of the problem. We proceed by approximating the infinite number of constraints using tools and techniques from semidefinite programming. We then show that, under appropriate conditions, the SDP approximation converges to the globally optimal solution of the problem. We also discuss the numerical performance of the method on some test problems. Financial support of EPSRC Grant GR/T02560/01 gratefully acknowledged.     

Projected Gradient Type Method of Centers for Constrained Optimization

本文提出了一种求解约束优化问题的新算法—投影梯度型中心方法．在连续可微和非退化的假设条件下，证明了其全局收敛性．本文算法计算简单且形式灵活．     

Coupled Aerostructural Design Optimization Using the Kriging Model and Integrated Multiobjective Optimization Algorithm

The paper develops and implements a highly applicable framework for the computation of coupled aerostructural design optimization. The multidisciplinary aerostructural design optimization is carried out and validated for a tested wing and can be easily extended to complex and practical design problems. To make the framework practical, the study utilizes a high-fidelity fluid/structure interface and robust optimization algorithms for an accurate determination of the design with the best performance. The aerodynamic and structural performance measures, including the lift coefficient, the drag coefficient, Von-Mises stress and the weight of wing, are precisely computed through the static aeroelastic analyses of various candidate wings. Based on these calculated performance, the design system can be approximated by using a Kriging interpolative model. To improve the design evenly for aerodynamic and structure performance, an automatic design method that determines appropriate weighting factors is developed. Multidisciplinary aerostructural design is, therefore, desirable and practical. The authors acknowledge the support of a Korea Research Foundation Grant funded by the Korean Government and the second stage of Brain Korea 21st project.     

Convergence of a Hybrid Projection Algorithm in Banach Spaces

In this paper, we consider a hybrid projection algorithm for two families of quasi-φ-nonexpansive mappings. We establish strong convergence theorems of common fixed points in the framework of Banach spaces. Our results improve and extend the corresponding results announced by many others.     

Conjugate Gradient Methods with Armijo-type Line Searches

Abstract Two Armijo-type line searches are proposed in this paper for nonlinear conjugate gradient methods.Under these line searches, global convergence results are established for several famous conjugate gradientmethods, including the Fletcher-Reeves method, the Polak-Ribiere-Polyak method, and the conjugate descentmethod.     

The Substitution Secant/Finite Difference Method for Large Scale Sparse Unconstrained Optimization

This paper studies a substitution secant/finite difference （SSFD） method for solving large scale sparse unconstrained optimization problems. This method is a combination of a secant method and a finite difference method, which depends on a consistent partition of the columns of the lower triangular part of the Hessian matrix. A q-superlinear convergence result and an r-convergence rate estimate show that this method has good local convergence properties. The numerical results show that this method may be competitive with some currently used algorithms.