Comment on “A new three-term conjugate gradient method for unconstrained problem”

Here, necessary corrections on computing an optimal parameter of the TTDES method of Andrei (Numer. Math. 68(2), 305–321 2015) are stated in brief. Throughout, we use the same notations and equation numbers as in N.Andrei.     

A modified Polak–Ribière–Polyak conjugate gradient algorithm for unconstrained optimization

A modified Polak–Ribière–Polyak conjugate gradient algorithm which satisfies both the sufficient descent condition and the conjugacy condition is presented. These properties are independent of the line search. The algorithms use the standard Wolfe line search. Under standard assumptions, we show the global convergence of the algorithm. Numerical comparisons with conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that this computational scheme outperforms the known Polak–Ribière–Polyak algorithm, as well as some other unconstrained optimization algorithms.     

A new predictor–corrector method for solving unconstrained minimization problems

This paper presents a new predictor–corrector method for finding a local minimum of a twice continuously differentiable function. The method successively constructs an approximation to the solution curve and determines a predictor on it using a technique similar to that used in trust region methods for unconstrained optimization. The proposed predictor is expected to be more effective than Euler's predictor in the sense that the former is usually much closer to the solution curve than the latter for the same step size. Results of numerical experiments are reported to demonstrate the effectiveness of the proposed method.     

A modified conjugate gradient algorithm with cyclic Barzilai–Borwein steplength for unconstrained optimization

For solving large-scale unconstrained minimization problems, the nonlinear conjugate gradient method is welcome due to its simplicity, low storage, efficiency and nice convergence properties. Among all the methods in the framework, the conjugate gradient descent algorithm — CG_DESCENT is very popular, in which the generated directions descend automatically, and this nice property is independent of any line search used. In this paper, we generalize CG_DESCENT with two Barzilai–Borwein steplength reused cyclically. We show that the resulting algorithm owns attractive sufficient descent property and converges globally under some mild conditions. We test the proposed algorithm by using a large set of unconstrained problems with high dimensions in CUTEr library. The numerical comparisons with the state-of-the-art algorithm CG_DESCENT illustrate that the proposed method is effective, competitive, and promising.     

An adaptive nonmonotone global Barzilai–Borwein gradient method for unconstrained optimization

The Barzilai–Borwein (BB) gradient method has received many studies due to its simplicity and numerical efficiency. By incorporating a nonmonotone line search, Raydan (SIAM J Optim. 1997;7:26–33) has successfully extended the BB gradient method for solving general unconstrained optimization problems so that it is competitive with conjugate gradient methods. However, the numerical results reported by Raydan are poor for very ill-conditioned problems because the effect of the degree of nonmonotonicity may be noticeable. In this paper, we focus more on the nonmonotone line search technique used in the global Barzilai–Borwein (GBB) gradient method. We improve the performance of the GBB gradient method by proposing an adaptive nonmonotone line search based on the morphology of the objective function. We also prove the global convergence and the R-linear convergence rate of the proposed method under reasonable assumptions. Finally, we give some numerical experiments made on a set of unconstrained optimization test problems of the CUTEr collection. The results show the efficiency of the proposed method in the sense of the performance profile introduced (Math Program. 2002;91:201–213) by Dolan and Moré.     

The Dai–Liao nonlinear conjugate gradient method with optimal parameter choices

Minimizing two different upper bounds of the matrix which generates search directions of the nonlinear conjugate gradient method proposed by Dai and Liao, two modified conjugate gradient methods are proposed. Under proper conditions, it is briefly shown that the methods are globally convergent when the line search fulfills the strong Wolfe conditions. Numerical comparisons between the implementations of the proposed methods and the conjugate gradient methods proposed by Hager and Zhang, and Dai and Kou, are made on a set of unconstrained optimization test problems of the CUTEr collection. The results show the efficiency of the proposed methods in the sense of the performance profile introduced by Dolan and Moré.     

An Eulerian–Lagrangian method for optimization problems governed by multidimensional nonlinear hyperbolic PDEs

We present a numerical method for solving tracking-type optimal control problems subject to scalar nonlinear hyperbolic balance laws in one and two space dimensions. Our approach is based on the formal optimality system and requires numerical solutions of the hyperbolic balance law forward in time and its nonconservative adjoint equation backward in time. To this end, we develop a hybrid method, which utilizes advantages of both the Eulerian finite-volume central-upwind scheme (for solving the balance law) and the Lagrangian discrete characteristics method (for solving the adjoint transport equation). Experimental convergence rates as well as numerical results for optimization problems with both linear and nonlinear constraints and a duct design problem are presented.     

A Barzilai–Borwein type method for stochastic linear complementarity problems

We consider the expected residual minimization (ERM) formulation of stochastic linear complementarity problem (SLCP). By employing the Barzilai–Borwein (BB) stepsize and active set strategy, we present a BB type method for solving the ERM problem. The global convergence of the proposed method is proved under mild conditions. Preliminary numerical results show that the method is promising.     

A reassessment of the Snyman–Fatti dynamic search trajectory method for unconstrained global optimization

The aim of this paper is to present a thorough reassessment of the Snyman–Fatti (SF) Multi-start Global Minimization Algorithm with Dynamic Search Trajectories, first published twenty years ago. The reassessment is done with reference to a slightly modified version of the original method, the essentials of which are summarized here. Results of the performance of the current code on an extensive set of standard test problems commonly in use today, are presented. This allows for a fair assessment to be made of the performance of the SF algorithm relative to that of the popular Differential Evolution (DE) method, for which test results on the same standard set of test problems used here for the SF algorithm, are also given. The tests show that the SF algorithm, that requires relatively few parameter settings, is a reliably robust and competitive method compared to the DE method. The results also indicate that the SF trajectory algorithm is particularly promising to solve minimum potential energy problems to determine the structure of atomic and molecular clusters.     

Convergence of a projected gradient method with trust region for nonlinear constrained optimization†

In this paper we describe a projected gradient algorithm with trust region, introducing a nondifferentiable merit function for solving nonlinear constrained optimization problems. We show that this method is globally convergent even if conditions are weak. It is also proved that, when the strict complementarity condition holds, the proposed algorithm can be solved by an equality constrained problem, allowing locally rate of superlinear convergence.     

A modified three–term conjugate gradient method with sufficient descent property

A hybridization of the three–term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Polak and Ribi`ere, and Polyak is suggested. Based on an eigenvalue analysis, it is shown that search directions of the proposed method satisfy the sufficient descent condition, independent of the line search and the objective function convexity. Global convergence of the method is established under an Armijo–type line search condition. Numerical experiments show practical efficiency of the proposed method.     

A hybrid conjugate gradient method based on a quadratic relaxation of the Dai–Yuan hybrid conjugate gradient parameter

To take advantage of the attractive features of the Hestenes–Stiefel and Dai–Yuan conjugate gradient (CG) methods, we suggest a hybridization of these methods using a quadratic relaxation of a hybrid CG parameter proposed by Dai and Yuan. In the proposed method, the hybridization parameter is computed based on a conjugacy condition. Under proper conditions, we show that our method is globally convergent for uniformly convex functions. We give a numerical comparison of the implementations of our method and two efficient hybrid CG methods proposed by Dai and Yuan using a set of unconstrained optimization test problems from the CUTEr collection. Numerical results show the efficiency of the proposed hybrid CG method in the sense of the performance profile introduced by Dolan and Moré.     

Scalarization method for Levitin–Polyak well-posedness of vectorial optimization problems

In this paper, we develop a method of study of Levitin?CPolyak well-posedness notions for vector valued optimization problems using a class of scalar optimization problems. We first introduce a non-linear scalarization function and consider its corresponding properties. We also introduce the Furi?CVignoli type measure and Dontchev?CZolezzi type measure to scalar optimization problems and vectorial optimization problems, respectively. Finally, we construct the equivalence relations between the Levitin?CPolyak well-posedness of scalar optimization problems and the vectorial optimization problems.     

A Jacobi–Davidson like method for nonlinear eigenvalue problems based on singularity theory

We present a Jacobi–Davidson like correction formula for left and right eigenvector approximations for non-Hermitian nonlinear eigenvalue problems. It exploits techniques from singularity theory for characterizing singular points of nonlinear equations. Unlike standard nonlinear Jacobi-Davidson, the correction formula does not contain derivative information and works with orthogonal projectors only. Moreover, the basic method is modified in that the new eigenvalue approximation is taken as a nonlinear Rayleigh functional obtained as root of a certain scalar nonlinear equation the existence of which – as well as a first order perturbation expansion – is shown. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A projected–gradient interior–point algorithm for complementarity problems

Interior–point algorithms are among the most efficient techniques for solving complementarity problems. In this paper, a procedure for globalizing interior–point algorithms by using the maximum stepsize is introduced. The algorithm combines exact or inexact interior–point and projected–gradient search techniques and employs a line–search procedure for the natural merit function associated with the complementarity problem. For linear problems, the maximum stepsize is shown to be acceptable if the Newton interior–point search direction is employed. Complementarity and optimization problems are discussed, for which the algorithm is able to process by either finding a solution or showing that no solution exists. A modification of the algorithm for dealing with infeasible linear complementarity problems is introduced which, in practice, employs only interior–point search directions. Computational experiments on the solution of complementarity problems and convex programming problems by the new algorithm are included.     

A note on the Douglas–Rachford splitting method for optimization problems involving hypoconvex functions

Recently, the convergence of the Douglas–Rachford splitting method (DRSM) was established for minimizing the sum of a nonsmooth strongly convex function and a nonsmooth hypoconvex function under the assumption that the strong convexity constant \(\beta \) is larger than the hypoconvexity constant \(\omega \). Such an assumption, implying the strong convexity of the objective function, precludes many interesting applications. In this paper, we prove the convergence of the DRSM for the case \(\beta =\omega \), under relatively mild assumptions compared with some existing work in the literature.     

Successive unconstrained dual optimization method for?rank-one approximation to tensors

A successive unconstrained dual optimization (SUDO) method is developed to solve the high order tensors?? best rank-one approximation problems, in the least-squares sense. The constrained dual program of tensors?? rank-one approximation is transformed into a sequence of unconstrained optimization problems, for where a fast gradient method is proposed. We introduce the steepest ascent direction, a initial step length strategy and a backtracking line search rule for each iteration. A proof of the global convergence of the SUDO algorithm is given. Preliminary numerical experiments show that our method outperforms the alternating least squares (ALS) method.     

Testing optimality for quadratic 0–1 unconstrained problems

This paper analyses a necessary and sufficient optimality condition for quadratic pseudo-Boolean unconstrained problems. It is proved that in general testing any necessary and sufficient optimality condition is a difficult task for anyNP-hard problem. An-optimality condition is derived together with an approximation scheme to test it.This work has been supported by the Progetto Finalizzato Trasporti 2, 93.01799.PF74.Professor Paolo Carraresi died unexpectedly on March 5, 1994. At the time of his death this paper had been completed. While undertaking the final revision, the other two authors were reminded just how much they were indebted to Professor Carraresi after many years of common work together.