A bundle type approach to the unconstrained minimization of convex nonsmooth functions

A numerical method for the unconstrained minimization of a convex nonsmooth function of several variables is presented. It is closely related to the bundle type approach and to the conjugate subgradient method. A way is suggested to reduce the amount of information to be stored during the computational procedure. Global convergence of the method to the minimum is proved.     

A globally convergent descent method for nonsmooth variational inequalities

We propose a descent method via gap functions for solving nonsmooth variational inequalities with a locally Lipschitz operator. Assuming monotone operator (not necessarily strongly monotone) and bounded domain, we show that the method with an Armijo-type line search is globally convergent. Finally, we report some numerical experiments. This work has been supported by the National Research Program PRIN/2005017083 “Innovative Problems and Methods in Nonlinear Optimization”.     

A generalized steepest descent method for continuously subdifferentiable functions

A class of methods for unconstrained minimization of quasidifferentiable, especially subdifferentiable functions is described, which includes well-known algorithms as special cases. Moreover, it is shown that an algorithm previously published fails to converge to an e-inf-stationary point in general. Some preliminary numerical results are reported on.     

On the superlinear convergence of a trust region algorithm for nonsmooth optimization

It is proved that the second order correction trust region algorithm of Fletcher [5] ensures superlinear convergence if some mild conditions are satisfied.     

Fast proximal algorithms for nonsmooth convex optimization

In the lines of our previous approach to devise proximal algorithms for nonsmooth convex optimization by applying Nesterov fast gradient concept to the Moreau–Yosida regularization of a convex function, we develop three new proximal algorithms for nonsmooth convex optimization. In these algorithms, the errors in computing approximate solutions for the Moreau–Yosida regularization are not fixed beforehand, while preserving the complexity estimates already established. We report some preliminary computational results to give a first estimate of their performance.     

A trust region algorithm for nonsmooth optimization

A trust region algorithm is proposed for minimizing the nonsmooth composite functionF(x) = h(f(x)), wheref is smooth andh is convex. The algorithm employs a smoothing function, which is closely related to Fletcher's exact differentiable penalty functions. Global and local convergence results are given, considering convergence to a strongly unique minimizer and to a minimizer satisfying second order sufficiency conditions.     

A descent method with linear programming subproblems for nondifferentiable convex optimization

Most of the descent methods developed so far suffer from the computational burden due to a sequence of constrained quadratic subproblems which are needed to obtain a descent direction. In this paper we present a class of proximal-type descent methods with a new direction-finding subproblem. Especially, two of them have a linear programming subproblem instead of a quadratic subproblem. Computational experience of these two methods has been performed on two well-known test problems. The results show that these methods are another very promising approach for nondifferentiable convex optimization.     

Constraint aggregation principle in convex optimization

A general constraint aggregation technique is proposed for convex optimization problems. At each iteration a set of convex inequalities and linear equations is replaced by a single surrogate inequality formed as a linear combination of the original constraints. After solving the simplified subproblem, new aggregation coefficients are calculated and the iteration continues. This general aggregation principle is incorporated into a number of specific algorithms. Convergence of the new methods is proved and speed of convergence analyzed. Next, dual interpretation of the method is provided and application to decomposable problems is discussed. Finally, a numerical illustration is given.     

The CoMirror algorithm for solving nonsmooth constrained convex problems

We introduce a first-order Mirror-Descent (MD) type algorithm for solving nondifferentiable convex problems having a combination of simple constraint set X (ball, simplex, etc.) and an additional functional constraint. The method is tuned to exploit the structure of X by employing an appropriate non-Euclidean distance-like function. Convergence results and efficiency estimates are derived. The performance of the algorithm is demonstrated by solving certain image deblurring problems.     

Merit functions for nonsmooth complementarity problems and related descent algorithm

Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal with nonlinear complementarity problem where the underlying function is assumed to be a continuous but not necessarily locally Lipschitzian map and gives a descent algorithm for solving the nonsmooth continuous complementarity problems. In addition, the global convergence of the derivative free descent algorithm is also proved.     

Strongly sub-feasible direction method for constrained optimization problems with nonsmooth objective functions

In this paper, we propose a strongly sub-feasible direction method for the solution of inequality constrained optimization problems whose objective functions are not necessarily differentiable. The algorithm combines the subgradient aggregation technique with the ideas of generalized cutting plane method and of strongly sub-feasible direction method, and as results a new search direction finding subproblem and a new line search strategy are presented. The algorithm can not only accept infeasible starting points but also preserve the “strong sub-feasibility” of the current iteration without unduly increasing the objective value. Moreover, once a feasible iterate occurs, it becomes automatically a feasible descent algorithm. Global convergence is proved, and some preliminary numerical results show that the proposed algorithm is efficient.     

Conditions for convergence of trust region algorithms for nonsmooth optimization

This paper discusses some properties of trust region algorithms for nonsmooth optimization. The problem is expressed as the minimization of a functionh(f(x), whereh(·) is convex andf is a continuously differentiable mapping from ℝ″ to ℝ‴. Bounds for the second order derivative approximation matrices are discussed. It is shown that Powel’s [7, 8] results hold for nonsmooth optimization.     

A descent algorithm without line search for unconstrained optimization

In this paper, a new descent algorithm for solving unconstrained optimization problem is presented. Its search direction is descent and line search procedure can be avoided except for the first iteration. It is globally convergent under mild conditions. The search direction of the new algorithm is generalized and convergence of corresponding algorithm is also proved. Numerical results show that the algorithm is efficient for given test problems.     

Multiple subgradient descent bundle method for convex nonsmooth multiobjective optimization

The aim of this paper is to propose a new multiple subgradient descent bundle method for solving unconstrained convex nonsmooth multiobjective optimization problems. Contrary to many existing multiobjective optimization methods, our method treats the objective functions as they are without employing a scalarization in a classical sense. The main idea of this method is to find descent directions for every objective function separately by utilizing the proximal bundle approach, and then trying to form a common descent direction for every objective function. In addition, we prove that the method is convergent and it finds weakly Pareto optimal solutions. Finally, some numerical experiments are considered.     

An algorithm for composite nonsmooth optimization problems

Nonsmooth optimization problems are divided into two categories. The first is composite nonsmooth problems where the generalized gradient can be approximated by information available at the current point. The second is basic nonsmooth problems where the generalized gradient must be approximated using information calculated at previous iterates.Methods for minimizing composite nonsmooth problems where the nonsmooth function is made up from a finite number of smooth functions, and in particular max functions, are considered. A descent method which uses an active set strategy, a nonsmooth line search, and a quasi-Newton approximation to the reduced Hessian of a Lagrangian function is presented. The Theoretical properties of the method are discussed and favorable numerical experience on a wide range of test problems is reported.This work was carried out at the University of Dundee from 1976–1979 and at the University of Kentucky at Lexington from 1979–1980. The provision of facilities in both universities is gratefully acknowledged, as well as the support of NSF Grant No. ECS-79-23272 for the latter period. The first author also wishes to acknowledge financial support from a George Murray Scholarship from the University of Adelaide and a University of Dundee Research Scholarship for the former period.     

An approximate subgradient algorithm for unconstrained nonsmooth,nonconvex optimization

In this paper a new algorithm for minimizing locally Lipschitz functions is developed. Descent directions in this algorithm are computed by solving a system of linear inequalities. The convergence of the algorithm is proved for quasidifferentiable semismooth functions. We present the results of numerical experiments with both regular and nonregular objective functions. We also compare the proposed algorithm with two different versions of the subgradient method using the results of numerical experiments. These results demonstrate the superiority of the proposed algorithm over the subgradient method.      

A superlinear space decomposition algorithm for constrained nonsmooth convex program

A class of constrained nonsmooth convex optimization problems, that is, piecewise C² convex objectives with smooth convex inequality constraints are transformed into unconstrained nonsmooth convex programs with the help of exact penalty function. The objective functions of these unconstrained programs are particular cases of functions with primal-dual gradient structure which has connection with VU space decomposition. Then a VU space decomposition method for solving this unconstrained program is presented. This method is proved to converge with local superlinear rate under certain assumptions. An illustrative example is given to show how this method works.     

Multiplier Rules and Saddle-Point Theorems for Helbig’s Approximate Solutions in Convex Pareto Problems

This paper deals with approximate Pareto solutions in convex multiobjective optimization problems. We relate two approximate Pareto efficiency concepts: one is already classic and the other is due to Helbig. We obtain Fritz John and Kuhn–Tucker type necessary and sufficient conditions for Helbig’s approximate solutions. An application we deduce saddle-point theorems corresponding to these solutions for two vector-valued Lagrangian functions.     

Mirror descent and nonlinear projected subgradient methods for convex optimization

The mirror descent algorithm (MDA) was introduced by Nemirovsky and Yudin for solving convex optimization problems. This method exhibits an efficiency estimate that is mildly dependent in the decision variables dimension, and thus suitable for solving very large scale optimization problems. We present a new derivation and analysis of this algorithm. We show that the MDA can be viewed as a nonlinear projected-subgradient type method, derived from using a general distance-like function instead of the usual Euclidean squared distance. Within this interpretation, we derive in a simple way convergence and efficiency estimates. We then propose an Entropic mirror descent algorithm for convex minimization over the unit simplex, with a global efficiency estimate proven to be mildly dependent in the dimension of the problem.     

A BFGS-IP algorithm for solving strongly convex optimization problems with feasibility enforced by an exact penalty approach

This paper introduces and analyses a new algorithm for minimizing a convex function subject to a finite number of convex inequality constraints. It is assumed that the Lagrangian of the problem is strongly convex. The algorithm combines interior point methods for dealing with the inequality constraints and quasi-Newton techniques for accelerating the convergence. Feasibility of the iterates is progressively enforced thanks to shift variables and an exact penalty approach. Global and q-superlinear convergence is obtained for a fixed penalty parameter; global convergence to the analytic center of the optimal set is ensured when the barrier parameter tends to zero, provided strict complementarity holds. Received: December 21, 2000 / Accepted: July 13, 2001?Published online February 14, 2002