New decomposition and convexification algorithm for nonconvex large-scale primal-dual optimization

A new algorithm for solving nonconvex, equality-constrained optimization problems with separable structures is proposed in the present paper. A new augmented Lagrangian function is derived, and an iterative method is presented. The new proposed Lagrangian function preserves separability when the original problem is separable, and the property of linear convergence of the new algorithm is also presented. Unlike earlier algorithms for nonconvex decomposition, the convergence ratio for this method can be made arbitrarily small. Furthermore, it is feasible to extend this method to algorithms suited for inequality-constrained optimization problems. An example is included to illustrate the method.This research was supported in part by the National Science Foundation under NSF Grant No. ECS-85-06249.     

Convexification procedures and decomposition methods for nonconvex optimization problems

In order for primal-dual methods to be applicable to a constrained minimization problem, it is necessary that restrictive convexity conditions are satisfied. In this paper, we consider a procedure by means of which a nonconvex problem is convexified and transformed into one which can be solved with the aid of primal-dual methods. Under this transformation, separability of the type necessary for application of decomposition algorithms is preserved. This feature extends the range of applicability of such algorithms to nonconvex problems. Relations with multiplier methods are explored with the aid of a local version of the notion of a conjugate convex function.This work was carried out at the Coordinated Science Laboratory, University of Illinois, Urbana, Illinois, and was supported by the National Science Foundation under Grant ENG 74-19332.     

Two-level primal–dual proximal decomposition technique to solve large scale optimization problems

We describe a primal–dual application of the proximal point algorithm to nonconvex minimization problems. Motivated by the work of Spingarn and more recently by the work of Hamdi et al. about the primal resource-directive decomposition scheme to solve nonlinear separable problems. This paper discusses some local results of a primal–dual regularization approach that leads to a decomposition algorithm.     

Zero duality gap for a class of nonconvex optimization problems

By an equivalent transformation using thepth power of the objective function and the constraint, a saddle point can be generated for a general class of nonconvex optimization problems. Zero duality gap is thus guaranteed when the primal-dual method is applied to the constructed equivalent form.The author very much appreciates the comments from Prof. Douglas J. White.     

A primal-dual trust region algorithm for nonlinear optimization

This paper concerns general (nonconvex) nonlinear optimization when first and second derivatives of the objective and constraint functions are available. The proposed method is based on finding an approximate solution of a sequence of unconstrained subproblems parameterized by a scalar parameter. The objective function of each unconstrained subproblem is an augmented penalty-barrier function that involves both primal and dual variables. Each subproblem is solved using a second-derivative Newton-type method that employs a combined trust region and line search strategy to ensure global convergence. It is shown that the trust-region step can be computed by factorizing a sequence of systems with diagonally-modified primal-dual structure, where the inertia of these systems can be determined without recourse to a special factorization method. This has the benefit that off-the-shelf linear system software can be used at all times, allowing the straightforward extension to large-scale problems. Numerical results are given for problems in the COPS test collection.Mathematics Subject Classification (2000): 49M37, 65F05, 65K05, 90C30This paper is dedicated to Roger Fletcher on the occasion of his 65th birthday     

A primal-dual Newton-type algorithm for geometric programs with equality constraints

An augmented Lagrangian algorithm is used to find local solutions of geometric programming problems with equality constraints (GPE). The algorithm is Newton's method for unconstrained minimization. The complexity of the algorithm is defined by the number of multiplications and divisions. By analyzing the algorithm we obtain results about the influence of each parameter in the GPE problem on the complexity of an iteration. An attempt is made to estimate the number of iterations needed for convergence. In practice, certain hypotheses are tested, such as the influence of the penalty coefficient update formula, the distance of the starting point from the optimum, and the stopping criterion. For these tests, a random problem generator was constructed, many problems were run, and the results were analyzed by statistical methods.The authors are grateful to Dr. J. Moré, Argonne National Laboratory for his valuable comments.This research was partially funded by the Fund for the Advancement of Research at the Technion and by the Applied Mathematical Sciences Research Program (KC-04-02), Office of Energy Research, US Department of Energy, Contract No. W-31-109-Eng-38.     

A primal-dual interior-point algorithm for nonlinear least squares constrained problems

This paper extends prior work by the authors on solving nonlinear least squares unconstrained problems using a factorized quasi-Newton technique. With this aim we use a primal-dual interior-point algorithm for nonconvex nonlinear programming. The factorized quasi-Newton technique is now applied to the Hessian of the Lagrangian function for the transformed problem which is based on a logarithmic barrier formulation. We emphasize the importance of establishing and maintaining symmetric quasi-definiteness of the reduced KKT system. The algorithm then tries to choose a step size that reduces a merit function, and to select a penalty parameter that ensures descent directions along the iterative process. Computational results are included for a variety of least squares constrained problems and preliminary numerical testing indicates that the algorithm is robust and efficient in practice.     

Augmented Lagrangian method for distributed optimal control problems with state constraints

We consider state-constrained optimal control problems governed by elliptic equations. Doing Slater-like assumptions, we know that Lagrange multipliers exist for such problems, and we propose a decoupled augmented Lagrangian method. We present the algorithm with a simple example of a distributed control problem.     

A convergent decomposition method for box-constrained optimization problems

In this work we consider the problem of minimizing a continuously differentiable function over a feasible set defined by box constraints. We present a decomposition method based on the solution of a sequence of subproblems. In particular, we state conditions on the rule for selecting the subproblem variables sufficient to ensure the global convergence of the generated sequence without convexity assumptions. The conditions require to select suitable variables (related to the violation of the optimality conditions) to guarantee theoretical convergence properties, and leave the degree of freedom of selecting any other group of variables to accelerate the convergence.     

Local search in problems with nonconvex constraints

Nonconvex optimization problems with an inequality constraint given by the difference of two convex functions (by a d.c. function) are considered. Two methods for finding local solutions to this problem are proposed that combine the solution of partially linearized problems and descent to a level surface of the d.c. function. The convergence of the methods is analyzed, and stopping criterions are proposed. The methods are compared by testing them in a numerical experiment.     

A new augmented Lagrangian function for inequality constraints in nonlinear programming problems

In this paper, a new augmented Lagrangian function is introduced for solving nonlinear programming problems with inequality constraints. The relevant feature of the proposed approach is that, under suitable assumptions, it enables one to obtain the solution of the constrained problem by a single unconstrained minimization of a continuously differentiable function, so that standard unconstrained minimization techniques can be employed. Numerical examples are reported.     

Vector optimization problems with quasiconvex constraints

Let X be a real linear space, a convex set, Y and Z topological real linear spaces. The constrained optimization problem min _C f(x), is considered, where f : X ₀→ Y and g : X ₀→ Z are given (nonsmooth) functions, and and are closed convex cones. The weakly efficient solutions (w-minimizers) of this problem are investigated. When g obeys quasiconvex properties, first-order necessary and first-order sufficient optimality conditions in terms of Dini directional derivatives are obtained. In the special case of problems with pseudoconvex data it is shown that these conditions characterize the global w-minimizers and generalize known results from convex vector programming. The obtained results are applied to the special case of problems with finite dimensional image spaces and ordering cones the positive orthants, in particular to scalar problems with quasiconvex constraints. It is shown, that the quasiconvexity of the constraints allows to formulate the optimality conditions using the more simple single valued Dini derivatives instead of the set valued ones.      

A novel neural network based on NCP function for solving constrained nonconvex optimization problems

This article presents a novel neural network (NN) based on NCP function for solving nonconvex nonlinear optimization (NCNO) problem subject to nonlinear inequality constraints. We first apply the p‐power convexification of the Lagrangian function in the NCNO problem. The proposed NN is a gradient model which is constructed by an NCP function and an unconstrained minimization problem. The main feature of this NN is that its equilibrium point coincides with the optimal solution of the original problem. Under a proper assumption and utilizing a suitable Lyapunov function, it is shown that the proposed NN is Lyapunov stable and convergent to an exact optimal solution of the original problem. Finally, simulation results on two numerical examples and two practical examples are given to show the effectiveness and applicability of the proposed NN. © 2015 Wiley Periodicals, Inc. Complexity 21: 130–141, 2016     

Two-stage stochastic optimization problems with stochastic ordering constraints on the recourse

We consider two-stage risk-averse stochastic optimization problems with a stochastic ordering constraint on the recourse function. Two new characterizations of the increasing convex order relation are provided. They are based on conditional expectations and on integrated quantile functions: a counterpart of the Lorenz function. We propose two decomposition methods to solve the problems and prove their convergence. Our methods exploit the decomposition structure of the risk-neutral two-stage problems and construct successive approximations of the stochastic ordering constraints. Numerical results confirm the efficiency of the methods.     

An alternating structured trust region algorithm for separable optimization problems with nonconvex constraints

In this paper, we propose a structured trust-region algorithm combining with filter technique to minimize the sum of two general functions with general constraints. Specifically, the new iterates are generated in the Gauss-Seidel type iterative procedure, whose sizes are controlled by a trust-region type parameter. The entries in the filter are a pair: one resulting from feasibility; the other resulting from optimality. The global convergence of the proposed algorithm is proved under some suitable assumptions. Some preliminary numerical results show that our algorithm is potentially efficient for solving general nonconvex optimization problems with separable structure.     

A quadratically convergent primal-dual algorithm with global convergence properties for solving optimization problems with equality constraints

This paper presents a globally convergent multiplier method which utilizes an explicit formula for the multiplier. The algorithm solves finite dimensional optimization problems with equality constraints. A unique feature of the algorithm is that it automatically calculates a value for the penalty coefficient, which, under certain assumptions, leads to global convergence.Research sponsored by the Joint Services Electronics Program, Contract F44620-71-C-0087 and the National Science Foundation, Grant GK-37672.     

An eigenvalue decomposition based branch-and-bound algorithm for nonconvex quadratic programming problems with convex quadratic constraints

In this paper, we propose a branch-and-bound algorithm for finding a global optimal solution for a nonconvex quadratic program with convex quadratic constraints (NQPCQC). We first reformulate NQPCQC by adding some nonconvex quadratic constraints induced by eigenvectors of negative eigenvalues associated with the nonconvex quadratic objective function to Shor’s semidefinite relaxation. Under the assumption of having a bounded feasible domain, these nonconvex quadratic constraints can be further relaxed into linear ones to form a special semidefinite programming relaxation. Then an efficient branch-and-bound algorithm branching along the eigendirections of negative eigenvalues is designed. The theoretic convergence property and the worst-case complexity of the proposed algorithm are proved. Numerical experiments are conducted on several types of quadratic programs to show the efficiency of the proposed method.     

Bucket elimination for multiobjective optimization problems

Multiobjective optimization deals with problems involving multiple measures of performance that should be optimized simultaneously. In this paper we extend bucket elimination (BE), a well known dynamic programming generic algorithm, from mono-objective to multiobjective optimization. We show that the resulting algorithm, MO-BE, can be applied to true multi-objective problems as well as mono-objective problems with knapsack (or related) global constraints. We also extend mini-bucket elimination (MBE), the approximation form of BE, to multiobjective optimization. The new algorithm MO-MBE can be used to obtain good quality multi-objective lower bounds or it can be integrated into multi-objective branch and bound in order to increase its pruning efficiency. Its accuracy is empirically evaluated in real scheduling problems, as well as in Max-SAT-ONE and biobjective weighted minimum vertex cover problems.     

Infeasible-start primal-dual methods and infeasibility detectors for nonlinear programming problems

ln) iterations, where ν is the parameter of a self-concordant barrier for the cone, ε is a relative accuracy and ρ_f is a feasibility measure. We also discuss the behavior of path-following methods as applied to infeasible problems. We prove that strict infeasibility (primal or dual) can be detected in O(ln) iterations, where ρ_· is a primal or dual infeasibility measure. Received April 25, 1996 / Revised version received March 4, 1998 Published online October 9, 1998     

Combining Lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems

A new algorithm for solving large-scale convex optimization problems with a separable objective function is proposed. The basic idea is to combine three techniques: Lagrangian dual decomposition, excessive gap and smoothing. The main advantage of this algorithm is that it automatically and simultaneously updates the smoothness parameters which significantly improves its performance. The convergence of the algorithm is proved under weak conditions imposed on the original problem. The rate of convergence is $O(\frac {1}{k})$ , where k is the iteration counter. In the second part of the paper, the proposed algorithm is coupled with a dual scheme to construct a switching variant in a dual decomposition framework. We discuss implementation issues and make a theoretical comparison. Numerical examples confirm the theoretical results.