Trust region affine scaling algorithms for linearly constrained convex and concave programs

We study a trust region affine scaling algorithm for solving the linearly constrained convex or concave programming problem. Under primal nondegeneracy assumption, we prove that every accumulation point of the sequence generated by the algorithm satisfies the first order necessary condition for optimality of the problem. For a special class of convex or concave functions satisfying a certain invariance condition on their Hessians, it is shown that the sequences of iterates and objective function values generated by the algorithm convergeR-linearly andQ-linearly, respectively. Moreover, under primal nondegeneracy and for this class of objective functions, it is shown that the limit point of the sequence of iterates satisfies the first and second order necessary conditions for optimality of the problem. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.The work of these authors was based on research supported by the National Science Foundation under grant INT-9600343 and the Office of Naval Research under grants N00014-93-1-0234 and N00014-94-1-0340.     

AN INTERIOR TRUST REGION ALGORITHM FOR NONLINEAR MINIMIZATION WITH LINEAR CONSTRAINTS

AbstractAn interior trust-region-based algorithm for linearly constrained minimization problems is proposed and analyzed. This algorithm is similar to trust region algorithms for unconstrained minimization: a trust region subproblem on a subspace is solved in each iteration. We establish that the proposed algorithm has convergence properties analogous to those of the trust region algorithms for unconstrained minimization. Namely, every limit point of the generated sequence satisfies the Krush-Kuhn-Tucker (KKT) conditions and at least one limit point satisfies second order necessary optimality conditions. In addition, if one limit point is a strong local minimizer and the Hessian is Lipschitz continuous in a neighborhood of that point, then the generated sequence converges globally to that point in the rate of at least 2-step quadratic. We are mainly concerned with the theoretical properties of the algorithm in this paper. Implementation issues and adaptation to large-scale problems will be addressed in a     

Convergence Analysis of a Class of Nonlinear Penalization Methods for Constrained Optimization via First-Order Necessary Optimality Conditions

We propose a scheme to solve constrained optimization problems by combining a nonlinear penalty method and a descent method. A sequence of nonlinear penalty optimization problems is solved to generate a sequence of stationary points, i.e., each point satisfies a first-order necessary optimality condition of a nonlinear penalty problem. Under some conditions, we show that any limit point of the sequence satisfies the first-order necessary condition of the original constrained optimization problem.     

Weak sharp solutions for variational inequalities in Banach spaces

In this paper, we introduce the notion of a weak sharp set of solutions to a variational inequality problem (VIP) in a reflexive, strictly convex and smooth Banach space, and present its several equivalent conditions. We also prove, under some continuity and monotonicity assumptions, that if any sequence generated by an algorithm for solving (VIP) converges to a weak sharp solution, then we can obtain solutions for (VIP) by solving a finite number of convex optimization subproblems with linear objective. Moreover, in order to characterize finite convergence of an iterative algorithm, we introduce the notion of a weak subsharp set of solutions to a variational inequality problem (VIP), which is more general than that of weak sharp solutions in Hilbert spaces. We establish a sufficient and necessary condition for the finite convergence of an algorithm for solving (VIP) which satisfies that the sequence generated by which converges to a weak subsharp solution of (VIP), and show that the proximal point algorithm satisfies this condition. As a consequence, we prove that the proximal point algorithm possesses finite convergence whenever the sequence generated by which converges to a weak subsharp solution of (VIP).     

A Smoothing Method for Zero–One Constrained Extremum Problems

In this paper, the zero–one constrained extremum problem is reformulated as an equivalent smooth mathematical program with complementarity constraints (MPCC), and then as a smooth ordinary nonlinear programming problem with the help of the Fischer–Burmeister function. The augmented Lagrangian method is adopted to solve the resulting problem, during which the non-smoothness may be introduced as a consequence of the possible inequality constraints. This paper incorporates the aggregate constraint method to construct a uniform smooth approximation to the original constraint set, with approximation controlled by only one parameter. Convergence results are established, showing that under reasonable conditions the limit point of the sequence of stationary points generated by the algorithm is a strongly stationary point of the original problem and satisfies the second order necessary conditions of the original problem. Unlike other penalty type methods for MPCC, the proposed algorithm can guarantee that the limit point of the sequence is feasible to the original problem.     

A log-exponential smoothing method for mathematical programs with complementarity constraints

In this paper a log-exponential smoothing method for mathematical programs with complementarity constraints (MPCC) is analyzed, with some new interesting properties and convergence results provided. It is shown that the stationary points of the resulting smoothed problem converge to the strongly stationary point of MPCC, under the linear independence constraint qualification (LICQ), the weak second-order necessary condition (WSONC), and some reasonable assumption. Moreover, the limit point satisfies the weak second-order necessary condition for MPCC. A notable fact is that the proposed convergence results do not restrict the complementarity constraint functions approach to zero at the same order of magnitude.     

A conditional gradient method for an optimal control problem involving a class of nonlinear second-order hyperbolic partial differential equations

The aim of this paper is to consider an optimal control problem involving a class of nonlinear hyperbolic partial differential equations. A conditional gradient method is used to obtain an algorithm for solving the optimal control problem iteratively. It is then shown that any accumulation point of the sequence of controls generated by the algorithm (if it exists) satisfies a necessary condition for optimality.     

Partial Augmented Lagrangian Method and Mathematical Programs with Complementarity Constraints

In this paper, we apply a partial augmented Lagrangian method to mathematical programs with complementarity constraints (MPCC). Specifically, only the complementarity constraints are incorporated into the objective function of the augmented Lagrangian problem while the other constraints of the original MPCC are retained as constraints in the augmented Lagrangian problem. We show that the limit point of a sequence of points that satisfy second-order necessary conditions of the partial augmented Lagrangian problems is a strongly stationary point (hence a B-stationary point) of the original MPCC if the limit point is feasible to MPCC, the linear independence constraint qualification for MPCC and the upper level strict complementarity condition hold at the limit point. Furthermore, this limit point also satisfies a second-order necessary optimality condition of MPCC. Numerical experiments are done to test the computational performances of several methods for MPCC proposed in the literature. This research was partially supported by the Research Grants Council (BQ654) of Hong Kong and the Postdoctoral Fellowship of The Hong Kong Polytechnic University. Dedicated to Alex Rubinov on the occassion of his 65th birthday.     

An Algorithm Using Trust Region Strategy for Minimization of a Nondifferentiable Function

In this article, we present a method for minimization of a nondifferentiable function. The method uses trust region strategy combined with a bundle method philosophy. It is proved that the sequence of points generated by the algorithm has an accumulation point that satisfies the first order necessary and sufficient conditions.     

A convergence analysis for a convex version of Dikin's algorithm

This paper is concerned with the convergence property of Dikin's algorithm applied to linearly constrained smooth convex programs. We study a version of Dikin's algorithm in which a second-order approximation of the objective function is minimized at each iteration together with an affine transformation of the variables. We prove that the sequence generated by the algorithm globally converges to a limit point at a local linear rate if the objective function satisfies a Hessian similarity condition. The result is of a theoretical nature in the sense that in order to ensure that the limit point is an -optimal solution, one may have to restrict the steplength to the order ofO(). The analysis does not depend on non-degeneracy assumptions.     

New Condition Characterizing the Solutions of Variational Inequality Problems

This paper is devoted to the study of a new necessary condition in variational inequality problems: approximated gradient projection (AGP). A feasible point satisfies such condition if it is the limit of a sequence of the approximated solutions of approximations of the variational problem. This condition comes from optimization where the error in the approximated solution is measured by the projected gradient onto the approximated feasible set, which is obtained from a linearization of the constraints with slack variables to make the current point feasible. We state the AGP condition for variational inequality problems and show that it is necessary for a point being a solution even without constraint qualifications (e.g., Abadie’s). Moreover, the AGP condition is sufficient in convex variational inequalities. Sufficiency also holds for variational inequalities involving maximal monotone operators subject to the boundedness of the vectors in the image of the operator (playing the role of the gradients). Since AGP is a condition verified by a sequence, it is particularly interesting for iterative methods. Research of R. Gárciga Otero was partially supported by CNPq, FAPERJ/Cientistas do Nosso Estado, and PRONEX Optimization. Research of B.F. Svaiter was partially supported by CNPq Grants 300755/2005-8 and 475647/2006-8 and by PRONEX Optimization.     

Global Convergence of a Smooth Approximation Method for Mathematical Programs with Complementarity Constraints

A new smoothing approach was given for solving the mathematical programs with complementarity constraints (MPCC) by using the aggregation technique. As the smoothing parameter tends to zero, if the KKT point sequence generated from the smoothed problems satisfies the second-order necessary condition, then any accumulation point of the sequence is a B-stationary point of MPCC if the linear independence constraint qualification (LICQ) and the upper level strict complementarity (ULSC) condition hold at the limit point. The ULSC condition is weaker than the lower level strict complementarity (LLSC) condition generally used in the literatures. Moreover, the method can be easily extended to the mathematical programs with general vertical complementarity constraints.     

A ROBUST SQP METHOD FOR OPTIMIZATION WITH INEQUALITY CONSTRAINTS

A new algorithm for inequality constrained optimization is presented, which solves a linear programming subproblem and a quadratic subproblem at each iteration. The algorithm can circumvent the difficulties associated with the possible inconsistency of QP subproblem of the original SQP method. Moreover, the algorithm can converge to a point which satisfies a certain first-order necessary condition even if the original problem is itself infeasible. Under certain condition, some global convergence results are proved and local superlinear convergence results are also obtained. Preliminary numerical results are reported.     

A Steepest Descent-Like Method for Variable Order Vector Optimization Problems

In some applications, the comparison between two elements may depend on the point leading to the so called variable order structure. Optimality concepts may be extended to this more general framework. In this paper, we extend the steepest descent-like method for smooth unconstrained vector optimization problems under a variable order structure. Roughly speaking, we see that every accumulation point of the generated sequence satisfies a necessary first order condition. We discuss the consequence of this fact in the convex case.     

A variant of SQP method for inequality constrained optimization and its global convergence

In this paper, a variant of SQP method for solving inequality constrained optimization is presented. This method uses a modified QP subproblem to generate a descent direction as each iteration and can overcome the possible difficulties that the QP subproblem of the standard SQP method is inconsistency. Furthermore, the method can start with an infeasible initial point. Under mild conditions, we prove that the algorithm either terminates as KKT point within finite steps or generates an infinite sequence whose accumulation point is a KKT point or satisfies certain first-order necessary condition. Finally, preliminary numerical results are reported.     

A Sequential Smooth Penalization Approach to Mathematical Programs with Complementarity Constraints

In this paper, a mathematical program with complementarity constraints (MPCC) is reformulated as a nonsmooth constrained mathematical program via the Fischer–Burmeister function. Smooth penalty functions are used to treat this nonsmooth constrained program. Under linear independence constraint qualification, and upper level strict complementarity condition, together with some other mild conditions, we prove that the limit point of stationary points satisfying second-order necessary conditions of unconstrained penalized problems is a strongly stationary point, hence a B-stationary point of the original MPCC. Furthermore, this limit point also satisfies a second-order necessary condition of the original MPCC. Numerical results are presented to test the performance of this method.     

A New Noninterior Predictor–Corrector Method for the P0 LCP

In this paper a new predictor-corrector noninterior method for LCP is presented, in which the predictor step is generated by the Levenberg-Marquadt method, which is new in the predictor-corrector-type methods, and the corrector step is generated as in [3]. The method has the following merits: (i) any cluster point of the iteration sequence is a solution of the P₀ LCP; (ii) if the generalized Jacobian is nonsingular at a solution point, then the whole sequence converges to the (unique) solution of the P₀ LCP superlinearly; (iii) for the P₀ LCP, if an accumulation point of the iteration sequence satisfies the strict complementary condition, then the whole sequence converges to this accumulation point superlinearly. Preliminary numerical experiments are reported to show the efficiency of the algorithm.     

A quasi-Newton trust-region method

The classical trust-region method for unconstrained minimization can be augmented with a line search that finds a point that satisfies the Wolfe conditions. One can use this new method to define an algorithm that simultaneously satisfies the quasi-Newton condition at each iteration and maintains a positive-definite approximation to the Hessian of the objective function. This new algorithm has strong global convergence properties and is robust and efficient in practice.     

An interior trust region algorithm for solving linearly constrained nonlinear optimization

In this paper, an interior point algorithm based on trust region techniques is proposed for solving nonlinear optimization problems with linear equality constraints and nonnegative variables. Unlike those existing interior-point trust region methods, this proposed method does not require that a general quadratic subproblem with a trust region bound be solved at each iteration. Instead, a system of linear equations is solved to get a search direction, and then a linesearch of Armijo type is performed in this direction to obtain a new iteration point. From a computational point of view, this approach may in general reduce a computational effort, and thus improve the computational efficiency. Under suitable conditions, it is proven that any accumulation of the sequence generated by the algorithm satisfies the first-order optimality condition.     

Convergence behavior of interior-point algorithms

We show that most interior-point algorithms for linear programming generate a solution sequence in which every limit point satisfies the strict complementarity condition. These algorithms include all path-following algorithms and some potential reduction algorithms. The result also holds for the monotone complementarity problem if a strict complementarity solution exists. In general, the limit point is a solution that maximizes the number of its nonzero components among all solutions.Research supported in part by NSF Grant DDM-8922636, the Iowa Business School Summer Grant, and the Interdisciplinary Research Grant of the University of Iowa Center for Advanced Studies.