Variable sample size method for equality constrained optimization problems

An equality constrained optimization problem with a deterministic objective function and constraints in the form of mathematical expectation is considered. The constraints are transformed into the Sample Average Approximation form resulting in deterministic problem. A method which combines a variable sample size procedure with line search is applied to a penalty reformulation. The method generates a sequence that converges towards first-order critical points. The final stage of the optimization procedure employs the full sample and the SAA problem is eventually solved with significantly smaller cost. Preliminary numerical results show that the proposed method can produce significant savings compared to SAA method and some heuristic sample update counterparts while generating a solution of the same quality.     

A method of multipliers for mathematical programming problems with equality and inequality constraints

This paper deals with the numerical solution of the general mathematical programming problem of minimizing a scalar functionf(x) subject to the vector constraints φ(x)=0 and ψ(x)≥0. The approach used is an extension of the Hestenes method of multipliers, which deals with the equality constraints only. The above problem is replaced by a sequence of problems of minimizing the augmented penalty function Ω(x, λ, μ,k)=f(x)+λ ^T φ(x)+kφ ^T (x)φ(x) ?μ ^T \(\tilde \psi \) (x)+k \(\tilde \psi \) ^T (x) \(\tilde \psi \) (x). The vectors λ and μ, μ ≥ 0, are respectively the Lagrange multipliers for φ(x) and \(\tilde \psi \) (x), and the elements of \(\tilde \psi \) (x) are defined by \(\tilde \psi \) _(j)(x)=min[ψ_(j)(x), (1/2k) μ_(j)]. The scalark>0 is the penalty constant, held fixed throughout the algorithm. Rules are given for updating the multipliers for each minimization cycle. Justification is given for trusting that the sequence of minimizing points will converge to the solution point of the original problem.     

A quadratically convergent primal-dual algorithm with global convergence properties for solving optimization problems with equality constraints

This paper presents a globally convergent multiplier method which utilizes an explicit formula for the multiplier. The algorithm solves finite dimensional optimization problems with equality constraints. A unique feature of the algorithm is that it automatically calculates a value for the penalty coefficient, which, under certain assumptions, leads to global convergence.Research sponsored by the Joint Services Electronics Program, Contract F44620-71-C-0087 and the National Science Foundation, Grant GK-37672.     

Exact penalties for optimization problems with 2-regular equality constraints

A new first-order sufficient condition for penalty exactness that includes neither the standard constraint qualification requirement nor the second-order sufficient optimality condition is proposed for optimization problems with equality constraints.     

Pseudo duality in mathematical programming: Unconstrained problems and problems with equality constraints

In this work non-convex programs are analyzed via Legendre transform. The first part includes definitions and the classification of programs that can be handled by the transformation. It is shown that differentiable functions that are represented as a sum of strictly concave and convex functions belong to this class. Conditions under which a function may have such representation are given. Pseudo duality is defined and the pseudo duality theorem for non linear programs with equality constraints is proved.The techniques described are constructive ones, and they enable tocalculate explicitly a pseudo dual once the primal program is given. Several examples are included. In the convex case these techniques enable the explicit calculation of the dual even in cases where direct calculation was not possible.     

Feasible directions algorithms for optimization problems with equality and inequality constraints

In this paper two algorithms, of the feasible-directions and dual feasible-directions type, are presented for optimization problems with equality and inequality constraints. An associated problem, having only inequality constraints, is defined, and shown to be equivalent to the original problem if a certain parameter is sufficiently large. The algorithms solve the associated problem, but incorporate a method for automatically increasing this parameter in order to ensure global convergence to a solution to the original problem. Any feasible directions algorithm can be similarly modified to enable it to handle equality constraints.Research sponsored by the US Army Research Office — Durham, Contract DAHCO4-73-C-0025 and the National Science Foundation Grant GK-37572.     

Neighboring extremals of dynamic optimization problems with path equality constraints

Neighboring extremals of dynamic optimization problems with path equality constraints and with an unknown parameter vector are considered in this paper. With some simplifications, the problem is reduced to solving a linear, time-varying two-point boundary-value problem with integral path equality constraints. A modified backward sweep method is used to solve this problem. Two example problems are solved to illustrate the validity and usefulness of the solution technique. This research was supported in part by the National Aeronautics and Space Administration under NASA Grant No. NCC-2-106. The author is indebted to Professor A. E. Bryson, Jr., Department of Aeronautics and Astronautics, Stanford University, for many stimulating discussions.     

An algorithm for solving global optimization problems with nonlinear constraints

In this paper we propose an algorithm using only the values of the objective function and constraints for solving one-dimensional global optimization problems where both the objective function and constraints are Lipschitzean and nonlinear. The constrained problem is reduced to an unconstrained one by the index scheme. To solve the reduced problem a new method with local tuning on the behavior of the objective function and constraints over different sectors of the search region is proposed. Sufficient conditions of global convergence are established. We also present results of some numerical experiments.     

ODE methods for optimization with equality constraints

In an optimization problem with equality constraints we define an accessory function that is similar but different from a normal penalty function. In the accessory function we demonstrate the need to use small values of the parameter associated with an equality constraint. Large values of the parameter create extraneous stationary points which destroy the global convergence properties of steepest descent methods. By using small values of the parameters in the accessory function, when the current point is far away from the solution and when the constraint violations are large we are led to a refined version of the established SUMT method.     

求解互补约束优化问题的乘子松弛法

利用互补问题的Lagrange函数, 给出了互补约束优化问题\,(MPCC)\,的一种新松弛问题. 在较弱的条件下, 新松弛问题满足线性独立约束规范. 在此基础上, 提出了求解互补约束优化问题的乘子松弛法. 在MPCC-LICQ条件下, 松弛问题稳定点的任何聚点都是MPCC的M-稳定点. 无需二阶必要条件, 只在ULSC条件下, 就可保证聚点是MPCC的B-稳定点. 另外, 给出了算法收敛于B-稳定点的新条件.     

A GENERALIZED GRADIENT PROJECTION METHOD FOR OPTIMIZATION PROBLEMS WITH EQUALITY AND INEQUALITY CONSTRAINTS ABOUT ARBITRARY INITIAL POINT

ＧＡＯＺＩＹＯＵ（高自友）（ＮｏｒｔｈｅｒｎＪｉａｏｔｏｎｇＵｎｉｖｅｒｓｉｔｙ，Ｂｅｉｊｉｎｇ１０００４４，Ｃｈｉｎａ）ＬＡＩＹＡＮＬＩＡＮ（赖炎连）（ＩｎｓｔｉｔｕｔｅｏｆＡｐｐｌｉｅｄＭａｔｈｅｍａｔｉｃｓ，ｔｈｅＣｈｉｎｅｓｅＡｃａｄｅｍｙｏ...     

On the weighting method for least squares problems with linear equality constraints

The weighting method for solving a least squares problem with linear equality constraints multiplies the constraints by a large number and appends them to the top of the least squares problem, which is then solved by standard techniques. In this paper we give a new analysis of the method, based on the QR decomposition, that exhibits many features of the algorithm. In particular it suggests a natural criterion for chosing the weighting factor. This work was supported in part by the National Science Foundation under grant CCR 95503126.     

Progressive genetic algorithm for solution of optimization problems with nonlinear equality and inequality constraints

A new approach, identified as progressive genetic algorithm (PGA), is proposed for the solutions of optimization problems with nonlinear equality and inequality constraints. Based on genetic algorithms (GAs) and iteration method, PGA divides the optimization process into two steps; iteration and search steps. In the iteration step, the constraints of the original problem are linearized using truncated Taylor series expansion, yielding an approximate problem with linearized constraints. In the search step, GA is applied to the problem with linearized constraints for the local optimal solution. The final solution is obtained from a progressive iterative process. Application of the proposed method to two simple examples is given to demonstrate the algorithm.     

A projection subgradient method for solving optimization with variational inequality constraints

In this paper, we propose a projection subgradient method for solving some classical variational inequality problem over the set of solutions of mixed variational inequalities. Under the conditions that $T$ is a $\Theta $ -pseudomonotone mapping and $A$ is a $\rho $ -strongly pseudomonotone mapping, we prove the convergence of the algorithm constructed by projection subgradient method. Our algorithm can be applied for instance to some mathematical programs with complementarity constraints.     

An effective method for solving linear programming problems with flexible constraints

For solving linear programming problems with flexible constraints being specific piecewise linear programs a new algorithm which is based on a systematic decomposition of the feasible set into linear constrained subsets is proposed and shown to converge.     

Theoretical justification of interior point algorithms for solving optimization problems with nonlinear constraints

A family of interior point algorithms is considered. These algorithms can be used for solving mathematical programming problems with nonlinear inequality constraints. Some weighted Euclidean norms are applied to finding a descent direction for improving the solution. These norms vary with iterations. A theoretical justification of the algorithms with some assumptions (including the nonsingularity of the problem) is presented.     

Second-order conditions for extremum problems with nonregular equality constraints

Combining results of Avakov about tangent directions to equality constraints given by smooth operators with results of Ben-Tal and Zowe, we formulate a second-order theory for optimality in the sense of Dubovitskii-Milyutin which gives nontrivial conditions also in the case of equality constraints given by nonregular operators. Secondorder feasible and tangent directions are defined to construct conical approximations to inequality and equality constraints which within a single construction lead to first- and second-order conditions of optimality for the problem also in the nonregular case. The definitions of secondorder feasible and tangent directions given in this paper allow for reparametrizations of the approximating curves and give approximating sets which form cones. The main results of the paper are a theorem which states second-order necessary condition of optimality and several corollaries which treat special cases. In particular, the paper generalizes the Avakov result in the smooth case.This research was supported by NSF Grant DMS-91-009324, NSF Grant DMS-91-00043, SIUE Research Scholar Award and Fourth Quarter Fellowship, Summer 1992.     

Global optimization of a quadratic functional with quadratic equality constraints

In this paper, we investigate a constrained optimization problem with a quadratic cost functional and two quadratic equality constraints. While it is obvious that, for a nonempty constraint set, there exists a global minimum cost, a method to determine if a given local solution yields the global minimum cost has not been established. We develop a necessary and sufficient condition that will guarantee that solutions of the optimization problem yield the global minimum cost. This constrained optimization problem occurs naturally in the computation of the phase margin for multivariable control systems. Our results guarantee that numerical routines can be developed that will converge to the global solution for the phase margin.     

The method of feasible directions for mathematical programming problems with preconvex constraints

The convergence of the method of feasible directions is proved for the case of the smooth objective function and a constraint in the form of the difference of convex sets (the so-called preconvex set). It is shown that the method converges to the set of stationary points, which generally is narrower than the corresponding set in the case of a smooth function and smooth constraints. The scheme of the proof is similar to that proposed earlier by Karmanov.