Penalty function methods for constrained stochastic approximation

This paper is concerned with sequential Monte Carlo methods for optimizing a system under constraints. We wish to minimize f(x), where qⁱ(x) ? 0 (i = 1, …, m) must hold. We can calculate the qⁱ(x), but f(x) can only be observed in the presence of noise. A general approach, based on an adaptation of a version of stochastic approximation to the penalty function method, is discussed and a convergence theorem is proved.     

An algorithm for linearly constrained nonlinear programming problems

In this paper an algorithm for solving a linearly constrained nonlinear programming problem is developed. Given a feasible point, a correction vector is computed by solving a least distance programming problem over a polyhedral cone defined in terms of the gradients of the “almost” binding constraints. Mukai's approximate scheme for computing the step size is generalized to handle the constraints. This scheme provides an estimate for the step size based on a quadratic approximation of the function. This estimate is used in conjunction with Armijo line search to calculate a new point. It is shown that each accumulation point is a Kuhn-Tucker point to a slight perturbation of the original problem. Furthermore, under suitable second order optimality conditions, it is shown that eventually only one trial is needed to compute the step size.     

Penalty function theory for general convex programming problems

A class of penalty functions for solving convex programming problems with general constraint sets is considered. Convergence theorems for penalty methods are established by utilizing the concept of infimal convergence of a sequence of functions. It is shown that most existing penalty functions are included in our class of penalty functions.     

Penalty methods in nonlinear programming (survey)

In this paper general interior and exterior penalty methods are considered and their essential properties are summarized. Penalty-shifting methods are also contained in this survey.     

Penalty formulation for zero-one nonlinear programming

Raghavachari has shown the equivalence of zero-one integer programming and a concave quadratic penalty function for a sufficiently large value of the penalty. A lower bound for this penalty was found by Kalantari and Rosen. It was also shown that this penalty could not be reduced in specific cases. We show that the results generalize to the case where the objective function is any concave function. Equivalent penalty formulation for non-concave functions is also considered.     

NLPQL: A fortran subroutine solving constrained nonlinear programming problems

NLPQL is a FORTRAN implementation of a sequential quadratic programming method for solving nonlinearly constrained optimization problems with differentiable objective and constraint functions. At each iteration, the search direction is the solution of a quadratic programming subproblem. This paper discusses the organization of NLPQL, including the formulation of the subproblem and the information that must be provided by a user. A summary is given of the performance of different algorithmic options of NLPQL on a collection of test problems (115 hand-selected or application problems, 320 randomly generated problems). The performance of NLPQL is compared with that of some other available codes.     

NLPQL: A fortran subroutine solving constrained nonlinear programming problems

NLPQL is a FORTRAN implementation of a sequential quadratic programming method for solving nonlinearly constrained optimization problems with differentiable objective and constraint functions. At each iteration, the search direction is the solution of a quadratic programming subproblem. This paper discusses the organization of NLPQL, including the formulation of the subproblem and the information that must be provided by a user. A summary is given of the performance of different algorithmic options of NLPQL on a collection of test problems (115 hand-selected or application problems, 320 randomly generated problems). The performance of NLPQL is compared with that of some other available codes.     

Stability achievement scalarization function for multiobjective nonlinear programming problems

In this paper, we present a method to determine the stability of nondominated criterion vectors using a modified weighted achievement scalarization metric. This method is based on the application of a particular objective function which scalarizes and parameterizes the original multiobjective nonlinear programming problem. Also, we show that this modified weighted achievement metric coincides with the metric introduced by Choo and Atkins [E.-U. Choo, D.R. Atkins, Proper efficiency in nonconvex multicriteria programming, Math. Oper. Res. 8 (1983) 467–470] and Kaliszewski [I. Kaliszewski, A modified weighted Tchebycheff metric for multiple objective programming, Comput. Oper. Res. 14 (1987) 315–323] in cases when sets of all criterion vectors are finite or polyhedral.     

Infeasible-start primal-dual methods and infeasibility detectors for nonlinear programming problems

ln) iterations, where ν is the parameter of a self-concordant barrier for the cone, ε is a relative accuracy and ρ_f is a feasibility measure. We also discuss the behavior of path-following methods as applied to infeasible problems. We prove that strict infeasibility (primal or dual) can be detected in O(ln) iterations, where ρ_· is a primal or dual infeasibility measure. Received April 25, 1996 / Revised version received March 4, 1998 Published online October 9, 1998     

A multi-parametric programming approach for constrained dynamic programming problems

In this work, we present a new algorithm for solving complex multi-stage optimization problems involving hard constraints and uncertainties, based on dynamic and multi-parametric programming techniques. Each echelon of the dynamic programming procedure, typically employed in the context of multi-stage optimization models, is interpreted as a multi-parametric optimization problem, with the present states and future decision variables being the parameters, while the present decisions the corresponding optimization variables. This reformulation significantly reduces the dimension of the original problem, essentially to a set of lower dimensional multi-parametric programs, which are sequentially solved. Furthermore, the use of sensitivity analysis circumvents non-convexities that naturally arise in constrained dynamic programming problems. The potential application of the proposed novel framework to robust constrained optimal control is highlighted.     

An exact penalty function approach for nonlinear integer programming problems

Nonlinear integer programming problems with bounded feasible sets are considered. It is shown how the number of constraints in such problems can be reduced with the aid of an exact penalty function approach. This approach can be used to construct an equivalent unconstrained problem, or a problem with a constraint set which makes it easier to solve. The application of this approach to various nonlinear integer programming problems is discussed.     

On a Lagrangian penalty function method for nonlinear programming problems

We modify a Lagrangian penalty function method proposed in [4] for constrained convex mathematical programming problems in order to obtain a geometric rate of convergence. For nonconvex problems we show that a special case of the algorithm in the above paper is still convergent without coercivity and convexity assumptions.On leave from the Institute of Mathematics, Hanoi, by a grant from Alexander-von-Humboldt-Stiftung.     

Algorithms for bound constrained quadratic programming problems

Summary We present an algorithm which combines standard active set strategies with the gradient projection method for the solution of quadratic programming problems subject to bounds. We show, in particular, that if the quadratic is bounded below on the feasible set then termination occurs at a stationary point in a finite number of iterations. Moreover, if all stationary points are nondegenerate, termination occurs at a local minimizer. A numerical comparison of the algorithm based on the gradient projection algorithm with a standard active set strategy shows that on mildly degenerate problems the gradient projection algorithm requires considerable less iterations and time than the active set strategy. On nondegenerate problems the number of iterations typically decreases by at least a factor of 10. For strongly degenerate problems, the performance of the gradient projection algorithm deteriorates, but it still performs better than the active set method.Work supported in part by the Applied Mathematical Sciences subprogram of the Office of Energy Research of the U.S. Department of Energy under Contract W-31-109-Eng-38     

On the numerical feasibility of continuation methods for nonlinear programming problems

In the present paper the radius of convergence of a class of locally convergent nonlinear programming algorithms (containing Robinson's and Wilson's methods) applied to a parametric nonlinear programming problem is estimated. A consequence is the numerical feasibility of globalizations of Robinson's and Wilson's methods by means of continuation techniques.     

An exact penalty function method for nonlinear mixed discrete programming problems

In this paper, we consider a general class of nonlinear mixed discrete programming problems. By introducing continuous variables to replace the discrete variables, the problem is first transformed into an equivalent nonlinear continuous optimization problem subject to original constraints and additional linear and quadratic constraints. Then, an exact penalty function is employed to construct a sequence of unconstrained optimization problems, each of which can be solved effectively by unconstrained optimization techniques, such as conjugate gradient or quasi-Newton methods. It is shown that any local optimal solution of the unconstrained optimization problem is a local optimal solution of the transformed nonlinear constrained continuous optimization problem when the penalty parameter is sufficiently large. Numerical experiments are carried out to test the efficiency of the proposed method.     

Penalty methods for a system of constrained variational inequalities

In this paper, using the penalty method in conjunction with graph-convergence, we study the existence of solutions for a class of generalized variational inequalities with variational problem constraints. Our results extend, improve and develop some known results in this field. Our method of proofs is very simple and does not use the hemicontinuity nor the pseudo-continuity properties of monotone operators.     

A language for nonlinear programming problems

An algebraic-like language for nonlinear programming problems is described. The rationale for the computation of the function values, gradients, and scond partial derivatives of the functions from their algebraic representation is developed. Each function is translated into an explicit factorable form or hierarchical representation which is used interpretively to compute the function value, gradient, and second partials of the function at each point for which such values are required. Computational efficiency is achieved by computing the matrix of second partials as the sum of a set of vector outer products, the vectors having resulted from the gradient computation, plus a diagonal matrix. An experimental computer program which implements the language and ties it to SUMT is described. In the experience with this program the computer times required have ranged from 4 to 30 times those times required by computer solutions to the same problems by using analyst-prepared programs to compute the function values, gradients, and second partial derivatives. A program based on a compiler approach to implementing the language, rather than the interpretative approach of the experimental program, will probably result in computer times between one and two times those required by using analyst-prepared programs.     

Parallel algorithms for nonlinear programming problems

This paper describes several parallel algorithms for solving nonlinear programming problems. Two approaches where parallelism can successfully be introduced have been explored: a quadratic approximation method based on penalty function and a dual method. These methods are improved by using two algorithms originally proposed for solving unconstrained problems: the parallel variable metric algorithm and the parallel Jacobson-Oksman algorithm. Even though general problems are dealt with, particular emphasis is placed on the potential of these parallel methods for separable programming problems. The numerical effectiveness of the algorithms is demonstrated on a set of test problems using a Cray-1S vector computer and serial computers (with respect to sequential versions of the same methods).These studies were sponsored in part by the CERT. The author would particularly like to thank Ph. Berger (LSI-ENSEEIHT), the researchers of the DERI (CERT) and of the Groupe Structures, Aerospatiale, for their assistance.     

Mesh-independence of semismooth Newton methods for Lavrentiev-regularized state constrained nonlinear optimal control problems

A class of nonlinear elliptic optimal control problems with mixed control-state constraints arising, e.g., in Lavrentiev-type regularized state constrained optimal control is considered. Based on its first order necessary optimality conditions, a semismooth Newton method is proposed and its fast local convergence in function space as well as a mesh-independence principle for appropriate discretizations are proved. The paper ends by a numerical verification of the theoretical results including a study of the algorithm in the case of vanishing Lavrentiev-parameter. The latter process is realized numerically by a combination of a nested iteration concept and an extrapolation technique for the state with respect to the Lavrentiev-parameter.     

A nonlinear Lagrangian for constrained optimization problems

A novel nonlinear Lagrangian is presented for constrained optimization problems with both inequality and equality constraints, which is nonlinear with respect to both functions in problem and Lagrange multipliers. The nonlinear Lagrangian inherits the smoothness of the objective and constraint functions and has positive properties. The algorithm on the nonlinear Lagrangian is demonstrated to possess local and linear convergence when the penalty parameter is less than a threshold (the penalty parameter in the penalty method has to approximate zero) under a set of suitable conditions, and be super-linearly convergent when the penalty parameter is decreased following Lagrange multiplier update. Furthermore, the dual problem based on the nonlinear Lagrangian is discussed and some important properties are proposed, which fail to hold for the dual problem based on the classical Lagrangian. At last, the preliminary and comparing numerical results for several typical test problems by using the new nonlinear Lagrangian algorithm and the other two related nonlinear Lagrangian algorithms, are reported, which show that the given nonlinear Lagrangian is promising.