A stable approach to Newton's method for general mathematical programming problems inR n

A modification to the algorithm of Ref. 1 is given.     

A feasible conjugate-direction method to solve linearly constrained minimization problems

An iterative procedure is presented which uses conjugate directions to minimize a nonlinear function subject to linear inequality constraints. The method (i) converges to a stationary point assuming only first-order differentiability, (ii) has ann-q step superlinear or quadratic rate of convergence with stronger assumptions (n is the number of variables,q is the number of constraints which are binding at the optimum), (iii) requires the computation of only the objective function and its first derivatives, and (iv) is experimentally competitive with well-known methods.For helpful suggestions, the author is much indebted to C. R. Glassey and K. Ritter.This research has been partially supported by the National Research Council of Canada under Grants Nos. A8189 and C1234.     

Differential dynamic programming and Newton's method for discrete optimal control problems

The purpose of this paper is to draw a detailed comparison between Newton's method, as applied to discrete-time, unconstrained optimal control problems, and the second-order method known as differential dynamic programming (DDP). The main outcomes of the comparison are: (i) DDP does not coincide with Newton's method, but (ii) the methods are close enough that they have the same convergence rate, namely, quadratic.The comparison also reveals some other facts of theoretical and computational interest. For example, the methods differ only in that Newton's method operates on a linear approximation of the state at a certain point at which DDP operates on the exact value. This would suggest that DDP ought to be more accurate, an anticipation borne out in our computational example. Also, the positive definiteness of the Hessian of the objective function is easy to check within the framework of DDP. This enables one to propose a modification of DDP, so that a descent direction is produced at each iteration, regardless of the Hessian.Efforts of the first author were partially supported by the South African Council for Scientific and Industrial Research, and those of the second author by NSF Grants Nos. CME-79-05010 and CEE-81-10778.     

An interactive method for multiple-objective mathematical programming problems

An interactive method is developed for solving the general nonlinear multiple objective mathematical programming problems. The method asks the decision maker to provide partial information (local tradeoff ratios) about his utility (preference) function at each iteration. Using the information, the method generates an efficient solution and presents it to the decision maker. In so doing, the best compromise solution is sought in a finite number of iterations. This method differs from the existing feasible direction methods in that (i) it allows the decision maker to consider only efficient solutions throughout, (ii) the requirement of line search is optional, and (iii) it solves the problems with linear objective functions and linear utility function in one iteration. Using various problems selected from the literature, five line search variations of the method are tested and compared to one another. The nonexisting decision maker is simulated using three different recognition levels, and their impact on the method is also investigated.     

The method of feasible directions for mathematical programming problems with preconvex constraints

The convergence of the method of feasible directions is proved for the case of the smooth objective function and a constraint in the form of the difference of convex sets (the so-called preconvex set). It is shown that the method converges to the set of stationary points, which generally is narrower than the corresponding set in the case of a smooth function and smooth constraints. The scheme of the proof is similar to that proposed earlier by Karmanov.     

Regularized dual method for nonlinear mathematical programming

For a nonlinear programming problem with equality constraints in a Hilbert space, a dual-type algorithm is constructed that is stable with respect to input data errors. The algorithm is based on a modified dual of the original problem that is solved directly by applying Tikhonov regularization. The algorithm is designed to determine a norm-bounded minimizing sequence of feasible elements. An iterative regularization of the dual algorithm is considered. A stopping rule for the iteration process is given in the case of a finite fixed error in the input data.     

Quadratic multiplier method convergence

In a recent paper (Ref. 1), the author briefly mentioned a variant of Hestenes' method of multipliers which would converge quadratically. This note examines that method in detail and provides some examples. In the quadratic-linear case, this algorithm converges in one iteration.     

A model trust-region modification of Newton's method for nonlinear two-point boundary-value problems

The method of quasilinearization for nonlinear two-point boundary-value problems is Newton's method for a nonlinear differential operator equation. A model trust-region approach to globalizing the quasilinearization algorithm is presented. A double-dogleg implementation yields a globally convergent algorithm that is robust in solving difficult problems.This work was supported in part by DOE Contract DE-AS05-82-ER13016 and NSF Grant RII-89-17691 and was part of the author's doctoral thesis at Rice University. It is a pleasure to thank the author's thesis advisors, Professor J. E. Dennis, Jr., and Professor R. A. Tapia.     

Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

Naive implementations of Newton's method for unconstrainedN-stage discrete-time optimal control problems with Bolza objective functions tend to increase in cost likeN ³ asN increases. However, if the inherent recursive structure of the Bolza problem is properly exploited, the cost of computing a Newton step will increase only linearly withN. The efficient Newton implementation scheme proposed here is similar to Mayne's DDP (differential dynamic programming) method but produces the Newton step exactly, even when the dynamical equations are nonlinear. The proposed scheme is also related to a Riccati treatment of the linear, two-point boundary-value problems that characterize optimal solutions. For discrete-time problems, the dynamic programming approach and the Riccati substitution differ in an interesting way; however, these differences essentially vanish in the continuous-time limit.This work was supported by the National Science Foundation, Grant No. DMS-85-03746.     

A second-order method for the general nonlinear programming problem

This paper presents a multiplier-type method for nonlinear programming problems with both equality and inequality constraints. Slack variables are used for the inequalities. The penalty coefficient is adjusted automatically, and the method converges quadratically to points satisfying second-order conditions.The work of the first author was supported by NSF RANN and JSEP Contract No. F44620-71-C-0087; the work of the second author was supported by the National Science Foundation Grant No. ENG73-08214A01 and US Army Research Office Durham Contract No. DAHC04-73-C-0025.     

Some investigations about a unified approach to quadratically convergent algorithms for function minimization

In 1969, Huang (Ref. 1) proposed a unified approach to quadratically convergent algorithms for function minimization without constraints; if we slightly modify a few points of his development, some other algorithms can be generated, among which several were published after 1969. It is also possible to generate a class of algorithms which provide quadratic convergence without linear search at each step.     

A polynomial-time algorithm,based on Newton's method,for linear programming

A new interior method for linear programming is presented and a polynomial time bound for it is proven. The proof is substantially different from those given for the ellipsoid algorithm and for Karmarkar's algorithm. Also, the algorithm is conceptually simpler than either of those algorithms.This research was supported by an NSF Mathematical Sciences Postdoctoral Research Fellowship and by NSF Grant 8120790. The research was performed at the Mathematical Sciences Research Institute in Berkeley, California.     

A limiting Lagrangian for infinitely constrained convex optimization inR n

For convex optimization inR ⁿ,we show how a minor modification of the usual Lagrangian function (unlike that of the augmented Lagrangians), plus a limiting operation, allows one to close duality gaps even in the absence of a Kuhn-Tucker vector [see the introductory discussion, and see the discussion in Section 4 regarding Eq. (2)]. The cardinality of the convex constraining functions can be arbitrary (finite, countable, or uncountable).In fact, our main result (Theorem 4.3) reveals much finer detail concerning our limiting Lagrangian. There are affine minorants (for any value 0<1 of the limiting parameter ) of the given convex functions, plus an affine form nonpositive onK, for which a general linear inequality holds onR ⁿAfter substantial weakening, this inequality leads to the conclusions of the previous paragraph.This work is motivated by, and is a direct outgrowth of, research carried out jointly with R. J. Duffin.This research was supported by NSF Grant No. GP-37510X1 and ONR Contract No. N00014-75-C0621, NR-047-048. This paper was presented at Constructive Approaches to Mathematical Models, a symposium in honor of R. J. Duffin, Pittsburgh, Pennsylvania, 1978. The author is grateful to Professor Duffin for discussions relating to the work reported here.The author wishes to thank R. J. Duffin for reading an earlier version of this paper and making numerous suggestions for improving it, which are incorporated here. Our exposition and proofs have profited from comments of C. E. Blair and J. Borwein.     

An accelerated multiplier method for nonlinear programming

This paper describes an accelerated multiplier method for solving the general nonlinear programming problem. The algorithm poses a sequence of unconstrained optimization problems. The unconstrained problems are solved using a rank-one recursive algorithm described in an earlier paper. Multiplier estimates are obtained by minimizing the error in the Kuhn-Tucker conditions using a quadratic programming algorithm. The convergence of the sequence of unconstrained problems is accelerated by using a Newton-Raphson extrapolation process. The numerical effectiveness of the algorithm is demonstrated on a relatively large set of test problems.This work was supported by the US Air Force under Contract No. F04701-74-C-0075.     

A multi-parametric programming approach for constrained dynamic programming problems

In this work, we present a new algorithm for solving complex multi-stage optimization problems involving hard constraints and uncertainties, based on dynamic and multi-parametric programming techniques. Each echelon of the dynamic programming procedure, typically employed in the context of multi-stage optimization models, is interpreted as a multi-parametric optimization problem, with the present states and future decision variables being the parameters, while the present decisions the corresponding optimization variables. This reformulation significantly reduces the dimension of the original problem, essentially to a set of lower dimensional multi-parametric programs, which are sequentially solved. Furthermore, the use of sensitivity analysis circumvents non-convexities that naturally arise in constrained dynamic programming problems. The potential application of the proposed novel framework to robust constrained optimal control is highlighted.     

A method for solving fuzzy goal programming problems based on MINMAX approach

Narasimhan incorporated fuzzy set theory within goal programming formulation in 1980. Since then numerous research has been carried out in this field. One of the well-known models for solving fuzzy goal programming problems was proposed by Hannan in 1981. In this paper the conventional MINMAX approach in goal programming is applied to solve fuzzy goal programming problems. It is proved that the proposed model is an extension to Hannan model that deals with unbalanced triangular linear membership functions. In addition, it is shown that the new model is equivalent to a model proposed in 1991 by Yang et al. Moreover, a weighted model of the new approach is introduced and is compared with Kim and Whang’s model presented in 1998. A numerical example is given to demonstrate the validity and strengths of the new models.     

An exterior point simplex algorithm for (general) linear programming problems

We present an exterior point simplex type algorithm that possesses a new monotonic property. A dual feasible basic solution is required to start with. Intermediate solutions are neither primal nor dual feasible. Cycling-free pivoting rules and an exponentional example are presented.     

An interior point method for general large-scale quadratic programming problems

In this paper, we present an interior point algorithm for solving both convex and nonconvex quadratic programs. The method, which is an extension of our interior point work on linear programming problems efficiently solves a wide class of largescale problems and forms the basis for a sequential quadratic programming (SQP) solver for general large scale nonlinear programs. The key to the algorithm is a three-dimensional cost improvement subproblem, which is solved at every interation. We have developed an approximate recentering procedure and a novel, adaptive big-M Phase I procedure that are essential to the sucess of the algorithm. We describe the basic method along with the recentering and big-M Phase I procedures. Details of the implementation and computational results are also presented.Contribution of the National Institute of Standards and Tedchnology and not subject to copyright in the United States. This research was supported in part by ONR Contract N-0014-87-F0053.     

A constraint shifting homotopy method for general non-linear programming

In this article, a constraint shifting homotopy method (CSHM) is proposed for solving non-linear programming with both equality and inequality constraints. A new homotopy is constructed, and existence and global convergence of a homotopy path determined by it are proven. All problems that can be solved by the combined homotopy interior point method (CHIPM) can also be solved by the proposed method. In contrast to the combined homotopy infeasible interior point method (CHIIPM), it needs a weaker regularity condition. And the starting point in the proposed method is not necessarily a feasible point or an interior point, so it is more convenient to be implemented than CHIPM and CHIIPM. Numerical results show that the proposed algorithm is feasible and effective.