Global multi-parametric optimal value bounds and solution estimates for separable parametric programs

In this tutorial, a strategy is described for calculating parametric piecewise-linear optimal value bounds for nonconvex separable programs containing several parameters restricted to a specified convex set. The methodology is based on first fixing the value of the parameters, then constructing sequences of underestimating and overestimating convex programs whose optimal values respectively increase or decrease to the global optimal value of the original problem. Existing procedures are used for calculating parametric lower bounds on the optimal value of each underestimating problem and parametric upper bounds on the optimal value of each overestimating problem in the sequence, over the given set of parameters. Appropriate updating of the bounds leads to a nondecreasing sequence of lower bounds and a nonincreasing sequence of upper bounds, on the optimal value of the original problem, continuing until the bounds satisfy a specified tolerance at the value of the parameter that was fixed at the outset. If the bounds are also sufficiently tight over the entire set of parameters, according to criteria specified by the user, then the calculation is complete. Otherwise, another parameter value is selected and the procedure is repeated, until the specified criteria are satisfied over the entire set of parameters. A parametric piecewise-linear solution vector approximation is also obtained. Results are expected in the theory, computations, and practical applications. The general idea of developing results for general problems that are limits of results that hold for a sequence of well-behaved (e.g., convex) problems should be quite fruitful.     

Computational aspects of two-segment separable programming

Recursive separable programming algorithms based on local, two-segment approximations are described for the solution of separable convex programs. Details are also given for the computation of lower bounds on the optimal value by both a primal and a dual approach, and these approaches are compared. Computational comparisons of the methods are provided for a variety of test problems, including a water supply application (with more than 600 constraints and more than 900 variables) and an econometric modelling problem (with more than 200 variables). Research supported by National Science Foundation Grants MCS74-20584 A02 and MCS-7901066.     

Nonanticipative duality,relaxations, and formulations for chance-constrained stochastic programs

We propose two new Lagrangian dual problems for chance-constrained stochastic programs based on relaxing nonanticipativity constraints. We compare the strength of the proposed dual bounds and demonstrate that they are superior to the bound obtained from the continuous relaxation of a standard mixed-integer programming (MIP) formulation. For a given dual solution, the associated Lagrangian relaxation bounds can be calculated by solving a set of single scenario subproblems and then solving a single knapsack problem. We also derive two new primal MIP formulations and demonstrate that for chance-constrained linear programs, the continuous relaxations of these formulations yield bounds equal to the proposed dual bounds. We propose a new heuristic method and two new exact algorithms based on these duals and formulations. The first exact algorithm applies to chance-constrained binary programs, and uses either of the proposed dual bounds in concert with cuts that eliminate solutions found by the subproblems. The second exact method is a branch-and-cut algorithm for solving either of the primal formulations. Our computational results indicate that the proposed dual bounds and heuristic solutions can be obtained efficiently, and the gaps between the best dual bounds and the heuristic solutions are small.     

Direct methods with maximal lower bound for mixed-integer optimal control problems

Many practical optimal control problems include discrete decisions. These may be either time-independent parameters or time-dependent control functions as gears or valves that can only take discrete values at any given time. While great progress has been achieved in the solution of optimization problems involving integer variables, in particular mixed-integer linear programs, as well as in continuous optimal control problems, the combination of the two is yet an open field of research. We consider the question of lower bounds that can be obtained by a relaxation of the integer requirements. For general nonlinear mixed-integer programs such lower bounds typically suffer from a huge integer gap. We convexify (with respect to binary controls) and relax the original problem and prove that the optimal solution of this continuous control problem yields the best lower bound for the nonlinear integer problem. Building on this theoretical result we present a novel algorithm to solve mixed-integer optimal control problems, with a focus on discrete-valued control functions. Our algorithm is based on the direct multiple shooting method, an adaptive refinement of the underlying control discretization grid and tailored heuristic integer methods. Its applicability is shown by a challenging application, the energy optimal control of a subway train with discrete gears and velocity limits.      

Dual gradient method for linearly constrained,strongly convex,separable mathematical programming problems

A new dual gradient method is given to solve linearly constrained, strongly convex, separable mathematical programming problems. The dual problem can be decomposed into one-dimensional problems whose solutions can be computed extremely easily. The dual objective function is shown to have a Lipschitz continuous gradient, and therefore a gradient-type algorithm can be used for solving the dual problem. The primal optimal solution can be obtained from the dual optimal solution in a straightforward way. Convergence proofs and computational results are given.     

Unconstrained duals to partially separable constrained programs

In this note partially separable convex programs are dualized in such a way that, under certain assumptions, unconstrained concave duals arise. A return formula is given by which the solution of the primal is directly computed if a solution of the dual is known. Further, the solvability of both the primal and the dual is shown to depend essentially on the behaviour of the lower dimensional programs for determining the Fenchel conjugates.     

A tighter variant of Jensen’s lower bound for stochastic programs and separable approximations to recourse functions

In this paper, we propose a new method to compute lower bounds on the optimal objective value of a stochastic program and show how this method can be used to construct separable approximations to the recourse functions. We show that our method yields tighter lower bounds than Jensen’s lower bound and it requires a reasonable amount of computational effort even for large problems. The fundamental idea behind our method is to relax certain constraints by associating dual multipliers with them. This yields a smaller stochastic program that is easier to solve. We particularly focus on the special case where we relax all but one of the constraints. In this case, the recourse functions of the smaller stochastic program are one dimensional functions. We use these one dimensional recourse functions to construct separable approximations to the original recourse functions. Computational experiments indicate that our lower bounds can significantly improve Jensen’s lower bound and our recourse function approximations can provide good solutions.     

Complexity and algorithms for convex network optimization and other nonlinear problems

Nonlinear optimization algorithms are rarely discussed from a complexity point of view. Even the concept of solving nonlinear problems on digital computers is not well defined. The focus here is on a complexity approach for designing and analyzing algorithms for nonlinear optimization problems providing optimal solutions with prespecified accuracy in the solution space. We delineate the complexity status of convex problems over network constraints, dual of flow constraints, dual of multi-commodity, constraints defined by a submodular rank function (a generalized allocation problem), tree networks, diagonal dominant matrices, and nonlinear Knapsack problem's constraint. All these problems, except for the latter in integers, have polynomial time algorithms which may be viewed within a unifying framework of a proximity-scaling technique or a threshold technique. The complexity of many of these algorithms is furthermore best possible in that it matches lower bounds on the complexity of the respective problems. In general nonseparable optimization problems are shown to be considerably more difficult than separable problems. We compare the complexity of continuous versus discrete nonlinear problems and list some major open problems in the area of nonlinear optimization. MSC classification: 90C30, 68Q25     

Convex separable minimization with box constraints

A minimization problem with convex separable objective function subject to a convex separable inequality constraint of the form “less than or equal to” and bounds on the variables (box constraints) is considered. Necessary and sufficient condition is proved for a feasible solution to be an optimal solution to this problem. An iterative algorithm of polynomial complexity for solving problems of the considered form is suggested and its convergence is proved. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Recovering an optimal LP basis from an optimal dual solution

Given a linear program, we describe an approach for crossing over from an optimal dual solution to an optimal basic primal solution. It consists in restricting the dual problem to a small box around the available optimal dual solution then, resolving the associated modified primal problem.     

Relaxed variants of Karmarkar's algorithm for linear programs with unknown optimal objective value

Variants of Karmarkar's algorithm are given for solving linear programs with unknown optimal objective valuez ^*. These new methods combine the approach of Goldfarb and Mehrotra for relaxing the requirement that certain projections be computed exactly with the approach of Todd and Burrell for generating an improving sequence of lower bounds forz ^* using dual feasible solutions. These methods retain the polynomial-time complexity of Karmarkar's algorithm.This research was supported in part by NSF Grants DMS-85-12277 and CDR-84-21402, and ONR Contract N0014-87-K0214.     

A dual dynamic programming for minimax optimal control problems governed by parabolic equation

In the article, minimax optimal control problems governed by parabolic equations are considered. We apply a new dual dynamic programming approach to derive sufficient optimality conditions for such problems. The idea is to move all the notions from a state space to a dual space and to obtain a new verification theorem providing the conditions, which should be satisfied by a solution of the dual partial differential equation of dynamic programming. We also give sufficient optimality conditions for the existence of an optimal dual feedback control and some approximation of the problem considered, which seems to be very useful from a practical point of view.     

Continuity of the optimal value function and optimal solutions of parametric mixed-integer quadratic programs

To properly describe and solve complex decision problems, research on theoretical properties and solution of mixed-integer quadratic programs is becoming very important. We establish in this paper different Lipschitz-type continuity results about the optimal value function and optimal solutions of mixed-integer parametric quadratic programs with parameters in the linear part of the objective function and in the right-hand sides of the linear constraints. The obtained results extend some existing results for continuous quadratic programs, and, more importantly, lay the foundation for further theoretical study and corresponding algorithm analysis on mixed-integer quadratic programs.     

Analysis of transient queues with semidefinite optimization

We derive upper bounds on the tail distribution of the transient waiting time in the GI/GI/1 queue, given a truncated sequence of the moments of the service time and that of the interarrival time. Our upper bound is given as the objective value of the optimal solution to a semidefinite program (SDP) and can be calculated numerically by solving the SDP. We also derive the upper bounds in closed form for the case when only the first two moments of the service time and those of the interarrival time are given. The upper bounds in closed form are constructed by formulating the dual problem associated with the SDP. Specifically, we obtain the objective value of a feasible solution of the dual problem in closed from, which turns out to be the upper bound that we derive. In addition, we study bounds on the maximum waiting time in the first busy period.     

Relaxation methods for problems with strictly convex separable costs and linear constraints

We consider the minimization problem with strictly convex, possibly nondifferentiable, separable cost and linear constraints. The dual of this problem is an unconstrained minimization problem with differentiable cost which is well suited for solution by parallel methods based on Gauss-Seidel relaxation. We show that these methods yield the optimal primal solution and, under additional assumptions, an optimal dual solution. To do this it is necessary to extend the classical Gauss-Seidel convergence results because the dual cost may not be strictly convex, and may have unbounded level sets. Work supported by the National Science Foundation under grant NSF-ECS-3217668.     

An efficient algorithm for the Lagrangean dual of nonlinear knapsack problems with additional nested constraints

This paper presents an efficient algorithm for solving the Lagrangean dual of nonlinear knapsack problems with additional nested constraints. The dual solution provides a feasible primal solution (if it exists) and associated lower and upper bounds on the optimal objective function value of the primal problem. Computational experience is cited indicating computation time, number of dual iterations, and “tightness” of the bounds.     

Complexity and algorithms for nonlinear optimization problems

Nonlinear optimization algorithms are rarely discussed from a complexity point of view. Even the concept of solving nonlinear problems on digital computers is not well defined. The focus here is on a complexity approach for designing and analyzing algorithms for nonlinear optimization problems providing optimal solutions with prespecified accuracy in the solution space. We delineate the complexity status of convex problems over network constraints, dual of flow constraints, dual of multi-commodity, constraints defined by a submodular rank function (a generalized allocation problem), tree networks, diagonal dominant matrices, and nonlinear knapsack problem’s constraint. All these problems, except for the latter in integers, have polynomial time algorithms which may be viewed within a unifying framework of a proximity-scaling technique or a threshold technique. The complexity of many of these algorithms is furthermore best possible in that it matches lower bounds on the complexity of the respective problems. In general nonseparable optimization problems are shown to be considerably more difficult than separable problems. We compare the complexity of continuous versus discrete nonlinear problems and list some major open problems in the area of nonlinear optimization. An earlier version of this paper appeared in 4OR, 3:3, 171–216, 2005.     

Optima of dual integer linear programs

We consider dual pairs of packing and covering integer linear programs. Best possible bounds are found between their optimal values. Tight inequalities are obtained relating the integral optima and the optimal rational solutions.     

Sequential location of two facilities: comparing random to optimal location of the first facility

We review our recent results in the development of optimal algorithms for the minimization of a strictly convex quadratic function subject to separable convex inequality constraints and/or linear equality constraints. A unique feature of our algorithms is the theoretically supported bound on the rate of convergence in terms of the bounds on the spectrum of the Hessian of the cost function, independent of representation of the constraints. When applied to the class of convex QP or QPQC problems with the spectrum in a given positive interval and a sparse Hessian matrix, the algorithms enjoy optimal complexity, i.e., they can find an approximate solution at the cost that is proportional to the number of unknowns. The algorithms do not assume representation of the linear equality constraints by full rank matrices. The efficiency of our algorithms is demonstrated by the evaluation of the projection of a point to the intersection of the unit cube and unit sphere with hyperplanes.     

Nonasymptotic Estimates for the Closeness of Gaussian Measures on Balls

Upper bounds for the closeness of two centered Gaussian measures in the class of balls in a separable Hilbert space are obtained. The bounds are optimal with respect to the dependence on the spectra of the covariance operators of the Gaussian measures. The inequalities cannot be improved in the general case.