Progressive hedging and tabu search applied to mixed integer (0,1) multistage stochastic programming

Many problems faced by decision makers are characterized by a multistage decision process with uncertainty about the future and some decisions constrained to take on values of either zero or one (for example, either open a facility at a location or do not open it). Although some mathematical theory exists concerning such problems, no general-purpose algorithms have been available to address them. In this article, we introduce the first implementation of general purpose methods for finding good solutions to multistage, stochastic mixed-integer (0, 1) programming problems. The solution method makes use of Rockafellar and Wets' progressive hedging algorithm that averages solutions rather than data. Solutions to the induced quadratic (0,1) mixed-integer subproblems are obtained using a tabu search algorithm. We introduce the notion of integer convergence for progressive hedging. Computational experiments verify that the method is effective. The software that we have developed reads standard (SMPS) data files.     

Variational analysis of functions of the roots of polynomials

The Gauss-Lucas Theorem on the roots of polynomials nicely simplifies the computation of the subderivative and regular subdifferential of the abscissa mapping on polynomials (the maximum of the real parts of the roots). This paper extends this approach to more general functions of the roots. By combining the Gauss-Lucas methodology with an analysis of the splitting behavior of the roots, we obtain characterizations of the subderivative and regular subdifferential for these functions as well. In particular, we completely characterize the subderivative and regular subdifferential of the radius mapping (the maximum of the moduli of the roots). The abscissa and radius mappings are important for the study of continuous and discrete time linear dynamical systems. Dedicated to R. Tyrrell Rockafellar on the occasion of his 70th birthday. Terry is one of those rare individuals who combine a broad vision, deep insight, and the outstanding writing and lecturing skills crucial for engaging others in his subject. With these qualities he has won universal respect as a founding father of our discipline. We, and the broader mathematical community, owe Terry a great deal. But most of all we are personally thankful to Terry for his friendship and guidance. Research supported in part by the National Science Foundation Grant DMS-0203175. Research supported in part by the Natural Sciences and Engineering Research Council of Canada. Research supported in part by the National Science Foundation Grant DMS-0412049.     

An Inexact Proximal-Type Algorithm in Banach Spaces

In this paper, we investigate the strong convergence of an inexact proximal-point algorithm. It is known that the proximal-point algorithm converges weakly to a solution of a maximal monotone operator, but fails to converge strongly. Solodov and Svaiter (Math. Program. 87:189–202, 2000) introduced a new proximal-type algorithm to generate a strongly convergent sequence and established a convergence result in Hilbert space. Subsequently, Kamimura and Takahashi (SIAM J. Optim. 13:938–945, 2003) extended the Solodov and Svaiter result to the setting of uniformly convex and uniformly smooth Banach space. On the other hand, Rockafellar (SIAM J. Control Optim. 14:877–898, 1976) gave an inexact proximal-point algorithm which is more practical than the exact one. Our purpose is to extend the Kamimura and Takahashi result to a new inexact proximal-type algorithm. Moreover, this result is applied to the problem of finding the minimizer of a convex function on a uniformly convex and uniformly smooth Banach space. L.C. Zeng’s research was partially supported by the Teaching and Research Award Fund for Outstanding Young Teachers in Higher Education Institutions of MOE, China and by the Dawn Program Foundation in Shanghai. J.C. Yao’s research was partially supported by the National Science Council of the Republic of China.     

The Elicited Progressive Decoupling Algorithm: A Note on the Rate of Convergence and a Preliminary Numerical Experiment on the Choice of Parameters

Set-Valued and Variational Analysis - The paper studies the progressive decoupling algorithm (PDA) of Rockafellar and focuses on the elicited version of the algorithm. Based on a generalized...     

Progressive hedging as a meta-heuristic applied to stochastic lot-sizing

In a great many situations, the data for optimization problems cannot be known with certainty and furthermore the decision process will take place in multiple time stages as the uncertainties are resolved. This gives rise to a need for stochastic programming (SP) methods that create solutions that are hedged against future uncertainty. The progressive hedging algorithm (PHA) of Rockafellar and Wets is a general method for SP. We cast the PHA in a meta-heuristic framework with the sub-problems generated for each scenario solved heuristically. Rather than using an approximate search algorithm for the exact problem as is typically the case in the meta-heuristic literature, we use an algorithm for sub-problems that is exact in its usual context but serves as a heuristic for our meta-heuristic. Computational results reported for stochastic lot-sizing problems demonstrate that the method is effective.     

Weak sharp minima revisited, part II: application to linear regularity and error bounds

The notion of weak sharp minima is an important tool in the analysis of the perturbation behavior of certain classes of optimization problems as well as in the convergence analysis of algorithms designed to solve these problems. It has been studied extensively by several authors. This paper is the second of a series on this subject where the basic results on weak sharp minima in Part I are applied to a number of important problems in convex programming. In Part II we study applications to the linear regularity and bounded linear regularity of a finite collection of convex sets as well as global error bounds in convex programming. We obtain both new results and reproduce several existing results from a fresh perspective. We dedicate this paper to our friend and mentor Terry Rockafellar on the occasion of his 70th birthday. He has been our guide in mathematics as well as in the backcountry and waterways of the Olympic and Cascade mountains. Research supported in part by the National Science Foundation Grant DMS-0203175.     

Collusive game solutions via optimization

A Nash-based collusive game among a finite set of players is one in which the players coordinate in order for each to gain higher payoffs than those prescribed by the Nash equilibrium solution. In this paper, we study the optimization problem of such a collusive game in which the players collectively maximize the Nash bargaining objective subject to a set of incentive compatibility constraints. We present a smooth reformulation of this optimization problem in terms of a nonlinear complementarity problem. We establish the convexity of the optimization problem in the case where each player's strategy set is unidimensional. In the multivariate case, we propose upper and lower bounding procedures for the collusive optimization problem and establish convergence properties of these procedures. Computational results with these procedures for solving some test problems are reported. It is with great honor that we dedicate this paper to Professor Terry Rockafellar on the occasion of his 70th birthday. Our work provides another example showing how Terry's fundamental contributions to convex and variational analysis have impacted the computational solution of applied game problems. This author's research was partially supported by the National Science Foundation under grant ECS-0080577. This author's research was partially supported by the National Science Foundation under grant CCR-0098013.     

Optimality conditions for non-finite valued convex composite functions

Burke (1987) has recently developed second-order necessary and sufficient conditions for convex composite optimization in the case where the convex function is finite valued. In this note we present a technique for reducing the infinite valued case to the finite valued one. We then use this technique to extend the results in Burke (1987) to the case in which the convex function may take infinite values. We conclude by comparing these results with those established by Rockafellar (1989) for the piecewise linear-quadratic case.Dedicated to the memory of Robin W. ChaneyResearch supported in part by the National Science Foundation under grants DMS-8602399 and DMS-8803206, and by the Air Force Office of Scientific Research under grant ISSA-860080.Research supported in part by the Natural Sciences and Engineering Research Council of Canada under grant OGP41983.     

Generalized monotonicity and generalized convexity

Generalized monotonocity of bifunctions or multifunctions is a rather new concept in optimization and nonsmooth analysis. It is shown in the present paper how quasiconvexity, pseudoconvexity, and strict pseudoconvexity of lower semicontinuous functions can be characterized via the quasimonotonicity, pseudomonotonicity, and strict pseudomonotonicity of different types of generalized derivatives, including the Dini, Dini-Hadamard, Clarke, and Rockafellar derivatives as well.This research was supported by the National Science Foundation of Hungary, Grant No. OTKA 1313/1991.     

An SQP algorithm for extended linear-quadratic problems in stochastic programming

Extended Linear-Quadratic Programming (ELQP) problems were introduced by Rockafellar and Wets for various models in stochastic programming and multistage optimization. Several numerical methods with linear convergence rates have been developed for solving fully quadratic ELQP problems, where the primal and dual coefficient matrices are positive definite. We present a two-stage sequential quadratic programming (SQP) method for solving ELQP problems arising in stochastic programming. The first stage algorithm realizes global convergence and the second stage algorithm realizes superlinear local convergence under a condition calledB-regularity.B-regularity is milder than the fully quadratic condition; the primal coefficient matrix need not be positive definite. Numerical tests are given to demonstrate the efficiency of the algorithm. Solution properties of the ELQP problem underB-regularity are also discussed.Supported by the Australian Research Council.     

An algorithm for minimax

An algorithm is proposed for computing an unconstrained minimax, based on differential equations with suitable stabilization terms. Methods for accelerating the convergence are discussed. For computing a constrained minimax, the augmented Lagrangian algorithm of Powell, Hestenes and Rockafellar is generalized to minimax, assuming the unconstrained minimax algorithm as a subroutine. An estimate of the convergence rate is obtained.     

Two general methods for computing saddle points with applications for decomposing convex programming problems

Two general methods are presented for computing saddle points in finite dimensional spaces. One of these methods is related to the column method given by Dantzig and Wolfe and also to an algorithm given recently by Rockafellar and Wets. The other method is a modification of the above in a more symmetric way. The principal application of these methods is to give new decomposition methods for solving separable convex programming.     

A scenario aggregation algorithm for the solution of stochastic dynamic minimax problems

In this paper, we present a scenario aggregation algorithm for the solution of the dynamic minimax problem in stochastic programming. We consider the case where the joint probability distribution has a known finite support. The algorithm applies the Alternating Direction of Multipliers Method on a reformulation of the minimax problem using a double duality framework. The problem is solved by decomposition into scenario sub-problems, which are deterministic multi-period problems. Convergence properties are deduced from the Alternating Direction of Multipliers. The resulting algorithm can be seen as an extension of Rockafellar and Wets Progressive Hedging algorithm to the dynamic minimax context.     

The perturbed proximal point algorithm and some of its applications

Following the works of R. T. Rockafellar, to search for a zero of a maximal monotone operator, and of B. Lemaire, to solve convex optimization problems, we present a perturbed version of the proximal point algorithm. We apply this new algorithm to convex optimization and to variational inclusions or, more particularly, to variational inequalities.     

Generalized proximal point algorithm for convex optimization

Ha (Ref. 1) recently introduced a generalized proximal point algorithm for solving a generalized equation. In this note, we present a generalized proximal point algorithm for convex optimization problems based on Ha's work. The idea behind this algorithm is that, instead of adding a quadratic term to all the variables, we add a quadratic term to a subset of the variables. We extend the criteria for approximate solutions given by Rockafellar (Ref. 2) and Auslender (Ref. 3) and present convergence results. Finally, we show how this algorithm can be applied to solve block-angular linear and quadratic programming problems.     

Hybrid inexact proximal point algorithms based on RMM frameworks with applications to variational inclusion problems

A new application-oriented notion of relatively A-maximal monotonicity (RMM) framework is introduced, and then it is applied to the approximation solvability of a general class of inclusion problems, while generalizing other existing results on linear convergence, including Rockafellar’s theorem (1976) on linear convergence using the proximal point algorithm in a real Hilbert space setting. The obtained results not only generalize most of the existing investigations, but also reduce smoothly to the case of the results on maximal monotone mappings and corresponding classical resolvent operators. Furthermore, our proof approach differs significantly to that of Rockafellar’s celebrated work, where the Lipschitz continuity of M ^?1, the inverse of M:X→2 ^X , at zero is assumed to achieve a linear convergence of the proximal point algorithm. Note that the notion of relatively A-maximal monotonicity framework seems to be used to generalize the classical Yosida approximation (which is applied and studied mostly based on the classical resolvent operator in the literature) that in turn can be applied to first-order evolution equations as well as evolution inclusions.     

General A-P iterative algorithm in shift-invariant spaces

A general A-P iterative algorithm in a shift-invariant space is presented. We use the algorithm to show reconstruction of signals from weighted samples and also show that the general improved algorithm has better convergence rate than the existing one. An explicit estimate for a guaranteed rate of convergence is given.     

Epsilon-proximal decomposition method

We propose a modification of the proximal decomposition method investigated by Spingarn [30] and Mahey et al. [19] for minimizing a convex function on a subspace. For the method to be favorable from a computational point of view, particular importance is the introduction of approximations in the proximal step. First, we couple decomposition on the graph of the epsilon-subdifferential mapping and cutting plane approximations to get an algorithmic pattern that falls in the general framework of Rockafellar inexact proximal-point algorithms [26]. Recently, Solodov and Svaiter [27] proposed a new proximal point-like algorithm that uses improved error criteria and an enlargement of the maximal monotone operator defining the problem. We combine their idea with bundle mecanism to devise an inexact proximal decomposition method with error condition which is not hard to satisfy in practice. Then, we present some applications favorable to our development. First, we give a new regularized version of Benders decomposition method in convex programming called the proximal convex Benders decomposition algorithm. Second, we derive a new algorithm for nonlinear multicommodity flow problems among which the message routing problem in telecommunications data networks.     

Extended scenario analysis

Scenario analysis, as proposed by Rockafellar and Wets, is a stochastic programming technique employing discrete scenarios with known probabilities, usually covering several time periods. The requirement of nonaticipativity (not using future information to make present decisions) is enforced during the computational solution by using Spingarn's method of partial inverses. The scenario analysis method as proposed relies on separability (with respect to scenarios) of all problem elements except the nonanticipativity constraint.We show how, by making a little more use of the partial inverse technique, one can include nonseparable convex constraints in such a procedure. As an illustrative example, we show how to analyze a portfolio optimization problem of Markowitz type (minimize variance for a given return) using scenarios. This offers the prospect of extending classical portfolio analysis from models based on historical behavior to models incorporating future scenarios of any desired type.The research reported here was sponsored by the National Science Foundation under Grant CCR-8801489, and by the Air Force Systems Command, USAF, under Grant No. AFOSR-89-0058. The US Government has certain rights in this material, and is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.     

A superlinearly and quadratically convergent SQP type feasible method for constrained optimization

A new SQP type feasible method for inequality constrained optimization is presented, it is a combination of a master algorithm and an auxiliary algorithm which is taken only in finite iterations. The directions of the master algorithm are generated by only one quadratic programming, and its step-size is always one, the directions of the auxiliary algorithm are new “secondorder“ feasible descent. Under suitable assumptions, the algorithm is proved to possess global and strong convergence, superlinear and quadratic convergence.