Duality and statistical tests of optimality for two stage stochastic programs

We present alternative methods for verifying the quality of a proposed solution to a two stage stochastic program with recourse. Our methods revolve around implications of a dual problem in which dual multipliers on the nonanticipativity constraints play a critical role. Using randomly sampled observations of the stochastic elements, we introduce notions of statistical dual feasibility and sampled error bounds. Additionally, we use the nonanticipativity multipliers to develop connections to reduced gradient methods. Finally, we propose a statistical test based on directional derivatives. We illustrate the applicability of these tests via some examples. This work was supported in part by Grant No. NSF-DMI-9414680 from the National Science Foundation     

On a mixture of the fix-and-relax coordination and Lagrangian substitution schemes for multistage stochastic mixed integer programming

We present a framework for solving large-scale multistage mixed 0–1 optimization problems under uncertainty in the coefficients of the objective function, the right-hand side vector, and the constraint matrix. A scenario tree-based scheme is used to represent the Deterministic Equivalent Model of the stochastic mixed 0–1 program with complete recourse. The constraints are modeled by a splitting variable representation via scenarios. So, a mixed 0–1 model for each scenario cluster is considered, plus the nonanticipativity constraints that equate the 0–1 and continuous so-called common variables from the same group of scenarios in each stage. Given the high dimensions of the stochastic instances in the real world, it is not realistic to obtain the optimal solution for the problem. Instead we use the so-called Fix-and-Relax Coordination (FRC) algorithm to exploit the characteristics of the nonanticipativity constraints of the stochastic model. A mixture of the FRC approach and the Lagrangian Substitution and Decomposition schemes is proposed for satisfying, both, the integrality constraints for the 0–1 variables and the nonanticipativity constraints. This invited paper is discussed in the comments available at: doi:, doi:, doi:, doi:.     

Feasible Interior Methods Using Slacks for Nonlinear Optimization

A slack-based feasible interior point method is described which can be derived as a modification of infeasible methods. The modification is minor for most line search methods, but trust region methods require special attention. It is shown how the Cauchy point, which is often computed in trust region methods, must be modified so that the feasible method is effective for problems containing both equality and inequality constraints. The relationship between slack-based methods and traditional feasible methods is discussed. Numerical results using the KNITRO package show the relative performance of feasible versus infeasible interior point methods.     

Duality Gaps in Stochastic Integer Programming

In this note, we explore the implications of a result that suggests that the duality gap caused by a Lagrangian relaxation of the nonanticipativity constraints in a stochastic mixed integer (binary) program diminishes as the number of scenarios increases. By way of an example, we illustrate that this is not the case. In general, the duality gap remains bounded away from zero.     

A Riccati-based primal interior point solver for multistage stochastic programming

We propose a new method for certain multistage stochastic programs with linear or nonlinear objective function, combining a primal interior point approach with a linear-quadratic control problem over the scenario tree. The latter problem, which is the direction finding problem for the barrier subproblem is solved through dynamic programming using Riccati equations. In this way we combine the low iteration count of interior point methods with an efficient solver for the subproblems. The computational results are promising. We have solved a financial problem with 1,000,000 scenarios, 15,777,740 variables and 16,888,850 constraints in 20 hours on a moderate computer.     

Two-stage linear decision rules for multi-stage stochastic programming

Multi-stage stochastic linear programs (MSLPs) are notoriously hard to solve in general. Linear decision rules (LDRs) yield an approximation of an MSLP by restricting the decisions at each stage to be an affine function of the observed uncertain parameters. Finding an optimal LDR is a static optimization problem that provides an upper bound on the optimal value of the MSLP, and, under certain assumptions, can be formulated as an explicit linear program. Similarly, as proposed by Kuhn et al. (Math Program 130(1):177–209, 2011) a lower bound for an MSLP can be obtained by restricting decisions in the dual of the MSLP to follow an LDR. We propose a new approximation approach for MSLPs, two-stage LDRs. The idea is to require only the state variables in an MSLP to follow an LDR, which is sufficient to obtain an approximation of an MSLP that is a two-stage stochastic linear program (2SLP). We similarly propose to apply LDR only to a subset of the variables in the dual of the MSLP, which yields a 2SLP approximation of the dual that provides a lower bound on the optimal value of the MSLP. Although solving the corresponding 2SLP approximations exactly is intractable in general, we investigate how approximate solution approaches that have been developed for solving 2SLP can be applied to solve these approximation problems, and derive statistical upper and lower bounds on the optimal value of the MSLP. In addition to potentially yielding better policies and bounds, this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static LDR approach. A computational study on two example problems demonstrates that using a two-stage LDR can yield significantly better primal policies and modestly better dual policies than using policies based on a static LDR.

     

A Parallel Interior Point Method and Its Application to Facility Location Problems

We present a parallel interior point algorithm to solve block structured linear programs. This algorithm can solve block diagonal linear programs with both side constraints (common rows) and side variables (common columns). The performance of the algorithm is investigated on uncapacitated, capacitated and stochastic facility location problems. The facility location problems are formulated as mixed integer linear programs. Each subproblem of the branch and bound phase of the MIP is solved using the parallel interior point method. We compare the total time taken by the parallel interior point method with the simplex method to solve the complete problems, as well as the various costs of reoptimisation of the non-root nodes of the branch and bound. Computational results on two parallel computers (Fujitsu AP1000 and IBM SP2) are also presented in this paper.     

Risk exposure and Lagrange multipliers of nonanticipativity constraints in multistage stochastic problems

We take advantage of the interpretation of stochastic capacity expansion problems as stochastic equilibrium models for assessing the risk exposure of new equipment in a competitive electricity economy. We develop our analysis on a standard multistage generation capacity expansion problem. We focus on the formulation with nonanticipativity constraints and show that their dual variables can be interpreted as the net margin accruing to plants in the different states of the world. We then propose a procedure to estimate the distribution of the Lagrange multipliers of the nonanticipativity constraints associated with first stage decisions; this gives us the distribution of the discounted cash flow of profitable plants in that stage.     

A constraint shifting homotopy method for general non-linear programming

In this article, a constraint shifting homotopy method (CSHM) is proposed for solving non-linear programming with both equality and inequality constraints. A new homotopy is constructed, and existence and global convergence of a homotopy path determined by it are proven. All problems that can be solved by the combined homotopy interior point method (CHIPM) can also be solved by the proposed method. In contrast to the combined homotopy infeasible interior point method (CHIIPM), it needs a weaker regularity condition. And the starting point in the proposed method is not necessarily a feasible point or an interior point, so it is more convenient to be implemented than CHIPM and CHIIPM. Numerical results show that the proposed algorithm is feasible and effective.     

The augmented system variant of IPMs in two-stage stochastic linear programming computation

The application of interior point methods (IPM) to solve the deterministic equivalent of two-stage stochastic linear programming problems is a known and natural idea. Experiments have proved that among the interior point methods, the augmented system approach gives the best performance on these problems. However, most of their implementations encounter numerical difficulties in certain cases, which can result in loss of efficiency. We present a new approach for the decomposition of the augmented system, which ‘automatically’ exploits the special behavior of the problems. We show that the suggested approach can be implemented in a fast and numerically robust way by solving a number of large-scale two-stage stochastic linear programming problems. The comparison of our solver with fo1aug, which is considered as a state-of-the-art augmented system implementation of interior point methods, is also given.     

Subregular recourse in nonlinear multistage stochastic optimization

We consider nonlinear multistage stochastic optimization problems in the spaces of integrable functions. We allow for nonlinear dynamics and general objective functionals, including dynamic risk measures. We study causal operators describing the dynamics of the system and derive the Clarke subdifferential for a penalty function involving such operators. Then we introduce the concept of subregular recourse in nonlinear multistage stochastic optimization and establish subregularity of the resulting systems in two formulations: with built-in nonanticipativity and with explicit nonanticipativity constraints. Finally, we derive optimality conditions for both formulations and study their relations.

     

Projected orthogonal vectors in two-dimensional search interior point algorithms for linear programming

The vast majority of linear programming interior point algorithms successively move from an interior solution to an improved interior solution by following a single search direction, which corresponds to solving a one-dimensional subspace linear program at each iteration. On the other hand, two-dimensional search interior point algorithms select two search directions, and determine a new and improved interior solution by solving a two-dimensional subspace linear program at each step. This paper presents primal and dual two-dimensional search interior point algorithms derived from affine and logarithmic barrier search directions. Both search directions are determined by randomly partitioning the objective function into two orthogonal vectors. Computational experiments performed on benchmark instances demonstrate that these new methods improve the average CPU time by approximately 12% and the average number of iterations by 14%.

     

A Scalable Parallel Interior Point Algorithm for Stochastic Linear Programming and Robust Optimization

We present a computationally efficient implementation of an interior point algorithm for solving large-scale problems arising in stochastic linear programming and robust optimization. A matrix factorization procedure is employed that exploits the structure of the constraint matrix, and it is implemented on parallel computers. The implementation is perfectly scalable. Extensive computational results are reported for a library of standard test problems from stochastic linear programming, and also for robust optimization formulations.The results show that the codes are efficient and stable for problems with thousands of scenarios. Test problems with 130 thousand scenarios, and a deterministic equivalent linear programming formulation with 2.6 million constraints and 18.2 million variables, are solved successfully.     

A Note on the Recursive and Parallel Structure of the Birge and Qi Factorization for Tree Structured Linear Programs

In this note, we derive the complete recursive structure of the Birge and Qi factorization for interior point methods (IPM) for tree structured linear programs as they appear in multistage stochastic programs. This recursive structure allows for an elegant implementation on parallel hardware, since multiple versions of the same program may be run on on different processors. Our preliminary computational experiment, conducted on a Beowulf cluster, demonstrates the scalability of this approach.     

Efficient solution of two-stage stochastic linear programs using interior point methods

Solving deterministic equivalent formulations of two-stage stochastic linear programs using interior point methods may be computationally difficult due to the need to factorize quite dense search direction matrices (e.g., AA ^T ). Several methods for improving the algorithmic efficiency of interior point algorithms by reducing the density of these matrices have been proposed in the literature. Reformulating the program decreases the effort required to find a search direction, but at the expense of increased problem size. Using transpose product formulations (e.g., A ^T A) works well but is highly problem dependent. Schur complements may require solutions with potentially near singular matrices. Explicit factorizations of the search direction matrices eliminate these problems while only requiring the solution to several small, independent linear systems. These systems may be distributed across multiple processors. Computational experience with these methods suggests that substantial performance improvements are possible with each method and that, generally, explicit factorizations require the least computational effort.     

Generalized stationary points and an interior-point method for mathematical programs with equilibrium constraints

Generalized stationary points of the mathematical program with equilibrium constraints (MPEC) are studied to better describe the limit points produced by interior point methods for MPEC. A primal-dual interior-point method is then proposed, which solves a sequence of relaxed barrier problems derived from MPEC. Global convergence results are deduced under fairly general conditions other than strict complementarity or the linear independence constraint qualification for MPEC (MPEC-LICQ). It is shown that every limit point of the generated sequence is a strong stationary point of MPEC if the penalty parameter of the merit function is bounded. Otherwise, a point with certain stationarity can be obtained. Preliminary numerical results are reported, which include a case analyzed by Leyffer for which the penalty interior-point algorithm failed to find a stationary point.Mathematics Subject Classification (1991):90C30, 90C33, 90C55, 49M37, 65K10     

求解退化单调线性互补问题极大互补解的复杂性

本文考虑求解退化单调线性互补问题的一类不可行内点算法，其中嵌入一个恢复算法，给出了用这类算法产生所考虑问题的一个精确极大互补解的复杂性．     

Response surface methodology with stochastic constraints for expensive simulation

This article investigates simulation-based optimization problems with a stochastic objective function, stochastic output constraints, and deterministic input constraints. More specifically, it generalizes classic response surface methodology (RSM) to account for these constraints. This Generalized RSM—abbreviated to GRSM—generalizes the estimated steepest descent—used in classic RSM—applying ideas from interior point methods, especially affine scaling. This new search direction is scale independent, which is important for practitioners because it avoids some numerical complications and problems commonly encountered. Furthermore, the article derives a heuristic that uses this search direction iteratively. This heuristic is intended for problems in which simulation runs are expensive, so that the search needs to reach a neighbourhood of the true optimum quickly. The new heuristic is compared with OptQuest, which is the most popular heuristic available with several simulation software packages. Numerical illustrations give encouraging results.     

Logarithmic barrier decomposition-based interior point methods for stochastic symmetric programming

We introduce and study two-stage stochastic symmetric programs with recourse to handle uncertainty in data defining (deterministic) symmetric programs in which a linear function is minimized over the intersection of an affine set and a symmetric cone. We present a Benders’ decomposition-based interior point algorithm for solving these problems and prove its polynomial complexity. Our convergence analysis proved by showing that the log barrier associated with the recourse function of stochastic symmetric programs behaves a strongly self-concordant barrier and forms a self-concordant family on the first stage solutions. Since our analysis applies to all symmetric cones, this algorithm extends Zhao’s results [G. Zhao, A log barrier method with Benders’ decomposition for solving two-stage stochastic linear programs, Math. Program. Ser. A 90 (2001) 507–536] for two-stage stochastic linear programs, and Mehrotra and Özevin’s results [S. Mehrotra, M.G. Özevin, Decomposition-based interior point methods for two-stage stochastic semidefinite programming, SIAM J. Optim. 18 (1) (2007) 206–222] for two-stage stochastic semidefinite programs.     

Certificates of Primal or Dual Infeasibility in Linear Programming

In general if a linear program has an optimal solution, then a primal and dual optimal solution is a certificate of the solvable status. Furthermore, it is well known that in the solvable case, then the linear program always has an optimal basic solution. Similarly, when a linear program is primal or dual infeasible then by Farkas's Lemma a certificate of the infeasible status exists. However, in the primal or dual infeasible case then there is not an uniform definition of what a suitable basis certificate of the infeasible status is.In this work we present a definition of a basis certificate and develop a strongly polynomial algorithm which given a Farkas type certificate of infeasibility computes a basis certificate of infeasibility. This result is relevant for the recently developed interior-point methods because they do not compute a basis certificate of infeasibility in general. However, our result demonstrates that a basis certificate can be obtained at a moderate computational cost.