The achievable region method in the optimal control of queueing systems; formulations,bounds and policies

We survey a new approach that the author and his co-workers have developed to formulate stochastic control problems (predominantly queueing systems) asmathematical programming problems. The central idea is to characterize the region of achievable performance in a stochastic control problem, i.e., find linear or nonlinear constraints on the performance vectors that all policies satisfy. We present linear and nonlinear relaxations of the performance space for the following problems: Indexable systems (multiclass single station queues and multiarmed bandit problems), restless bandit problems, polling systems, multiclass queueing and loss networks. These relaxations lead to bounds on the performance of an optimal policy. Using information from the relaxations we construct heuristic nearly optimal policies. The theme in the paper is the thesis that better formulations lead to deeper understanding and better solution methods. Overall the proposed approach for stochastic control problems parallels efforts of the mathematical programming community in the last twenty years to develop sharper formulations (polyhedral combinatorics and more recently nonlinear relaxations) and leads to new insights ranging from a complete characterization and new algorithms for indexable systems to tight lower bounds and nearly optimal algorithms for restless bandit problems, polling systems, multiclass queueing and loss networks.     

Computational complexity of stochastic programming problems

Stochastic programming is the subfield of mathematical programming that considers optimization in the presence of uncertainty. During the last four decades a vast quantity of literature on the subject has appeared. Developments in the theory of computational complexity allow us to establish the theoretical complexity of a variety of stochastic programming problems studied in this literature. Under the assumption that the stochastic parameters are independently distributed, we show that two-stage stochastic programming problems are ♯P-hard. Under the same assumption we show that certain multi-stage stochastic programming problems are PSPACE-hard. The problems we consider are non-standard in that distributions of stochastic parameters in later stages depend on decisions made in earlier stages. Supported by the EPSRC grant ``Phase Transitions in the Complexity of Randomised Algorithms', by the EC IST project RAND-APX, and by the MRT Network ADONET of the European Community (MRTN-CT-2003-504438).     

Probabilistic programming with discrete distributions and precedence constrained knapsack polyhedra

 We consider stochastic programming problems with probabilistic constraints involving random variables with discrete distributions. They can be reformulated as large scale mixed integer programming problems with knapsack constraints. Using specific properties of stochastic programming problems and bounds on the probability of the union of events we develop new valid inequalities for these mixed integer programming problems. We also develop methods for lifting these inequalities. These procedures are used in a general iterative algorithm for solving probabilistically constrained problems. The results are illustrated with a numerical example. Received: October 8, 2000 / Accepted: August 13, 2002 Published online: September 27, 2002 Key words. stochastic programming – integer programming – valid inequalities     

Modeling, simulation and optimization in logistics

This text summarizes the PhD thesis defended by the author in March 2009 under the supervision of Pasquale Legato at the University of Calabria, Italy. The thesis is written in English and is available for download at the following URL: . It aims to explore friendly Operations Research tools for modeling and simulation of logistics processes, with particular interest for mathematical programming models combined with stochastic simulation tools. In particular key assignment and scheduling problems that arise in maritime container terminals are explored. Initially it is presented a study on different modeling paradigms devoted to the representation of logistical processes and the formalization of problems with complex scheduling/assignment constraints. Successively an IP model for managing the assignment of a pool of rail-mounted gantry cranes to berthed vessels is proposed. Then, according to a functional integration approach, a second model centered on the intra-ship scheduling of vessel container bulks to the assigned cranes is further formulated. Finally a simulation-based optimization approach is investigated and the effectiveness of recent search methods is evaluated by comparison with a commercial solver.     

On portfolio optimization using fuzzy decisions

We consider stochastic optimization problems involving stochastic dominance constraints. We develop portfolio optimization model involving stochastic dominance constrains using fuzzy decisions and we concentrate on fuzzy linear programming problems with only fuzzy technological coefficients and aplication/implementation of modified subgradient method to fuzy linear programming problems. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

An optimal method for stochastic composite optimization

This paper considers an important class of convex programming (CP) problems, namely, the stochastic composite optimization (SCO), whose objective function is given by the summation of general nonsmooth and smooth stochastic components. Since SCO covers non-smooth, smooth and stochastic CP as certain special cases, a valid lower bound on the rate of convergence for solving these problems is known from the classic complexity theory of convex programming. Note however that the optimization algorithms that can achieve this lower bound had never been developed. In this paper, we show that the simple mirror-descent stochastic approximation method exhibits the best-known rate of convergence for solving these problems. Our major contribution is to introduce the accelerated stochastic approximation (AC-SA) algorithm based on Nesterov’s optimal method for smooth CP (Nesterov in Doklady AN SSSR 269:543–547, 1983; Nesterov in Math Program 103:127–152, 2005), and show that the AC-SA algorithm can achieve the aforementioned lower bound on the rate of convergence for SCO. To the best of our knowledge, it is also the first universally optimal algorithm in the literature for solving non-smooth, smooth and stochastic CP problems. We illustrate the significant advantages of the AC-SA algorithm over existing methods in the context of solving a special but broad class of stochastic programming problems.     

Approximating infinite horizon stochastic optimal control in discrete time with constraints

Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are retained, and numerical solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimization problems with discounted cost criterion. Constraints on both states and controls are permitted, and modeled in the objective function by allowing it to take infinite values. Approximating techniques are developed using variational analysis, and intuitive lower bounds are obtained via averaging the future. These bounds could be used in a finite-time horizon stochastic programming setting to find solutions numerically. Research supported in part by a grant of the National Science Foundation. AMS Classification 46N10, 49N15, 65K10, 90C15, 90C46     

PySP: modeling and solving stochastic programs in Python

Although stochastic programming is a powerful tool for modeling decision-making under uncertainty, various impediments have historically prevented its wide-spread use. One factor involves the ability of non-specialists to easily express stochastic programming problems as extensions of their deterministic counterparts, which are typically formulated first. A second factor relates to the difficulty of solving stochastic programming models, particularly in the mixed-integer, non-linear, and/or multi-stage cases. Intricate, configurable, and parallel decomposition strategies are frequently required to achieve tractable run-times on large-scale problems. We simultaneously address both of these factors in our PySP software package, which is part of the Coopr open-source Python repository for optimization; the latter is distributed as part of IBM’s COIN-OR repository. To formulate a stochastic program in PySP, the user specifies both the deterministic base model (supporting linear, non-linear, and mixed-integer components) and the scenario tree model (defining the problem stages and the nature of uncertain parameters) in the Pyomo open-source algebraic modeling language. Given these two models, PySP provides two paths for solution of the corresponding stochastic program. The first alternative involves passing an extensive form to a standard deterministic solver. For more complex stochastic programs, we provide an implementation of Rockafellar and Wets’ Progressive Hedging algorithm. Our particular focus is on the use of Progressive Hedging as an effective heuristic for obtaining approximate solutions to multi-stage stochastic programs. By leveraging the combination of a high-level programming language (Python) and the embedding of the base deterministic model in that language (Pyomo), we are able to provide completely generic and highly configurable solver implementations. PySP has been used by a number of research groups, including our own, to rapidly prototype and solve difficult stochastic programming problems.     

A scenario decomposition algorithm for 0–1 stochastic programs

We propose a scenario decomposition algorithm for stochastic 0–1 programs. The algorithm recovers an optimal solution by iteratively exploring and cutting-off candidate solutions obtained from solving scenario subproblems. The scheme is applicable to quite general problem structures and can be implemented in a distributed framework. Illustrative computational results on standard two-stage stochastic integer programming and nonlinear stochastic integer programming test problems are presented.     

Applying oracles of on-demand accuracy in two-stage stochastic programming – A computational study

Traditionally, two variants of the L-shaped method based on Benders’ decomposition principle are used to solve two-stage stochastic programming problems: the aggregate and the disaggregate version. In this study we report our experiments with a special convex programming method applied to the aggregate master problem. The convex programming method is of the type that uses an oracle with on-demand accuracy. We use a special form which, when applied to two-stage stochastic programming problems, is shown to integrate the advantages of the traditional variants while avoiding their disadvantages. On a set of 105 test problems, we compare and analyze parallel implementations of regularized and unregularized versions of the algorithms. The results indicate that solution times are significantly shortened by applying the concept of on-demand accuracy.     

变分不等式与互补问题、双层规划与平衡约束数学规划问题的若干进展

考虑有限维变分不等式与互补问题、双层规划以及均衡约束的数学规划问题. 在简单介绍这些问题之后,重点介绍近年来这些领域中发展迅速的几个研究方向,包括对称锥互补问题的理论与算法、变分不等式的投影收缩算法、随机变分不等式与随机互补问题的模型与方法、双层规划以及均衡约束数学规划问题的新方法. 最后提出几个进一步研究的方向.     

Approximations for chance-constrained programming problems

This paper proposes an approximation approach to the solution of chance-constrained stochastic programming problems. The results rely in a fundamental way on the theory of convergence of sequences of measurable multifunctions. Particular results are presented for stochastic linear programming problems.     

Probability maximization models for portfolio selection under ambiguity

This paper considers several probability maximization models for multi-scenario portfolio selection problems in the case that future returns in possible scenarios are multi-dimensional random variables. In order to consider occurrence probabilities and decision makers’ predictions with respect to all scenarios, a portfolio selection problem setting a weight with flexibility to each scenario is proposed. Furthermore, by introducing aspiration levels to occurrence probabilities or future target profit and maximizing the minimum aspiration level, a robust portfolio selection problem is considered. Since these problems are formulated as stochastic programming problems due to the inclusion of random variables, they are transformed into deterministic equivalent problems introducing chance constraints based on the stochastic programming approach. Then, using a relation between the variance and absolute deviation of random variables, our proposed models are transformed into linear programming problems and efficient solution methods are developed to obtain the global optimal solution. Furthermore, a numerical example of a portfolio selection problem is provided to compare our proposed models with the basic model.     

Stochastic programming approach to optimization under uncertainty

In this paper we discuss computational complexity and risk averse approaches to two and multistage stochastic programming problems. We argue that two stage (say linear) stochastic programming problems can be solved with a reasonable accuracy by Monte Carlo sampling techniques while there are indications that complexity of multistage programs grows fast with increase of the number of stages. We discuss an extension of coherent risk measures to a multistage setting and, in particular, dynamic programming equations for such problems.      

L-shaped decomposition of two-stage stochastic programs with integer recourse

We consider two-stage stochastic programming problems with integer recourse. The L-shaped method of stochastic linear programming is generalized to these problems by using generalized Benders decomposition. Nonlinear feasibility and optimality cuts are determined via general duality theory and can be generated when the second stage problem is solved by standard techniques. Finite convergence of the method is established when Gomory’s fractional cutting plane algorithm or a branch-and-bound algorithm is applied.     

Application-oriented mixed integer non-linear programming

This is a summary of the author’s PhD thesis supervised by Andrea Lodi and defended on 16 April 2009 at the University of Bologna. The thesis is written in English and available for download at . The main topic of the thesis is Mixed Integer Non-Linear Programming, with focus on non-convex problems (i.e., problems for which the feasible region of the continuous relaxation is a non-convex set) and real-world applications. Different kinds of algorithms are presented: linearization methods, heuristic and global optimization algorithms. Also, different kinds of real-world applications are solved, arising, for example, from Hydraulic and Electrical Engineering problems. The last part of the thesis is devoted to software and tools for mixed integer non-linear programming problems.     

极大极小随机规划逼近问题最优解集和最优值的稳定性

研究了特殊的二层极大极小随机规划逼近收敛问题. 首先将下层初始随机规划最优解集拓展到非单点集情形, 且可行集正则的条件下, 讨论了下层随机规划逼近问题最优解集关于上层决策变量参数的上半收敛性和最优值函数的连续性. 然后把下层随机规划的epsilon-最优解向量函数反馈到上层随机规划的目标函数中, 得到了上层随机规划逼近问题的最优解集关于最小信息概率度量收敛的上半收敛性和最优值的连续性.     

Two-Stage Stochastic Variational Inequality Arising from Stochastic Programming

We consider a two-stage stochastic variational inequality arising from a general convex two-stage stochastic programming problem, where the random variables have continuous distributions. The equivalence between the two problems is shown under some moderate conditions, and the monotonicity of the two-stage stochastic variational inequality is discussed under additional conditions. We provide a discretization scheme with convergence results and employ the progressive hedging method with double parameterization to solve the discretized stochastic variational inequality. As an application, we show how the water resources management problem under uncertainty can be transformed from a two-stage stochastic programming problem to a two-stage stochastic variational inequality, and how to solve it, using the discretization scheme and the progressive hedging method with double parameterization.

     

Decomposition with branch-and-cut approaches for two-stage stochastic mixed-integer programming

Decomposition has proved to be one of the more effective tools for the solution of large-scale problems, especially those arising in stochastic programming. A decomposition method with wide applicability is Benders' decomposition, which has been applied to both stochastic programming as well as integer programming problems. However, this method of decomposition relies on convexity of the value function of linear programming subproblems. This paper is devoted to a class of problems in which the second-stage subproblem(s) may impose integer restrictions on some variables. The value function of such integer subproblem(s) is not convex, and new approaches must be designed. In this paper, we discuss alternative decomposition methods in which the second-stage integer subproblems are solved using branch-and-cut methods. One of the main advantages of our decomposition scheme is that Stochastic Mixed-Integer Programming (SMIP) problems can be solved by dividing a large problem into smaller MIP subproblems that can be solved in parallel. This paper lays the foundation for such decomposition methods for two-stage stochastic mixed-integer programs.