Robustness of perturbation analysis estimators for queueing systems with unknown distributions

Sample-path-based stochastic gradient estimators for performance measures of queueing systems rely on the assumption that a probability distribution of the random vector of interest (e.g., a service or interarrival time sequence) is given. In this paper, we address the issue of dealing with unknown probability distributions and investigate the robustness of such estimators with respect to possibly erroneous distribution choices. We show that infinitesimal perturbation analysis (IPA) can be robust in this sense and, in some cases, provides distribution-independent estimates. Comparisons with other gradient estimators are provided, including experimental results. We also show that finite perturbation analysis (FPA), though only providing gradient approximations, possesses some attractive robustness properties with respect to unknown distribution parameters. An application of FPA estimation is included for a queueing system performance optimization problem involving customers with real-time constraints.This work was supported in part by the National Science Foundation Grant ECS-88-01912 and by the Office of Naval Research Contract N00014-87-K-0304.The authors wish to thank Dr. Jack Holtzman for several useful comments and suggestions.     

Resource contention games in multiclass stochastic flow models

We consider resource contention games in a stochastic hybrid system setting using Stochastic Flow Models (SFM) with multiple classes and class-dependent objectives. We present a general modeling framework for such games, where Infinitesimal Perturbation Analysis (IPA) estimators are derived for the derivatives of various class-dependent objectives. This allows us to study these games from the point of view of system-centric optimization of a performance metric and compare it to the user-centric approach where each user optimizes its own performance metric. We derive explicit solutions for a specific model in which the competing user classes employ threshold control policies and service is provided on a First Come First Serve (FCFS) basis. The unbiasedness of the IPA estimators is established in this case and it is shown that under certain conditions the system-centric and user-centric optimization solutions coincide.     

Error analysis for regenerative queueing estimators with special reference to gradient estimators via likelihood ratio

The performance of telecommunications systems is typically estimated (either analytically or by simulation) via queueing theoretic models. The gradient of the expected performance with respect to the various parameters (such as arrival rate or service rate) is very important as it not only measures the sensitivity to change, but is also needed for the solution of optimization problems. While the estimator for the expected performance is the sample mean of the simulation experiment, there are several possibilities for the estimator of the gradient. They include the obvious finite difference approximation, but also other recently advocated techniques, such as estimators derived from likelihood ratio transformations or from infinitesimal perturbations. A major problem in deciding upon which estimator to use and in planning the length of the simulation has been the scarcity of analytical error calculations for estimators of queueing models. It is this question that we answer in this paper for the waiting time moments (of arbitrary order) of theM / G / 1 queue by using the queueing analysis technique developed by Shalmon. We present formulas for the error variance of the estimators of expectation and of its gradient as a function of the simulation length; at arbitrary traffic intensity the formulas are recursive, while the heavy traffic approximations are explicit and of very simple form. For the gradient of the mean waiting time with respect to the arrival (or service) rate, and at 1 percent relative precision, the heavy traffic formulas show that the likelihood ratio estimator for the gradient reduces the length of the simulation required by the finite difference estimator by about one order of magnitude; further increasing the relative precision by a factor increases the reduction advantage by the same factor. At any relative precision, it exceeds the length of the simulation required for the estimation of the mean with the same relative precision by about one order of magnitude. While strictly true for theM / G / 1 queue, the formulas can also be used as guidelines in the planning of queueing simulations and of stochastic optimizations of complex queueing systems, particularly those with Poisson arivals.     

Infinitesimal perturbation analysis for second derivative estimation and design of manufacturing flow controllers

A simulation-based numerical technique for the design of near-optimal manufacturing flow controllers for unreliable flexible manufacturing systems uses quadratic approximations of the value functions that characterize the optimal policy and employs stochastic optimization to design the key coefficients of the quadratic approximations. First and second derivative estimates that drive the optimization algorithm are obtained from a single sample path of the system via infinitesimal perturbation analysis (IPA). Extensive computational experience is reported for one, two, and three-part-type production systems. The relative performance of first-order and second-order stochastic optimization algorithms is investigated. The computational efficiency of these algorithms is finally compared to conventional controller design algorithms based on state-space discretization and successive approximation.This research was supported by the National Science Foundation, Grant No. DDM-89-14277 and DDM-9215368.     

A probability metrics approach for reducing the bias of optimality gap estimators in two-stage stochastic linear programming

Monte Carlo sampling-based estimators of optimality gaps for stochastic programs are known to be biased. When bias is a prominent factor, estimates of optimality gaps tend to be large on average even for high-quality solutions. This diminishes our ability to recognize high-quality solutions. In this paper, we present a method for reducing the bias of the optimality gap estimators for two-stage stochastic linear programs with recourse via a probability metrics approach, motivated by stability results in stochastic programming. We apply this method to the Averaged Two-Replication Procedure (A2RP) by partitioning the observations in an effort to reduce bias, which can be done in polynomial time in sample size. We call the resulting procedure the Averaged Two-Replication Procedure with Bias Reduction (A2RP-B). We provide conditions under which A2RP-B produces strongly consistent point estimators and an asymptotically valid confidence interval. We illustrate the effectiveness of our approach analytically on a newsvendor problem and test the small-sample behavior of A2RP-B on a number of two-stage stochastic linear programs from the literature. Our computational results indicate that the procedure effectively reduces bias. We also observe variance reduction in certain circumstances.     

Uniform exponential convergence of sample average random functions under general sampling with applications in stochastic programming

Sample average approximation (SAA) is one of the most popular methods for solving stochastic optimization and equilibrium problems. Research on SAA has been mostly focused on the case when sampling is independent and identically distributed (iid) with exceptions (Dai et al. (2000) [9], Homem-de-Mello (2008) [16]). In this paper we study SAA with general sampling (including iid sampling and non-iid sampling) for solving nonsmooth stochastic optimization problems, stochastic Nash equilibrium problems and stochastic generalized equations. To this end, we first derive the uniform exponential convergence of the sample average of a class of lower semicontinuous random functions and then apply it to a nonsmooth stochastic minimization problem. Exponential convergence of estimators of both optimal solutions and M-stationary points (characterized by Mordukhovich limiting subgradients (Mordukhovich (2006) [23], Rockafellar and Wets (1998) [32])) are established under mild conditions. We also use the unform convergence result to establish the exponential rate of convergence of statistical estimators of a stochastic Nash equilibrium problem and estimators of the solutions to a stochastic generalized equation problem.     

Variance comparison between infinitesimal perturbation analysis and likelihood ratio estimators to stochastic gradient

We theoretically compare variances between the Infinitesimal Perturbation Analysis (IPA) estimator and the Likelihood Ratio (LR) estimator to Monte Carlo gradient for stochastic systems. The results presented in Cui et al. (2020) [2] on variance comparison between these two estimators are substantially improved. We also prove a practically interesting result that the IPA estimators to European vanilla and arithmetic Asian options' Delta, respectively, have smaller variance when the underlying asset's return process is independent with the initial price and square integrable.     

How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths

Stochastic optimization/approximation algorithms are widely used to recursively estimate the optimum of a suitable function or its root under noisy observations when this optimum or root is a constant or evolves randomly according to slowly time-varying continuous sample paths. In comparison, this paper analyzes the asymptotic properties of stochastic optimization/approximation algorithms for recursively estimating the optimum or root when it evolves rapidly with nonsmooth (jump-changing) sample paths. The resulting problem falls into the category of regime-switching stochastic approximation algorithms with two-time scales. Motivated by emerging applications in wireless communications, and system identification, we analyze asymptotic behavior of such algorithms. Our analysis assumes that the noisy observations contain a (nonsmooth) jump process modeled by a discrete-time Markov chain whose transition frequency varies much faster than the adaptation rate of the stochastic optimization algorithm. Using stochastic averaging, we prove convergence of the algorithm. Rate of convergence of the algorithm is obtained via bounds on the estimation errors and diffusion approximations. Remarks on improving the convergence rates through iterate averaging, and limit mean dynamics represented by differential inclusions are also presented. The research of G. Yin was supported in part by the National Science Foundation under DMS-0603287, in part by the National Security Agency under MSPF-068-029, and in part by the National Natural Science Foundation of China under #60574069. The research of C. Ion was supported in part by the Wayne State University Rumble Fellowship. The research of V. Krishnamurthy was supported in part by NSERC (Canada).     

A new regularized stochastic approximation framework for stochastic inverse problems

We study the nonlinear inverse problem of estimating stochastic parameters in the fourth-order partial differential equation with random data. The primary focus is on developing a novel stochastic approximation framework for inverse problems consisting of three key components. As a first step, we reformulate the inverse problem into a stochastic convex optimization problem. The second step includes developing a new regularized stochastic extragradient framework for a nonlinear variational inequality, which subsumes the optimality conditions for the optimization formulation of the inverse problem. The third step involves modeling random variables by a Karhunen–Loève type finite-dimensional noise representation, allowing the direct and the inverse problems to be conveniently discretized. We show that the regularized extragradient methods are strongly convergent in a Hilbert space setting, and we also provide several auxiliary results for the inverse problem, including Lipschitz continuity and a derivative characterization of the solution map. We provide the outcome of computational experiments to estimate stochastic and deterministic parameters. The numerical results demonstrate the feasibility and effectiveness of the developed framework and validate stochastic approximation as an effective method for stochastic inverse problems.     

Online IPA Gradient Estimators in Stochastic Continuous Fluid Models

This paper applies infinitesimal perturbation analysis (IPA) to loss-related and workload-related metrics in a class of stochastic flow models (SFM). It derives closed-form formulas for the gradient estimators of these metrics with respect to various parameters of interest, such as buffer size, service rate, and inflow rate. The IPA estimators derived are simple and fast to compute, and are further shown to be unbiased and nonparametric, in the sense that they can be computed directly from the observed data without any knowledge of the underlying probability law. These properties hold out the promise of utilizing IPA gradient estimates as ingredients of online management and control of telecommunications networks. While this paper considers single-node SFMs, the analysis method developed is amenable to extensions to networks of SFM nodes with more general topologies.     

Monte-Carlo estimation of time-dependent statistical characteristics of random dynamical systems

The problem of estimating the time-dependent statistical characteristics of a random dynamical system is studied under two different settings. In the first, the system dynamics is governed by a differential equation parameterized by a random parameter, while in the second, this is governed by a differential equation with an underlying parameter sequence characterized by a continuous time Markov chain. We propose, for the first time in the literature, stochastic approximation algorithms for estimating various time-dependent process characteristics of the system. In particular, we provide efficient estimators for quantities such as the mean, variance and distribution of the process at any given time as well as the joint distribution and the autocorrelation coefficient at different times.     

The influence and selection of subspaces for a posteriori error estimators

Summary. The element residual method for a posteriori error estimation is analyzed for degree finite element approximation on quadrilateral elements. The influence of the choice of subspace used to solve the element residual problem is studied. It is shown that the resulting estimators will be consistent (or asymptotically exact) for all if and only if the mesh is parallel. Moreover, even if the mesh consists of rectangles, then the estimators can be inconsistent when . The results provide concrete guidelines for the selection of a posteriori error estimators and establish the limits of their performance. In particular, the use of the element residual method for high orders of approximation (such as those arising in the - version finite element method) is vindicated. The mechanism behind the rather poor performance of the estimators is traced back to the basic formulation of the residual problem. The investigations reveal a deficiency in the formulation, leading, as it does, to spurious modes in the true solution of the residual problem. The recommended choice of subspaces may be viewed as being sufficient to guarantee that the spurious modes are filtered out from the approximate solution while at the same time retaining a sufficient degree of approximation to represent the true modes. Received February 27, 1995 / Revised version received June 7, 1995     

Probability density estimation for survival data with censoring indicators missing at random

In this paper, some nonparametric approaches of density function estimation are developed when censoring indicators are missing at random. A conditional mean score based estimator and a mean score estimator are suggested, respectively. The two estimators are proved to be asymptotically normal and uniformly strongly consistent. The bandwidth selection problem is also discussed. A simulation study is conducted to compare finite-sample behaviors of the proposed estimators.     

On Extending the LP Computable Risk Measures to Account Downside Risk

A mathematical model of portfolio optimization is usually quantified with mean-risk models offering a lucid form of two criteria with possible trade-off analysis. In the classical Markowitz model the risk is measured by a variance, thus resulting in a quadratic programming model. Following Sharpe’s work on linear approximation to the mean-variance model, many attempts have been made to linearize the portfolio optimization problem. There were introduced several alternative risk measures which are computationally attractive as (for discrete random variables) they result in solving linear programming (LP) problems. Typical LP computable risk measures, like the mean absolute deviation (MAD) or the Gini’s mean absolute difference (GMD) are symmetric with respect to the below-mean and over-mean performances. The paper shows how the measures can be further combined to extend their modeling capabilities with respect to enhancement of the below-mean downside risk aversion. The relations of the below-mean downside stochastic dominance are formally introduced and the corresponding techniques to enhance risk measures are derived.The resulting mean-risk models generate efficient solutions with respect to second degree stochastic dominance, while at the same time preserving simplicity and LP computability of the original models. The models are tested on real-life historical data.The research was supported by the grant PBZ-KBN-016/P03/99 from The State Committee for Scientific Research.     

Convergence of a recursive robust algorithm with strongly regular observations

Robust estimation of parameters may be obtained via stochastic approximation algorithms. This paper deals with the properties of a recursive estimator of a location parameter in a stationary strongly regular process. Adaptive estimators of particular interest are also studied.     

Optimization challenges in the structured low rank approximation problem

In this paper we illustrate some optimization challenges in the structured low rank approximation (SLRA) problem. SLRA can be described as the problem of finding a low rank approximation of an observed matrix which has the same structure as this matrix (such as Hankel). We demonstrate that the optimization problem arising is typically very difficult: in particular, the objective function is multiextremal even for simple cases. The main theme of the paper is to suggest that the difficulties described in approximating a solution of the SLRA problem open huge possibilities for the application of stochastic methods of global optimization.     

Remarks on the optimization process of a manufacturing system with stochastic breakdown and rework

This paper reexamines the optimization process of a manufacturing system with stochastic breakdown and rework proposed by Chiu [S.W. Chiu, An optimization problem of manufacturing systems with stochastic machine breakdown and rework process, Applied Stochastic Models in Business and Industry 24 (2008) 203–219]. The proof of convexity of the long-run average cost function for the aforementioned manufacturing system is provided in this note. It can be used to replace the conditional convexity given in Theorem 1 of Chiu (2008) [1]. Therefore, when determining the optimal solution for such a real-life system, computational efforts in verifying the conditional convexity can now be omitted, due to the improved quality of the optimization process.     

Estimation of two ordered bivariate mean residual life functions

Situations occur frequently in which the mean residual life (mrl) functions of two populations must be ordered. For example, if a mechanical device is improved, the mrl function for the improved device should not be less than that of the original device. Also, mrl functions for medical patients should often be ordered depending on the status of concomitant variables. This paper proposes nonparametric estimators of the bivariate mrl function under a mrl ordering. The estimators are shown to be asymptotically unbiased, strongly uniformly consistent and weakly convergent to a bivariate Gaussian process. The estimators are shown to be the projections, in a sense to be made precise, of the empirical mrl function onto an appropriate convex set of mrl functions. In the one-sample problem, the new estimators dominate the empirical mrl function in terms of risk with respect to a wide class of loss functions.     

IPA Derivatives for Make-to-Stock Production-Inventory Systems with Backorders

A single-stage Make-to-Stock (MTS) production-inventory system consists of a production facility coupled to an inventory facility, and is subject to a policy that aims to maintain a prescribed inventory level (called base stock) by modulating production capacity. This paper considers a class of single-stage, single-product MTS systems with backorders, driven by random demand and production capacity, and subject to a continuous-review base-stock policy. A model from this class is formulated as a stochastic fluid model (SFM), where all flows are described by stochastic rate processes with piecewise constant sample paths, subject to very mild regularity assumptions that merely preclude accumulation points of jumps with probability 1. Other than that, the MTS model in SFM setting is nonparametric in that it assumes no specific form for the underlying probability law, and as such is quite general. The paper proceeds to derive formulas for the (stochastic) IPA (Infinitesimal Perturbation Analysis) derivatives of the sample-path time averages of the inventory level and backorders level with respect to the base-stock level and a parameter of the production rate. These formulas are comprehensive in that they are exhibited for any initial condition of the system, and include right and left derivatives (when they do not coincide). The derivatives derived are then shown to be unbiased and their formulas are seen to be amenable to fast computation. The generality of the model and comprehensiveness of the IPA derivative formulas hold out the promise of gradient-based applications. More specifically, since the base-stock level and production rate are the key control parameters of MTS systems, the results provide the theoretical underpinnings for optimizing the design of MTS systems and for devising prospective on-line adaptive control algorithms that employ IPA derivatives. The paper concludes with a discussion of those issues.     

Modeling Stochastic Dominance as Infinite-Dimensional Constraint Systems via the Strassen Theorem

We use the Strassen theorem to solve stochastic optimization problems with stochastic dominance constraints. First, we show that a dominance-constrained problem on general probability spaces can be expressed as an infinite-dimensional optimization problem with a convenient representation of the dominance constraints provided by the Strassen theorem. This result generalizes earlier work which was limited to finite probability spaces. Second, we derive optimality conditions and a duality theory to gain insight into this optimization problem. Finally, we present a computational scheme for constructing finite approximations along with a convergence rate analysis on the approximation quality.