A stochastic decision model with vector-valued reward

In this note we consider a non-stationary stochastic decision model with vector-valued reward. Based on Pareto-optimality we define the maximal total reward as a set of vector valued total rewards, which have not a successor with respect to the underlying partially order relation. The principle of optimality is derived. Using the well-known von-Neumann-Morgenstern-property we formulate a Bellman-equation, which consists in a system of iterative set relations.     

Functional limit theorems for a new class of non-stationary shot noise processes

We study a class of non-stationary shot noise processes which have a general arrival process of noises with non-stationary arrival rate and a general shot shape function. Given the arrival times, the shot noises are conditionally independent and each shot noise has a general (multivariate) cumulative distribution function (c.d.f.) depending on its arrival time. We prove a functional weak law of large numbers and a functional central limit theorem for this new class of non-stationary shot noise processes in an asymptotic regime with a high intensity of shot noises, under some mild regularity conditions on the shot shape function and the conditional (multivariate) c.d.f. We discuss the applications to a simple multiplicative model (which includes a class of non-stationary compound processes and applies to insurance risk theory and physics) and the queueing and work-input processes in an associated non-stationary infinite-server queueing system. To prove the weak convergence, we show new maximal inequalities and a new criterion of existence of a stochastic process in the space $\mathbb{D}$ given its consistent finite dimensional distributions, which involve a finite set function with the superadditive property.     

Bias optimality of admission control in a non-stationary repairable queue

We study a non-stationary repairable queue with a single server and multiple customers’ types. The difference between types of customers is defined by the offered rewards. We show that the bias optimal policy has the trunk reservation (threshold) form. Furthermore, under some given conditions, we also prove that the control level of the bias optimal policy is monotone about time.     

On the Non-parametric Prediction of Conditionally Stationary Sequences

We prove the strong consistency of estimators of the conditional distribution function and conditional expectation of a future observation of a discrete time stochastic process given a fixed number of past observations. The results apply to conditionally stationary processes (a class of processes including Markov and stationary processes) satisfying a strong mixing condition, and they extend and bring together the work of several authors in the area of non-parametric estimation. One of our goals is to provide further justification for the growing practical application of non-parametric estimators in non-stationary time series and in other `non-i.i.d.' settings. Some arguments as to why such estimators should work very generally in practice, often in a nearly `optimal' way, are given. Two numerical illustrations are included, one with simulated data and the other with oceanographic data. An erratum to this article is available at .     

Target-level criterion in Markov decision processes

The Markov decision process is studied under the maximization of the probability that total discounted rewards exceed a target level. We focus on and study the dynamic programing equations of the model. We give various properties of the optimal return operator and, for the infinite planning-horizon model, we characterize the optimal value function as a maximal fixed point of the previous operator. Various turnpike results relating the finite and infinite-horizon models are also given.     

Optimal task sequencing with precedence constraints

A given finite set of tasks, having known nonnegligible failure probabilities and known costs (or rewards) for their performance, can be performed sequentially until either one of the tasks fails or all tasks have been executed. The allowable task performance sequences are constrained only by certain precedence requirements, which specify that certain tasks must be performed before certain other tasks. Given the individual task failure probabilities and task costs, along with the intertask precedence requirements, the problem is to determine an optimal task performance sequence having minimal expected cost (or maximal expected reward). A number of potential applications of such “task ordering” problems are described, including R&D project organization, design of screening procedures, and determining testing points for sequential manufacturing processes.The main results of this paper are a number of reduction theorems which lead to a very efficient optimization algorithm for a large class of task ordering problems. Though these theorems are not quite sufficient for us to give a fast optimization algorithm, we do show how their use can improve upon exhaustive search techniques.     

A new strong optimality criterion for nonstationary Markov decision processes

This paper deals with a new optimality criterion consisting of the usual three average criteria and the canonical triplet (totally so-called strong average-canonical optimality criterion) and introduces the concept of a strong average-canonical policy for nonstationary Markov decision processes, which is an extension of the canonical policies of Herna′ndez-Lerma and Lasserre [16] (pages: 77) for the stationary Markov controlled processes. For the case of possibly non-uniformly bounded rewards and denumerable state space, we first construct, under some conditions, a solution to the optimality equations (OEs), and then prove that the Markov policies obtained from the OEs are not only optimal for the three average criteria but also optimal for all finite horizon criteria with a sequence of additional functions as their terminal rewards (i.e. strong average-canonical optimal). Also, some properties of optimal policies and optimal average value convergence are discussed. Moreover, the error bound in average reward between a rolling horizon policy and a strong average-canonical optimal policy is provided, and then a rolling horizon algorithm for computing strong average ε(>0)-optimal Markov policies is given.     

Games with frequency-dependent stage payoffs

Games with frequency-dependent stage payoffs (FD-games), are infinitely repeated non-cooperative games played at discrete moments in time called stages. The stage payoffs depend on the action pair actually chosen, and on the relative frequencies with which all actions were chosen before. We assume that players wish to maximize their expected (limiting) average rewards over the entire time-horizon. We prove an analogy to, as well as an extension of the (perfect) Folk Theorem. Each pair of rewards in the convex hull of all individually-rational jointly-convergent pure-strategy rewards can be supported by an equilibrium. Moreover, each pair of rewards in same set giving each player strictly more than the threat-point-reward, can be supported by a subgame-perfect equilibrium. Under a pair of jointly-convergent strategies, the relative frequency of each action pair converges in the long run. Received: March 2002/Revised: January 2003     

工序能力Bayes推断

本文从Ｂａｙｅｓ观点研究工序能力，对无信息先验和共轭先验，给出了Ｃｐ的后验分布、条件期望估计和最大后验估计、Ｂａｙｅｓ置信下限和判断工序是否有能力的临界值，适于对相似工序作统计推断。     

Further Results for Closed Cyclic Queues with Balanced Exponential Servers

This note presents some new results for closed cyclic queues with balanced exponential servers. The probability density function, expected value, variance and coefficient of variation of conditional idle time, residual idle time, time in a stage, cycle time and waiting time are presented. Conditional waiting time, residual waiting time and interdeparture time results are obtained from the dual.     

Efficient simulation and conditional functional limit theorems for ruinous heavy-tailed random walks

The contribution of this paper is to introduce change of measure based techniques for the rare-event analysis of heavy-tailed random walks. Our changes of measures are parameterized by a family of distributions admitting a mixture form. We exploit our methodology to achieve two types of results. First, we construct Monte Carlo estimators that are strongly efficient (i.e. have bounded relative mean squared error as the event of interest becomes rare). These estimators are used to estimate both rare-event probabilities of interest and associated conditional expectations. We emphasize that our techniques allow us to control the expected termination time of the Monte Carlo algorithm even if the conditional expected stopping time (under the original distribution) given the event of interest is infinity–a situation that sometimes occurs in heavy-tailed settings. Second, the mixture family serves as a good Markovian approximation (in total variation) of the conditional distribution of the whole process given the rare event of interest. The convenient form of the mixture family allows us to obtain functional conditional central limit theorems that extend classical results in the literature.     

Bias Optimality versus Strong 0-Discount Optimality in Markov Control Processes with Unbounded Costs

This paper deals with expected average cost (EAC) and discount-sensitive criteria for discrete-time Markov control processes on Borel spaces, with possibly unbounded costs. Conditions are given under which (a) EAC optimality and strong –1-discount optimality are equivalent; (b) strong 0-discount optimality implies bias optimality; and, conversely, under an additional hypothesis, (c) bias optimality implies strong 0-discount optimality. Thus, in particular, as the class of bias optimal policies is nonempty, (c) gives the existence of a strong 0-discount optimal policy, whereas from (b) and (c) we get conditions for bias optimality and strong 0-discount optimality to be equivalent. A detailed example illustrates our results.     

条件交半格中的相对极大滤子

引入偏序集的相对极大滤子的概念,证明在任意条件交半格中一个滤子是相对极大滤子当且仅当它是滤子格的完全交不可约元.一个格是分配的当且仅当每一个相对极大滤子都是素滤子.随后研究了Heyting代数中相对极大滤子的刻画,最后定义和研究了完全并既约生成格.     

Stochastic Variational Inequality Approaches to the Stochastic Generalized Nash Equilibrium with Shared Constraints

In this paper, we consider the generalized Nash equilibrium with shared constraints in the stochastic environment, and we call it the stochastic generalized Nash equilibrium. The stochastic variational inequalities are employed to solve this kind of problems, and the expected residual minimization model and the conditional value-at-risk formulations defined by the residual function for the stochastic variational inequalities are discussed. We show the risk for different kinds of solutions for the stochastic generalized Nash equilibrium by the conditional value-at-risk formulations. The properties of the stochastic quadratic generalized Nash equilibrium are shown. The smoothing approximations for the expected residual minimization formulation and the conditional value-at-risk formulation are employed. Moreover, we establish the gradient consistency for the measurable smoothing functions and the integrable functions under some suitable conditions, and we also analyze the properties of the formulations. Numerical results for the applications arising from the electricity market model illustrate that the solutions for the stochastic generalized Nash equilibrium given by the ERM model have good properties, such as robustness, low risk and so on.     

Stochastic Model of Conditional Non-stationary Time Series of the Wind Chill Index in West Siberia

In this paper, we propose a stochastic model of the conditional time series of the wind chill index. The model is based on the inverse distribution function method and on the normalization method for simulation of the non-Gaussian non-stationary random processes as well as on the method of conditional distributions for simulation of the conditional Gaussian processes. In the framework of the approach considered, two types of conditions (point conditions and interval conditions) are imposed on the time series. The model in question was verified using the real data collected at the weather stations located in West Siberia (Russia). It is shown that the simulated trajectories are close in their statistical properties to the real time series. The model proposed was used for stochastic forecasting of the wind chill index and the results of the numerical experiments have shown that the accuracy of the short-term forecasts is high enough.

     

The empirical bayes rules with floating optimal sample size for exponential conditional distributions

Summary We consider the empirical Bayes solution in such a situation where the sample size is successively determined by a rule which includes the Bayes risks and the observation costs. The empirical Bayes floating optimal sample size depends on current as well as on previous information assumed to be collected from earlier performances of similar decisions. The sampling is done from an exponential conditional distribution, with a single parameter. The proofs, which show the asymptotic optimality of the empirical Bayes solution, are presented for a hypotheses-testing problem. A straight generalization to a multiple decision problem is also given.     

Nonzero-sum non-stationary discounted Markov game model

The goal of this paper is provide a theory of K-person non-stationary Markov games with unbounded rewards, for a countable state space and action spaces. We investigate both the finite and infinite horizon problems. We define the concept of strong Nash equilibrium and present conditions for both problems for which strong Nash or Nash equilibrium strategies exist for all players within the Markov strategies, and show that the rewards in equilibrium satisfy the optimality equations.     

Chow-Type Maximal Inequality for Conditional Demimartingales and Its Applications

In this paper, the Chow-type maximal inequality for conditional demimartingales is established. By using the Chow-type maximal inequality, the authors provide the maximal inequality for conditional demimartingales based on {concave Young functions}. At last, the moment inequalities for conditional demimartingales are established.     

Queue length estimation from probe vehicle location and the impacts of sample size

Probe vehicles are increasingly receiving more attention as an alternative means of collecting real-time traffic data needed for system optimization. This paper focuses on real-time estimation of queue lengths from the location information of probe vehicles in a queue at an isolated and undersaturated intersection. The paper also addresses the evaluation of the accuracy of such estimates as a function of the market penetration of probe vehicles. An analytical formulation based on conditional probability distributions is developed for estimating the expected queue length and its variance. It is found that, for the given settings, only the location information of the last probe vehicle in the queue is sufficient for the estimation. Exact expressions for the conditional mean and variance of queue length are derived. Various numerical results are documented to show how estimation errors behave by the volume to capacity ratio and by market penetration.