Infinite horizon stopping problems with (nearly) total reward criteria

We study an infinite horizon optimal stopping Markov problem which is either undiscounted (total reward) or with a general Markovian discount rate. Using ergodic properties of the underlying Markov process, we establish the feasibility of the stopping problem and prove the existence of optimal and ε$\epsilon$-optimal stopping times. We show the continuity of the value function and its variational characterisation (in the viscosity sense) under different sets of assumptions satisfied by large classes of diffusion and jump–diffusion processes. In the case of a general discounted problem we relax a classical assumption that the discount rate is uniformly separated from zero.     

Optimal stopping for partially observed piecewise-deterministic Markov processes

This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an ?$?$-optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence.     

One-armed bandit models with continuous and delayed responses

One-armed bandit processes with continuous delayed responses are formulated as controlled stochastic processes following the Bayesian approach. It is shown that under some regularity conditions, a Gittins-like index exists which is the limit of a monotonic sequence of break-even values characterizing optimal initial selections of arms for finite horizon bandit processes. Furthermore, there is an optimal stopping solution when all observations on the unknown arm are complete. Results are illustrated with a bandit model having exponentially distributed responses, in which case the controlled stochastic process becomes a Markov decision process, the Gittins-like index is the Gittins index and the Gittins index strategy is optimal. Acknowledgement.We thank an anonymous referee for constructive and insightful comments, especially those related to the notion of the Gittins index.Both authors are funded by the Natural Sciences and Engineering Research Council (NSERC) of Canada.     

The value of foresight

Suppose you have one unit of stock, currently worth 1, which you must sell before time $T$. The Optional Sampling Theorem tells us that whatever stopping time we choose to sell, the expected discounted value we get when we sell will be 1. Suppose however that we are able to see $a$ units of time into the future, and base our stopping rule on that; we should be able to do better than expected value 1. But how much better can we do? And how would we exploit the additional information? The optimal solution to this problem will never be found, but in this paper we establish remarkably close bounds on the value of the problem, and we derive a fairly simple exercise rule that manages to extract most of the value of foresight.     

Sensitivity of optimal prices to system parameters in a steady-state service facility

We consider the problem of maximizing the long-run average reward in a service facility with dynamic pricing. We investigate sensitivity of optimal pricing policies to the parameters of the service facility which is modelled as an M/M/s/K$M/M/s/K$ queueing system. Arrival process to the facility is Poisson with arrival rate a decreasing function of the price currently being charged by the facility. We prove structural results on the optimal pricing policies when the parameters in the facility change. Namely, we show that optimal prices decrease when the capacity of the facility or the number of servers in the facility increase. Under a reasonable assumption, we also show that optimal prices increase as the overall demand for the service provided by the facility increases or when the service rate of the facility decreases. We illustrate how these structural results simplify the required computational effort while finding the optimal policy.     

Time discretization and quantization methods for optimal multiple switching problem

In this paper, we study probabilistic numerical methods based on optimal quantization algorithms for computing the solution to optimal multiple switching problems with regime-dependent state process. We first consider a discrete-time approximation of the optimal switching problem, and analyse its rate of convergence. Given a time step h$h$, the error is in general of order (hlog(1/h))^1/2${(hlog(1/h))}^{1/2}$, and of order h^1/2$h^{1/2}$ when the switching costs do not depend on the state process. We next propose quantization numerical schemes for the space discretization of the discrete-time Euler state process. A Markovian quantization approach relying on the optimal quantization of the normal distribution arising in the Euler scheme is analysed. In the particular case of uncontrolled state process, we describe an alternative marginal quantization method, which extends the recursive algorithm for optimal stopping problems as in Bally (2003) [1]. A priori L^p$L^p$-error estimates are stated in terms of quantization errors. Finally, some numerical tests are performed for an optimal switching problem with two regimes.     

Stochastic approximations of constrained discounted Markov decision processes

We consider a discrete-time constrained Markov decision process under the discounted cost optimality criterion. The state and action spaces are assumed to be Borel spaces, while the cost and constraint functions might be unbounded. We are interested in approximating numerically the optimal discounted constrained cost. To this end, we suppose that the transition kernel of the Markov decision process is absolutely continuous with respect to some probability measure μ  . Then, by solving the linear programming formulation of a constrained control problem related to the empirical probability measure _μn${\mu }_n$ of μ, we obtain the corresponding approximation of the optimal constrained cost. We derive a concentration inequality which gives bounds on the probability that the estimation error is larger than some given constant. This bound is shown to decrease exponentially in n. Our theoretical results are illustrated with a numerical application based on a stochastic version of the Beverton–Holt population model.     

Multi-player stopping games with redistribution of payoffs and BSDEs with oblique reflection

We examine the connections between a novel class of multi-person stopping games with redistribution of payoffs and multi-dimensional reflected BSDEs in discrete- and continuous-time frameworks. Our goal is to provide an essential extension of classic results for two-player stopping games (Dynkin games) to the multi-player framework. We show the link between certain multi-period m$m$-player stopping games and a new kind of m$m$-dimensional reflected BSDEs. The existence and uniqueness of a solution to continuous-time reflected BSDEs are established. Continuous-time redistribution games are constructed with the help of reflected BSDEs and a characterization of the value of such stopping games is provided.     

Trimmed stable AR(1) processes

In this paper we investigate the distribution of trimmed sums of dependent observations with heavy tails. We consider the case of autoregressive processes of order one with independent innovations in the domain of attraction of a stable law. We show if the $d$ largest (in magnitude) terms are removed from the sample, then the sum of the remaining elements satisfies a functional central limit theorem with random centering provided $d=d\left(n\right)\ge n^{\gamma }$ (for some $\gamma >0$) and $d\left(n\right)/n\to 0$. This result is used to get asymptotics for the widely used CUSUM process in case of dependent heavy tailed observations.     

Finite approximation schemes for Lévy processes,and their application to optimal stopping problems

This paper proposes two related approximation schemes, based on a discrete grid on a finite time interval [0,T]$[0,T]$, and having a finite number of states, for a pure jump Lévy process _Lt$L_t$. The sequences of discrete processes converge to the original process, as the time interval becomes finer and the number of states grows larger, in various modes of weak and strong convergence, according to the way they are constructed. An important feature is that the filtrations generated at each stage by the approximations are sub-filtrations of the filtration generated by the continuous time Lévy process. This property is useful for applications of these results, especially to optimal stopping problems, as we illustrate with an application to American option pricing. The rates of convergence of the discrete approximations to the underlying continuous time process are assessed in terms of a “complexity” measure for the option pricing algorithm.     

The coding complexity of diffusion processes under supremum norm distortion

We investigate the high resolution quantization and entropy coding problem for solutions of stochastic differential equations under supremum norm distortion. Tight asymptotic formulas are found under mild regularity assumptions. The main technical tool is a decoupling method which allows us to relate the complexity of the diffusion process to that of the Wiener process. The technique is also applicable when considering the L^p[0,1]$L^p[0,1]$-norm distortion.     

Path transformations for local times of one-dimensional diffusions

Let $X$ be a regular one-dimensional transient diffusion and $L^y$ be its local time at $y$. The stochastic differential equation (SDE) whose solution corresponds to the process $X$ conditioned on $[L_{\infty }^y=a]$ for a given $a\ge 0$ is constructed and a new path decomposition result for transient diffusions is given. In the course of the construction Bessel-type motions as well as their SDE representations are studied. Moreover, the Engelbert–Schmidt theory for the weak solutions of one dimensional SDEs is extended to the case when the initial condition is an entrance boundary for the diffusion. This extension was necessary for the construction of the Bessel-type motion which played an essential part in the SDE representation of $X$ conditioned on $[L_{\infty }^y=a]$.     

Extremes of space–time Gaussian processes

Let Z={_Zt(h);h∈R^d,t∈R}$Z=\{Z_t\left(h\right);h\in {\mathbb{R}}^d,t\in \mathbb{R}\}$ be a space–time Gaussian process which is stationary in the time variable t$t$. We study _Mn(h)=_{supt∈[0,n]Zt}(_snh)$M_n\left(h\right)={sup}_{t\in [0,n]}{Z}_t\left(s_{n}h\right)$, the supremum of Z$Z$ taken over t∈[0,n]$t\in [0,n]$ and rescaled by a properly chosen sequence _sn→0$s_n\to 0$. Under appropriate conditions on Z$Z$, we show that for some normalizing sequence _bn→∞$b_n\to \infty$, the process _bn(_Mn−_bn)$b_n(M_n-b_n)$ converges as n→∞$n\to \infty$ to a stationary max-stable process of Brown–Resnick type. Using strong approximation, we derive an analogous result for the empirical process.     

Fluctuations of Omega-killed spectrally negative Lévy processes

In this paper we solve the exit problems for (reflected) spectrally negative Lévy processes, which are exponentially killed with a killing intensity dependent on the present state of the process and analyze respective resolvents. All identities are given in terms of new generalizations of scale functions. For the particular cases $\omega \left(x\right)=q$ and $\omega \left(x\right)=q{\mathbf{1}}_{(a,b)}\left(x\right)$, we obtain results for the classical exit problems and the Laplace transforms of the occupation times in a given interval, until first passage times, respectively. Our results can also be applied to find the bankruptcy probability in the so-called Omega model, where bankruptcy occurs at rate $\omega \left(x\right)$ when the Lévy surplus process is at level $x<0$. Finally, we apply these results to obtain some exit identities for spectrally positive self-similar Markov processes. The main method throughout all the proofs relies on the classical fluctuation identities for Lévy processes, the Markov property and some basic properties of a Poisson process.     

Weak Dirichlet processes with jumps

This paper develops systematically the stochastic calculus via regularization in the case of jump processes. In particular one continues the analysis of real-valued càdlàg weak Dirichlet processes with respect to a given filtration. Such a process is the sum of a local martingale and an adapted process $A$ such that $[N,A]=0$, for any continuous local martingale $N$. Given a function $u:[0,T]\times \mathbb{R}\to \mathbb{R}$, which is of class $C^{0,1}$ (or sometimes less), we provide a chain rule type expansion for $u(t,X_t)$ which stands in applications for a chain Itô type rule.     

Quadratic reflected BSDEs with unbounded obstacles

In this paper, we analyze a real-valued reflected backward stochastic differential equation (RBSDE) with an unbounded obstacle and an unbounded terminal condition when its generator f$f$ has quadratic growth in the z$z$-variable. In particular, we obtain existence, uniqueness, and stability results, and consider the optimal stopping for quadratic g$g$-evaluations. As an application of our results we analyze the obstacle problem for semi-linear parabolic PDEs in which the non-linearity appears as the square of the gradient. Finally, we prove a comparison theorem for these obstacle problems when the generator is concave in the z$z$-variable.