Numerical solution of quasi-variational inequalities arising in stochastic game theory

We study the finite-difference approximation for the quasi-variational inequalities for a stochastic game involving discrete actions of the players and continuous and discrete payoff. We prove convergence of iterative schemes for the solution of the discretized quasi-variational inequalities, with estimates of the rate of convergence (via contraction mappings) in two particular cases. Further, we prove stability of the finite-difference schemes, and convergence of the solution of the discrete problems to the solution of the continuous problem as the discretization mesh goes to zero. We provide a direct interpretation of the discrete problems in terms of finite-state, continuous-time Markov processes.     

On solving continuous-time dynamic network flows

Temporal dynamics is a crucial feature of network flow problems occurring in many practical applications. Important characteristics of real-world networks such as arc capacities, transit times, transit and storage costs, demands and supplies etc. are subject to fluctuations over time. Consequently, also flow on arcs can change over time which leads to so-called dynamic network flows. While time is a continuous entity by nature, discrete-time models are often used for modeling dynamic network flows as the resulting problems are in general much easier to handle computationally. In this paper, we study a general class of dynamic network flow problems in the continuous-time model, where the input functions are assumed to be piecewise linear or piecewise constant. We give two discrete approximations of the problem by dividing the considered time range into intervals where all parameters are constant or linear. We then present two algorithms that compute, or at least converge to optimum solutions. Finally, we give an empirical analysis of the performance of both algorithms.     

Geometric ergodicity of affine processes on cones

For affine processes on finite-dimensional cones, we give criteria for geometric ergodicity — that is exponentially fast convergence to a unique stationary distribution. Ergodic results include both the existence of exponential moments of the limiting distribution, where we exploit the crucial affine property, and finite moments, where we invoke the polynomial property of affine semigroups. Furthermore, we elaborate sufficient conditions for aperiodicity and irreducibility. Our results are applicable to Wishart processes with jumps on the positive semidefinite matrices, continuous-time branching processes with immigration in high dimensions, and classical term-structure models for credit and interest rate risk.     

Two-time-scale Jump-Diffusion Models with Markovian Switching Regimes

This work is concerned with two-time-scale jump diffusion models modulated by continuous-time Markov chains. One of our motivations stems from generalization of insurance risk models. The models are hybrid in the sense that they involve both continuous dynamics and discrete events. Two cases are considered. One of them has a fast-varying switching process, and the other contains a rapidly fluctuating diffusion. Two-time scale is used for complexity reduction. Using weak convergence methods, we derive their limit processes. The insight and implication provided by the analysis are: to reduce the complexity, one can ignore the detailed variations and concentrate on the limit or the reduced models.     

On the link between infinite horizon control and quasi-stationary distributions

We study infinite horizon control of continuous-time non-linear branching processes with almost sure extinction for general (positive or negative) discount. Our main goal is to study the link between infinite horizon control of these processes and an optimization problem involving their quasi-stationary distributions and the corresponding extinction rates. More precisely, we obtain an equivalent of the value function when the discount parameter is close to the threshold where the value function becomes infinite, and we characterize the optimal Markov control in this limit. To achieve this, we present a new proof of the dynamic programming principle based upon a pseudo-Markov property for controlled jump processes. We also prove the convergence to a unique quasi-stationary distribution of non-linear branching processes controlled by a Markov control conditioned on non-extinction.     

盈余过程的马氏性与索赔到达间隔分布为离散型的连续时间风险模型

本文研究广泛的一类连续时间风险模型盈余过程的马氏性,得到了盈余过程成为马氏过程的充分必要条件．首次建立了索赔到达间隔为离散型分布的连续时间风险模型．并对两个基本特例得到了破产概率的准确表达式．     

Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes

We obtain sufficient criteria for central limit theorems (CLTs) for ergodic continuous-time Markov chains (CTMCs). We apply the results to establish CLTs for continuous-time single birth processes. Moreover, we present an explicit expression of the time average variance constant for a single birth process whenever a CLT exists. Several examples are given to illustrate these results.     

Power utility maximization in exponential Lévy models: convergence of discrete-time to continuous-time maximizers

We consider power utility maximization of terminal wealth in a 1-dimensional continuous-time exponential Lévy model with finite time horizon. We discretize the model by restricting portfolio adjustments to an equidistant discrete time grid. Under minimal assumptions we prove convergence of the optimal discrete-time strategies to the continuous-time counterpart. In addition, we provide and compare qualitative properties of the discrete-time and continuous-time optimizers.     

HARA frontiers of optimal portfolios in stochastic markets

In this paper, we consider the optimal portfolio selection problem in continuous-time settings where the investor maximizes the expected utility of the terminal wealth in a stochastic market. The utility function has the structure of the HARA family and the market states change according to a Markov process. The states of the market describe the prevailing economic, financial, social and other conditions that affect the deterministic and probabilistic parameters of the model. This includes the distributions of the random asset returns as well as the utility function. We analyzed Black–Scholes type continuous-time models where the market parameters are driven by Markov processes. The Markov process that affects the state of the market is independent of the underlying Brownian motion that drives the stock prices. The problem of maximizing the expected utility of the terminal wealth is investigated and solved by stochastic optimal control methods for exponential, logarithmic and power utility functions. We found explicit solutions for optimal policy and the associated value functions. We also constructed the optimal wealth process explicitly and discussed some of its properties. In particular, it is shown that the optimal policy provides linear frontiers.     

Compound Markov counting processes and their applications to modeling infinitesimally over-dispersed systems

We propose an infinitesimal dispersion index for Markov counting processes. We show that, under standard moment existence conditions, a process is infinitesimally (over-)equi-dispersed if, and only if, it is simple (compound), i.e. it increases in jumps of one (or more) unit(s), even though infinitesimally equi-dispersed processes might be under-, equi- or over-dispersed using previously studied indices. Compound processes arise, for example, when introducing continuous-time white noise to the rates of simple processes resulting in Lévy-driven SDEs. We construct multivariate infinitesimally over-dispersed compartment models and queuing networks, suitable for applications where moment constraints inherent to simple processes do not hold.     

Using the Dinkelbach-type algorithm to solve the continuous-time linear fractional programming problems

A Dinkelbach-type algorithm is proposed in this paper to solve a class of continuous-time linear fractional programming problems. We shall transform this original problem into a continuous-time non-fractional programming problem, which unfortunately happens to be a continuous-time nonlinear programming problem. In order to tackle this nonlinear problem, we propose the auxiliary problem that will be formulated as parametric continuous-time linear programming problem. We also introduce a dual problem of this parametric continuous-time linear programming problem in which the weak duality theorem also holds true. We introduce the discrete approximation method to solve the primal and dual pair of parametric continuous-time linear programming problems by using the recurrence method. Finally, we provide two numerical examples to demonstrate the usefulness of this practical algorithm.     

Using the parametric approach to solve the continuous-time linear fractional max–min problems

A numerical algorithm based on parametric approach is proposed in this paper to solve a class of continuous-time linear fractional max-min programming problems. We shall transform this original problem into a continuous-time non-fractional programming problem, which unfortunately happens to be a continuous-time nonlinear programming problem. In order to tackle this nonlinear problem, we propose the auxiliary problem that will be formulated as a parametric continuous-time linear programming problem. We also introduce a dual problem of this parametric continuous-time linear programming problem in which the weak duality theorem also holds true. We introduce the discrete approximation method to solve the primal and dual pair of parametric continuous-time linear programming problems by using the recurrence method. Finally, we provide two numerical examples to demonstrate the usefulness of this algorithm.     

Variance or spectral density in sampled data filtering?

Most physical systems operate in continuous time. However, to interact with such systems one needs to take samples. This raises the question of the relationship between the sampled response and the response of the underlying continuous-time system. In this paper we review several aspects of the sampling process. In particular, we examine the role played by variance and spectral density in describing discrete random processes. We argue that spectral density has several advantages over variance. We illustrate the ideas by reference to the problem of state estimation using the discrete-time Kalman filter.     

Switching systems with dwell time: Computing the maximal Lyapunov exponent

We study asymptotic stability of continuous-time systems with mode-dependent guaranteed dwell time. These systems are reformulated as special cases of a general class of mixed (discrete–continuous) linear switching systems on graphs, in which some modes correspond to discrete actions and some others correspond to continuous-time evolutions. Each discrete action has its own positive weight which accounts for its time-duration. We develop a theory of stability for the mixed systems; in particular, we prove the existence of an invariant Lyapunov norm for mixed systems on graphs and study its structure in various cases, including discrete-time systems for which discrete actions have inhomogeneous time durations. This allows us to adapt recent methods for the joint spectral radius computation (Gripenberg’s algorithm and the Invariant Polytope Algorithm) to compute the Lyapunov exponent of mixed systems on graphs.     

Discrete-continuous model conversion

This paper presents methods for model conversions of continuous-time state-space equations and discrete-time state-space equations. An improved geometric-series method is presented for converting continuous-time models to equivalent discrete-time models. Also, a direct truncation method, a matrix continued fraction method and a geometric-series method are presented for converting discrete models to equivalent continuous models. As a result, many well-developed theorems and methods in either continuous or discrete domains can be effectively applied to a suitable model in either domain.     

Linear Stieltjes equation with generalized riemann integral and existence of regulated solutions

1. IntroductiollIn the last two decades the memann generalized integral, having values in B-spaces,has been increasingly studied.The development in this area mainly concerns the HenstoCk-Kurzweil (see e.g. I1, 2]),and the Dushnik and the Young integrals (see e.g. I3]). Recently there appeared in the1iterature many proper app1ications in this field ("proper" here being considered in thesense that the results are not disguises of an essentially finite dimensional frame) (see e.g.[4, 5]).In t…     

Models and algorithms for skip-free Markov decision processes on trees

We introduce a class of models for multidimensional control problems that we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in the negative direction. Starting with the finite average cost case, we show that the algorithm combines the advantages of both value iteration and policy iteration—it is guaranteed to converge to an optimal policy and optimal value function after a finite number of iterations but the computational effort required for each iteration step is comparable with that for value iteration. We show that the algorithm can also be used to solve discounted cost models and continuous-time models, and that a suitably modified algorithm can be used to solve communicating models.     

随机线性二次最优控制:从离散到连续时间模型

在一般情形下,分析了离散时间LQ问题与连续时间情形两者之间的自然联系.首先回顾了连续时间和离散时间随机LQ问题及对应Riccati微分/差分方程的相关结论.接下来在假设Riccati微分方程有解的前提下,证明了离散化步长足够小时,Riccati差分方程有解.然后针对连续和离散时间模型,采用配对问题最优控制的反馈形式,分别构造了一个辅助反馈控制,并证明该控制可驱使对应模型的性能指标逼近于配对问题的值函数,以此得到了关于两个模型之间联系的初步结论.最后藉由前述结论以及控制问题的特性,揭晓了连续时间和离散时间模型之间的自然联系,并给出了Riccati差分方程和微分方程的解之间的误差估计.由此联系,可构造相应离散系统和LQ问题,以适当的阶估计连续时间LQ问题的解,抑或为离散时间模型构造一个近似最优控制.无论哪种思路,都旨在降低直接求解原问题的难度和复杂性.     

Average stochastic games for continuous-time jump processes

We consider nonzero-sum games for continuous-time jump processes with unbounded transition rates under expected average payoff criterion. The state and action spaces are Borel spaces and reward rates are unbounded. We introduce an approximating sequence of stochastic game models with extended state space, for which the uniform exponential ergodicity is obtained. Moreover, we prove the existence of a stationary almost Markov Nash equilibrium by introducing auxiliary static game models. Finally, a cash flow model is employed to illustrate the results.