On Numerical Solution of the Markov Renewal Equation: Tight Upper and Lower Kernel Bounds

We develop tight bounds and a fast parallel algorithm to compute the Markov renewal kernel. Knowledge of the kernel allows us to solve Markov renewal equations numerically to study non-steady state behavior in a finite state Markov renewal process. Computational error and numerical stability for computing the bounds in parallel are discussed using well-known results from numerical analysis. We use our algorithm and computed bounds to study the expected number of departures as a function of time for a two node overflow queueing network.     

Random walk centrality and a partition of Kemeny’s constant

We consider an accessibility index for the states of a discrete-time, ergodic, homogeneous Markov chain on a finite state space; this index is naturally associated with the random walk centrality introduced by Noh and Reiger (2004) for a random walk on a connected graph. We observe that the vector of accessibility indices provides a partition of Kemeny’s constant for the Markov chain. We provide three characterizations of this accessibility index: one in terms of the first return time to the state in question, and two in terms of the transition matrix associated with the Markov chain. Several bounds are provided on the accessibility index in terms of the eigenvalues of the transition matrix and the stationary vector, and the bounds are shown to be tight. The behaviour of the accessibility index under perturbation of the transition matrix is investigated, and examples exhibiting some counter-intuitive behaviour are presented. Finally, we characterize the situation in which the accessibility indices for all states coincide.     

随机有序马尔可夫链收敛速度显式界的注记

弱化Scott与Tweedie在计算马氏链收敛速度界时的条件,即变一步转移概率为m(m≥1)步转移概率,并运用不同于Scott与Tweedie的方法,计算出马氏链几何收敛速度r~n的界,从而推广了已有的结论.     

Heat kernels of non-symmetric jump processes with exponentially decaying jumping kernel

In this paper we study the transition densities for a large class of non-symmetric Markov processes whose jumping kernels decay exponentially or subexponentially. We obtain their upper bounds which also decay at the same rate as their jumping kernels. When the lower bounds of jumping kernels satisfy the weak upper scaling condition at zero, we also establish lower bounds for the transition densities, which are sharp.     

A refinement of multivariate value set bounds

Over finite fields, if the image of a polynomial map is not the entire field, then its cardinality can be bounded above by a significantly smaller value. Earlier results bound the cardinality of the value set using the degree of the polynomial, but more recent results make use of the powers of all monomials.In this paper, we explore the geometric properties of the Newton polytope and show how they allow for tighter upper bounds on the cardinality of the multivariate value set. We then explore a method which allows for even stronger upper bounds, regardless of whether one uses the multivariate degree or the Newton polytope to bound the value set. Effectively, this provides improvement of a degree matrix-based result given by Zan and Cao, making our new bound the strongest upper bound thus far.     

Polytope bounds on multivariate value sets

We improve the upper bounds for the cardinality of the value set of a multivariable polynomial map over a finite field using the polytope of the polynomial. This generalizes earlier bounds only dependent on the degree of a polynomial.     

Local approximations of Markov random walks by diffusions

We consider triangular arrays of Markov random walks that can be approximated by an accompanying sequence of diffusion processes. We give uniform bounds for approximation of scaled transition probabilities by transition densities of the diffusion process. In particular, we state local limit theorems for the case that the Markov random walks converge weakly to a diffusion process.     

Markov Chains Competing for Transitions: Application to Large-Scale Distributed Systems

We consider the behavior of a stochastic system composed of several identically distributed, but non independent, discrete-time absorbing Markov chains competing at each instant for a transition. The competition consists in determining at each instant, using a given probability distribution, the only Markov chain allowed to make a transition. We analyze the first time at which one of the Markov chains reaches its absorbing state. We obtain its distribution and its expectation and we propose an algorithm to compute these quantities. We also exhibit the asymptotic behavior of the system when the number of Markov chains goes to infinity. Actually, this problem comes from the analysis of large-scale distributed systems and we show how our results apply to this domain.     

A new asymptotic bound of the minimum possible odd cardinality of spherical (2k−1)-designs

We apply polynomial techniques to investigate the structure of spherical designs in an asymptotic process with fixed odd strength while the dimension and odd cardinality tend to infinity in a certain relation. Our bounds for the extreme inner products of special points in such designs allow new lower bounds on the minimum possible odd cardinality.     

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Abstract

In this article, we study continuous-time Markov decision processes in Polish spaces. The optimality criterion to be maximized is the expected discounted criterion. The transition rates may be unbounded, and the reward rates may have neither upper nor lower bounds. We provide conditions on the controlled system's primitive data under which we prove that the transition functions of possibly non-homogeneous continuous-time Markov processes are regular by using Feller's construction approach to such transition functions. Then, under continuity and compactness conditions we prove the existence of optimal stationary policies by using the technique of extended infinitesimal operators associated with the transition functions of possibly non-homogeneous continuous-time Markov processes, and also provide a recursive way to compute (or at least to approximate) the optimal reward values. The conditions provided in this paper are different from those used in the previous literature, and they are illustrated with an example.     

Perturbed Markov Chains with Damping Component

The paper is devoted to studies of regularly and singularly perturbed Markov chains with damping component. In such models, a matrix of transition probabilities is regularised by adding a special damping matrix multiplied by a small damping (perturbation) parameter ε. We perform a detailed perturbation analysis for such Markov chains, particularly, give effective upper bounds for the rate of approximation for stationary distributions of unperturbed Markov chains by stationary distributions of perturbed Markov chains with regularised matrices of transition probabilities, asymptotic expansions for approximating stationary distributions with respect to damping parameter, explicit coupling type upper bounds for the rate of convergence in ergodic theorems for n-step transition probabilities, as well as ergodic theorems in triangular array mode.

     

Convergence of stochastic search algorithms to finite size pareto set approximations

In this work we investigate the convergence of stochastic search algorithms toward the Pareto set of continuous multi-objective optimization problems. The focus is on obtaining a finite approximation that should capture the entire solution set in a suitable sense, which will be defined using the concept of ε-dominance. Under mild assumptions about the process to generate new candidate solutions, the limit approximation set will be determined entirely by the archiving strategy. We propose and analyse two different archiving strategies which lead to a different limit behavior of the algorithms, yielding bounds on the obtained approximation quality as well as on the cardinality of the resulting Pareto set approximation.      

Exact point-distributions over the complex sphere

We study point-distributions over the surface of the unit sphere in unitary space that generate quadrature rules which are exact for spherical polynomials up to a certain bi-degree. In this first stage, we explore several different characterizations for this type of point sets using standard tools such as, positive definiteness, reproducing kernel techniques, linearization formulas, etc. We find bounds on the cardinality of a point-distribution, without discussing the deeper question regarding best bounds. We include examples, construction methods and explain, via isometric embeddings from real to complex spheres, the proper connections with the so-called spherical designs.     

A Note on Varying Cardinality in the Average Case Setting

We study how much information with varying cardinality can be better than information with fixed cardinality for approximating linear operators in the average case setting with Gaussian measure. It has been known that adaptive choice of functionals forming information is not better than nonadaptive, and that the only gain may be obtained by using varying cardinality. We prove that the lower bounds from Traub (J. F. Traub, G. W. Wasilkowski, and H. Woźniakowski, "Information-Based Complexity," Academic Press, San Diego, 1988) et al. on the efficiency of varying cardinality are sharp. In particular, we show that information whose cardinality assumes at most two different values can significantly help in approximating any linear operator with infinite dimensional domain space.     

Data dependent worst case bounds for weighted set packing

We develop data dependent worst case bounds for three simple greedy algorithms for the maximum weighted independent set problem applied to maximum weighted set packing. We exploit the property that the generated output will attain at least a certain weight. These weight quantities are a function of the individual weights corresponding to the vertices of the problem. By using an argument based on linear programming duality we develop a priori bounds that are a function of the minimum guaranteed weight quantities, the highest average reward for a ground item, and cardinality of the ground set. This extends the current bounds which are only a function of the maximum vertex degree in the associated conflict graph. Examples are given that show the benefits of incorporating this data dependent information into bounds.     

Exponential bounds for discrete-time singularly perturbed Markov chains

This paper develops exponential type upper bounds for scaled occupation measures of singularly perturbed Markov chains in discrete time. By considering two-time scale in the Markov chains, asymptotic analysis is carried out. The cases of the fast changing transition probability matrix is irreducible and that are divisible into l ergodic classes are examined first; the upper bounds of a sequence of scaled occupation measures are derived. Then extensions to Markov chains involving transient states and/or nonhomogeneous transition probabilities are dealt with. The results enable us to further our understanding of the underlying Markov chains and related dynamic systems, which is essential for solving many control and optimization problems.     

Markov approximations of chains of infinite order

We consider chains whose transition probabilities depend on the whole past, with summable continuity rates. We show that Ornstein's -distance between one such chain and its canonical Markov approximations of different orders is at worst proportional to the continuity rate of the chain. The result generalizes previous bounds obtained by X. Bressaud and ourselves, while relying on a similar coupling argument. Received: 9 April 2002     

On Weierstrass semigroups and sets: a review with new results

In this work we present a survey of the main results in the theory of Weierstrass semigroups at several points, with special attention to the determination of bounds for the cardinality of its set of gaps. We also review results on applications to the theory of error correcting codes. We then recall a generalization of the concept of Weierstrass semigroup, which is the Weierstrass set associated to a linear system and several points. We finish by presenting new results on this Weierstrass set, including some on the cardinality of its set of gaps.      

On lower bounds on the size of designs in compact symmetric spaces of rank 1

The concept of t-designs in compact symmetric spaces of rank 1 is a generalization of the theory of classical t-designs. In this paper we obtain new lower bounds on the cardinality of designs in projective compact symmetric spaces of rank 1. With one exception our bounds are the first improvements of the classical bounds by more than one. We use the linear programming technique and follow the approach we have proposed for spherical codes and designs. Some examples are shown and compared with the classical bounds.     

A polyhedral study on 0–1 knapsack problems with disjoint cardinality constraints: Strong valid inequalities by sequence-independent lifting

We study the set of 0–1 integer solutions to a single knapsack constraint and a set of non-overlapping cardinality constraints (MCKP), which generalizes the classical 0–1 knapsack polytope and the 0–1 knapsack polytope with generalized upper bounds. We derive strong valid inequalities for the convex hull of its feasible solutions using sequence-independent lifting. For problems with a single cardinality constraint, we derive two-dimensional superadditive lifting functions and prove that they are maximal and non-dominated under some mild conditions. We then show that these functions can be used to build strong valid inequalities for problems with multiple disjoint cardinality constraints. Finally, we present preliminary computational results aimed at evaluating the strength of the cuts obtained from sequence-independent lifting with respect to those obtained from sequential lifting.