报酬函数及转移速率族均非一致有界的连续时间折扣马氏决策规划

本文首次在报酬函数及转移速率族均非一致有界的条件下，对可数状态空间，可地动集的连续时间折扣马氏决策规划进行研究，文中引入一类新的无界报酬函数，在一类新的马氏策略中，讨论了最优策略的存在性及春结构，除证明了在有界报酬和一致有界转移速率族下成立的主要结果外，本文还得到一些重要结论。     

Strong 0-discount optimal policies in a Markov decision process with a Borel state space

We consider a Markov decision process with a Borel state space, a countable action space, finite action sets, bounded rewards and a bounded transition density satisfying a simultaneous Doeblin condition. The existence of stationary strong 0-discount optimal polices is proved.Supported by NSF grant DMS-9404177.     

Existence of optimal policy for time non-homogeneous discounted Markovian decision programming

In this paper we discuss the discrete time non-homogeneous discounted Markovian decision programming, where the state space and all action sets are countable. Suppose that the optimum value function is finite. We give the necessary and sufficient conditions for the existence of an optimal policy. Suppose that the absolute mean of rewards is relatively bounded. We also give the necessary and sufficient conditions for the existence of an optimal policy.     

Finite-state approximations for denumerable state discounted markov decision processes

A finite-state iterative scheme introduced by White [9] to approximate the optimal value function of denumerable-state Markov decision processes with bounded rewards, is extended to the case of unbounded rewards. Convergence theorems that, when applied to the case of bounded rewards, give stronger results than those in [9] are proved. Moreover, bounds on the rates of convergence under several assumptions are given and the extended scheme is used to obtain policies with asymptotic optimality properties.This research was supported in part by the Consejo Nacional de Ciencia y Tecnología under Grant PCCBBNA 020630, and in part by the Universidad Autónoma Agraria Antonio Narro.     

Optimality equations and sensitive optimality in bounded Markov decision processes 1

This paper consider Markov decision processes with countable state space, compact action spaces and a bounded reward function. Under some recurrence and connectedness condition, including the simultaneous Döblin condition, we prove the existence of bounded solutions of the optimality equations which arise for the multichain case in connection with the average reward criterion and sensitive optimality criteria, and we give a characterization of the sets of n-average optimal decision rules.     

Denumerable semi-Markov decision chains with small interest rates

In this paper we investigate denumerable state semi-Markov decision chains with small interest rates. We consider average and Blackwell optimality and allow for multiple closed sets and unbounded immediate rewards. Our analysis uses the existence of a Laurent series expansion for the total discounted rewards and the continuity of its terms. The assumptions are expressed in terms of a weighted supremum norm. Our method is based on an algebraic treatment of Laurent series; it constructs an appropriate linear space with a lexicographic ordering. Using two operators and a positiveness property we establish the existence of bounded solutions to optimality equations. The theory is illustrated with an example of aK-dimensional queueing system. This paper is strongly based on the work of Denardo [11] and Dekker and Hordijk [7].This research has partially been sponsored by the Netherlands Organization for Scientific Research (NWO).     

Nonstationary denumerable state Markov decision processes – with average variance criterion

In this paper, we consider the nonstationary Markov decision processes (MDP, for short) with average variance criterion on a countable state space, finite action spaces and bounded one-step rewards. From the optimality equations which are provided in this paper, we translate the average variance criterion into a new average expected cost criterion. Then we prove that there exists a Markov policy, which is optimal in an original average expected reward criterion, that minimizies the average variance in the class of optimal policies for the original average expected reward criterion.     

On the Chebyshev Center and the Nonemptiness of the Intersection of Nested Sets

Mathematical Notes - We show that if every bounded set in a Banach space has a Chebyshev center, then the intersection of nested closed bounded sets in this space is nonempty in the case of a...     

Bounded VC-Dimension Implies a Fractional Helly Theorem

We prove that every set system of bounded VC-dimension has a fractional Helly property. More precisely, if the dual shatter function of a set system $\FF$ is bounded by $o(m^k)$, then $\FF$ has fractional Helly number $k$. This means that for every $\alpha>0$ there exists a $\beta>0$ such that if $F_1,F_2,\ldots,F_n\in\FF$ are sets with $\bigcap_{i\in I}F_i\neq\emptyset$ for at least $\alpha{n\choose k}$ sets $I\subseteq\{1,2,\ldots,n\}$ of size $k$, then there exists a point common to at least $\beta n$ of the $F_i$. This further implies a $(p,k)$-theorem: for every $\FF$ as above and every $p\geq k$ there exists $T$ such that if $\GG\subseteq\FF$ is a finite subfamily where among every $p$ sets, some $k$ intersect, then $\GG$ has a transversal of size $T$. The assumption about bounded dual shatter function applies, for example, to families of sets in $\Rd$ definable by a bounded number of polynomial inequalities of bounded degree; in this case we obtain fractional Helly number $d{+}1$.     

N-Person stochastic games with upper semi-continuous payoffs

Every n-person stochastic game with a countable state space, finite action sets for the players and bounded, upper semi-continuous payoffs has an ε-equilibrium for every ε>0.     

Localization of invariant compact sets of nonautonomous systems

We generalize the localization method for invariant compact sets of an autonomous dynamical system to the case of a nonautonomous system of differential equations. By using this method, we solve the localization problem for the Vallis third-order dynamical system governing some processes in atmosphere dynamics over the Pacific Ocean. For this system, we construct a one-parameter family of localizing sets bounded by second-order surfaces and find the intersection of all sets of the family.     

Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards

This paper is the first part of a study of Blackwell optimal policies in Markov decision chains with a Borel state space and unbounded rewards. We prove here the existence of deterministic stationary policies which are Blackwell optimal in the class of all, in general randomized, stationary policies. We establish also a lexicographical policy improvement algorithm leading to Blackwell optimal policies and the relation between such policies and the Blackwell optimality equation. Our technique is a combination of the weighted norms approach developed in Dekker and Hordijk (1988) for countable models with unbounded rewards and of the weak-strong topology approach used in Yushkevich (1997a) for Borel models with bounded rewards.     

Bornologies and metrically generated theories

Bornologies axiomatize an abstract notion of bounded sets and are introduced as collections of subsets satisfying a number of consistency properties. Bornological spaces form a topological construct, the morphisms of which are those functions which preserve bounded sets. A typical example is a bornology generated by a metric, i.e. the collection of all bounded sets for that metric. In a recent paper [E. Colebunders, R. Lowen, Metrically generated theories, Proc. Amer. Math. Soc. 133 (2005) 1547-1556] the authors noted that many examples are known of natural functors describing the transition from categories of metric spaces to the “metrizable” objects in some given topological construct such that, in some natural way, the metrizable objects generate the whole construct. These constructs can be axiomatically described and are called metrically generated. The construct of bornological spaces is not metrically generated, but an important large subconstruct is. We also encounter other important examples of metrically generated constructs, the constructs of Lipschitz spaces, of uniform spaces and of completely regular spaces. In this paper, the unified setting of metrically generated theories is used to study the functorial relationship between these constructs and the one of bornological spaces.     

Convergence of multiple fourier series for functions of bounded variation

For functions of bounded variation in the sense of Hardy, we consider the pointwise convergence of the partial sums of Fourier series over a given sequence of bounded sets in the space of harmonics. We obtain sufficient conditions for convergence; necessary and sufficient conditions are obtained for the case in which these sets are convex with respect to each coordinate direction. The Pringsheim convergence of Fourier series in this problem was established by Hardy. Translated fromMatematicheskie Zametki, Vol. 61, No. 4, pp. 583–595, April, 1997. Translated by S. A. Telyakovskii and V. N. Temlyakov     

Total boundedness and bornologies

A set A in a metric space is called totally bounded if for each ε>0 the set can be ε-approximated by a finite set. If this can be done, the finite set can always be chosen inside A. If the finite sets are replaced by an arbitrary approximating family of sets, this coincidence may disappear. We present necessary and sufficient conditions for the coincidence assuming only that the family is closed under finite unions. A complete analysis of the structure of totally bounded sets is presented in the case that the approximating family is a bornology, where approximation in either sense amounts to approximation in Hausdorff distance by members of the bornology.     

Unified Treatment of Algebraic and Geometric Difference by a New Difference Scheme and its Continuity Properties

We introduce a new operation for the difference of two sets A and C of R ⁿ depending on a parameter . This new operation may yield as special cases the classical difference and the Minkowski difference, if the sets A and C are closed, convex sets, if int(C) is nonempty, and if A or C bounded. Continuity properties with respect to both the operands and the parameter of this operation are studied. Lipschitz properties of the Minkowski difference between two sets of a normed vector space are proved in the bounded case as well as in the unbounded case without condition on the dimension of the space.     

Contractive Markov Systems

Certain discrete-time Markov processes on locally compact metricspaces which arise from graph-directed constructions of fractalsets with place-dependent probabilities are studied. Such systemsnaturally extend finite Markov chains and inherit some of theirproperties. It is shown that the Markov operator defined bysuch a system has a unique invariant probability measure inthe irreducible case and an attractive probability measure inthe aperiodic case if the vertex sets form an open partitionof the state space, the restrictions of the probability functionson their vertex sets are Dini-continuous and bounded away fromzero, and the system satisfies a condition of contractivenesson average.     

Rough Isometry and <Emphasis Type="Italic">p</Emphasis>-Harmonic Boundaries of Complete Riemannian Manifolds

In this paper, we describe the behavior of bounded energy finite solutions for certain nonlinear elliptic operators on a complete Riemannian manifold in terms of its p-harmonic boundary. We also prove that if two complete Riemannian manifolds are roughly isometric to each other, then their p-harmonic boundaries are homeomorphic to each other. In the case, there is a one to one correspondence between the sets of bounded energy finite solutions on such manifolds. In particular, in the case of the Laplacian, it becomes a linear isomorphism between the spaces of bounded harmonic functions with finite Dirichlet integral on the manifolds. This work was supported by grant No. R06-2002-012-01001-0(2002) from the Basic Research Program of the Korea Science & Engineering Foundation.