Deviation Matrix,Laurent Series and Blackwell Optimality in Countable State Markov Decision Processes

This paper presents a recurrent condition on Markov decision processes with a countable state space and bounded rewards. The condition is sufficient for the existence of a Blackwell optimal stationary policy, having the Laurent series expansion with continuous coefficients. It is so relaxed that the Markov chain corresponding to a stationary policy may have countably many periodic recurrent classes. Our method finds the deviation matrix in an explicit form.     

On formal Laurent series

Several kinds of formal Laurent series have been introduced with some restrictions so far. This paper systematically sets up a natural definition and structure of formal Laurent series without those restrictions, including introducing a multiplication between formal Laurent series. This paper also provides some results on the algebraic structure of the space of formal Laurent series, denoted by \mathbbL\mathbb{L}. By means of the results of the generalized composition of formal power series, we define a composition of a Laurent series with a formal power series and provide a necessary and sufficient condition for the existence of such compositions. The calculus about formal Laurent series is also introduced.     

Second order optimality conditions for theL 1-minimization problem

In this paper we establish necessary and sufficient second order optimality conditions for theL ₁-problem. The approach is based on optimality criteria in terms of a curved second directional derivative, discussed in [3]. Our conditions generalize conditions for theL ₁-problem given in [6]. An example demonstrates the usefulness of our criteria.This research was supported by NSF Grant No. ECS-8214081 and the Fund for Promotion of Research at the Technion, andDeutsche Forschungsgemeinschaft.     

Laurent-Padé approximants to four kinds of Chebyshev polynomial expansions. Part II: Clenshaw-Lord type approximants

Laurent-Padé (Chebyshev) rational approximantsP _m (w, w ⁻¹)/Q _n (w, w ⁻¹) of Clenshaw-Lord type [2,1] are defined, such that the Laurent series ofP _m /Q _n matches that of a given functionf(w, w ⁻¹) up to terms of orderw ^±(m+n) , based only on knowledge of the Laurent series coefficients off up to terms inw ^±(m+n) . This contrasts with the Maehly-type approximants [4,5] defined and computed in part I of this paper [6], where the Laurent series ofP _m matches that ofQ _n f up to terms of orderw ^±(m+n ), but based on knowledge of the series coefficients off up to terms inw ^±(m+2n). The Clenshaw-Lord method is here extended to be applicable to Chebyshev polynomials of the 1st, 2nd, 3rd and 4th kinds and corresponding rational approximants and Laurent series, and efficient systems of linear equations for the determination of the Padé-Chebyshev coefficients are obtained in each case. Using the Laurent approach of Gragg and Johnson [4], approximations are obtainable for allm≥0,n≥0. Numerical results are obtained for all four kinds of Chebyshev polynomials and Padé-Chebyshev approximants. Remarkably similar results of formidable accuracy are obtained by both Maehly-type and Clenshaw-Lord type methods, thus validating the use of either.     

Laurent-Padé approximants to four kinds of Chebyshev polynomial expansions. Part I. Maehly type approximants

Laurent Padé-Chebyshev rational approximants,A _m (z,z ⁻¹)/B _n (z, z ⁻¹), whose Laurent series expansions match that of a given functionf(z,z ⁻¹) up to as high a degree inz, z ⁻¹ as possible, were introduced for first kind Chebyshev polynomials by Clenshaw and Lord [2] and, using Laurent series, by Gragg and Johnson [4]. Further real and complex extensions, based mainly on trigonometric expansions, were discussed by Chisholm and Common [1]. All of these methods require knowledge of Chebyshev coefficients off up to degreem+n. Earlier, Maehly [5] introduced Padé approximants of the same form, which matched expansions betweenf(z,z ⁻¹)B _n (z, z ⁻¹)). The derivation was relatively simple but required knowledge of Chebyshev coefficients off up to degreem+2n. In the present paper, Padé-Chebyshev approximants are developed not only to first, but also to second, third and fourth kind Chebyshev polynomial series, based throughout on Laurent series representations of the Maehly type. The procedures for developing the Padé-Chebyshev coefficients are similar to that for a traditional Padé approximant based on power series [8] but with essential modifications. By equating series coefficients and combining equations appropriately, a linear system of equations is successfully developed into two sub-systems, one for determining the denominator coefficients only and one for explicitly defining the numerator coefficients in terms of the denominator coefficients. In all cases, a type (m, n) Padé-Chebyshev approximant, of degreem in the numerator andn in the denominator, is matched to the Chebyshev series up to terms of degreem+n, based on knowledge of the Chebyshev coefficients up to degreem+2n. Numerical tests are carried out on all four Padé-Chebyshev approximants, and results are outstanding, with some formidable improvements being achieved over partial sums of Laurent-Chebyshev series on a variety of functions. In part II of this paper [7] Padé-Chebyshev approximants of Clenshaw-Lord type will be developed for the four kinds of Chebyshev series and compared with those of the Maehly type.     

Non-commutative integrability, paths and quasi-determinants

In previous work, we showed that the solution of certain systems of discrete integrable equations, notably Q and T-systems, is given in terms of partition functions of positively weighted paths, thereby proving the positive Laurent phenomenon of Fomin and Zelevinsky for these cases. This method of solution is amenable to generalization to non-commutative weighted paths. Under certain circumstances, these describe solutions of discrete evolution equations in non-commutative variables: Examples are the corresponding quantum cluster algebras (Berenstein and Zelevinsky (2005) [3]), the Kontsevich evolution (Di Francesco and Kedem (2010) [10]) and the T-systems themselves (Di Francesco and Kedem (2009) [8]). In this paper, we formulate certain non-commutative integrable evolutions by considering paths with non-commutative weights, together with an evolution of the weights that reduces to cluster algebra mutations in the commutative limit. The general weights are expressed as Laurent monomials of quasi-determinants of path partition functions, allowing for a non-commutative version of the positive Laurent phenomenon. We apply this construction to the known systems, and obtain Laurent positivity results for their solutions in terms of initial data.     

On the asymptotic behavior of the Laurent coefficients of a class of Dirichlet series

Matsuoka showed an asymptotic formula for the coefficients of the Laurent expansion of ζ (s) at s = 1. In the present paper we extend this result to a large class of Dirichlet series which was first studied by Chandrasekharan and Narasimhan. Our proofs are based on a saddle point argument and use the fact that the Dirichlet series under consideration admit a functional equation. The second author was supported by the FWF project S381O.     

Simultaneous approximation by universal series

We show that, if individual universal series exist, then we can choose a sequence of universal series performing simultaneous universal approximation with the same sequence of indices. As an application we derive the existence of universal Laurent Series on an annulus using only the existence of universal Taylor Series on discs. Our results are generic (© 2010 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

A new strong optimality criterion for nonstationary Markov decision processes

This paper deals with a new optimality criterion consisting of the usual three average criteria and the canonical triplet (totally so-called strong average-canonical optimality criterion) and introduces the concept of a strong average-canonical policy for nonstationary Markov decision processes, which is an extension of the canonical policies of Herna′ndez-Lerma and Lasserre [16] (pages: 77) for the stationary Markov controlled processes. For the case of possibly non-uniformly bounded rewards and denumerable state space, we first construct, under some conditions, a solution to the optimality equations (OEs), and then prove that the Markov policies obtained from the OEs are not only optimal for the three average criteria but also optimal for all finite horizon criteria with a sequence of additional functions as their terminal rewards (i.e. strong average-canonical optimal). Also, some properties of optimal policies and optimal average value convergence are discussed. Moreover, the error bound in average reward between a rolling horizon policy and a strong average-canonical optimal policy is provided, and then a rolling horizon algorithm for computing strong average ε(>0)-optimal Markov policies is given.     

The Classification of Representations f Unramified Hecke Algebras

We give two characterizations of the isolated singularities of the local resolvent function of an operator T ε L(X) at a point ε of a complex Banach space X: in terms of a suitable decomposition of ε, and in terms of the existence of a sequence in X related with the Laurent series of the local resolvent function. Moreover, we introduce the locally chain-finite operators at a point ε and show that T is chain-finite if and only if T is locally chain-finite at every χ ε X.     

Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards

This paper is the first part of a study of Blackwell optimal policies in Markov decision chains with a Borel state space and unbounded rewards. We prove here the existence of deterministic stationary policies which are Blackwell optimal in the class of all, in general randomized, stationary policies. We establish also a lexicographical policy improvement algorithm leading to Blackwell optimal policies and the relation between such policies and the Blackwell optimality equation. Our technique is a combination of the weighted norms approach developed in Dekker and Hordijk (1988) for countable models with unbounded rewards and of the weak-strong topology approach used in Yushkevich (1997a) for Borel models with bounded rewards.     

Semiprimeness,quasi-Baerness and prime radical of skew generalized power series rings

Let R be a ring, (S,≤) a strictly ordered monoid and ω:S→End(R) a monoid homomorphism. The skew generalized power series ring R[[S,ω]] is a common generalization of (skew) polynomial rings, (skew) power series rings, (skew) Laurent polynomial rings, (skew) group rings, and Mal’cev-Neumann Laurent series rings. In this paper we obtain necessary and sufficient conditions for the skew generalized power series ring R[[S,ω]] to be a semiprime, prime, quasi-Baer, or Baer ring. Furthermore, we study the prime radical of a skew generalized power series ring R[[S,ω]]. Our results extend and unify many existing results. In particular, we obtain new theorems on (skew) group rings, Mal’cev-Neumann Laurent series rings and the ring of generalized power series.     

Laurent–Padé Approximants to Four Kinds of Chebyshev Polynomial Expansions. Part I. Maehly Type Approximants

Laurent Padé–Chebyshev rational approximants, A _m (z,z ^–1)/B _n (z,z ^–1), whose Laurent series expansions match that of a given function f(z,z ^–1) up to as high a degree in z,z ^–1 as possible, were introduced for first kind Chebyshev polynomials by Clenshaw and Lord [2] and, using Laurent series, by Gragg and Johnson [4]. Further real and complex extensions, based mainly on trigonometric expansions, were discussed by Chisholm and Common [1]. All of these methods require knowledge of Chebyshev coefficients of f up to degree m+n. Earlier, Maehly [5] introduced Padé approximants of the same form, which matched expansions between f(z,z ^–1)B _n (z,z ^–1) and A _m (z,z ^–1). The derivation was relatively simple but required knowledge of Chebyshev coefficients of f up to degree m+2n. In the present paper, Padé–Chebyshev approximants are developed not only to first, but also to second, third and fourth kind Chebyshev polynomial series, based throughout on Laurent series representations of the Maehly type. The procedures for developing the Padé–Chebyshev coefficients are similar to that for a traditional Padé approximant based on power series [8] but with essential modifications. By equating series coefficients and combining equations appropriately, a linear system of equations is successfully developed into two sub-systems, one for determining the denominator coefficients only and one for explicitly defining the numerator coefficients in terms of the denominator coefficients. In all cases, a type (m,n) Padé–Chebyshev approximant, of degree m in the numerator and n in the denominator, is matched to the Chebyshev series up to terms of degree m+n, based on knowledge of the Chebyshev coefficients up to degree m+2n. Numerical tests are carried out on all four Padé–Chebyshev approximants, and results are outstanding, with some formidable improvements being achieved over partial sums of Laurent–Chebyshev series on a variety of functions. In part II of this paper [7] Padé–Chebyshev approximants of Clenshaw–Lord type will be developed for the four kinds of Chebyshev series and compared with those of the Maehly type.     

On stopped decision processes with discrete time parameter

A non-stationary stopped decision process is investigated under rather weak convergence assumptions on the expected total rewards. Sufficient conditions are given for the approximation of the maximal conditional expected rewards from infinite stage play by the maximal conditional expected rewards from finite stage play. General criteria of optimality are derived. The results are essentially based on two lemmas given in this paper. The existence of optimal plans is established using results of non-stationary dynamic programming.     

Finite-stage stochastic decision processes with recursive reward structure I: optimality equations and deterministic strategies

In this paper we examine an N-stage stochastic decision model with a recursive reward structure whose state and action spaces are standard Borel ones. The central results relate to the validity of the optimality equations and to the sufficiency of deterministic strategies. The results expand statements known for classical dynamic programming problems with additive total rewards to a wide class of recursive reward functions under certain monotonicity and continuity assumptions. The existence of an expected utility representation of the total rewards is generally not presupposed     

Laurent–Padé Approximants to Four Kinds of Chebyshev Polynomial Expansions. Part II: Clenshaw–Lord Type Approximants

Laurent–Padé (Chebyshev) rational approximants P _m (w,w ^–1)/Q _n (w,w ^–1) of Clenshaw–Lord type [2,1] are defined, such that the Laurent series of P _m /Q _n matches that of a given function f(w,w ^–1) up to terms of order w ^±(m+n), based only on knowledge of the Laurent series coefficients of f up to terms in w ^±(m+n). This contrasts with the Maehly-type approximants [4,5] defined and computed in part I of this paper [6], where the Laurent series of P _m matches that of Q _n f up to terms of order w ^±(m+n), but based on knowledge of the series coefficients of f up to terms in w ^±(m+2n). The Clenshaw–Lord method is here extended to be applicable to Chebyshev polynomials of the 1st, 2nd, 3rd and 4th kinds and corresponding rational approximants and Laurent series, and efficient systems of linear equations for the determination of the Padé–Chebyshev coefficients are obtained in each case. Using the Laurent approach of Gragg and Johnson [4], approximations are obtainable for all m0, n0. Numerical results are obtained for all four kinds of Chebyshev polynomials and Padé–Chebyshev approximants. Remarkably similar results of formidable accuracy are obtained by both Maehly-type and Clenshaw–Lord type methods, thus validating the use of either.     

Markov control processes with pathwise constraints

This paper deals with discrete-time Markov control processes in Borel spaces, with unbounded rewards. The criterion to be optimized is a long-run sample-path (or pathwise) average reward subject to constraints on a long-run pathwise average cost. To study this pathwise problem, we give conditions for the existence of optimal policies for the problem with “expected” constraints. Moreover, we show that the expected case can be solved by means of a parametric family of optimality equations. These results are then extended to the problem with pathwise constraints.     

Recent results on conditions for the existence of average optimal stationary policies

This paper concerns countable state space Markov decision processes endowed with a (long-run expected)average reward criterion. For these models we summarize and, in some cases,extend some recent results on sufficient conditions to establish the existence of optimal stationary policies. The topics considered are the following: (i) the new assumptions introduced by Sennott in [20–23], (ii)necessary and sufficient conditions for the existence of a bounded solution to the optimality equation, and (iii) equivalence of average optimality criteria. Some problems are posed.This research was partially supported by the Third World Academy of Sciences (TWAS) under Grant No. TWAS RG MP 898-152.     

On primeness of general skew inverse Laurent series ring

Ever since the introduction, skew inverse Laurent series rings have kept growing in importance, as researchers characterized their properties (such as Noetherianness, Armendarizness, McCoyness, etc.) in terms of intrinsic properties of the base ring and studied their relations with other fields of mathematics, as for example quantum mechanics theory. The goal of our paper is to study the primeness and semiprimeness of general skew inverse Laurent series rings R((x^?1;σ,δ)), where R is an associative ring equipped with an automorphism σ and a σ-derivation δ.