A convex analytic approach to Markov decision processes

Summary This paper develops a new framework for the study of Markov decision processes in which the control problem is viewed as an optimization problem on the set of canonically induced measures on the trajectory space of the joint state and control process. This set is shown to be compact convex. One then associates with each of the usual cost criteria (infinite horizon discounted cost, finite horizon, control up to an exit time) a naturally defined occupation measure such that the cost is an integral of some function with respect to this measure. These measures are shown to form a compact convex set whose extreme points are characterized. Classical results about existence of optimal strategies are recovered from this and several applications to multicriteria and constrained optimization problems are briefly indicated.Research supported by NSF Grant CDR-85-00108     

Bayesian stratified random sampling using auxiliary information

Summary Smith (1976,J. R. Statist. Soc., A,139, 183–204) has argued that survey statisticians should attempt to model finite population structures in the same way that statisticians in other disciplines have to provide models of finite or infinite populations. Following this argument, we suggest in this paper that an obvious model for a stratified population when auxiliary information regarding variate values is available, is the one way analysis of covariance model with unequal variances and we consider the problem of estimating the finite population mean. Finally a possible extension of this result is discussed.     

Risk-Sensitive Control and an Abstract Collatz–Wielandt Formula

The ‘value’ of infinite horizon risk-sensitive control is the principal eigenvalue of a certain positive operator. For the case of compact domain, Chang has built upon a nonlinear version of the Krein–Rutman theorem to give a ‘min–max’ characterization of this eigenvalue which may be viewed as a generalization of the classical Collatz–Wielandt formula for the Perron–Frobenius eigenvalue of a nonnegative irreducible matrix. We apply this formula to the Nisio semigroup associated with risk-sensitive control and derive a variational characterization of the optimal risk-sensitive cost. For the linear, i.e., uncontrolled case, this is seen to reduce to the celebrated Donsker–Varadhan formula for principal eigenvalue of a second-order elliptic operator.     

Sc(OTf)3-catalyzed intramolecular aza-Prins cyclization for the synthesis of heterobicycles

The coupling of (E)- and (Z)-hex-3-ene-1,6-ditosylamide with various aldehydes in the presence of 10 mol % Sc(OTf)₃ gave the corresponding trans- and cis-fused 1,5-ditosyl-octahydro-1H-pyrrolo[3,2-c]pyridines, respectively, in good yields via intramolecular aza-Prins cyclization, whereas the coupling of (E)- and (Z)-N-(6-hydroxyhex-3-enyl)-4-methylbenzenesulfonamide afforded the corresponding trans- and cis-fused octahydro-1-tosylpyrano[4,3-b]pyrroles derivatives, respectively, via intramolecular Prins-cyclization.     

Siderophore-Mediated Iron Uptake Promotes Yeast–Bacterial Symbiosis

In the present study, siderophore produced by the marine yeast Aureobasidium pullulans was characterized as hydroxamate by chemical and bioassays. The hydroxamate assignment was supported by the appearance of peaks at 1,647.21?C1,625.99?cm^?1 and at 1,435.04?cm^?1 in the infrared spectrum. The purified siderophore exhibited specific growth-promoting activity under iron-limited conditions for siderophore auxotrophic probiotic bacteria. Cross-utilization of siderophore indicates a symbiotic relationship between the yeast A. pullulans and the selected probiotic bacterial strains. Statistical optimization of medium components for improved siderophore production in A. pullulans was depicted by response surface methodology. The shift in UV?CVis spectroscopy indicates the photoreactive property and subsequent oxidative cleavage of purified siderophore on exposure to sunlight.     

The effect of titanium dioxide exposure on the thermal properties of Zebrafish (<Emphasis Type="Italic">Danio rerio</Emphasis>) bones

This article presents the changes in the thermal properties of the control and titanium dioxide (TiO₂), both nano and bulk exposed Zebrafish bones by using thermo analytical techniques. The result shows that the mass loss due to the thermal decomposition occurs in three distinct steps due to loss of water, organic and inorganic materials. The titanium dioxide exposed bones present a different thermal behaviour compared to the control bones. The residue masses are found to be increased due to titanium dioxide exposure. In particular, nano titanium dioxide exposure increases the residue mass level significantly (three fold) when compared to titanium dioxide bulk exposure. These thermal characteristics can be used as a qualitative method to check the metal oxide intoxication in biological samples.     

Markov chains, Hamiltonian cycles and volumes of convex bodies

In this note the Hamiltonian cycle problem is mapped into an infinite horizon discounted cost constrained Markov decision problem. The occupation measure based linear polytope associated with this control problem defines a convex set which either strictly contains or is equal to another convex set, depending on whether the underlying graph has a Hamiltonian cycle or not. This allows us to distinguish Hamiltonian graphs from non-Hamiltonian graphs by comparing volumes of two convex sets.     

Dissociative photoionization mechanism of methanol isotopologues (CH3OH, CD3OH, CH3OD and CD3OD) by iPEPICO: energetics, statistical and non-statistical kinetics and isotope effects

The dissociative photoionization of energy selected methanol isotopologue (CH(3)OH, CD(3)OH, CH(3)OD and CD(3)OD) cations was investigated using imaging Photoelectron Photoion Coincidence (iPEPICO) spectroscopy. The first dissociation is an H/D-atom loss from the carbon, also confirmed by partial deuteration. Somewhat above 12 eV, a parallel H(2)-loss channel weakly asserts itself. At photon energies above 15 eV, in a consecutive hydrogen molecule loss to the first H-atom loss, the formation of CHO(+)/CDO(+) dominates as opposed to COH(+)/COD(+) formation. We see little evidence for H-atom scrambling in these processes. In the photon energy range corresponding to the B[combining tilde] and C[combining tilde] ion states, a hydroxyl radical loss appears yielding CH(3)(+)/CD(3)(+). Based on the branching ratios, statistical considerations and ab initio calculations, this process is confirmed to take place on the first electronically excited ?(2)A' ion state. Uncharacteristically, internal conversion is outcompeted by unimolecular dissociation due to the apparently weak Renner-Teller-like coupling between the X[combining tilde] and the ? ion states. The experimental 0 K appearance energies of the ions CH(2)OH(+), CD(2)OH(+), CH(2)OD(+) and CD(2)OD(+) are measured to be 11.646 ± 0.003 eV, 11.739 ± 0.003 eV, 11.642 ± 0.003 eV and 11.737 ± 0.003 eV, respectively. The E(0)(CH(2)OH(+)) = 11.6454 ± 0.0017 eV was obtained based on the independently measured isotopologue results and calculated zero point effects. The 0 K heat of formation of CH(2)OH(+), protonated formaldehyde, was determined to be 717.7 ± 0.7 kJ mol(-1). This yields a 0 K heat of formation of CH(2)OH of -11.1 ± 0.9 kJ mol(-1) and an experimental 298 K proton affinity of formaldehyde of 711.6 ± 0.8 kJ mol(-1). The reverse barrier to homonuclear H(2)-loss from CH(3)OH(+) is determined to be 36 kJ mol(-1), whereas for heteronuclear H(2)-loss from CH(2)OH(+) it is found to be 210 kJ mol(-1).     

Distributed computation of fixed points of ∞-nonexpansive maps

The distributed implementation of an algorithm for computing fixed points of an ∞-nonexpansive map is shown to converge to the set of fixed points under very general conditions.