Jensen's inequality relative to matrix-valued measures

A Bochner-integral formulation of Jensen's inequality is presented for Hermitian matrix-valued functions and measures.     

Sufficiency and Jensen's inequality for conditional expectations

For finite sets of probability measures, sufficiency is characterized by means of certain positively homogeneous convex functions. The essential tool is a discussion of equality in Jensen's inequality for conditional expectations. In particular, it is shown that characterizations of sufficiency by Csiszár's f-divergence (1963, Publ. Math. Inst. Hung. Acad. Sci. Ser. A, 8, 85–107) and by optimal solutions of a Bayesian decision problem used by Morse and Sacksteder (1966, Ann. Math. Statist., 37, 203–214) can be proved by the same method.     

Multifunctions of faces for conditional expectations of selectors and Jensen's inequality

Let (T, , P) be a probability space, a P-complete sub-δ-algebra of and X a Banach space. Let multifunction t → Γ(t), t T, have a (X)-measurable graph and closed convex subsets of X for values. If x(t) ε Γ(t) P-a.e. and y(·) ε E_p x(·), then y(t) ε Γ(t) P-a.e. Conversely, x(t) ε F(Γ(t), y(t)) P-a.e., where F(Γ(t), y(t)) is the face of point y(t) in Γ(t). If X = , then the same holds true if Γ(t) is Borel and convex, only. These results imply, in particular, extensions of Jensen's inequality for conditional expectations of random convex functions and provide a complete characterization of the cases when the equality holds in the extended Jensen inequality.     

Jensen's inequality for the lower semicontinuous quasiconvex envelope and relaxation of multidimensional control problems

Assume that K⊂Rnm is a convex body with o∈int(K) and is a function with f|K∈C⁰(K,R) and f|(Rnm?K)≡+∞. We show that its lower semicontinuous quasiconvex envelope

     

Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

ABSTRACT

The main goal of this paper is to study the infinite-horizon long run average continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. We provide conditions for the existence of a solution to an integro-differential optimality inequality, the so called Hamilton-Jacobi-Bellman (HJB) equation, and for the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called vanishing discount approach, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.     

Adaptive control of Markov processes with incomplete state information and unknown parameters

Recent results for parameter-adaptive Markov decision processes (MDP's) are extended to partially observed MDP's depending on unknown parameters. These results include approximations converging uniformly to the optimal reward function and asymptotically optimal adaptive policies.This research was supported in part by the Consejo del Sistema Nacional de Educación Tecnologica (COSNET) under Grant 178/84, in part by the Air Force Office of Scientific Research under Grant AFOSR-84-0089, in part by the National Science Foundation under Grant ECS-84-12100, and in part by the Joint Services Electronics Program under Contract F49602-82-C-0033.     

Application of stochastic optimal control to the suboptimal design of thrust-mass profiles

Stochastic optimal control techniques are applied to compare the performance of identical medium-range air-to-air missiles which have different thrust-mass profiles. The measure of the performance is the probability of reaching a lock-on-point with a favorable range of guidance and flight parameters, during a fixed time interval [0,t _f ], given that, during the flight, the trajectories of the missile are subjected to a variety of constraints including dynamic pressure constraints.     

Application of stochastic optimal control to the suboptimal design of thrust-mass profiles

A printing error in Ref. 1 is corrected.     

On the optimal control of arrivals to a single queue with arbitrary feedback delay

We consider a problem of admission control to a single queue in discrete time. The controller has access to k step old queue lengths only, where k can be arbitrary. The problem is motivated, in particular, by recent advances in high-speed networking where information delays have become prominent. We formulate the problem in the framework of Completely Observable Controlled Markov Chains, in terms of a multi-dimensional state variable. Exploiting the structure of the problem, we show that under appropriate conditions, the multi-dimensional Dynamic Programming Equation (DPE) can be reduced to a unidimensional one. We then provide simple computable upper and lower bounds to the optimal value function corresponding to the reduced unidimensional DPE. These upper and lower bounds, along with a certain relationship among the parameters of the problem, enable us to deduce partially the structural features of the optimal policy. Our approach enables us to recover simply, in part, the recent results of Altman and Stidham, who have shown that a multiple-threshold-type policy is optimal for this problem. Further, under the same relationship among the parameters of the problem, we provide easily computable upper bounds to the multiple thresholds and show the existence of simple relationships among these upper bounds. These relationships allow us to gain very useful insights into the nature of the optimal policy. In particular, the insights obtained are of great importance for the problem of actually computing an optimal policy because they reduce the search space enormously. This revised version was published online in June 2006 with corrections to the Cover Date.     

Application of an Extremization Method to a Linear Integral of a Statistical Decision Problem

We consider the variational inequality that represents the first-order optimality condition for the class of variational problems with the property that the integrand in the objective functional does not depend on the derivative of the unknown function. This allows the development of an iterative method for solving the statistical decision problem of testing simple hypotheses.     

马尔可夫过程在生产运营项目实施中的应用研究

运用马氏决策规划方法,对企业产品的销售和利润状况进行分析和研究,建立了实施企业生产运营项目的预决策模型,为降低企业项目实施的风险,实现决策的长期效益趋于最优提供了有价值的理论与方法.     

Interpolation inequality of logarithmic type in abstract Besov spaces and an application to semilinear evolution equations

An abstract version of Besov spaces is introduced by using the resolvent of nonnegative operators. Interpolation inequalities with respect to abstract Besov spaces and generalized Lorentz spaces are obtained. These inequalities provide a generalization of Sobolev inequalities of logarithmic type. Uniqueness problems to abstract semilinear evolution equations are also discussed (© 2010 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Optimality of monotonic policies for two-action Markovian decision processes,with applications to control of queues with delayed information

We consider a discrete-time Markov decision process with a partially ordered state space and two feasible control actions in each state. Our goal is to find general conditions, which are satisfied in a broad class of applications to control of queues, under which an optimal control policy is monotonic. An advantage of our approach is that it easily extends to problems with both information and action delays, which are common in applications to high-speed communication networks, among others. The transition probabilities are stochastically monotone and the one-stage reward submodular. We further assume that transitions from different states are coupled, in the sense that the state after a transition is distributed as a deterministic function of the current state and two random variables, one of which is controllable and the other uncontrollable. Finally, we make a monotonicity assumption about the sample-path effect of a pairwise switch of the actions in consecutive stages. Using induction on the horizon length, we demonstrate that optimal policies for the finite- and infinite-horizon discounted problems are monotonic. We apply these results to a single queueing facility with control of arrivals and/or services, under very general conditions. In this case, our results imply that an optimal control policy has threshold form. Finally, we show how monotonicity of an optimal policy extends in a natural way to problems with information and/or action delay, including delays of more than one time unit. Specifically, we show that, if a problem without delay satisfies our sufficient conditions for monotonicity of an optimal policy, then the same problem with information and/or action delay also has monotonic (e.g., threshold) optimal policies.     

On truncations and perturbations of Markov decision problems with an application to queueing network overflow control

Conditions are provided to derive error bounds on the effect of truncations and perturbations in Markov decision problems. Both the average and finite horizon case are studied. As an application, an explicit error bound is obtained for a truncation of a Jacksonian queueing network with overflow control.     

Application of wavelet transforms to the solution of boundary value problems for linear parabolic equations

A method based on wavelet transforms is proposed for finding weak solutions to initial-boundary value problems for linear parabolic equations with discontinuous coefficients and inexact data. In the framework of multiresolution analysis, the general scheme for finite-dimensional approximation in the regularization method is combined with the discrepancy principle. An error estimate is obtained for the stable approximate solution obtained by solving a set of linear algebraic equations for the wavelet coefficients of the desired solution.