Risk sensitive control of pure jump processes on a general state space

We study stochastic control problem for pure jump processes on a general state space with risk sensitive discounted and ergodic cost criteria. For the discounted cost criterion we prove the existence and Hamilton–Jacobi–Bellman characterization of optimal α-discounted control for bounded cost function. For the ergodic cost criterion we assume a Lyapunov type stability assumption and a small cost condition. Under these assumptions we show the existence of the optimal risk-sensitive ergodic control.     

Optimal cash management using impulse control

We consider the impulse control of Lévy processes under the infinite horizon, discounted cost criterion. Our motivating example is the cash management problem in which a controller is charged a fixed plus proportional cost for adding to or withdrawing from his/her reserve, plus an opportunity cost for keeping any cash on hand. Our main result is to provide a verification theorem for the optimality of control band policies in this scenario. We also analyze the transient and steady-state behavior of the controlled process under control band policies and explicitly solve for an optimal policy in the case in which the Lévy process to be controlled is the sum of a Brownian motion with drift and a compound Poisson process with exponentially distributed jump sizes.     

Value iteration in average cost Markov control processes on Borel spaces

This paper deals with discrete-time Markov control processes withBorel state and control spaces, with possiblyunbounded costs andnoncompact control constraint sets, and the average cost criterion. Conditions are given for the convergence of the value iteration algorithm to the optimal average cost, and for a sequence of finite-horizon optimal policies to have an accumulation point which is average cost optimal.This research was partially supported by the Consejo Nacional de Ciencia y Tecnología (CONACyT) under grant 1332-E9206.     

Discrete and sampled-data stochastic control problems with complete and incomplete state information

An unconstrained stochastic optimization problem involving a discrete-time linear process with a normally distributed initial condition and subject to additive gaussian state and measurement noise is formulated in terms of a quite general finite horizon, discrete-time quadratic cost criterion and solved when there is either complete or incomplete state information. It is shown that both the stochastic sampled-data optimal tracker and the stochastic sampled-data optimal regulator are special cases of this problem. A breakdown of the minimum cost for both sampled-data controllers is given.     

Stochastic adaptive control for continuous-time linear systems with quadratic cost

An adaptive control problem is formulated and solved for a completely observed, continuous-time, linear stochastic system with an ergodic quadratic cost criterion. The linear transformationsA of the state,B of the control, andC of the noise are assumed to be unknown. Assuming only thatA is stable and that the pair (A, C) is controllable and using a diminishing excitation control that is asymptotically negligible for an ergodic, quadratic cost criterion it is shown that a family of least-squares estimates is strongly consistent. Furthermore, an adaptive control is given using switchings that is self-optimizing for an ergodic, quadratic cost criterion.This research was partially supported b y NSF Grants ECS-9102714, ECS-9113029, and DMS-9305936.     

Intersections of extremals in linear-quadratic control problems

In this paper, we study the structure of intersections of extremals in linear control systems with quadratic cost. The structure of the intersections is determined. In particular, it is shown that, in contrast with the classical calculus of variations, these points are not necessarily isolated. The results obtained extend a known criterion from the calculus of variations involving the Jacobi necessary condition.The author would like to thank the referee, whose valuable remarks much improved the exposition of the results.     

Optimal control in light traffic Markov decision processes

We consider Markov Decision Processes under light traffic conditions. We develop an algorithm to obtain asymptotically optimal policies for both the total discounted and the average cost criterion. This gives a general framework for several light traffic results in the literature. We illustrate the method by deriving the asymptotically optimal control of a simple ATM network.     

Dynamic control of a single-server system with abandonments

In this paper, we discuss the dynamic server control in a two-class service system with abandonments. Two models are considered. In the first case, rewards are received upon service completion, and there are no abandonment costs (other than the lost opportunity to gain rewards). In the second, holding costs per customer per unit time are accrued, and each abandonment involves a fixed cost. Both cases are considered under the discounted or average reward/cost criterion. These are extensions of the classic scheduling question (without abandonments) where it is well known that simple priority rules hold.     

Homogenization of low-cost control problems on perforated domains

The aim of this paper is to study the asymptotic behaviour of some low-cost control problems in periodically perforated domains with Neumann condition on the boundary of the holes. The optimal control problems considered here are governed by a second order elliptic boundary value problem with oscillating coefficients. It is assumed that the cost of the control is of the same order as that describing the oscillations of the coefficients. The asymptotic analysis of small cost problem is more delicate and need the H-convergence result for weak data. In this connection, an H-convergence result for weak data under some hypotheses is also proved.     

Continuous time control of Markov processes on an arbitrary state space: Average return criterion

The paper deals with continuous time Markov decision processes on a fairly general state space. The economic criterion is the long-run average return. A set of conditions is shown to be sufficient for a constant g to be optimal average return and a stationary policy π¹ to be optimal. This condition is shown to be satisfied under appropriate assumptions on the optimal discounted return function. A policy improvement algorithm is proposed and its convergence to an optimal policy is proved.     

Optimal multi-threshold control by the BMAP/SM/1 retrial system

A single server retrial system having several operation modes is considered. The modes are distinguished by the transition rate of the batch Markovian arrival process (BMAP), kernel of the semi-Markovian (SM) service process and the intensity of retrials. Stationary state distribution is calculated under the fixed value of the multi-threshold control strategy. Dependence of the cost criterion, which includes holding and operation cost, on the thresholds is derived. Numerical results illustrating the work of the computer procedure for calculation of the optimal values of thresholds are presented.     

On semirings satisfying the Baer criterion

It is well-known that for modules over rings the Baer injectivity criterion takes place. In this paper we prove that under one additional condition this criterion is also valid for modules over semirings. We prove that a semiring S satisfies the Baer criterion if and only if all injective (with respect to one-sided ideals of S) semimodules satisfy the above condition. We propose a newmethod for constructing semirings satisfying the Baer criterion.     

Optimal control problems for a class of nonlinear evolution equations

In this paper we study optimal control problems for systems governed by nonlinear evolution equations. First we develop an existence theory for systems with a priori feedback using the reduction technique and a convexity-type hypothesis involving property Q. In doing this we also establish the nonemptiness of the set of admissible state-control pairs, by solving a nonlinear evolution inclusion. Then we obtain necessary conditions for optimality for a class of problems with terminal cost criterion and initial condition which is not a priori given but is only required to belong to a given set (systems with insufficient data in the terminology of Lions). This revised version was published online in June 2006 with corrections to the Cover Date.     

Min-max optimal control of systems approximated by finite-dimensional models: Nonquadratic cost functional

An open-loop control problem with nonquadratic performance criterion for a dynamical system, described by an abstract linear equation of evolution and approximated by a finite-dimensional model, is solved. A min-max approach is taken: the value of the cost functional in theworst-case output error between the system and the model, under the assumption that the norm of this output error is estimated by the norm of the input, is minimized.The form of the cost functional reproduces itself under maximization, so that the min-max control problem, when only thedistance between the model and the system is given, has the same features and proprieties of the control problem when the system is thorougly known.Existence and uniqueness theorems for the optimal control are proven, using the spectral proprieties of the model transfer function; and, in the case of a time-invariant model, the min-max control computation is reduced to the solution of a constant-coefficients Sturm-Liouville problem followed by the search for the zeros of a very simple numerical function.This work was made within the Gruppo Nazionale per l'Analisi Funzionale e le sue Applicazioni, Consiglio Nazionale delle Ricerche.     

Practical criteria for positive-definite matrix,M-matrix and Hurwitz matrix

All the conventional criterions to verify if a matrix is positive definite, Hurwitz or an M-matrix, such as Sylvester condition and Hurwitz theorem, require us to compute n determinants. This paper develops a new and simple criterion, based on a new type of Gaussian elimination process. In contrast to the traditional criterions, the new criterion only computes one determinate. Thus, the computing cost is greatly reduced. An illustrative example is given to show the application of the new criterion in the stability analysis of Hopfied-type neural networks with time delays.     

Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chains

This note concerns controlled Markov chains on a denumerable sate space. The performance of a control policy is measured by the risk-sensitive average criterion, and it is assumed that (a) the simultaneous Doeblin condition holds, and (b) the system is communicating under the action of each stationary policy. If the cost function is bounded below, it is established that the optimal average cost is characterized by an optimality inequality, and it is to shown that, even for bounded costs, such an inequality may be strict at every state. Also, for a nonnegative cost function with compact support, the existence an uniqueness of bounded solutions of the optimality equation is proved, and an example is provided to show that such a conclusion generally fails when the cost is negative at some state.     

道德风险影响下的保险最优投资研究

在考虑道德风险的情况下,以均值方差准则为目标研究保险人最优投资问题.假设保险盈余过程服从C-L模型,金融市场上存在一种无风险资产和一种风险资产可供投资,其中风险资产的价格过程服从几何布朗运动.在纯道德风险保险契约设计中,借鉴相关研究对努力水平和效用化努力成本的假设,量化道德风险对盈余过程的影响.在均值方差目标下,建立保险人最优投资问题的广义Hamilton-Jacobi-Bellman(HJB)方程,给出保险人时间一致的均衡投资策略和价值函数.结果显示累计索赔比例参数越大,公司对最优努力水平越敏感,采取措施降低道德风险有利于公司收益提升;努力成本参数越大,公司会降低努力水平减少支出,避免损失.     

Euler discretization for a class of nonlinear optimal control problems with control appearing linearly

We investigate Euler discretization for a class of optimal control problems with a nonlinear cost functional of Mayer type, a nonlinear system equation with control appearing linearly and constraints defined by lower and upper bounds for the controls. Under the assumption that the cost functional satisfies a growth condition we prove for the discrete solutions Hölder type error estimates w.r.t. the mesh size of the discretization. If a stronger second-order optimality condition is satisfied the order of convergence can be improved. Numerical experiments confirm the theoretical findings.     

Controlled Markov processes on the infinite planning horizon: Weighted and overtaking cost criteria

Stochastic control problems for controlled Markov processes models with an infinite planning horizon are considered, under some non-standard cost criteria. The classical discounted and average cost criteria can be viewed as complementary, in the sense that the former captures the short-time and the latter the long-time performance of the system. Thus, we study a cost criterion obtained as weighted combinations of these criteria, extending to a general state and control space framework several recent results by Feinberg and Shwartz, and by Krass et al. In addition, a functional characterization is given for overtaking optimal policies, for problems with countable state spaces and compact control spaces; our approach is based on qualitative properties of the optimality equation for problems with an average cost criterion.Research partially supported by the Engineering Foundation under grant RI-A-93-10, in part by the National Science Foundation under grant NSF-INT 9201430, and in part by a grant from the AT&T Foundation.Research partially supported by the Air Force Office of Scientific Research under Grant F49620-92-J-0045, and in part by the National Science Foundation under Grant CDR-8803012.     

Production capacity planning and control in multi-stage manufacturing

This paper develops a mathematical model for determining the optimum lot-sizes for a set of products and the capacity required to produce them in a multi-stage production system. The purpose of the modelling is to support capacity planning at the production function level and the basic criterion considered for the optimisation is the minimisation of the total system cost (TSC) per unit time. The TSC consists of (i) set-up cost, (ii) cost due to the quenching of batches, and (iii) hiring cost of the machines. An example is presented to explain the model.