Optimal inspection and replacement policies for multi-state deteriorating systems

In this paper, a phase-type approach is proposed to derive optimal inspection and replacement policies for semi-Markovian deteriorating systems. In this approach, the general sojourn time distributions of a semi-Markovian maintenance model are approximated by acyclic phase-type distributions. Using the approximation, a semi-Markovian maintenance model can be transformed into a Markovian maintenance model such that the analytical tractability of Markov processes can be preserved. Based on the Markovian model, algorithms are provided to derive the optimal state-dependent and state-age-dependent inspection and replacement policies such that the expected long-run cost rate is minimized. Furthermore, procedures are developed to implement the optimal policies on semi-Markovian deteriorating systems. The implementation of the optimal policies are illustrated by numerical examples.     

Monotone Control of Queueing Systems with Heterogeneous Servers

A multi-server controllable queueing system with heterogeneous servers is considered. Several monotonicity properties of optimal policies for such a system are proved.     

Optimally designed renewal reward processes: General framework and applications

For physical and organizational systems which are regenerative in nature, certain mechanisms may be utilized to effect the time between regenerations. Those scenarios give rise to renewal processes with renewal distributions that are controllable at cost. We provide a unifying framework for optimal design of such processes in situations where certain parameter(s) of the renewal distributions can be manipulated at cost. Following the formulation of the general problem and discussion of some of its properties, we discuss three application areas where such situations arise. Those involve stochastic dispatching, labour turnover control, and design of repairable systems.     

OPTIMAL CONTROL OF MARKOVIAN SWITCHING SYSTEMS WITH APPLICATIONS TO PORTFOLIO DECISIONS UNDER INFLATION

This article is concerned with a class of control systems with Markovian switching,in which an ltd formula for Markov-modulated processes is derived.Moreover,an optimal control law satisfying the generalized Hamilton-Jacobi-Bellman(HJB) equation with Markovian switching is characterized.Then,through the generalized HJB equation,we study an optimal consumption and portfolio problem with the financial markets of Markovian switching and inflation.Thus,we deduce the optimal policies and show that a modified Mutual Fund Theorem consisting of three funds holds.Finally,for the CRRA utility function,we explicitly give the optimal consumption and portfolio policies.Numerical examples are included to illustrate the obtained results.     

Two-person game with nash equilibrium in optimal control problems

A two-person game with a Nash equilibrium is formulated for optimal control problems with a free right end and a linear differential system. The game is reduced to the calculation of a fixed point of an extremal mapping, which in turn is reduced to a variational inequality with linear constraints generated by systems of linear differential controllable processes. An extra-gradient iterative method is proposed for calculating the Nash equilibrium of the dynamic game. The convergence of the method is proved.     

带有相同输入函数双系统的同时近似可控性

研究了一类具有相同输入函数抽象双线性系统的同时近似可控性.在此,我们考虑双系统都是无穷维的而且其中一个为Riesz-Spectral系统,证明了如果两个系统在$T_0$时刻都是精确可控的而且系统的生成元没有共同特征值,那么对于任何时刻$T>T_0$这两个系统是同时近似可控的.此外,对于特殊的控制算子,如果系统$(A_1,~B_1)$在时刻$T_0$是近似可控的而系统$(A_2,B_2)$在时刻$T_0$是精确可控的,并且生成元算子$A_1$和$A_2$的谱集满足$\sigma(A_2)\subset \rho_{\infty}(A_1)$,那么一定存在某个时刻$T>0$,使得双系统在时刻$T$是同时近似可控的.最后,给出定理的一些应用以及说明同时近似可控时刻$T$是最优的.     

Optimal time-consistent investment and reinsurance policies for mean-variance insurers

This paper investigates the optimal time-consistent policies of an investment-reinsurance problem and an investment-only problem under the mean-variance criterion for an insurer whose surplus process is approximated by a Brownian motion with drift. The financial market considered by the insurer consists of one risk-free asset and multiple risky assets whose price processes follow geometric Brownian motions. A general verification theorem is developed, and explicit closed-form expressions of the optimal polices and the optimal value functions are derived for the two problems. Economic implications and numerical sensitivity analysis are presented for our results. Our main findings are: (i) the optimal time-consistent policies of both problems are independent of their corresponding wealth processes; (ii) the two problems have the same optimal investment policies; (iii) the parameters of the risky assets (the insurance market) have no impact on the optimal reinsurance (investment) policy; (iv) the premium return rate of the insurer does not affect the optimal policies but affects the optimal value functions; (v) reinsurance can increase the mean-variance utility.     

Estimate of Periodic Suboptimal Controls

We consider the infinite-time optimal control problem of minimizing an average functional for nonlinear control systems. For controllable systems, we give an explicit estimate for the required period T of -suboptimal T-periodic orbits. Moreover, we show that controllable systems are characterized by the existence of periodic suboptimal trajectories for any average functional.     

Modelling replenishment and transshipment decisions in periodic review multilocation inventory systems

Effective models of key operational decisions in multilocation inventory systems are important for a successful retail sector. This paper argues that much of the existing research in this area is not applicable to a highly competitive retail environment, particularly if periodic review replenishment policies are used. The paper develops a model of a periodic review multilocation inventory system that is suitable for this environment and investigates the characteristics of optimal replenishment and transshipment decisions. This motivates the development of three simple heuristic transshipment policies that are practical for systems with many locations. The results of a numerical study involving systems with five locations suggest that the performance of these heuristic policies is often close to optimal and can be considerably better than the performance of commonly used policies.     

Output controllability and optimal output control of positive fractional order linear discrete system with multiple delays in state, input and output

The article concerns output controllability and optimal output control of positive fractional order discrete linear systems with multiple delays in state, input and output. Necessary and sufficient conditions for output reachability (output controllability from zero initial conditions) and null output controllability (output controllability to zero final output) are given and proven. We also prove that the positive system is output controllable if it is output reachable and null output controllable with the output reachability index is equal or less than the null output controllability index. Sufficient conditions for the solvability of the optimal output control problem are given. Numerical examples are presented to illustrate the theoretical results.     

Optimal ergodic control of Markov diffusion processes with minimum variance

In this paper, we study the optimal ergodic control problem with minimum variance for a general class of controlled Markov diffusion processes. To this end, we follow a lexicographical approach. Namely, we first identify the class of average optimal control policies, and then within this class, we search policies that minimize the limiting average variance. To do this, a key intermediate step is to show that the limiting average variance is a constant independent of the initial state. Our proof of this latter fact gives a result stronger than the central limit theorem for diffusions. An application to manufacturing systems illustrates our results.     

Finding the K best policies in a finite-horizon Markov decision process

Directed hypergraphs represent a general modelling and algorithmic tool, which have been successfully used in many different research areas such as artificial intelligence, database systems, fuzzy systems, propositional logic and transportation networks. However, modelling Markov decision processes using directed hypergraphs has not yet been considered.In this paper we consider finite-horizon Markov decision processes (MDPs) with finite state and action space and present an algorithm for finding the K best deterministic Markov policies. That is, we are interested in ranking the first K deterministic Markov policies in non-decreasing order using an additive criterion of optimality. The algorithm uses a directed hypergraph to model the finite-horizon MDP. It is shown that the problem of finding the optimal policy can be formulated as a minimum weight hyperpath problem and be solved in linear time, with respect to the input data representing the MDP, using different additive optimality criteria.     

Recursive adaptive control of Markov decision processes with the average reward criterion

We are concerned with Markov decision processes with Borel state and action spaces; the transition law and the reward function depend on anunknown parameter. In this framework, we study therecursive adaptive nonstationary value iteration policy, which is proved to be optimal under thesame conditions usually imposed to obtain the optimality of other well-knownnonrecursive adaptive policies. The results are illustrated by showing the existence of optimal adaptive policies for a class of additive-noise systems with unknown noise distribution.This research was supported in part by the Consejo Nacional de Ciencia y Tecnología under Grants PCEXCNA-050156 and A128CCOEO550, and in part by the Third World Academy of Sciences under Grant TWAS RG MP 898-152.     

A new strong optimality criterion for nonstationary Markov decision processes

This paper deals with a new optimality criterion consisting of the usual three average criteria and the canonical triplet (totally so-called strong average-canonical optimality criterion) and introduces the concept of a strong average-canonical policy for nonstationary Markov decision processes, which is an extension of the canonical policies of Herna′ndez-Lerma and Lasserre [16] (pages: 77) for the stationary Markov controlled processes. For the case of possibly non-uniformly bounded rewards and denumerable state space, we first construct, under some conditions, a solution to the optimality equations (OEs), and then prove that the Markov policies obtained from the OEs are not only optimal for the three average criteria but also optimal for all finite horizon criteria with a sequence of additional functions as their terminal rewards (i.e. strong average-canonical optimal). Also, some properties of optimal policies and optimal average value convergence are discussed. Moreover, the error bound in average reward between a rolling horizon policy and a strong average-canonical optimal policy is provided, and then a rolling horizon algorithm for computing strong average ε(>0)-optimal Markov policies is given.     

Optimal risk probability for first passage models in semi-Markov decision processes

This paper studies the risk minimization problem in semi-Markov decision processes with denumerable states. The criterion to be optimized is the risk probability (or risk function) that a first passage time to some target set doesn't exceed a threshold value. We first characterize such risk functions and the corresponding optimal value function, and prove that the optimal value function satisfies the optimality equation by using a successive approximation technique. Then, we present some properties of optimal policies, and further give conditions for the existence of optimal policies. In addition, a value iteration algorithm and a policy improvement method for obtaining respectively the optimal value function and optimal policies are developed. Finally, two examples are given to illustrate the value iteration procedure and essential characterization of the risk function.     

Optimal consumption and portfolio under inflation and Markovian switching

In this paper, a financial market with Markovian switching and inflation is described, and the problem of maximizing expected utility of consumption discounted by inflation is given. Then, by a generalized Itô formula for Markov-modulated processes, the verification theorems of optimal policies are derived. Finally, the optimal consumption and portfolio policies for the constant relative risk aversion utility are given explicitly, and an economic analysis is made by numerical examples.     

Innovation diffusion uncertainty,advertising and pricing policies

We develop and analyze a normative and structurally stochastic model of innovation diffusion by depicting the market at an aggregate level. Model dynamics are defined through the flow pattern of individuals that move from the innovation unaware stage, to the innovation aware, and ultimately to the adopter stages. The stochastic evolution of this stage-wise transition unfolds according to tractable stochastic processes and is influenced by such factors as price, word of mouth, and advertisement efforts. In this environment, techniques of contingent claims analysis and stochastic control theory are employed to obtain optimal pricing or advertising policies that maximize the value of the innovation. To account for their optimal adjustment over time, these policies are modeled as positive real-valued adapted processes. Given this setting, policy adjustments over time (i.e. advertising or pricing) are viewed as a value additive sequence of nested real options. We present closed-form analytic results regarding the optimal policies. Simulations provide a numeric insight to the models' behavior.     

A-调和逼近方法和具有可控增长条件的非线性椭圆方程组最优内部部分正则性

考虑具有可控增长条件的非线性椭圆方程组弱解的部分正则性．利用Duzaar和Grotowski引进的弱解部分正则性证明的新方法，该方法是建立在调和逼近技巧一般形式的基础上的，我们把前人的结果由自然增长条件推广到了可控增长条件，并且所得到的弱解导数的Hoelder指标是最优的．