Average sample-path optimality for continuous-time Markov decision processes in Polish spaces

In this paper we study the average sample-path cost(ASPC) problem for continuous-time Markov decision processes in Polish spaces.To the best of our knowledge,this paper is a first attempt to study the ASPC criterion on continuous-time MDPs with Polish state and action spaces.The corresponding transition rates are allowed to be unbounded,and the cost rates may have neither upper nor lower bounds.Under some mild hypotheses,we prove the existence of ε(ε≥ 0)-ASPC optimal stationary policies based on two differe...     

Criteria of optimality in the infinite-time optimal control problem

The present paper gives a systematic presentation of different definitions of optimality in the infinite-time optimal control problem. Some of these definitions are new, while others have been used throughout the literature, sometimes with different names. The logical implications between them are clearly stated, corresponding comparison criteria for solutions are defined, and other relations as well as two types of equivalence relations are established.This work was supported in part by Simmons College Fund for Research, Grant No. 201.     

Sufficient optimality conditions for multidimensional optimal control problems with mixed constraints governed by an elliptic PDE

In this article, the idea of a dual dynamic programming is applied to the optimal control problems with multiple integrals governed by a semi-linear elliptic PDE and mixed state-control constraints. The main result called a verification theorem provides the new sufficient conditions for optimality in terms of a solution to the dual equation of a multidimensional dynamic programming. The optimality conditions are also obtained by using the concept of an optimal dual feedback control. Besides seeking the exact minimizers of problems considered some kind of an approximation is given and the sufficient conditions for an approximated optimal pair are derived.     

On necessary optimality conditions in vector optimization problems

Necessary conditions of the multiplier rule type for vector optimization problems in Banach spaces are proved by using separation theorems and Ljusternik's theorem. The Pontryagin maximum principle for multiobjective control problems with state constraints is derived from these general conditions. The paper extends to vector optimization results established in the scalar case by Ioffe and Tihomirov.     

Necessary optimality conditions for minimax programming problems with mathematical constraints

In this paper, necessary optimality conditions in terms of upper and/or lower subdifferentials of both cost and constraint functions are derived for minimax optimization problems with inequality, equality and geometric constraints in the setting of non-differentiatiable and non-Lipschitz functions in Asplund spaces. Necessary optimality conditions in the fuzzy form are also presented. An application of the fuzzy necessary optimality condition is shown by considering minimax fractional programming problem.     

Stability and accuracy functions for a multicriteria Boolean linear programming problem with parameterized principle of optimality “from Condorcet to Pareto”

A multicriteria Boolean programming problem with linear cost functions in which initial coefficients of the cost functions are subject to perturbations is considered. For any optimal alternative, with respect to parameterized principle of optimality “from Condorcet to Pareto”, appropriate measures of the quality are introduced. These measures correspond to the so-called stability and accuracy functions defined earlier for optimal solutions of a generic multicriteria combinatorial optimization problem with Pareto and lexicographic optimality principles. Various properties of such functions are studied and maximum norms of perturbations for which an optimal alternative preserves its optimality are calculated. To illustrate the way how the stability and accuracy functions can be used as efficient tools for post-optimal analysis, an application from the voting theory is considered.     

Overtaking optimality for controlled Markov-modulated diffusions

This article concerns controlled Markov-modulated diffusions (also known as piecewise diffusions or switching diffusions or diffusions with Markovian switchings). Our main objective is to give conditions for the existence and characterization of overtaking optimal policies. To this end, first, we use fact that the average reward Hamilton–Jacobi–Bellman equation gives that the family of the so-called canonical control policies is nonempty. Then, within this family, we search policies with some special feature, for instance, canonical policies that in addition maximize the bias, which turn out to be overtaking optimal.     

Infinite horizon asymptotic average optimality for large-scale parallel server networks

We study infinite-horizon asymptotic average optimality for parallel server networks with multiple classes of jobs and multiple server pools in the Halfin–Whitt regime. Three control formulations are considered: (1) minimizing the queueing and idleness cost, (2) minimizing the queueing cost under constraints on idleness at each server pool, and (3) fairly allocating the idle servers among different server pools. For the third problem, we consider a class of bounded-queue, bounded-state (BQBS) stable networks, in which any moment of the state is bounded by that of the queue only (for both the limiting diffusion and diffusion-scaled state processes). We show that the optimal values for the diffusion-scaled state processes converge to the corresponding values of the ergodic control problems for the limiting diffusion. We present a family of state-dependent Markov balanced saturation policies (BSPs) that stabilize the controlled diffusion-scaled state processes. It is shown that under these policies, the diffusion-scaled state process is exponentially ergodic, provided that at least one class of jobs has a positive abandonment rate. We also establish useful moment bounds, and study the ergodic properties of the diffusion-scaled state processes, which play a crucial role in proving the asymptotic optimality.     

Near optimality of stochastic control in systems with unknown parameter processes

Stochastic control for systems with an unknown parameter is considered in this paper. The underlying problem is to minimize a functional subject to a system described by a singularly perturbed differential equation with an unknown parameter process driven by fast fluctuating random disturbances. This problem arises in the context of stochastic adaptive control, adaptive signal processing, and failure-prone manufacturing systems. Due to the nature of the wide-bandwidth noise processes, identifying the parameter process for eacht is very hard since the driving noise changes very rapidly. An alternative approach is used, and an auxiliary control problem is introduced to overcome the difficulties. By means of weak convergence methods and comparison control techniques, nearly optimal controls are obtained.This research was supported in part by the National Science Foundation under Grant DMS-9022139 and DMS-9224372.     

Average optimality inequality for continuous-time Markov decision processes in Polish spaces

In this paper, we study the average optimality for continuous-time controlled jump Markov processes in general state and action spaces. The criterion to be minimized is the average expected costs. Both the transition rates and the cost rates are allowed to be unbounded. We propose another set of conditions under which we first establish one average optimality inequality by using the well-known “vanishing discounting factor approach”. Then, when the cost (or reward) rates are nonnegative (or nonpositive), from the average optimality inequality we prove the existence of an average optimal stationary policy in all randomized history dependent policies by using the Dynkin formula and the Tauberian theorem. Finally, when the cost (or reward) rates have neither upper nor lower bounds, we also prove the existence of an average optimal policy in all (deterministic) stationary policies by constructing a “new” cost (or reward) rate. Research partially supported by the Natural Science Foundation of China (Grant No: 10626021) and the Natural Science Foundation of Guangdong Province (Grant No: 06300957).     

Sufficient conditions for preference optimality

Preference optimality is an optimality concept in multicriteria problems, that is, in problems where several criteria are to beoptimized simultaneously. Formally, one wishes to optimizeN criteriag _i(·) or, equivalently, a criterion vectorg(·) ^N , subject to either functional constraints in programming or to side conditions which are differential equations in optimal control. Subject to these constraints, one obtains forg(·) a set of attainable values in ^N . This set is preordered by the introduction of a complete preordering ; a controlu*(·) or a decisionx*, then, is preference-optimal if it results ing(u*(·))g(u(·)) for all admissible controlsu(·) or ifg(x*)g(x) for all feasible decisionsx. The present paper concerns sufficient conditions for preference-optimal control and for preference-optimal decisions.     

一类变结构动态系统的非光滑最优性条件

研究了一类事件驱动的变结构动态系统的非光滑最优性条件. 通过引入一个新的时间变量, 将变结构动态系统的最优性问题转化为古典动态系统的最优性问题. 基于广义微分和古典动态系统的最优性理论, 得到了该系统的Frechet上微分形式的必要性条件, 推广了已有文献的相关结论. 结果表明, 在系统的连续运行过程中, 控制变量、协态变量和状态变量满足最小值原理和协态方程. 在系统的运行模型发生改变时, 协态变量产生一定的跳跃, 哈密尔顿函数连续. 最后通过一个算例说明了该结论的有效性.     

On sufficient optimality conditions for mathematical programs with equaiity constraints

A number of sufficiency theorems in the mathematical programming literature, concerning problems with equality constraints, are shown to be trivial consequences of the corresponding results for inequality constraints.This work was supported by NSF Grant No. ECS-8214081. Research by the first author was done while a visitor at La Trobe University.     

Positional strengthenings of the maximum principle and sufficient optimality conditions

We derive nonlocal necessary optimality conditions, which efficiently strengthen the classical Pontryagin maximum principle and its modification obtained by B. Ka?kosz and S. ?ojasiewicz as well as our previous result of a similar kind named the “feedback minimum principle.” The strengthening of the feedback minimum principle (and, hence, of the Pontryagin principle) is owing to the employment of two types of feedback controls “compatible” with a reference trajectory (i.e., producing this trajectory as a Carath´eodory solution). In each of the versions, the strengthened feedback minimum principle states that the optimality of a reference process implies the optimality of its trajectory in a certain family of variational problems generated by cotrajectories of the original and compatible controls. The basic construction of the feedback minimum principle—a perturbation of a solution to the adjoint system—is employed to prove an exact formula for the increment of the cost functional. We use this formula to obtain sufficient conditions for the strong and global minimum of Pontryagin’s extremals. These conditions are much milder than their known analogs, which require the convexity in the state variable of the functional and of the lower Hamiltonian. Our study is focused on a nonlinear smooth Mayer problem with free terminal states. All assertions are illustrated by examples.     

Solutions of the average cost optimality equation for finite Markov decision chains: risk-sensitive and risk-neutral criteria

This work is concerned with controlled Markov chains with finite state and action spaces. It is assumed that the decision maker has an arbitrary but constant risk sensitivity coefficient, and that the performance of a control policy is measured by the long-run average cost criterion. Within this framework, the existence of solutions of the corresponding risk-sensitive optimality equation for arbitrary cost function is characterized in terms of communication properties of the transition law.     

Global optimality of extremals: An example

The question of the existence and the location of Darboux points (beyond which global optimality is lost) is crucial for minimal sufficient conditions for global optimality and for computation of optimal trajectories. Here, we investigate numerically the Darboux points and their relationship with conjugate points for a problem of minimum fuel, constant velocity, horizontal aircraft turns to capture a line. This simple second-order optimal control problem shows that ignoring the possible existence of Darboux points may play havoc with the computation of optimal trajectories.The authors are indebted to G. Moyer for his constructive comments. This research was supported, for the first author, by a National Research Council Associateship at NASA Ames Research Center.on leave from the Technion, Israel Institute of Technology, Haifa, Israel.     

Second-order optimality principle for singular optimal control problems

A new necessary condition for singular optimal control problems is presented in this paper. The condition is simpler to apply than existing conditions and is easily derived from a Taylor series expansion of the performance index.     

双值约束非凸三次规化问题的全局最优性条件

考虑一类带有双值约束的非凸三次优化问题, 给出了该问题的一个全局最优充分必要条件. 结果改进并推广了一些文献中所给出的全局最优性条件, 同时还通过数值例子来说明所给出的全局最优充要条件是易验证的.