Continuous time control of Markov processes on an arbitrary state space: Average return criterion

The paper deals with continuous time Markov decision processes on a fairly general state space. The economic criterion is the long-run average return. A set of conditions is shown to be sufficient for a constant g to be optimal average return and a stationary policy π¹ to be optimal. This condition is shown to be satisfied under appropriate assumptions on the optimal discounted return function. A policy improvement algorithm is proposed and its convergence to an optimal policy is proved.     

The derivation of cubic splines with obstacles by methods of optimization and optimal control

Summary Interpolating splines which are restricted in their movement by the presence of obstacles are investigated. For simplicity we mainly treat cubic splines which are required to be non-negative. The extension to splines of higher order and to certain other forms of obstacles is straightforward. Methods of optimization and of optimal control are used to obtain necessary optimality criteria. These criteria are applied to derive an algorithm to compute splines which are restricted to constant lower or upper bounds. There is a numerical example which illustrates the method presented.Dedicated to Günter Meinardus on the occasion of his 60th birthday     

Graphical methods in first and second-order differentiability theory of integral functionals

We discuss several notions of first and second-order differentiability for integral functionals on a Hilbert space.Supported in part by Deutsche Forschungsgemeinschaft (DFG) and Natural Sciences and Engineering Research Council of Canada (NSERC).     

A Linear PDE Approach to the Bellman Equation of Ergodic Control with Periodic Structure

   Abstract. In this paper we give a new proof of the existence result of Bensoussan [1, Theorem II-6.1] for the Bellman equation of ergodic control with periodic structure. This Bellman equation is a nonlinear PDE, and he constructed its solution by using the solution of a nonlinear PDE. On the contrary, our key idea is to solve two linear PDEs. Hence, we propose a linear PDE approach to this Bellman equation.     

Extremal solutions of quasilinear parabolic inclusions with generalized Clarke's gradient

In this paper we consider an initial boundary value problem for a parabolic inclusion whose multivalued nonlinearity is characterized by Clarke's generalized gradient of some locally Lipschitz function, and whose elliptic operator may be a general quasilinear operator of Leray-Lions type. Recently, extremality results have been obtained in case that the governing multivalued term is of special structure such as, multifunctions given by the usual subdifferential of convex functions or subgradients of so-called dc-functions. The main goal of this paper is to prove the existence of extremal solutions within a sector of appropriately defined upper and lower solutions for quasilinear parabolic inclusions with general Clarke's gradient. The main tools used in the proof are abstract results on nonlinear evolution equations, regularization, comparison, truncation, and special test function techniques as well as tools from nonsmooth analysis.     

Formulation and numerical solution of finite-level quantum optimal control problems

Optimal control of finite-level quantum systems is investigated, and iterative solution schemes for the optimization of a control representing laser pulses are developed. The purpose of this external field is to channel the system's wavefunction between given states in its most efficient way. Physically motivated constraints, such as limited laser resources or population suppression of certain states, are accounted for through an appropriately chosen cost functional. First-order necessary optimality conditions and second-order sufficient optimality conditions are investigated. For solving the optimal control problems, a cascadic non-linear conjugate gradient scheme and a monotonic scheme are discussed. Results of numerical experiments with a representative finite-level quantum system demonstrate the effectiveness of the optimal control formulation and efficiency and robustness of the proposed approaches.     

A hybrid iterative scheme for mixed equilibrium problems and fixed point problems

The purpose of this paper is to investigate the problem of finding a common element of the set of solutions of a mixed equilibrium problem (MEP) and the set of common fixed points of finitely many nonexpansive mappings in a real Hilbert space. First, by using the well-known KKM technique we derive the existence and uniqueness of solutions of the auxiliary problems for the MEP. Second, by virtue of this result we introduce a hybrid iterative scheme for finding a common element of the set of solutions of MEP and the set of common fixed points of finitely many nonexpansive mappings. Furthermore, we prove that the sequences generated by the hybrid iterative scheme converge strongly to a common element of the set of solutions of MEP and the set of common fixed points of finitely many nonexpansive mappings.     

A family of projective splitting methods for the sum of two maximal monotone operators

A splitting method for two monotone operators A and B is an algorithm that attempts to converge to a zero of the sum A + B by solving a sequence of subproblems, each of which involves only the operator A, or only the operator B. Prior algorithms of this type can all in essence be categorized into three main classes, the Douglas/Peaceman-Rachford class, the forward-backward class, and the little-used double-backward class. Through a certain “extended” solution set in a product space, we construct a fundamentally new class of splitting methods for pairs of general maximal monotone operators in Hilbert space. Our algorithms are essentially standard projection methods, using splitting decomposition to construct separators. We prove convergence through Fejér monotonicity techniques, but showing Fejér convergence of a different sequence to a different set than in earlier splitting methods. Our projective algorithms converge under more general conditions than prior splitting methods, allowing the proximal parameter to vary from iteration to iteration, and even from operator to operator, while retaining convergence for essentially arbitrary pairs of operators. The new projective splitting class also contains noteworthy preexisting methods either as conventional special cases or excluded boundary cases. Dedicated to Clovis Gonzaga on the occassion of his 60th birthday.     

A Linear PDE Approach to the Bellman Equation of Ergodic Control with Periodic Structure

Abstract. In this paper we give a new proof of the existence result of Bensoussan [1, Theorem II-6.1] for the Bellman equation of ergodic control with periodic structure. This Bellman equation is a nonlinear PDE, and he constructed its solution by using the solution of a nonlinear PDE. On the contrary, our key idea is to solve two linear PDEs. Hence, we propose a linear PDE approach to this Bellman equation.