Infinitesimal methods in control theory: Deterministic and stochastic

In Part I, methods of nonstandard analysis are applied to deterministic control theory, extending earlier work of the author. Results established include compactness of relaxed controls, continuity of solution and cost as functions of the controls, and existence of optimal controls. In Part II, the methods are extended to obtain similar results for partially observed stochastic control. Systems considered take the form:where the feedback control u depends on information from a digital read-out of the observation process y. The noise in the state equation is controlled along with the drift. Similar methods are applied to a Markov system in the final section.     

Stochastic optimal control and algorithm of the trajectory of horizontal wells

This paper presents a nonlinear, multi-phase and stochastic dynamical system according to engineering background. We show that the stochastic dynamical system exists a unique solution for every initial state. A stochastic optimal control model is constructed and the sufficient and necessary conditions for optimality are proved via dynamic programming principle. This model can be converted into a parametric nonlinear stochastic programming by integrating the state equation. It is discussed here that the local optimal solution depends in a continuous way on the parameters. A revised Hooke–Jeeves algorithm based on this property has been developed. Computer simulation is used for this paper, and the numerical results illustrate the validity and efficiency of the algorithm.     

Second order Hamilton–Jacobi–Bellman equations with an unbounded operator

This work is devoted to the study of a class of Hamilton–Jacobi–Bellman equations associated to an optimal control problem where the state equation is a stochastic differential inclusion with a maximal monotone operator. We show that the value function minimizing a Bolza-type cost functional is a viscosity solution of the HJB equation. The proof is based on the perturbation of the initial problem by approximating the unbounded operator. Finally, by providing a comparison principle we are able to show that the solution of the equation is unique.     

Random relaxed controls and partially observed stochastic systems

We introduce a class of generalized controls called random relaxed controls, and show that under quite general conditions, a partially observed, controlled diffusion will have an optimal random relaxed control whose cost equals the infimum over the costs of all ordinary controls. We also show that the optimal admissible control can be approximated arbitrarily well by very simple, ordinary controls. The proofs are based on a close analysis of the standard parts of nonstandard controls.     

Stochastic Lotka-Volterra system with infinite delay

This paper investigates a stochastic Lotka-Volterra system with infinite delay, whose initial data comes from an admissible Banach space C_r. We show that, under a simple hypothesis on the environmental noise, the stochastic Lotka-Volterra system with infinite delay has a unique global positive solution, and this positive solution will be asymptotic bounded. The asymptotic pathwise of the solution is also estimated by the exponential martingale inequality. Finally, two examples with their numerical simulations are provided to illustrate our result.     

The problem of optimal control with reflection studied through a linear optimization problem stated on occupational measures

We obtain a linear programming characterization for the minimum cost associated with finite dimensional reflected optimal control problems. In order to describe the value functions, we employ an infinite dimensional dual formulation instead of using the characterization via Hamilton-Jacobi partial differential equations. In this paper we consider control problems with both infinite and finite horizons. The reflection is given by the normal cone to a proximal retract set.     

Minimizing the probability of lifetime ruin under borrowing constraints

We determine the optimal investment strategy of an individual who targets a given rate of consumption and who seeks to minimize the probability of going bankrupt before she dies, also known as lifetime ruin. We impose two types of borrowing constraints: First, we do not allow the individual to borrow money to invest in the risky asset nor to sell the risky asset short. However, the latter is not a real restriction because in the unconstrained case, the individual does not sell the risky asset short. Second, we allow the individual to borrow money but only at a rate that is higher than the rate earned on the riskless asset.We consider two forms of the consumption function: (1) The individual consumes at a constant (real) dollar rate, and (2) the individual consumes a constant proportion of her wealth. The first is arguably more realistic, but the second is closely connected with Merton’s model of optimal consumption and investment under power utility. We demonstrate that connection in this paper, as well as include a numerical example to illustrate our results.     

Stochastic realization of a Gaussian stochastic control system

The stochastic realization problem is considered of representing a stationary Gaussian process as the observation process of a Gaussian stochastic control system. The problem formulation includes that the lastm components of the observation process form the Gaussian white noise input process to the system. Identifiability of this class of systems motivates the problem. The results include a necessary and sufficient condition for the existence of a stochastic realization. A subclass of Gaussian stochastic control systems is defined that is almost a canonical form for this stochastic realization problem. For a structured Gaussian stochastic control system an equivalent condition for identifiability of the parametrization is stated.The research of this paper is supported in part by the Commission of the European Communities through the SCIENCE Program by the projectSystem Identification with contract number SC1-CT92-0779.     

A “maximum principle for semicontinuous functions” applicable to integro-partial differential equations

We formulate and prove a non-local “maximum principle for semicontinuous functions” in the setting of fully nonlinear and degenerate elliptic integro-partial differential equations with integro operators of second order. Similar results have been used implicitly by several researchers to obtain compare/uniqueness results for integro-partial differential equations, but proofs have so far been lacking.     

Bounded-from-below solutions of the Hamilton-Jacobi equation for optimal control problems with exit times: vanishing lagrangians,eikonal equations,and shape-from-shading

We study the Hamilton-Jacobi equation for undiscounted exit time control problems with general nonnegative Lagrangians using the dynamic programming approach. We prove theorems characterizing the value function as the unique bounded-from-below viscosity solution of the Hamilton-Jacobi equation that is null on the target. The result applies to problems with the property that all trajectories satisfying a certain integral condition must stay in a bounded set. We allow problems for which the Lagrangian is not uniformly bounded below by positive constants, in which the hypotheses of the known uniqueness results for Hamilton-Jacobi equations are not satisfied. We apply our theorems to eikonal equations from geometric optics, shape-from-shading equations from image processing, and variants of the Fuller Problem.     

Convergence and stability of implicit runge-kutta methods for systems with multiplicative noise

A class ofimplicit Runge-Kutta schemes for stochastic differential equations affected bymultiplicative Gaussian white noise is shown to be optimal with respect to global order of convergence in quadratic mean. A test equation is proposed in order to investigate the stability of discretization methods for systems of this kind. Herestability is intended in a truly probabilistic sense, as opposed to the recently introduced extension of A-stability to the stochastic context, given for systems with additive noise. Stability regions for the optimal class are also given.Partially supported by the Italian Consiglio Nazionale delle Ricerche.     

Convexity properties of the minimum time function

This paper studies some regularity properties of the minimum time functionT for a nonlinear control system with a general targetK. Under a Petrov type controllability assumption,T is shown to be semiconcave if the distance fromK is semiconcave. A semiconvexity result also holds for linear control systems with convex targets. These properties are then applied to study the structure of the set of nondifferentiability points ofT. Partially supported by the Italian National Project MURST 40% Problemi nonlineari....     

Construction of asymptotically optimal controls for control and game problems

Recently the connection between control and game problems and Backward Stochastic Differential Equations has been established. This allows us to use an approximation scheme for such equations in order to construct an ɛ-optimal control. Received: 13 November 1995 / Revised version: 11 February 1998     

Constrained control of a nonlinear two point boundary value problem,I

In this paper we consider an optimal control problem for a nonlinear second order ordinary differential equation with integral constraints. A necessary optimality condition in form of the Pontryagin minimum principle is derived. The proof is based on McShane-variations of the optimal control, a thorough study of their behaviour in dependence of some denning parameters, a generalized Green formula for second order ordinary differential equations with measurable coefficients and certain tools of convex analysis.Dedicated to Lothar von Wolfersdorf on the occasion of his 60th birthday     

A Zubov’s method for stochastic differential equations

We consider a stochastic differential equation with an asymptotically stable equilibrium point. We show that the domain of attraction of the equilibrium, i.e. the set of points which are attracted with positive probability to it, can be characterized by the solution of a suitable partial differential equation.     

Discontinuous control problems for non-convex dynamics and near viability for singularly perturbed control systems

The aim of this paper is to study two classes of discontinuous control problems without any convexity assumption on the dynamics. In the first part we characterize the value function for the Mayer problem and the supremum cost problem using viscosity tools and the notion of ε-viability (near viability). These value functions are given with respect to discontinuous cost functionals. In the second part we obtain results describing the ε-viability (near viability) of singularly perturbed control systems.     

Stochastic control under progressive enlargement of filtrations and applications to multiple defaults risk management

We formulate and investigate a general stochastic control problem under a progressive enlargement of filtration. The global information is enlarged from a reference filtration and the knowledge of multiple random times together with associated marks when they occur. By working under a density hypothesis on the conditional joint distribution of the random times and marks, we prove a decomposition of the original stochastic control problem under the global filtration into classical stochastic control problems under the reference filtration, which is determined in a finite backward induction. Our method revisits and extends in particular stochastic control of diffusion processes with a finite number of jumps. This study is motivated by optimization problems arising in default risk management, and we provide applications of our decomposition result for the indifference pricing of defaultable claims, and the optimal investment under bilateral counterparty risk. The solutions are expressed in terms of BSDEs involving only Brownian filtration, and remarkably without jump terms coming from the default times and marks in the global filtration.     

Asymptotic almost-equivalence of Lipschitz evolution systems in Banach spaces

In this paper we introduce a notion of asymptotic almost-equivalence of two evolution systems and provide simple tests that guarantee that two evolution systems have the same qualitative asymptotic properties. In this way we are able to unify and extend many previously known results and also to understand what is behind equally behaved systems. In particular, we establish convergence, ergodic convergence and almost-convergence of almost-orbits both for the weak and the strong topologies based on the behavior of the orbits.