Optimal Control of Point Processes with Noisy Observations: The Maximum Principle

This paper studies the optimal control problem for point processes with Gaussian white-noised observations. A general maximum principle is proved for the partially observed optimal control of point processes, without using the associated filtering equation . Adjoint flows—the adjoint processes of the stochastic flows of the optimal system—are introduced, and their relations are established. Adjoint vector fields , which are observation-predictable, are introduced as the solutions of associated backward stochastic integral-partial differential equtions driven by the observation process. In a heuristic way, their relations are explained, and the adjoint processes are expressed in terms of the adjoint vector fields, their gradients and Hessians, along the optimal state process. In this way the adjoint processes are naturally connected to the adjoint equation of the associated filtering equation . This shows that the conditional expectation in the maximum condition is computable through filtering the optimal state, as usually expected. Some variants of the partially observed stochastic maximum principle are derived, and the corresponding maximum conditions are quite different from the counterpart for the diffusion case. Finally, as an example, a quadratic optimal control problem with a free Poisson process and a Gaussian white-noised observation is explicitly solved using the partially observed maximum principle. Accepted 8 August 2001. Online publication 17 December, 2001.     

The Nehari problem for infinite-dimensional linear systems of parabolic type

A complete solution is obtained to the suboptimal Nehari extension problem for transfer functions of parabolic systems with Dirichlet boundary control and smooth observations. The solutions are given in terms of the realization (–A, B, C), whereA is a uniformly strongly elliptic operator of order two with smooth coefficients defined on a bounded open domain ofR _d ,B=AB _D andB _D is the Dirichlet map associated with Dirichlet boundary conditions andC is a bounded observation map fromL ₂() to the output spaceY. The approach is to solve an equivalentJ-spectral factorization problem for this particular realization.     

The Nehari problem for the Pritchard-Salamon class of infinite-dimensional linear systems: a direct approach

A complete solution is obtained to the Nehari problem for symbols which have a realization as an exponentially stable Pritchard-Salamon system (A, B, C). This allows for the possibility thatB andC be unbounded and have infinite rank. The approach is to solve an equivalentJ-spectral factorization problem for this particular realization.     

Perfect cocycles through stochastic differential equations

Summary We prove that if is a random dynamical system (cocycle) for whicht(t, )x is a semimartingale, then it is generated by a stochastic differential equation driven by a vector field valued semimartingale with stationary increment (helix), and conversely. This relation is succinctly expressed as semimartingale cocycle=exp(semimartingale helix). To implement it we lift stochastic calculus from the traditional one-sided time to two-sided timeT= and make this consistent with ergodic theory. We also prove a general theorem on the perfection of a crude cocycle, thus solving a problem which was open for more than ten years.This article was processed by the author using the latex style filepljour Im from Springer-Verlag.     

The infinite-time admissibility of observation operators and operator Lyapunov equations

Let (A, –, C) be an abstract dynamical system withA being the generator of aC ₀-semigroup on a Hilbert spaceH, C:D(A)Y a linear operator,Y another Hilbert space. In this paper, some sufficient and necessary conditions are obtained for the observation operatorC to be infinite-time admissible. For a control system (A, B, –), due to duality argument, some sufficient and necessary conditions are also given for the control operatorB to be extended admissible. It is wellknown that observation operatorC is admissible if and only if the operator Lyapunov equation associated with the system has a nonnegative solution. In this paper, all nonnegative solutions to this equation are represented parametrically.This project is supported by the NNSF of China, and the Youth Science and Technique Foundation of Shanxi Province.     

The canonical Cauchy problem for linear systems of partial difference equations with constant coefficients over the completer-dimensional integral lattice ℕ2r

We canonically define and algorithmically solve the problem of the title. Such algorithms are of great significance for the method of finite differences for the solution of partial differential equations and for many technical applications such as image processing. In contrast to the wide (system theoretic) literature for ordinary difference equations and in spite of the great theoretical and practical significance of this problem, until now, there was no systematic theory of these systems and in particular of the corresponding Cauchy problem, let alone an algorithm. In this paper, we give both. The method consists in a transformation of this problem into a naturally associated problem which is defined over the 2r-dimensional natural number lattice ^2r (the upper quadrant in ^2r ) and for which the canonical initial value or Cauchy problem was defined and constructively solved by the second author.     

Infinitesimal methods in control theory: Deterministic and stochastic

In Part I, methods of nonstandard analysis are applied to deterministic control theory, extending earlier work of the author. Results established include compactness of relaxed controls, continuity of solution and cost as functions of the controls, and existence of optimal controls. In Part II, the methods are extended to obtain similar results for partially observed stochastic control. Systems considered take the form:where the feedback control u depends on information from a digital read-out of the observation process y. The noise in the state equation is controlled along with the drift. Similar methods are applied to a Markov system in the final section.     

Multidimensional constant linear systems

A continuous resp. discrete r-dimensional (r1) system is the solution space of a system of linear partial differential resp. difference equations with constant coefficients for a vector of functions or distributions in r variables resp. of r-fold indexed sequences. Although such linear systems, both multidimensional and multivariable, have been used and studied in analysis and algebra for a long time, for instance by Ehrenpreis et al. thirty years ago, these systems have only recently been recognized as objects of special significance for system theory and for technical applications. Their introduction in this context in the discrete one-dimensional (r=1) case is due to J. C. Willems. The main duality theorem of this paper establishes a categorical duality between these multidimensional systems and finitely generated modules over the polynomial algebra in r indeterminates by making use of deep results in the areas of partial differential equations, several complex variables and algebra. This duality theorem makes many notions and theorems from algebra available for system theoretic considerations. This strategy is pursued here in several directions and is similar to the use of polynomial algebra in the standard one-dimensional theory, but mathematically more difficult. The following subjects are treated: input-output structures of systems and their transfer matrix, signal flow spaces and graphs of systems and block diagrams, transfer equivalence and (minimal) realizations, controllability and observability, rank singularities and their connection with the integral respresentation theorem, invertible systems, the constructive solution of the Cauchy problem and convolutional transfer operators for discrete systems. Several constructions on the basis of the Gröbner basis algorithms are executed. The connections with other approaches to multidimensional systems are established as far as possible (to the author).Partially supported by US Air Force Grant AFOSR-87-0249 and by Office of Naval Research Grant N 00014-86-K-0538 through the Center for Mathematical System Theory, University of Florida, Gainesville, Florida, U.S.A.     

Adaptive Stabilization of Nonlinear Stochastic Systems

The purpose of this paper is to study the problem of asymptotic stabilization in probability of nonlinear stochastic differential systems with unknown parameters. With this aim, we introduce the concept of an adaptive control Lyapunov function for stochastic systems and we use the stochastic version of Artstein's theorem to design an adaptive stabilizer. In this framework the problem of adaptive stabilization of a nonlinear stochastic system is reduced to the problem of asymptotic stabilization in probability of a modified system. The design of an adaptive control Lyapunov function is illustrated by the example of adaptively quadratically stabilizable in probability stochastic differential systems. Accepted 9 December 1996     

Optimal stopping for partially observed piecewise-deterministic Markov processes

This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an ?$?$-optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence.     

On Bellman Equations in Quadratic Ergodic Control with Controller Constraints

We consider the Bellman equation related to the quadratic ergodic control problem for stochastic differential systems with controller constraints. We solve this equation rigidly in C ² -class, and give the minimal value and the optimal control. Accepted 9 January 1997     

Nonlinearity inH ∞-control theory,causality in the commutant lifting theorem,and extension of intertwining operators

The problems studied in this note have been motivated by our work in generalizing linearH control theory to nonlinear systems. These ideas have led to a design procedure applicable to analytic nonlinear plants. Our technique is a generalization of the linearH theory. In contrast to previous work on this topic ([9], [10]), we now are able to explicitly incorporate a causality constraint into the theory. In fact, we show that it is possible to reduce a causal optimal design problem (for nonlinear systems) to a classical interpolation problem solvable by the commutant lifting theorem [8]. Here we present the complete operator theoretical background of our research together with a short control theoretical motivation.This work was supported in part by grants from the Research Fund of Indiana University, the National Science Foundation DMS-8811084 and ECS-9122106, by the Air Force Office of Scientific Research F49620-94-1-0098DEF, and by the Army Research Office DAAL03-91-G-0019 and DAAH04-93-G-0332     

An obstruction to the simultaneous stabilization of two n-D plants

This paper calculates an obstruction to the simultaneous stabilization of twon-D plants. This obstruction is topological and lies in the (singular) cohomology of a subset of ⁿ that arises naturally in the problem. Given any family ofn-D plants, the vanishing of the corresponding cohomology classes of every pair of plants in the family is thus a necessary condition for the simultaneous stabilization of the family.     

Convergence and stability of implicit runge-kutta methods for systems with multiplicative noise

A class ofimplicit Runge-Kutta schemes for stochastic differential equations affected bymultiplicative Gaussian white noise is shown to be optimal with respect to global order of convergence in quadratic mean. A test equation is proposed in order to investigate the stability of discretization methods for systems of this kind. Herestability is intended in a truly probabilistic sense, as opposed to the recently introduced extension of A-stability to the stochastic context, given for systems with additive noise. Stability regions for the optimal class are also given.Partially supported by the Italian Consiglio Nazionale delle Ricerche.     

Filtering on manifolds

We coasider a partially observable diffusion process (x _t,y_t)t0 whose unobservable componentx _t lives on a submanifold M ofR ⁿ . We present some general conditions under which the conditional law ofx _t, given the observationsy _s ,s [0,t], admits a density w.r.t. a given measure on M. We characterize the analytical properties of this density by using appropriate Sobolev spaces.Research supported by the Hungarian National Foundation of Scientific Research No. 2290.     

Regularity properties of viscosity solutions of integro-partial differential equations of Hamilton–Jacobi–Bellman type

We study the regularity properties of integro-partial differential equations of Hamilton–Jacobi–Bellman type with the terminal condition, which can be interpreted through a stochastic control system, composed of a forward and a backward stochastic differential equation, both driven by a Brownian motion and a compensated Poisson random measure. More precisely, we prove that, under appropriate assumptions, the viscosity solution of such equations is jointly Lipschitz and jointly semiconcave in (t,x)∈Δ×R^d$(t,x)\in \Delta \times {\mathbb{R}}^d$, for all compact time intervals Δ$\Delta$ excluding the terminal time. Our approach is based on the time change for the Brownian motion and on Kulik’s transformation for the Poisson random measure.     

Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem

We study a stochastic optimal control problem for a partially observed diffusion. By using the control randomization method in Bandini et al. (2018), we prove a corresponding randomized dynamic programming principle (DPP) for the value function, which is obtained from a flow property of an associated filter process. This DPP is the key step towards our main result: a characterization of the value function of the partial observation control problem as the unique viscosity solution to the corresponding dynamic programming Hamilton–Jacobi–Bellman (HJB) equation. The latter is formulated as a new, fully non linear partial differential equation on the Wasserstein space of probability measures. An important feature of our approach is that it does not require any non-degeneracy condition on the diffusion coefficient, and no condition is imposed to guarantee existence of a density for the filter process solution to the controlled Zakai equation. Finally, we give an explicit solution to our HJB equation in the case of a partially observed non Gaussian linear–quadratic model.     

Hierarchical decomposition of symmetric discrete systems by matroid and group theories

An algebraic method is proposed for the hierarchical decomposition of large-scale group-symmetric discrete systems into partially ordered subsystems. It aims at extracting substructures and hierarchy for such systems as electrical networks and truss structures.The mathematical problem considered is: given a parametrized family of group invariant structured matricesA, we are to find two constant (=parameter-independent) nonsingular matricesS _r andS _c such thatS _r ^-1 AS _c takes a (common) block-triangular form.The proposed method combines two different decomposition principles developed independently in matroid theory and in group representation theory. The one is the decomposition principle for submodular functions, which has led to the Dulmage—Mendelsohn (DM-) decomposition and further to the combinatorial canonical form (CCF) of layered mixed (LM-) matrices. The other is the full reducibility of group representations, which yields the block-diagonal decomposition of group invariant matrices. The optimality of the proposed method is also discussed.