A dynamic programming approach to the maximum principle of distributed-parameter systems

This paper provides a dynamic programming approach to the maximum principle for the optimal control of systems with distributed parameters. The process of the systems under consideration is governed by a partial differential equation.This paper is based on Chapter 2 of the author's PhD Thesis under the supervision of Professor S. E. Dreyfus to whom the author wishes to express his appreciation.     

An infinite-horizon multistage dynamic optimization problem

Multiprocess problems are dynamic optimization problems in which there is a collection of control systems coupled through constraints in the endpoints of the constituent trajectories and through the cost function. Optimality conditions for such problems posed over a finite interval have already been derived. However, multiprocess problems arise, for example in the mathematical economics literature, in which one of the component processes operates over an infinite horizon. We give a proof of the relevant necessary conditions in the form of a maximum principle under mild and verifiable hypotheses and apply the theory to a two-stage problem in which an explicit dependence on the intermediate time (taken as a choice variable) is incorporated in the integrands of the cost functional.This research was carried out while the author was a Graduate Student at the Department of Electrical Engineering, Imperial College of Science, Technology, and Medicine, London, England. The author is grateful to Professor R. B. Vinter for his advice and helpful discussions.     

Dynamics of quantum entropy maximization for finite-level systems

The problem of constructing models for the statistical dynamics of finite-level quantum mechanical systems is considered. The maximum entropy principle formulated by E.T. Jaynes in 1957 and asserting that the entropy of any physical system increases until it attains its maximum value under constraints imposed by other physical laws is applied. In accordance with this principle, the von Neumann entropy is taken for the objective function; a dynamical equation describing the evolution of the density operator in finite-level systems is derived by using the speed gradient principle. In this case, physical constraints are the mass conservation law and the energy conservation law. The stability of the equilibrium points of the system thus obtained is investigated. By using LaSalle’s theorem, it is shown that the density function tends to a Gibbs distribution, under which the entropy attains its maximum. The method is exemplified by analyzing a finite system of identical particles distributed between cells. Results of numerical simulation are presented.     

Smooth and Nonsmooth Lipschitz Controls for a Class of Vector Differential Equations

This paper is devoted to the study of a class of control problems associated to a nonlinear second-order vector differential equation with pointwise state constraints. The control is realized via a function of the state. We extend the results of Akkouchi, Bounabat, and Goebel to vector differential equations and furthermore consider the more general case. Under proper conditions, we prove the existence of optimal controls in the class of Lipschitz functions and obtain an optimality condition which looks somehow like the Pontryagin maximum principle for a smooth optimal control function. For a nonsmooth optimal control function, we derive a suboptimality condition by means of the Ekeland variational principle.Communicated by M. J. BalasThis work was supported by 985 Project of Jilin University. The author thanks Professor Yong Li for valuable suggestions. He also thanks Professor M. J. Balas and the anonymous referees for their comments.     

Directionally differentiable multiobjective optimization involving discrete inclusions

This paper is devoted to multiobjective optimization problems involving discrete inclusions. The objective functions are assumed to be directionally differentiable and the domination structure is defined by a closed convex cone. The directional derivatives are not assumed to be linear or convex. Several concepts of optimal solutions are analyzed, and the corresponding necessary conditions are obtained as well in maximum principle form. As an application of the main results, a maximum principle is also derived for multiobjective optimization with extremalvalue fucctions involving discrete inclusions.The authors are indebted to the referee for detailed comments.The paper was written while the second author was visiting the laboratory of Prof. S. Suzuki, Department of Mechanical Engineering, Sophia University, Tokyo, Japan.     

Maximum likelihood least squares identification for systems with autoregressive moving average noise

Maximum likelihood methods are important for system modeling and parameter estimation. This paper derives a recursive maximum likelihood least squares identification algorithm for systems with autoregressive moving average noises, based on the maximum likelihood principle. In this derivation, we prove that the maximum of the likelihood function is equivalent to minimizing the least squares cost function. The proposed algorithm is different from the corresponding generalized extended least squares algorithm. The simulation test shows that the proposed algorithm has a higher estimation accuracy than the recursive generalized extended least squares algorithm.     

Optimal control: Nonlocal conditions,computational methods,and the variational principle of maximum

This paper surveys theoretical results on the Pontryagin maximum principle (together with its conversion) and nonlocal optimality conditions based on the use of the Lyapunov-type functions (solutions to the Hamilton-Jacobi inequalities). We pay special attention to the conversion of the maximum principle to a sufficient condition for the global and strong minimum without assumptions of the linear convexity, normality, or controllability. We give the survey of computational methods for solving classical optimal control problems and describe nonstandard procedures of nonlocal improvement of admissible processes in linear and quadratic problems. Furthermore, we cite some recent results on the variational principle of maximum in hyperbolic control systems. This principle is the strongest first order necessary optimality condition; it implies the classical maximum principle as a consequence.     

An application of Neustadt's abstract maximum principle to probability kinematics

An important class of problems in philosophy can be formulated as mathematical programming problems in an infinite-dimensional vector space. One such problem is that of probability kinematics: the study of how an individual ought to adjust his degree-of-belief function in response to new information. Much work has recently been done to establish maximum principles for these generalized programming problems (Refs. 3–4). Perhaps, the most general treatment of the problem presented to date is that by Neustadt (Ref. 1). In this paper, the problem of probability kinematics is formulated as a generalized mathematical programming problem and necessary conditions for the optimal revised degree-of-belief function are derived from an abstract maximum principle contained in Neustadt's paper.This work was supported by the National Research Council of Canada.The author is grateful to G. J. Lastman and J. A. Baker of the University of Waterloo for numerous suggestions made for improvement of this paper. The problem of probability kinematics was brought to the author's attention by W. L. Harper of the University of Western Ontario.     

The connection between thermodynamic entropy and probability

Ergodic Hamiltonian systems with an arbitrary number of degrees of freedom n are considered. A relation is derived connecting the distribution function of the system characteristics with the entropy. It is shown that as n → ∞ it reduces to Einstein's formula /1/. A variational principle for the distribution function, which reduces to the maximum-uncertainty principle as n → ∞ is derived. The principle of maximum entropy for Hamiltonian systems is formulated.     

Stochastic Relations and the Problem of Prior in the Principle of Maximum Entropy

In this paper we discuss the problem of prior for the maximum entropy principle. We show that stochastic relations can be used to constrain priors and in some case uniquely determine them. The principle of maximum entropy turns stochastic relations into (over)determined systems of partial difference equations for the partition function. All statistical consequences of the stochastic relations are determined by the space of solutions of the system.     

Necessary conditions for optimality of singular controls

The present paper is concerned with the study of controls which are singular in the sense of the maximum principle. We obtain necessary conditions for optimality of singular controls in systems governed by ordinary differential equations. A useful feature of the method considered here is that it can be applied to optimal control problems with distributed parameters.this research was supported in part by the National Science Foundation under Grant No. NSF-MCS-80-02337 at the University of Michigan.The author wishes to express his deep gratitude to Professor L. Cesari for his valuable guidance and constant encouragement during the preparation of this paper.     

Herleitung eines stochastischen maximumprinzips mit variationsmethoden

Using variation techniques a stochastic maximum principle is proved for control systems, which are described by stochastic differential equations over a fixed time interval. Thereby the diffusion matrix is a function of the state of the system. The principle of optimality has the form of an inequality of the conditional expectation of a HAMILTON-function with respect to the so-called б-algebra of informations.     

Maximum principle for optimal control problems involving impulse controls with nonsmooth data

This paper is concerned with the stochastic maximum principle for impulse optimal control problems of forward–backward systems, where the coefficients of the forward part are Lipschitz continuous. The domain of the regular controls is not necessarily convex. We establish a Pontryagins maximum principle for this control problem by applying Ekelands variational principle to a sequence of approximated control problems with smooth coefficients of the initial problems.     

Optimal control of time-delay systems with distributed parameters

An optimal control problem with a prescribed performance index for parabolic systems with time delays is investigated. A necessary condition for optimality is formulated and proved in the form of a maximum principle. Under additional conditions, the maximum principle gives sufficient conditions for optimality. It is also shown that the optimal control is unique. As an illustration of the theoretical consideration, an analytic solution is obtained for a time-delayed diffusion system.The author wishes to express his deep gratitude to Professors J. M. Sloss and S. Adali for the valuable guidance and constant encouragement during the preparation of this paper.     

Maximum principle for differential games of forward–backward stochastic systems with applications

This paper is concerned with a maximum principle for both zero-sum and nonzero-sum games. The most distinguishing feature, compared with the existing literature, is that the game systems are described by forward–backward stochastic differential equations. This kind of games is motivated by linear-quadratic differential game problems with generalized expectation. We give a necessary condition and a sufficient condition in the form of maximum principle for the foregoing games. Finally, an example of a nonzero-sum game is worked out to illustrate that the theories may find interesting applications in practice. In terms of the maximum principle, the explicit form of an equilibrium point is obtained.     

UNIQUENESS OF VISCOSITY SOLUTIONS OF FULLY NONLINEAR SECOND ORDER PARABOLIC PDE''''S

This paper investigates the maximum principle for viscosity solutions of fully nonlinear, second order parabolic, possibly degenerate partial differential equations. Using this maximum principle the anthor prove that viscosity solutions of initial and bo unoary value problem for parabolic equations are unique.     

Boundary arcs for integral equations

Behavior of boundary arcs for control systems is investigated when the systems are governed by integral equations of the Volterra type. The main result is in the form of a maximum principle. This result is then used to obtain necessary conditions for a minimum control problem.     

Spektrum,numerischer Wertebereich und IHRE Maximumprinzipien in Banachalgebren

By a function of support a numerical range is defined for elements of complex Banach algebras. The geometric character of the definition allows to proof some results about the numerical range in a very natural way.- In the second part holomorphic perturbations of the numerical range and the spectrum are treated as perturbations of the function of support to get a maximum principle for the numerical range and the convex hull of the spectrum, whereas there is in general no maximum principle for the spectrum. This answers a question raised by A.Brown and R.G.Douglas.

---

---

Ich danke Herrn Prof.Dr.S.Hildebrandt für viele Anregungen. Herrn K.P.Steffen verdanke ich wertvolle Hinweise. Von ihm stammen Satz 2.1 und Beispiel 2.13.     

Maximum principle of optimal control for functional differential systems

We show that the maximum principle holds for optimal periodic control problems governed by functional differential equations under a Lipschitz condition on the value functional. Generalizations to other boundary conditions are also considered.This research was partially supported by NSF Grant No. DMS-84-01719.The first author was partially supported by the Science Fund of the Chinese Academy of Sciences, Beijing, China.     

A unified theory of necessary conditions for nonlinear nonconvex control systems

We consider optimal problems for a general nonlinear nonconvex input-output relation for Banach space valued functions. A maximum principle is obtained using Ekeland's variational principle. The formulation applies to systems described by ordinary differential equations, functional differential equations, and partial differential equations (both for distributed and boundary control systems).This work was supported in part by the National Science Foundation under Grant No. DMS-8200645.