Maximum principle,dynamic programming,and their connection in deterministic control

Two major tools for studying optimally controlled systems are Pontryagin's maximum principle and Bellman's dynamic programming, which involve the adjoint function, the Hamiltonian function, and the value function. The relationships among these functions are investigated in this work, in the case of deterministic, finite-dimensional systems, by employing the notions of superdifferential and subdifferential introduced by Crandall and Lions. Our results are essentially non-smooth versions of the classical ones. The connection between the maximum principle and the Hamilton-Jacobi-Bellman equation (in the viscosity sense) is thereby explained by virtue of the above relationship.This research was supported by the Natural Science Fund of China.This paper was written while the author visited Keio University, Japan. The author is indebted to Professors H. Tanaka and M. Nisio for their helpful suggestions and discussions. Thanks are also due to Professor X. J. Li for his comments and criticism.