Value function,relaxation, and transversality conditions in infinite horizon optimal control

We investigate the value function $V:{\mathbb{R}}_{+}\times {\mathbb{R}}^n\to {\mathbb{R}}_{+}\cup \{+\infty \}$ of the infinite horizon problem in optimal control for a general—not necessarily discounted—running cost and provide sufficient conditions for its lower semicontinuity, continuity, and local Lipschitz regularity. Then we use the continuity of $V(t,?)$ to prove a relaxation theorem and to write the first order necessary optimality conditions in the form of a, possibly abnormal, maximum principle whose transversality condition uses limiting/horizontal supergradients of $V(0,?)$ at the initial point. When $V(0,?)$ is merely lower semicontinuous, then for a dense subset of initial conditions we obtain a normal maximum principle augmented by sensitivity relations involving the Fréchet subdifferentials of $V(t,?)$. Finally, when V is locally Lipschitz, we prove a normal maximum principle together with sensitivity relations involving generalized gradients of V for arbitrary initial conditions. Such relations simplify drastically the investigation of the limiting behavior at infinity of the adjoint state.