Normality of the maximum principle for nonconvex constrained Bolza problems

We consider a Bolza optimal control problem with state constraints. It is well known that under some technical assumptions every strong local minimizer of this problem satisfies first order necessary optimality conditions in the form of a constrained maximum principle. In general, the maximum principle may be abnormal or even degenerate and so does not provide a sufficient information about optimal controls. In the recent literature some sufficient conditions were proposed to guarantee that at least one maximum principle is nondegenerate, cf. [A.V. Arutyanov, S.M. Aseev, Investigation of the degeneracy phenomenon of the maximum principle for optimal control problems with state constraints, SIAM J. Control Optim. 35 (1997) 930–952; F. Rampazzo, R.B. Vinter, A theorem on existence of neighbouring trajectories satisfying a state constraint, with applications to optimal control, IMA 16 (4) (1999) 335–351; F. Rampazzo, R.B. Vinter, Degenerate optimal control problems with state constraints, SIAM J. Control Optim. 39 (4) (2000) 989–1007]. Our aim is to show that actually conditions of a similar nature guarantee normality of every nondegenerate maximum principle. In particular we allow the initial condition to be fixed and the state constraints to be nonsmooth. To prove normality we use J. Yorke type linearization of control systems and show the existence of a solution to a linearized control system satisfying new state constraints defined, in turn, by linearization of the original set of constraints along an extremal trajectory.     

A direct approach to second order conditions for mixed equality constraints

In this paper we consider an optimal control problem posed over piecewise continuous controls and involving state-control (mixed) equality constraints. We provide an explicit derivation of second order necessary conditions simpler than others available in the literature, yielding a clear understanding of how to define a set of “differentially admissible variations” where a certain quadratic form is nonnegative.     

Second-order necessary conditions for differential inclusion problems

We study second-order necessary conditions for optimality in the unbounded differential inclusion control problem and recover the accessory problem in optimal control theory.     

Structure of minimizers in linear-quadratic Bolza problems of optimal control

In this paper, we study intersections of extremals in a linear-quadratic Bolza problem of optimal control. The structure of the inter-sections is described. We show that this structure implies the semipositive definiteness of the quadratic cost functional. In addition, we derive necessary and sufficient conditions for the existence of minimizers.     

New approximation method in the proof of the Maximum Principle for nonsmooth optimal control problems with state constraints

Traditional proofs of the Pontryagin Maximum Principle (PMP) require the continuous differentiability of the dynamics with respect to the state variable on a neighborhood of the minimizing state trajectory, when arbitrary values of control variable are inserted into the dynamic equations. Sussmann has drawn attention to the fact that the PMP remains valid when the dynamics are differentiable with respect to the state variable, merely when the minimizing control is inserted into the dynamic equations. This weakening of earlier hypotheses has been referred to as the Lojasiewicz refinement. Arutyunov and Vinter showed that these extensions of early versions of the PMP can be simply proved by finite-dimensional approximations, application of a Lagrange multiplier rule in finite dimensions and passage to the limit. This paper generalizes the finite-dimensional approximation technique to a problem with state constraints, where the use of needle variations of the optimal control had not been successful. Moreover, the cost function and endpoint constraints are not assumed to be differentiable, but merely locally Lipschitz continuous. The Maximum Principle is expressed in terms of Michel-Penot subdifferential.     

Jacobi condition for elliptic forms in Hilbert spaces

In this paper, we give the definitions of conjugate and semi-conjugate points for a quadratic elliptic form in a Hilbert space, and we state the corresponding necessary and sufficient conditions (the Jacobi conditions) for the form to be positive (nonnegative). We apply the abstract results to a one-dimensional problem in the calculus of variations where both endpoints are allowed to vary. The conditions that we obtain complete and generalize previously known results.This research was partly supported by MURST Research Grant, Teoria del Controllo dei Sistemi Dinamici.     

Jacobi Condition for Elliptic Forms in Hilbert Spaces

A definition given in Ref. 1 is corrected together with its consequences.     

Lipschitzianity of optimal trajectories for the Bolza optimal control problem

In this paper we investigate Lipschitz continuity of optimal solutions for the Bolza optimal control problem under Tonelli’s type growth condition. Such regularity being a consequence of normal necessary conditions for optimality, we propose new sufficient conditions for normality of state-constrained nonsmooth maximum principles for absolutely continuous optimal trajectories. Furthermore we show that for unconstrained problems any minimizing sequence of controls can be slightly modified to get a new minimizing sequence with nice boundedness properties. Finally, we provide a sufficient condition for Lipschitzianity of optimal trajectories for Bolza optimal control problems with end point constraints and extend a result from (J. Math. Anal. Appl. 143, 301–316, 1989) on Lipschitzianity of minimizers for a classical problem of the calculus of variations with discontinuous Lagrangian to the nonautonomous case.     

An existence theorem for neutral variational problems of Bolza

This note proves an existence theorem for a generalized Bolza-type problem that has time delays in both the state and velocity variables. The assumptions are stated in terms of a modification of the classical Hamiltonian, and extend ideas of Rockafellar to the delay case.     

Optimal fields for problems with delays

The theory of optimal fields is developed for optimal control problems in which the state variables are solutions of integral equations with delayed arguments. The maximum principle obtained reflects the effects of the delay in the control argument. The Hamilton-Jacobi equations are derived for this problem.     

A nonmonotone truncated Newton–Krylov method exploiting negative curvature directions, for large scale unconstrained optimization

We propose a new truncated Newton method for large scale unconstrained optimization, where a Conjugate Gradient (CG)-based technique is adopted to solve Newton’s equation. In the current iteration, the Krylov method computes a pair of search directions: the first approximates the Newton step of the quadratic convex model, while the second is a suitable negative curvature direction. A test based on the quadratic model of the objective function is used to select the most promising between the two search directions. Both the latter selection rule and the CG stopping criterion for approximately solving Newton’s equation, strongly rely on conjugacy conditions. An appropriate linesearch technique is adopted for each search direction: a nonmonotone stabilization is used with the approximate Newton step, while an Armijo type linesearch is used for the negative curvature direction. The proposed algorithm is both globally and superlinearly convergent to stationary points satisfying second order necessary conditions. We carry out a significant numerical experience in order to test our proposal.     

A New Notion of Conjugacy for Isoperimetric Problems

For problems in the calculus of variations with isoperimetric side constraints, we provide in this paper a set of points whose emptiness, independently of nonsingularity assumptions, is equivalent to the nonnegativity of the second variation along admissible variations. The main objective of introducing a characterization of this condition should be, of course, to obtain a simpler way of verifying it. There are two other sets of points available in the literature, introduced by Loewen and Zheng (1994) and Zeidan (1996), for which this necessary condition implies their emptiness. However, we show that verifying membership of these sets may be more difficult than checking directly if that condition holds. Contrary to this behavior, we prove that the desired objective of characterizing that condition is achieved by means of the set introduced in this paper.     

On the order of singular optimal control problems

In singular optimal control problems, the functional form of the optimal control function is usually determined by solving the algebraic equation which results by successively differentiating the switching function until the control appears explicitly. This process defines the order of the singular problem. Order-related results are developed for singular linear-quadratic problems and for a bilinear example which gives new insights into the relationship between singular problem order and singular are order.Dedicated to R. BellmanThis work was supported by the National Science Foundation under Grant No. ENG-77-16660.     

Second order optimality conditions for a class of control problems governed by non-linear integral equations with application to parabolic boundary control

In the paper necessary and sufficient second order optimality conditions for optimal control problems governed by weakly singular non linear Hammerstein integral equations are derived. They are applied to a semilinear parabolic boundary control problem for the one dimensional heat equation.     

Necessary optimality conditions for abnormal problems with geometric constraints

The abnormal minimization problem with a finite-dimensional image and geometric constraints is examined. In particular, inequality constraints are included. Second-order necessary conditions for this problem are established that strengthen previously known results.     

Second order necessary conditions in set constrained differentiable vector optimization

We state second order necessary optimality conditions for a vector optimization problem with an arbitrary feasible set and an order in the final space given by a pointed convex cone with nonempty interior. We establish, in finite-dimensional spaces, second order optimality conditions in dual form by means of Lagrange multipliers rules when the feasible set is defined by a function constrained to a set with convex tangent cone. To pass from general conditions to Lagrange multipliers rules, a generalized Motzkin alternative theorem is provided. All the involved functions are assumed to be twice Fréchet differentiable. Mathematics subject classification 2000:90C29, 90C46This research was partially supported by Ministerio de Ciencia y Tecnología (Spain), project BMF2003-02194.     

Additional necessary conditions for optimal control with state-variable inequality constraints

It is shown that, when the set of necessary conditions for an optimal control problem with state-variable inequality constraints given by Bryson, Denham, and Dreyfus is appropriately augmented, it is equivalent to the (different) set of conditions given by Jacobson, Lele, and Speyer. Relationships among the various multipliers are given.This work was done at NASA Ames Research Center, Moffett Field, California, under a National Research Council Associateship.     

Successive approximations for Markov decision processes and Markov games with unbounded rewards

The aim of this paper is to give a survey of recent developments in the area of successive approximations for Markov decision processes and Markov games. We will emphasize two aspects, viz. the conditions under which successive approximations converge in some strong sense and variations of these methods which diminish the amount of computational work to be executed. With respect to the first aspect it will be shown how much unboundedness of the rewards may be allowed without violation of the convergence

With respect to the second aspect we will present four ideas, that can be applied in conjunction, which may diminish the amount of work to be done. These ideas are: 1. the use of the actual convergence of the iterates for the construction of upper and lower bounds (Macqueen bounds), 2. the use of alternative policy improvement procedures (based on stopping times), 3. a better evaluation of the values of actual policies in each iteration step by a value oriented approach, 4. the elimination of suboptimal actions not only permanently, but also temporarily. The general presentation is given for Markov decision processes with a final section devoted to the possibilities of extension to Markov games.     

Singular Normal Extremals and Conjugate Points for Bolza Functionals

In this paper, we study the problem of the optimal control for Bolza functionals by investigating extremals containing singular arcs. We use the Moore–Penrose generalized inverse, which allows one to determine normality criteria and sufficient conditions for the nonexistence of conjugate points.