Additional necessary conditions for optimal control with state-variable inequality constraints

It is shown that, when the set of necessary conditions for an optimal control problem with state-variable inequality constraints given by Bryson, Denham, and Dreyfus is appropriately augmented, it is equivalent to the (different) set of conditions given by Jacobson, Lele, and Speyer. Relationships among the various multipliers are given.This work was done at NASA Ames Research Center, Moffett Field, California, under a National Research Council Associateship.     

Minimal trajectories of nonconvex differential inclusions

We consider an optimization problem with endpoint constraints associated with a nonconvex differential inclusion. We give a necessary condition of the maximum principle type for a solution of the problem. Following the approach from Ref. 1, the condition is stated in terms of single-valued selections of the convexified right-hand side of the inclusion.This work was supported in part by the National Science Foundation, Grant No. DMS-86-01774.     

Constraint Qualifications for Nonsmooth Mathematical Programs with Equilibrium Constraints

We study nonsmooth mathematical programs with equilibrium constraints. First we consider a general disjunctive program which embeds a large class of problems with equilibrium constraints. Then, we establish several constraint qualifications for these optimization problems. In particular, we generalize the Abadie and Guignard-type constraint qualifications. Subsequently, we specialize these results to mathematical program with equilibrium constraints. In our investigation, we show that a local minimum results in a so-called M-stationary point under a very weak constraint qualification.      

Necessary conditions for functional differential inclusions

We consider an optimal control problem in which the dynamic equation and cost function depend on the recent past of the trajectory. The regularity assumed in the basic data is Lipschitz continuity with respect to the sup norm. It is shown that, for a given optimal solution, an adjoint arc of bounded variation exists that satisfies an associated Hamiltonian inclusion. From this result, known smooth versions of the Pontryagin maximum principle for hereditary problems can be easily derived. Problems with Euclidean endpoint constraints are also considered.     

On optimal control problems with general boundary conditions

It is shown that the necessary optimality conditions for optimal control problems with terminal constraints and with given initial state allow also to obtain in a straightforward way the necessary optimality conditions for problems involving parameters and general (mixed) boundary conditions. In a similar manner, the corresponding numerical algorithms can be adapted to handle this class of optimal control problems.This research was supported in part by the Commission on International Relations, National Academy of Sciences, under Exchange Visitor Program No. P-1-4174.The author is indebted to the anonymous reviewer bringing to his attention Ref. 9 and making him aware of the possible use of generalized inverse notation when formulating the optimality conditions.     

Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints

In this paper we consider a mathematical program with equilibrium constraints (MPEC) formulated as a mathematical program with complementarity constraints. Various stationary conditions for MPECs exist in literature due to different reformulations. We give a simple proof to the M-stationary condition and show that it is sufficient for global or local optimality under some MPEC generalized convexity assumptions. Moreover, we propose new constraint qualifications for M-stationary conditions to hold. These new constraint qualifications include piecewise MFCQ, piecewise Slater condition, MPEC weak reverse convex constraint qualification, MPEC Arrow-Hurwicz-Uzawa constraint qualification, MPEC Zangwill constraint qualification, MPEC Kuhn-Tucker constraint qualification, and MPEC Abadie constraint qualification.     

Necessary and sufficient conditions for discrete and differential inclusions of elliptic type

This paper deals for the first time with the Dirichlet problem for discrete (PD), discrete approximation problem on a uniform grid and differential (PC) inclusions of elliptic type. In the form of Euler-Lagrange inclusion necessary and sufficient conditions for optimality are derived for the problems under consideration on the basis of new concepts of locally adjoint mappings. The results obtained are generalized to the multidimensional case with a second order elliptic operator.     

Regularity of flows and optimal control of shear-thinning fluids

We study optimal control problems of systems describing the flow of incompressible shear-thinning fluids. Taking advantage of regularity properties of the flows, we derive necessary optimality conditions under a restriction on the optimal control.     

Maximum principle for optimal distributed control of the viscous Dullin-Gottwald-Holm equation

In this paper, the optimal distributed control of the viscous Dullin-Gottwald-Holm equation is investigated. Adopting the Dubovitskii and Milyutin functional analytical approach, we obtain the Pontryagin maximum principle of the system. The necessary optimality condition is established for an optimal control problem in fixed final horizon case. Finally, an illustrative example is also given.     

Successive approximations for Markov decision processes and Markov games with unbounded rewards

The aim of this paper is to give a survey of recent developments in the area of successive approximations for Markov decision processes and Markov games. We will emphasize two aspects, viz. the conditions under which successive approximations converge in some strong sense and variations of these methods which diminish the amount of computational work to be executed. With respect to the first aspect it will be shown how much unboundedness of the rewards may be allowed without violation of the convergence

With respect to the second aspect we will present four ideas, that can be applied in conjunction, which may diminish the amount of work to be done. These ideas are: 1. the use of the actual convergence of the iterates for the construction of upper and lower bounds (Macqueen bounds), 2. the use of alternative policy improvement procedures (based on stopping times), 3. a better evaluation of the values of actual policies in each iteration step by a value oriented approach, 4. the elimination of suboptimal actions not only permanently, but also temporarily. The general presentation is given for Markov decision processes with a final section devoted to the possibilities of extension to Markov games.     

Sufficient optimality conditions for nonconvex control problems with state constraints

The sufficient optimality conditions of Zeidan for optimal control problems (Refs. 1 and 2) are generalized such that they are applicable to problems with pure state-variable inequality constraints. We derive conditions which neither assume the concavity of the Hamiltonian nor the quasiconcavity of the constraints. Global as well as local optimality conditions are presented.     

Optimal Control Problems via Exact Penalty Functions

The nonsmoothness is viewed by many people as at least an undesirable (if not unavoidable) property. Our aim here is to show that recent developments in Nonsmooth Analysis (especially in Exact Penalization Theory) allow one to treat successfully even some quite smooth problems by tools of Nonsmooth Analysis and Nondifferentiable Optimization. Our approach is illustrated by one Classical Control Problem of finding optimal parameters in a system described by ordinary differential equations.     

Applying the Leitmann-Stalford sufficient conditions to maximization control problems with non-concave Hamiltonian

The main result in this short note is that the integral form of the Leitmann-Stalford sufficiency conditions can be verified for a class of optimal control problems whose Hamiltonian is not concave with respect to the state variable. The main requirement for this class of problems is that the dynamics is sufficiently dissipative. An application to a Stackelberg differential game between a producer and a developer is exemplified. Using our result we show that the necessary conditions implied by Pontryagin’s maximum principle are also sufficient. This allows a complete characterization of the solution.     

Weak maximum principle for optimal control problems of nonsmooth systems

In this paper, by considering vector-valued maximum type functions satisfying Lipschitz condition, and optimal control systems with continuous-time which is governed by systems of ordinary differential equation, we derive results similar to Pontryagin’s maximum principle and properties concerning the generalized Jacobian set for optimal control problems of these systems.     

State constrained optimal control problems with time delays

In a recent, related, paper, necessary conditions in the form of a Maximum Principle were derived for optimal control problems with time delays in both state and control variables. Different versions of the necessary conditions covered fixed end-time problems and, under additional hypotheses, free end-time problems. These conditions improved on previous conditions in the following respects. They provided the first fully non-smooth Pontryagin Maximum Principle for problems involving delays in both state and control variables, only special cases of which were previously available. They provide a strong version of the Weierstrass condition for general problems with possibly non-commensurate control delays, whereas the earlier literature does so only under structural assumptions about the dynamic constraint. They also provided a new ‘two-sided’ generalized transversality condition, associated with the optimal end-time. This paper provides an extension of the Pontryagin Maximum Principle of the earlier paper for time delay systems, to allow for the presence of a unilateral state constraint. The new results fully recover the necessary conditions of the earlier paper when the state constraint is absent, and therefore retain all their advantages but in a setting of greater generality.     

Optimal control of impulsive hybrid systems

In this paper, we deal with optimization techniques for a class of hybrid systems that comprise continuous controllable dynamics and impulses (jumps) in the state. Using the mathematical techniques of distributional derivatives and impulse differential equations, we rewrite the original hybrid control system as a system with autonomous location transitions. For the obtained auxiliary dynamical system and the corresponding optimal control problem (OCP), we apply the Lagrange approach and derive the reduced gradient formulas. Moreover, we formulate necessary optimality conditions for the above hybrid OCPs, and discuss the newly elaborated Pontryagin-type Maximum Principle for impulsive OCPs. As in the case of the conventional OCPs, the proposed first order optimization techniques provide a basis for constructive computational algorithms.     

Numerical solution of minimax optimal control problems by multiple shooting technique

This paper presents the application of the multiple shooting technique to minimax optimal control problems (optimal control problems with Chebyshev performance index). A standard transformation is used to convert the minimax problem into an equivalent optimal control problem with state variable inequality constraints. Using this technique, the highly developed theory on the necessary conditions for state-restricted optimal control problems can be applied advantageously. It is shown that, in general, these necessary conditions lead to a boundary-value problem with switching conditions, which can be treated numerically by a special version of the multiple shooting algorithm. The method is tested on the problem of the optimal heating and cooling of a house. This application shows some typical difficulties arising with minimax optimal control problems, i.e., the estimation of the switching structure which is dependent on the parameters of the problem. This difficulty can be overcome by a careful application of a continuity method. Numerical solutions for the example are presented which demonstrate the efficiency of the method proposed.     

Necessary conditions for super minimizers in constrained multiobjective optimization

This paper concerns the study of the so-called super minimizers related to the concept of super efficiency in constrained problems of multiobjective optimization, where cost mappings are generally set-valued. We derive necessary conditions for super minimizers on the base of advanced tools of variational analysis and generalized differentiation that are new in both finite-dimensional and infinite-dimensional settings for problems with single-valued and set-valued objectives.      

Necessary conditions for optimal control problems with bounded state by a penalty method

Necessary conditions are derived for a general relaxed control problem with unilateral state constraint. The results are also valid for ordinary controls that are solutions of the relaxed problem.A penalty is imposed to change the constrained problem into a sequence of unconstrained problems. The assumptions are on the data of the problem and do not requirea priori verification of hypotheses involving the optimal solution.