Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method

We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov??s approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPs associated to stochastic gene networks. Explicit computations are given for Cook??s model for gene expression.     

Infinite horizon noncooperative differential games

For a noncooperative differential game, the value functions of the various players satisfy a system of Hamilton-Jacobi equations. In the present paper, we consider a class of infinite horizon games with nonlinear costs exponentially discounted in time. By the analysis of the value functions, we establish the existence of Nash equilibrium solutions in feedback form and provide results and counterexamples on their uniqueness and stability.     

Saddle point methods,and alogorithms,for non-symmetric linear equations ∗

This paper describes methods for solving non-singular, non-symmetric linear equations whose symmetric part is positive definite. First, the solutions are characterized as saddle points of a convex-concave function. The associated primal and dual variational principles provide quadratic, strictly convex, functions whose minima are the solutions of the original equation and which generalize the energy function for symmetric problems.

Direct iterative methods for finding the saddle point are then developed and analyzed. A globally convergent algorithm for finding the saddle points is described. We show that requiring conjugacy of successive search directions with respect to the symmetric part of the equation is a poor strategy.     

Uniqueness results for a class of hamilton-jacobi equations with singular coefficients

We establish uniqueness or comparison results for a class of Hamilton-Jacobi equations and give characterizations of maximal solutions of Hamilton-Jacobi equations. The results are applied to characterizing value functions of exit time problems in optimal control.     

Explicit Hopf-Lax type formulas for Hamilton-Jacobi equations and conservation laws with discontinuous coefficients

We deal with a Hamilton-Jacobi equation with a Hamiltonian that is discontinuous in the space variable. This is closely related to a conservation law with discontinuous flux. Recently, an entropy framework for single conservation laws with discontinuous flux has been developed which is based on the existence of infinitely many stable semigroups of entropy solutions based on an interface connection. In this paper, we characterize these infinite classes of solutions in terms of explicit Hopf-Lax type formulas which are obtained from the viscosity solutions of the corresponding Hamilton-Jacobi equation with discontinuous Hamiltonian. This also allows us to extend the framework of infinitely many classes of solutions to the Hamilton-Jacobi equation and obtain an alternative representation of the entropy solutions for the conservation law. We have considered the case where both the Hamiltonians are convex (concave). Furthermore, we also deal with the less explored case of sign changing coefficients in which one of the Hamiltonians is convex and the other concave. In fact in convex-concave case we cannot expect always an existence of a solution satisfying Rankine-Hugoniot condition across the interface. Therefore the concept of generalised Rankine-Hugoniot condition is introduced and prove existence and uniqueness.     

Externality and Hamilton-Jacobi equations

The relationship between optimal control problems and Hamilton-Jacobi-Bellman equations is well known [9]. In fact the value function, defined as the infimum of the cost functional, satisfies in the viscosity sense an appropriate Hamilton-Jacobi-Bellman equation. In this paper we consider several control problems such that the cost functional associated to each problem depends explicitly on the value functions of the other problems. This leads to a system of Hamilton-Jacobi-Bellman equations. This is known, in economic context [14] cap XI, as an externality problem. In these problems may occur a lack of uniqueness of the value functions. We give conditions to ensure existence, uniqueness of the value functions and an implicit integral representation formula. Moreover, under uniqueness assumption, we prove that the variational solutions of the associated Hamilton-Jacobi system converge asymptotically to the value functions. We prove also an uniqueness theorem in the case of viscosity solutions of Hamilton-Jacobi-Bellman system.     

The problem of optimal control with reflection studied through a linear optimization problem stated on occupational measures

We obtain a linear programming characterization for the minimum cost associated with finite dimensional reflected optimal control problems. In order to describe the value functions, we employ an infinite dimensional dual formulation instead of using the characterization via Hamilton-Jacobi partial differential equations. In this paper we consider control problems with both infinite and finite horizons. The reflection is given by the normal cone to a proximal retract set.     

On the Bellman Equation for Infinite Horizon Problems with Unbounded Cost Functional

We study a class of infinite horizon control problems for nonlinear systems, which includes the Linear Quadratic (LQ) problem, using the Dynamic Programming approach. Sufficient conditions for the regularity of the value function are given. The value function is compared with sub- and supersolutions of the Bellman equation and a uniqueness theorem is proved for this equation among locally Lipschitz functions bounded below. As an application it is shown that an optimal control for the LQ problem is nearly optimal for a large class of small unbounded nonlinear and nonquadratic pertubations of the same problem. Accepted 8 October 1998     

Duality in infinite dimensional linear programming

We consider the class of linear programs with infinitely many variables and constraints having the property that every constraint contains at most finitely many variables while every variable appears in at most finitely many constraints. Examples include production planning and equipment replacement over an infinite horizon. We form the natural dual linear programming problem and prove strong duality under a transversality condition that dual prices are asymptotically zero. That is, we show, under this transversality condition, that optimal solutions are attained in both primal and dual problems and their optimal values are equal. The transversality condition, and hence strong duality, is established for an infinite horizon production planning problem.This material is based on work supported by the National Science Foundation under Grant No. ECS-8700836.     

Conjugate maps and duality in multiobjective optimization

This paper considers duality in convex vector optimization. A vector optimization problem requires one to find all the efficient points of the attainable value set for given multiple objective functions. Embedding the primal problem into a family of perturbed problems enables one to define a dual problem in terms of the conjugate map of the perturbed objective function. Every solution of the stable primal problem is associated with a certain solution of the dual problem, which is characterized as a subgradient of the perturbed efficient value map. This pair of solutions also provides a saddle point of the Lagrangian map.     

On the Interpretation of Optimal Dual Solutions in Convex Programming

The relationships between multiple optimal dual solutions of a convex programming problem and the corresponding primal optimal value function are established by straightforward arguments on known results in duality theory. The subsequent discussion includes a demonstration that one-sided shadow prices can be found by solving a linear programming problem, conditions for the uniqueness and validity of the classical shadow price interpretation and a specialization of the results to the linear programming case.     

Differential sensitivity of solutions of convex constrained optimal control problems for discrete systems

A family of optimal control problems for discrete systems that depend on a real parameter is considered. The problems are strongly convex and subject to state and control constraints. Some regularity conditions are imposed on the constraints.The control problems are reformulated as mathematical programming problems. It is shown that both the primal and dual optimal variables for these problems are right-differentiable functions of a parameter. The right-derivatives are characterized as solutions to auxiliary quadratic control problems. Conditions of continuous differentiability are discussed, and some estimates of the rate of convergence of the difference quotients to the respective derivatives are given.     

Bounded-from-below solutions of the Hamilton-Jacobi equation for optimal control problems with exit times: vanishing lagrangians,eikonal equations,and shape-from-shading

We study the Hamilton-Jacobi equation for undiscounted exit time control problems with general nonnegative Lagrangians using the dynamic programming approach. We prove theorems characterizing the value function as the unique bounded-from-below viscosity solution of the Hamilton-Jacobi equation that is null on the target. The result applies to problems with the property that all trajectories satisfying a certain integral condition must stay in a bounded set. We allow problems for which the Lagrangian is not uniformly bounded below by positive constants, in which the hypotheses of the known uniqueness results for Hamilton-Jacobi equations are not satisfied. We apply our theorems to eikonal equations from geometric optics, shape-from-shading equations from image processing, and variants of the Fuller Problem.     

A numerical approach to the infinite horizon problem of deterministic control theory

We are concerned with the Hamilton-Jacobi equation related to the infinite horizon problem of deterministic control theory. Approximate solutions are constructed by means of a discretization in time as well as in the state variable and we prove that their rate of convergence to the viscosity solution is of order 1, provided a semiconcavity assumption is satisfied. A computational algorithm, originally due to R. Gonzales and E. Rofman, is adapted and reformulated for the problem at hand in order to obtain an error estimate for the numerical approximate solutions.This work has been partially supported by CNR-GNAFA.     

Conjugate Duality and the Control of Linear Discrete Systems

In this paper we deal with the minimization of a convex function over the solution set of a range inclusion problem determined by a multivalued operator with convex graph. We attach a dual problem to it, provide regularity conditions guaranteeing strong duality and derive for the resulting primal–dual pair necessary and sufficient optimality conditions. We also discuss the existence of optimal solutions for the primal and dual problems by using duality arguments. The theoretical results are applied in the context of the control of linear discrete systems.     

A comparison of alternative c-conjugate dual problems in infinite convex optimization

In this work, we obtain a Fenchel–Lagrange dual problem for an infinite dimensional optimization primal one, via perturbational approach and using a conjugation scheme called c-conjugation instead of classical Fenchel conjugation. This scheme is based on the generalized convex conjugation theory. We analyse some inequalities between the optimal values of Fenchel, Lagrange and Fenchel–Lagrange dual problems and we establish sufficient conditions under which they are equal. Examples where such inequalities are strictly fulfilled are provided. Finally, we study the relations between the optimal solutions and the solvability of the three mentioned dual problems.     

Viscosity solutions for the dynamic programming equations

In a previous paper the author has introduced a new notion of a (generalized) viscosity solution for Hamilton-Jacobi equations with an unbounded nonlinear term. It is proved here that the minimal time function (resp. the optimal value function) for time optimal control problems (resp. optimal control problems) governed by evolution equations is a (generalized) viscosity solution for the Bellman equation (resp. the dynamic programming equation). It is also proved that the Neumann problem in convex domains may be viewed as a Hamilton-Jacobi equation with a suitable unbounded nonlinear term.     

Lower bounds and duality in the optimal control of distributed systems

We develop a duality theory for problems of minimization of a convex functional on a convex set and apply this theory to optimal control problems with non-differentiable functional and state as well as control constraints. The dual problem is sometimes found to be simpler than the primal problem. The equivalence of the primal and the dual problem is established for problems in which the functional is strongly convex. A posteriori error are also given for such problems.     

Uniqueness of unbounded viscosity solutions for impulse control problem

We study here the impulse control problem in infinite as well as finite horizon. We allow the cost functionals and dynamics to be unbounded and hence the value function can possibly be unbounded. We prove that the value function is the unique viscosity solution in a suitable subclass of continuous functions, of the associated quasivariational inequality. Our uniqueness proof for the infinite horizon problem uses stopping time problem and for the finite horizon problem, comparison method. However, we assume proper growth conditions on the cost functionals and the dynamics.     

Optimal Solutions of Linear Periodic Control Systems with Convex Integrands

In this work we study the existence and asymptotic behavior of overtaking optimal trajectories for linear control systems with convex integrands. We extend the results obtained by Artstein and Leizarowitz for tracking periodic problems with quadratic integrands [2] and establish the existence and uniqueness of optimal trajectories on an infinite horizon. The asymptotic dynamics of finite time optimizers is examined. Accepted 31 January 1996