Approximation of optimal feedback control: a dynamic programming approach

We consider the general continuous time finite-dimensional deterministic system under a finite horizon cost functional. Our aim is to calculate approximate solutions to the optimal feedback control. First we apply the dynamic programming principle to obtain the evolutive Hamilton–Jacobi–Bellman (HJB) equation satisfied by the value function of the optimal control problem. We then propose two schemes to solve the equation numerically. One is in terms of the time difference approximation and the other the time-space approximation. For each scheme, we prove that (a) the algorithm is convergent, that is, the solution of the discrete scheme converges to the viscosity solution of the HJB equation, and (b) the optimal control of the discrete system determined by the corresponding dynamic programming is a minimizing sequence of the optimal feedback control of the continuous counterpart. An example is presented for the time-space algorithm; the results illustrate that the scheme is effective.     

Optimal portfolio and consumption selection with default risk

We investigate an optimal portfolio and consumption choice problem with a defaultable security. Under the goal of maximizing the expected discounted utility of the average past consumption, a dynamic programming principle is applied to derive a pair of second-order parabolic Hamilton-Jacobi-Bellman (HJB) equations with gradient constraints. We explore these HJB equations by a viscosity solution approach and characterize the post-default and pre-default value functions as a unique pair of constrained viscosity solutions to the HJB equations.     

Optimal stopping of switching diffusions with state dependent switching rates

This paper is concerned with a continuous-time and infinite-horizon optimal stopping problem in switching diffusion models. In contrast to the assumption commonly made in the literature that the regime-switching is modeled by an independent Markov chain, we consider in this paper the case of state-dependent regime-switching. The Hamilton–Jacobi–Bellman (HJB) equation associated with the optimal stopping problem is given by a system of coupled variational inequalities. By means of the dynamic programming (DP) principle, we prove that the value function is the unique viscosity solution of the HJB system. As an interesting application in mathematical finance, we examine the problem of pricing perpetual American put options with state-dependent regime-switching. A numerical procedure is developed based on the DP approach and an efficient discrete tree approximation of the continuous asset price process. Numerical results are reported.     

A dynamical approach to the large-time behavior of solutions to weakly coupled systems of Hamilton–Jacobi equations

We investigate the large-time behavior of the value functions of the optimal control problems on the n-dimensional torus which appear in the dynamic programming for the system whose states are governed by random changes. From the point of view of the study on partial differential equations, it is equivalent to consider viscosity solutions of quasi-monotone weakly coupled systems of Hamilton–Jacobi equations. The large-time behavior of viscosity solutions of this problem has been recently studied by the authors and Camilli, Ley, Loreti, and Nguyen for some special cases, independently, but the general cases remain widely open. We establish a convergence result to asymptotic solutions as time goes to infinity under rather general assumptions by using dynamical properties of value functions.     

Conservative finite difference schemes for the generalized Zakharov–Kuznetsov equations

This paper is concerned with the construction of conservative finite difference schemes by means of discrete variational method for the generalized Zakharov–Kuznetsov equations and the numerical solvability of the two-dimensional nonlinear wave equations. A finite difference scheme is proposed such that mass and energy conservation laws associated with the generalized Zakharov–Kuznetsov equations hold. Our arguments are based on the procedure that D. Furihata has recently developed for real-valued nonlinear partial differential equations. Numerical results are given to confirm the accuracy as well as validity of the numerical solutions and then exhibit remarkable nonlinear phenomena of the interaction and behavior of pulse wave solutions.     

Explicit diffusive kinetic schemes for nonlinear degenerate parabolic systems

We design numerical schemes for nonlinear degenerate parabolic systems with possibly dominant convection. These schemes are based on discrete BGK models where both characteristic velocities and the source-term depend singularly on the relaxation parameter. General stability conditions are derived, and convergence is proved to the entropy solutions for scalar equations.

     

Trajectory-following algorithms for min-max optimization problems

In this paper, we consider a class of nonlinear minimum-maximum optimization problems subject to boundedness constraints on the decision vectors. Three algorithms are developed for finding the min-max point using the concept of solving an associated dynamical system. In the first and third algorithms, solutions are obtained by solving systems of differential equations. The second algorithm is a discrete version of the first algorithm. The trajectories generated by the first and second algorithms may move inside or on the boundary of the constraint set, while the third algorithm ensures that any trajectory that begins inside the constraint region remains in its interior. Sufficient conditions for global convergence of the two algorithms are also established. For illustration, four numerical examples are solved.This work was partially supported by a research grant from the Australian Research Committee.     

Finding all solutions of separable systems of piecewise-linear equations using integer programming

Finding all solutions of nonlinear or piecewise-linear equations is an important problem which is widely encountered in science and engineering. Various algorithms have been proposed for this problem. However, the implementation of these algorithms are generally difficult for non-experts or beginners. In this paper, an efficient method is proposed for finding all solutions of separable systems of piecewise-linear equations using integer programming. In this method, we formulate the problem of finding all solutions by a mixed integer programming problem, and solve it by a high-performance integer programming software such as GLPK, SCIP, or CPLEX. It is shown that the proposed method can be easily implemented without making complicated programs. It is also confirmed by numerical examples that the proposed method can find all solutions of medium-scale systems of piecewise-linear equations in practical computation time.     

Optimal Control for Utility Portfolio Selection with Liability

In this paper we use stochastic optimal control theory to investigate a dynamic portfolio selection problem with liability process, in which the liability process is assumed to be a geometric Brownian motion and completely correlated with stock prices. We apply dynamic programming principle to obtain Hamilton-Jacobi-Bellman (HJB) equations for the value function and systematically study the optimal investment strategies for power utility, exponential utility and logarithm utility. Firstly, the explicit expressions of the optimal portfolios for power utility and exponential utility are obtained by applying variable change technique to solve corresponding HJB equations. Secondly, we apply Legendre transform and dual approach to derive the optimal portfolio for logarithm utility. Finally, numerical examples are given to illustrate the results obtained and analyze the effects of the market parameters on the optimal portfolios.     

Optimal mean–variance efficiency of a family with life insurance under inflation risk

We study an optimization problem of a family under mean–variance efficiency. The market consists of cash, a zero-coupon bond, an inflation-indexed zero-coupon bond, a stock, life insurance and income-replacement insurance. The instantaneous interest rate is modeled as the Cox–Ingersoll–Ross (CIR) model, and we use a generalized Black–Scholes model to characterize the stock and labor income. We also take into account the inflation risk and consider our problem in the real market. The goal of the family is to maximize the mean of the surplus wealth at the retirement or death of the breadwinner and minimize its variance by finding a portfolio selection. The efficient frontier and optimal strategies are derived through the dynamic programming method and the technique of solving associated nonlinear HJB equations. We also present a numerical illustration to explore the impact of economical parameters on the efficient frontier.     

Nonlinear Potentials for Hamilton–Jacobi–Bellman Equations. II

A formal method of constructing the viscosity solutions for abstract nonlinear equations of Hamilton–Jacobi–Bellman (HJB) type was developed in the previous work of the author. A new advantage of this method (which was called an nonlinear potentials method) is that it gives a possibility to choose at the first step an expected regularity of the solution and then – to construct this solution. This makes the whole procedure more simple because an analysis of regularity of viscosity solutions is usually the most complicated step.Nonlinear potentials method is a generalization of Krylov's approach to study HJB equations.In this article nonlinear potentials method is applied to elliptic degenerate HJB equations in R^d with variable coefficients.     

Numerical solution of a minimax ergodic optimal control problem

In this work we consider an L^∞ minimax ergodic optimal control problem with cumulative cost. We approximate the cost function as a limit of evolutions problems. We present the associated Hamilton-Jacobi-Bellman equation and we prove that it has a unique solution in the viscosity sense. As this HJB equation is consistent with a numerical procedure, we use this discretization to obtain a procedure for the primitive problem. For the numerical solution of the ergodic version we need a perturbation of the instantaneous cost function. We give an appropriate selection of the discretization and penalization parameters to obtain discrete solutions that converge to the optimal cost. We present numerical results. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

VISCOSITY SOLUTIONS OF HJB EQUATIONS ARISING FROM THE VALUATION OF EUROPEAN PASSPORT OPTIONS

The passport option is a call option on the balance of a trading account. The option holder retains the gain from trading, while the issuer is liable for the net loss. In this article, the mathematical foundation for pricing the European passport option is established. The pricing equation which is a fully nonlinear equation is derived using the dynamic programming principle. The comparison principle, uniqueness and convexity preserving of the viscosity solutions of related H J13 equation are proved. A relationship between the passport and lookback options is discussed.     

Entropy solution theory for fractional degenerate convection–diffusion equations

We study a class of degenerate convection-diffusion equations with a fractional non-linear diffusion term. This class is a new, but natural, generalization of local degenerate convection-diffusion equations, and include anomalous diffusion equations, fractional conservation laws, fractional porous medium equations, and new fractional degenerate equations as special cases. We define weak entropy solutions and prove well-posedness under weak regularity assumptions on the solutions, e.g. uniqueness is obtained in the class of bounded integrable solutions. Then we introduce a new monotone conservative numerical scheme and prove convergence toward the entropy solution in the class of bounded integrable BV functions. The well-posedness results are then extended to non-local terms based on general Lévy operators, connections to some fully non-linear HJB equations are established, and finally, some numerical experiments are included to give the reader an idea about the qualitative behavior of solutions of these new equations.     

Optimal control of stochastic functional differential equations with a bounded memory

This paper treats a finite time horizon optimal control problem in which the controlled state dynamics are governed by a general system of stochastic functional differential equations with a bounded memory. An infinite dimensional Hamilton–Jacobi–Bellman (HJB) equation is derived using a Bellman-type dynamic programming principle. It is shown that the value function is the unique viscosity solution of the HJB equation.     

Optimal investment–consumption problem: Post-retirement with minimum guarantee

We study the optimal investment–consumption problem for a member of defined contribution plan during the decumulation phase. For a fixed annuitization time, to achieve higher final annuity, we consider a variable consumption rate. Moreover, to have a minimum guarantee for the final annuity, a safety level for the wealth process is considered. To solve the stochastic optimal control problem via dynamic programming, we obtain a Hamilton–Jacobi–Bellman (HJB) equation on a bounded domain. The existence and uniqueness of classical solutions are proved through the dual transformation. We apply the finite difference method to find numerical approximations of the solution of the HJB equation. Finally, the simulation results for the optimal investment–consumption strategies, optimal wealth process and the final annuity for different admissible ranges of consumption are given. Furthermore, by taking into account the market present value of the cash flows before and after the annuitization, we compare the outcomes of different scenarios.     

Improved interpolation inequalities,relative entropy and fast diffusion equations

We consider a family of Gagliardo–Nirenberg–Sobolev interpolation inequalities which interpolate between Sobolev?s inequality and the logarithmic Sobolev inequality, with optimal constants. The difference of the two terms in the interpolation inequalities (written with optimal constant) measures a distance to the manifold of the optimal functions. We give an explicit estimate of the remainder term and establish an improved inequality, with explicit norms and fully detailed constants. Our approach is based on nonlinear evolution equations and improved entropy–entropy production estimates along the associated flow. Optimizing a relative entropy functional with respect to a scaling parameter, or handling properly second moment estimates, turns out to be the central technical issue. This is a new method in the theory of nonlinear evolution equations, which can be interpreted as the best fit of the solution in the asymptotic regime among all asymptotic profiles.     

A Deterministic Approach To Stochastic Optimal Control With Application To Anticipative Control

Using the decomposition of solution of SDE, we consider the stochastic optimal control problem with anticipative controls as a family of deterministic control problems parametrized by the paths of the driving Wiener process and of a newly introduced Lagrange multiplier stochastic process (nonanticipativity equality constraint). It is shown that the value function of these problems is the unique global solution of a robust equation (random partial differential equation) associated to a linear backward Hamilton-Jacobi-Bellman stochastic partial differential equation (HJB SPDE). This appears as limiting SPDE for a sequence of random HJB PDE's when linear interpolation approximation of the Wiener process is used. Our approach extends the Wong-Zakai type results [20] from SDE to the stochastic dynamic programming equation by showing how this arises as average of the limit of a sequence of deterministic dynamic programming equations. The stochastic characteristics method of Kunita [13] is used to represent the value function. By choosing the Lagrange multiplier equal to its nonanticipative constraint value the usual stochastic (nonanticipative) optimal control and optimal cost are recovered. This suggests a method for solving the anticipative control problems by almost sure deterministic optimal control. We obtain a PDE for the “cost of perfect information” the difference between the cost function of the nonanticipative control problem and the cost of the anticipative problem which satisfies a nonlinear backward HJB SPDE. Poisson bracket conditions are found ensuring this has a global solution. The cost of perfect information is shown to be zero when a Lagrangian submanifold is invariant for the stochastic characteristics. The LQG problem and a nonlinear anticipative control problem are considered as examples in this framework     

Delta shock waves with Dirac delta function in both components for systems of conservation laws

We study a class of non-strictly and weakly hyperbolic systems of conservation laws which contain the equations of geometrical optics as a prototype. The Riemann problems are constructively solved. The Riemann solutions include two kinds of interesting structures. One involves a cavitation where both state variables tend to zero forming a singularity, the other is a delta shock wave in which both state variables contain Dirac delta function simultaneously. The generalized Rankine–Hugoniot relation and entropy condition are proposed to solve the delta shock wave. Moreover, with the limiting viscosity approach, we show all of the existence, uniqueness and stability of solution involving the delta shock wave. The generalized Rankine–Hugoniot relation is also confirmed. Then our theory is successfully applied to two typical systems including the geometric optics equations. Finally, we present the numerical results coinciding with the theoretical analysis.     

SEQUENTIAL SYSTEMS OF LINEAR EQUATIONS ALGORITHM FOR NONLINEAR OPTIMIZATION PROBLEMS-INEQUALITY CONSTRAINED PROBLEMS

AbstractIn this paper, a new superlinearly convergent algorithm of sequential systems of linear equations (SSLE) for nonlinear optimization problems with inequality constraints is proposed. Since the new algorithm only needs to solve several systems of linear equations having a same coefficient matrix per iteration, the computation amount of the algorithm is much less than that of the existing SQP algorithms per iteration. Moreover, for the SQP type algorithms, there exist so-called inconsistent problems, i.e., quadratic programming subproblems of the SQP algorithms may not have a solution at some iterations, but this phenomenon will not occur with the SSLE algorithms because the related systems of linear equations always have solutions. Some numerical results are reported.