A nonlinear stochastic growth model on discrete time domains

In this paper, we introduce nonlinear stochastic dynamic problems on discrete time domains where events may occur at unevenly spaced time points. We define Euler equation and transversality condition for the problem. We prove that the Euler equation and the transversality condition are sufficient for the existence of the optimal solution. Next we generalize discrete time Cagan type rational expectation model to multivariate case. As an application of the main results, we obtain an explicit solution to a log-linearized nonlinear stochastic growth model.     

Two Characterizations of Optimality in Dynamic Programming

It holds in great generality that a plan is optimal for a dynamic programming problem, if and only if it is “thrifty” and “equalizing.” An alternative characterization of an optimal plan, that applies in many economic models, is that the plan must satisfy an appropriate Euler equation and a transversality condition. Here we explore the connections between these two characterizations.     

Viscosity solutions for the dynamic programming equations

In a previous paper the author has introduced a new notion of a (generalized) viscosity solution for Hamilton-Jacobi equations with an unbounded nonlinear term. It is proved here that the minimal time function (resp. the optimal value function) for time optimal control problems (resp. optimal control problems) governed by evolution equations is a (generalized) viscosity solution for the Bellman equation (resp. the dynamic programming equation). It is also proved that the Neumann problem in convex domains may be viewed as a Hamilton-Jacobi equation with a suitable unbounded nonlinear term.     

ON THE VISCOSITY SPLITT NG METHOD FOR INITIAL BOUNDARY VALUE PROBLEMS OF THE NAVIAR-STOKES EQUATIONS

The viscosily splitting method for the Navier-Stokes equations on two dimensional multi-connected domains is considered. The equation is split into an Euler equation and a non-stationary Stekes equation within each time step. The author proves the convergence theorem as he has done for the problem on simply connected domains, and the rate of convergence is improved from loss than 1/4 to 1.     

A note on the dynamic liquidity trading problem with a mean-variance objective

This note studies the dynamic liquidity trader’s problem with a mean-variance objective function. Independent of the market impact functions and the market price dynamics, we provide a necessary and sufficient condition under which the dynamic programming equation (Bellman equation) can be extended to mean-variance objectives. Evaluation of this condition involves solving an optimization problem and taking variance of its optimal value. This computation may be difficult even when random disturbances in the market price dynamics follow a well-known distribution. To avoid this pitfall, we then provide some sufficient condition which can be assessed very easily.     

Optimal control of stochastic functional differential equations with a bounded memory

This paper treats a finite time horizon optimal control problem in which the controlled state dynamics are governed by a general system of stochastic functional differential equations with a bounded memory. An infinite dimensional Hamilton–Jacobi–Bellman (HJB) equation is derived using a Bellman-type dynamic programming principle. It is shown that the value function is the unique viscosity solution of the HJB equation.     

A note on continuous dependence of solutions of dynamic equations on time scales

We give a precise formulation and a proof as constructive as possible of the widely accepted claim that solutions of a dynamic equation depend continuously on the base time scale. Our approach to this problem is via Euler polygons which opens possibilities for development of numerical analysis of dynamic equations on time scales.     

Maximum principle for optimal distributed control of viscous weakly dispersive Degasperis–Procesi equation

This paper is concerned with the optimal distributed control of the viscous weakly dispersive Degasperis–Procesi equation in nonlinear shallow water dynamics. It is well known that the Pontryagin maximum principle, which unifies calculus of variations and control theory of ordinary differential equations, sets up the theoretical basis of the modern optimal control theory along with the Bellman dynamic programming principle. In this paper, we commit ourselves to infinite dimensional generalizations of the maximum principle and aim at the optimal control theory of partial differential equations. In contrast to the finite dimensional setting, the maximum principle for the infinite dimensional system does not generally hold as a necessary condition for optimal control. By the Dubovitskii and Milyutin functional analytical approach, we prove the Pontryagin maximum principle of the controlled viscous weakly dispersive Degasperis–Procesi equation. The necessary optimality condition is established for the problem in fixed final horizon case. Finally, a remark on how to utilize the obtained results is also made. Copyright © 2015 John Wiley & Sons, Ltd.     

American option valuation in a stochastic volatility model with transaction costs

In the present paper we analyse the American option valuation problem in a stochastic volatility model when transaction costs are taken into account. We shall show that it can be formulated as a singular stochastic optimal control problem, proving the existence and uniqueness of the viscosity solution for the associated Hamilton–Jacobi–Bellman partial differential equation. Moreover, after performing a dimensionality reduction through a suitable choice of the utility function, we shall provide a numerical example illustrating how American options prices can be computed in the present modelling framework.     

Regularity of Nash payoffs of Markovian nonzero-sum stochastic differential games

In this paper we deal with the problem of existence of a smooth solution of the Hamilton–Jacobi–Bellman–Isaacs (HJBI for short) system of equations associated with nonzero-sum stochastic differential games. We consider the problem in unbounded domains either in the case of continuous generators or for discontinuous ones. In each case we show the existence of a smooth solution of the system. As a consequence, we show that the game has smooth Nash payoffs which are given by means of the solution of the HJBI system and the stochastic process which governs the dynamic of the controlled system.     

On Neumann problems for nonlocal Hamilton–Jacobi equations with dominating gradient terms

We are concerned with the well-posedness of Neumann boundary value problems for nonlocal Hamilton–Jacobi equations related to jump processes in general smooth domains. We consider a nonlocal diffusive term of censored type of order strictly less than 1 and Hamiltonians both in coercive form and in noncoercive Bellman form, whose growth in the gradient make them the leading term in the equation. We prove a comparison principle for bounded sub-and supersolutions in the context of viscosity solutions with generalized boundary conditions, and consequently by Perron’s method we get the existence and uniqueness of continuous solutions. We give some applications in the evolutive setting, proving the large time behaviour of the associated evolutive problem under suitable assumptions on the data.     

Mean square stability for systems on stochastically generated discrete time scales

We consider dynamic systems which evolve on discrete time domains where the time steps form a sequence of independent, identically distributed random variables. In particular, we classify the mean-square stability of linear systems on these time domains using quadratic Lyapunov functionals. In the case where the system matrix is a function of the time step, our results agree with and generalize stability results found in the Markov jump linear systems literature. In the case where the system matrix is constant, our results generalize, illuminate, and extend to the stochastic realm results in the field of dynamic equations on time scales. In order to help see the factors that contribute to stability, we prove a sufficient condition for the solvability of the Lyapunov equation by appealing to a fixed point theorem of Ran and Reurings. Finally, an example using observer-based feedback control is presented to demonstrate the utility of the results to control engineers who cannot guarantee uniform timing of the system.     

The 3D compressible Euler equations with damping in a bounded domain

We proved global existence and uniqueness of classical solutions to the initial boundary value problem for the 3D damped compressible Euler equations on bounded domain with slip boundary condition when the initial data is near its equilibrium. Time asymptotically, the density is conjectured to satisfy the porous medium equation and the momentum obeys to the classical Darcy's law. Based on energy estimate, we showed that the classical solution converges to steady state exponentially fast in time. We also proved that the same is true for the related initial boundary value problem of porous medium equation and thus justified the validity of Darcy's law in large time.     

A Long Wave Approximation for Capillary-Gravity Waves and an Effect of the Bottom

The Korteweg–de Vries (KdV) equation is known as a model of long waves in an infinitely long canal over a flat bottom and approximates the 2-dimensional water wave problem, which is a free boundary problem for the incompressible Euler equation with the irrotational condition. In this article, we consider the validity of this approximation in the case of the presence of the surface tension. Moreover, we consider the case where the bottom is not flat and study an effect of the bottom to the long wave approximation. We derive a system of coupled KdV like equations and prove that the dynamics of the full problem can be described approximately by the solution of the coupled equations for a long time interval. We also prove that if the initial data and the bottom decay at infinity in a suitable sense, then the KdV equation takes the place of the coupled equations.     

Probabilistic interpretation of a coupled system of Hamilton–Jacobi–Bellman equations

In this paper, we give a probabilistic interpretation for a coupled system of Hamilton–Jacobi–Bellman equations using the value function of a stochastic control problem. First we introduce this stochastic control problem. Then we prove that the value function of this problem is deterministic and satisfies a (strong) dynamic programming principle. And finally, the value function is shown to be the unique viscosity solution of the coupled system of Hamilton–Jacobi–Bellman equations.     

A Time Finite Element Method Based on the Generalized Variational Calculus

In this contribution, a new finite element method in the temporal domain is presented, in which the time step size is introduced as an additional variable. Thus, the variation of the time integral of the Lagrangean resulting from Hamilton's principle has to be carried out with respect to the rules of the generalized variational calculus. Apart from the usual time integral of the Euler‐Lagrange differential equations, the so‐called transversality condition is obtained as an additional result representing a time‐boundary term, which is used to obtain an optimal step size in the time domain.     

Contractivity of Bellman operator in risk averse dynamic programming with infinite horizon

The paper deals with a risk averse dynamic programming problem with infinite horizon. First, the required assumptions are formulated to have the problem well defined. Then the Bellman equation is derived, which may be also seen as a standalone reinforcement learning problem. The fact that the Bellman operator is contraction is proved, guaranteeing convergence of various solution algorithms used for dynamic programming as well as reinforcement learning problems, which we demonstrate on the value iteration and the policy iteration algorithms.     

Nonsmooth maximum principle for infinite-horizon problems

In this paper, we consider a class of infinite-horizon discounted optimal control problems with nonsmooth problem data. A maximum principle in terms of differential inclusions with a Michel type transversality condition is given. It is shown that, when the discount rate is sufficiently large, the problem admits normal multipliers and a strong transversality condition holds. A relationship between dynamic programming and the maximum principle is also given.The author is indebted to Francis Clarke for helpful suggestions and discussions.     

On the existence of an optimal feedback control for stochastic systems

We prove the existence of an optimal control for systems of stochastic differential equations without solving the Bellman dynamic programming equation. Instead, we use direct methods for solving extremal problems.     

Homogenization of Incompressible Euler Equations

In this paper, we perform a nonlinear multiscale analysis for incompressible Euler equations with rapidly oscillating initial data. The initial condition for velocity field is assumed to have two scales. The fast scale velocity component is periodic and is of order one.One of the important questions is how the two-scale velocity structure propagates in time and whether nonlinear interaction will generate more scales dynamically. By using a Lagrangian framework to describe the propagation of small scale solution, we show that the two-scale structure is preserved dynamically. Moreover, we derive a well-posed homogenized equation for the incompressible Euler equations. Preliminary numerical experiments are presented to demonstrate that the homogenized equation captures the correct averaged solution of the incompressible Euler equation.