Optimization of Nonlinear Systems of Stochastic Difference Equations

We present new results concerning the synthesis of optimal control for systems of difference equations that depend on a semi-Markov or Markov stochastic process. We obtain necessary conditions for the optimality of solutions that generalize known conditions for the optimality of deterministic systems of control. These necessary optimality conditions are obtained in the form convenient for the synthesis of optimal control. On the basis of Lyapunov stochastic functions, we obtain matrix difference equations of the Riccati type, the integration of which enables one to synthesize an optimal control. The results obtained generalize results obtained earlier for deterministic systems of difference equations.     

A discrete optimality system for an optimal harvesting problem

In this paper, we obtain the discrete optimality system of an optimal harvesting problem. While maximizing a combination of the total expected utility of the consumption and of the terminal size of a population, as a dynamic constraint, we assume that the density of the population is modeled by a stochastic quasi-linear heat equation. Finite-difference and symplectic partitioned Runge–Kutta (SPRK) schemes are used for space and time discretizations, respectively. It is the first time that a SPRK scheme is employed for the optimal control of stochastic partial differential equations. Monte-Carlo simulation is applied to handle expectation appearing in the cost functional. We present our results together with a numerical example. The paper ends with a conclusion and an outlook to future studies, on further research questions and applications.     

Multilevel Monte Carlo method for parabolic stochastic partial differential equations

We analyze the convergence and complexity of multilevel Monte Carlo discretizations of a class of abstract stochastic, parabolic equations driven by square integrable martingales. We show under low regularity assumptions on the solution that the judicious combination of low order Galerkin discretizations in space and an Euler–Maruyama discretization in time yields mean square convergence of order one in space and of order 1/2 in time to the expected value of the mild solution. The complexity of the multilevel estimator is shown to scale log-linearly with respect to the corresponding work to generate a single path of the solution on the finest mesh, resp. of the corresponding deterministic parabolic problem on the finest mesh.     

Brownian and fractional Brownian stochastic currents via Malliavin calculus

By using Malliavin calculus and multiple Wiener-Itô integrals, we study the existence and the regularity of stochastic currents defined as Skorohod (divergence) integrals with respect to the Brownian motion and to the fractional Brownian motion. We consider also the multidimensional multiparameter case and we compare the regularity of the current as a distribution in negative Sobolev spaces with its regularity in the Watanabe spaces.     

Optimal greedy policies for stochastic control models

We introduce the notion of a greedy policy for general stochastic control models. Sufficient conditions for the optimality of the greedy policy for finite and infinite horizon are given. Moreover, we derive error bounds if the greedy policy is not optimal. The main results are illustrated by Bayesian information models, discounted Bayesian search problems, stochastic scheduling problems, single-server queueing networks and deterministic dynamic programs.     

Analysis and finite element approximations for distributed optimal control problems for implicit parabolic equations

This work concerns analysis and error estimates for optimal control problems related to implicit parabolic equations. The minimization of the tracking functional subject to implicit parabolic equations is examined. Existence of an optimal solution is proved and an optimality system of equations is derived. Semi-discrete (in space) error estimates for the finite element approximations of the optimality system are presented. These estimates are symmetric and applicable for higher-order discretizations. Finally, fully-discrete error estimates of arbitrarily high-order are presented based on a discontinuous Galerkin (in time) and conforming (in space) scheme. Two examples related to the Lagrangian moving mesh Galerkin formulation for the convection-diffusion equation are described.     

Operator differential-algebraic equations with noise arising in fluid dynamics

We study linear semi-explicit stochastic operator differential algebraic equations (DAEs) for which the constraint equation is given in an explicit form. In particular, this includes the Stokes equations arising in fluid dynamics. We combine a white noise polynomial chaos expansion approach to include stochastic perturbations with deterministic regularization techniques. With this, we are able to include Gaussian noise and stochastic convolution terms as perturbations in the differential as well as in the constraint equation. By the application of the polynomial chaos expansion method, we reduce the stochastic operator DAE to an infinite system of deterministic operator DAEs for the stochastic coefficients. Since the obtained system is very sensitive to perturbations in the constraint equation, we analyze a regularized version of the system. This then allows to prove the existence and uniqueness of the solution of the initial stochastic operator DAE in a certain weighted space of stochastic processes.     

Itô’s stochastic calculus and Heisenberg commutation relations

Stochastic calculus and stochastic differential equations for Brownian motion were introduced by K. Itô in order to give a pathwise construction of diffusion processes. This calculus has deep connections with objects such as the Fock space and the Heisenberg canonical commutation relations, which have a central role in quantum physics. We review these connections, and give a brief introduction to the noncommutative extension of Itô’s stochastic integration due to Hudson and Parthasarathy. Then we apply this scheme to show how finite Markov chains can be constructed by solving stochastic differential equations, similar to diffusion equations, on the Fock space.     

Wick-stochastic finite element solution of reaction–diffusion problems

Real life reaction–diffusion problems are characterized by their inherent or externally induced uncertainties in the design parameters. This paper presents a finite element solution of reaction–diffusion equations of Wick type. Using the Wick-product properties and the Wiener–Itô chaos expansion, the stochastic variational problem is reformulated to a set of deterministic variational problems. To obtain the chaos coefficients in the corresponding deterministic reaction–diffusion, we implement the usual Galerkin finite element method using standard techniques. Once this representation is computed, the statistics of the numerical solution can be easily evaluated. Computational results are shown for one- and two-dimensional test examples.     

Stochastic asymptotical stability for stochastic impulsive differential equations and it is application to chaos synchronization

In this paper, the stochastic asymptotical stability of stochastic impulsive differential equations is studied, and a comparison theory about the stochastic asymptotical stability of trivial solution is established. From the comparison theory, we can find out whether the stochastic impulsive differential system is stochastic asymptotically stable by studying the stability of a deterministic comparison system. As an application of this theory, we study the problem of chaos synchronization in Chua circuit using impulsive method. Finally, numerical simulation is employed to verify the feasibility of our method.     

A maximum principle for fully coupled stochastic control systems of mean-field type

The present paper considers an optimal control problem for fully coupled forward–backward stochastic differential equations (FBSDEs) of mean-field type in the case of controlled diffusion coefficient. Moreover, the control domain is not assumed to be convex. By virtue of a reduction method, we establish the necessary optimality conditions of Pontryagin's type. As an application, a linear–quadratic stochastic control problem is studied.     

Malliavin calculus and decoupling inequalities in Banach spaces

We develop a theory of Malliavin calculus for Banach space-valued random variables. Using radonifying operators instead of symmetric tensor products we extend the Wiener-Itô isometry to Banach spaces. In the white noise case we obtain two sided Lp-estimates for multiple stochastic integrals in arbitrary Banach spaces. It is shown that the Malliavin derivative is bounded on vector-valued Wiener-Itô chaoses. Our main tools are decoupling inequalities for vector-valued random variables. In the opposite direction we use Meyer's inequalities to give a new proof of a decoupling result for Gaussian chaoses in UMD Banach spaces.     

The maximum principle for the nonlinear stochastic optimal control problem of switching systems

The aim of this paper is to present a stochastic maximum principle for an optimal control problem of switching systems. It presents necessary conditions of optimality in the form of a maximum principle for stochastic switching systems, in which the dynamic of the constituent processes takes the form of stochastic differential equations. The restrictions on transitions for the system are described through equality constraints.     

Optimal feedback control policies for stochastic,distributed-parameter systems

This paper considers optimal feedback control policies for a class of discrete stochastic distributed-parameter systems. The class under consideration has the property that the random variable in the dynamic systems depends only on the time and possesses the Markovian property with stationary transition probabilities. A necessary condition for optimality of a feedback control policy, which has form similar to the Hamiltonian form in the deterministic case, is derived via a dynamic programming approach.     

Necessary conditions for stochastic optimal control problems in infinite dimensions

The purpose of this paper is to establish the first and second order necessary conditions for stochastic optimal controls in infinite dimensions. The control system is governed by a stochastic evolution equation, in which both drift and diffusion terms may contain the control variable and the set of controls is allowed to be nonconvex. Only one adjoint equation is introduced to derive the first order necessary optimality condition either by means of the classical variational analysis approach or, under an additional assumption, by using differential calculus of set-valued maps. More importantly, in order to avoid the essential difficulty with the well-posedness of higher order adjoint equations, using again the classical variational analysis approach, only the first and the second order adjoint equations are needed to formulate the second order necessary optimality condition, in which the solutions to the second order adjoint equation are understood in the sense of the relaxed transposition.     

Large deviations for stochastic PDE with Lévy noise

We prove a large deviation principle result for solutions of abstract stochastic evolution equations perturbed by small Lévy noise. We use general large deviations theorems of Varadhan and Bryc coupled with the techniques of Feng and Kurtz (2006) [15], viscosity solutions of integro-partial differential equations in Hilbert spaces, and deterministic optimal control methods. The Laplace limit is identified as a viscosity solution of a Hamilton-Jacobi-Bellman equation of an associated control problem. We also establish exponential moment estimates for solutions of stochastic evolution equations driven by Lévy noise. General results are applied to stochastic hyperbolic equations perturbed by subordinated Wiener process.     

A new regularized stochastic approximation framework for stochastic inverse problems

We study the nonlinear inverse problem of estimating stochastic parameters in the fourth-order partial differential equation with random data. The primary focus is on developing a novel stochastic approximation framework for inverse problems consisting of three key components. As a first step, we reformulate the inverse problem into a stochastic convex optimization problem. The second step includes developing a new regularized stochastic extragradient framework for a nonlinear variational inequality, which subsumes the optimality conditions for the optimization formulation of the inverse problem. The third step involves modeling random variables by a Karhunen–Loève type finite-dimensional noise representation, allowing the direct and the inverse problems to be conveniently discretized. We show that the regularized extragradient methods are strongly convergent in a Hilbert space setting, and we also provide several auxiliary results for the inverse problem, including Lipschitz continuity and a derivative characterization of the solution map. We provide the outcome of computational experiments to estimate stochastic and deterministic parameters. The numerical results demonstrate the feasibility and effectiveness of the developed framework and validate stochastic approximation as an effective method for stochastic inverse problems.     

Necessary conditions for optimality in relaxed stochastic control problems

In this paper, we are concerned with optimal control problems where the system is driven by a stochastic differential equation of the Ito type. We study the relaxed model for which an optimal solution exists. This is an extension of the initial control problem, where admissible controls are measure valued processes. Using Ekeland's variational principle and some stability properties of the corresponding state equation and adjoint processes, we establish necessary conditions for optimality satisfied by an optimal relaxed control. This is the first version of the stochastic maximum principle that covers relaxed controls.     

Opticon: An algorithm for the optimal control of nonlinear stochastic models

In this paper we describe the algorithm OPTCON which has been developed for the optimal control of nonlinear stochastic models. It can be applied to obtain approximate numerical solutions of control problems where the objective function is quadratic and the dynamic system is nonlinear. In addition to the usual additive uncertainty, some or all of the parameters of the model may be stochastic variables. The optimal values of the control variables are computed in an iterative fashion: First, the time-invariant nonlinear system is linearized around a reference path and approximated by a time-varying linear system. Second, this new problem is solved by applying Bellman's principle of optimality. The resulting feedback equations are used to project expected optimal state and control variables. These projections then serve as a new reference path, and the two steps are repeated until convergence is reached. The algorithm has been implemented in the statistical programming system GAUSS. We derive some mathematical results needed for the algorithm and give an overview of the structure of OPTCON. Moreover, we report on some tentative applications of OPTCON to two small macroeconometric models for Austria.     

Dynamic programming in stochastic control of systems with delay

We consider optimal control problems for systems described by stochastic differential equations with delay (SDDE). We prove a version of Bellman's principle of optimality (the dynamic programming principle) for a general class of such problems. That the class in general means that both the dynamics and the cost depends on the past in a general way. As an application, we study systems where the value function depends on the past only through some weighted average. For such systems we obtain a Hamilton-Jacobi-Bellman partial differential equation that the value function must solve if it is smooth enough. The weak uniqueness of the SDDEs we consider is our main tool in proving the result. Notions of strong and weak uniqueness for SDDEs are introduced, and we prove that strong uniqueness implies weak uniqueness, just as for ordinary stochastic differential equations.